We Keep Coding

RandomAccessFile doesn't work with Minecraft Forge

I'm working at the moment on a Mod for Minecraft with a dedicated Gui system written in C++ and Qt5. I let my GUI and Minecraft communicate through a named pipe, but I have there a small problem. I can read and write with a simple Java and C++(Qt) program into the pipe. But when I create a new instance of my Pipeendpoint class in post init of Minecraft Forge it can't read anything from the Pipe. In a standalone system, it can read stuff.
Not working Forge Implementation:
package de.CoderDE.CodersAnimationEditor;
import java.io.FileNotFoundException;
import de.CoderDE.CodersAnimationEditor.Pipe.PipeEndpoint;
import net.minecraft.client.Minecraft;
import net.minecraftforge.fml.client.registry.ClientRegistry;
import net.minecraftforge.fml.common.event.FMLInitializationEvent;
import net.minecraftforge.fml.common.event.FMLPostInitializationEvent;
import net.minecraftforge.fml.common.event.FMLPreInitializationEvent;
import net.minecraftforge.fml.relauncher.Side;
import net.minecraftforge.fml.relauncher.SideOnly;
#SideOnly(Side.CLIENT)
public class ClientProxy extends CommonProxy {
static PipeEndpoint pendpoint;
#Override
public void preInit(FMLPreInitializationEvent e) {
super.preInit(e);
}
#Override
public void init(FMLInitializationEvent e) {
super.init(e);
}
#Override
public void postInit(FMLPostInitializationEvent e) {
super.postInit(e);
try {
pendpoint = new PipeEndpoint();
} catch (FileNotFoundException e1) {
e1.printStackTrace();
}
try {
Thread.sleep(10000);
} catch (InterruptedException e1) {
e1.printStackTrace();
}
}
}
Working standalone implementation:
import java.io.FileNotFoundException;
import de.CoderDE.CodersAnimationEditor.Pipe.PipeEndpoint;
public class Main {
static PipeEndpoint pipe;
public static void main(String[] args) {
try {
pipe = new PipeEndpoint();
} catch (FileNotFoundException e1) {
e1.printStackTrace();
}
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
And the important PipeEndpoint class:
package de.CoderDE.CodersAnimationEditor.Pipe;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.RandomAccessFile;
public class PipeEndpoint {
private Thread reciever;
private RandomAccessFile pipe;
public PipeEndpoint() throws FileNotFoundException {
pipe = new RandomAccessFile("\\\\.\\pipe\\CodersAnimationEditor", "rw");
reciever = new Thread(new PipeEndpointReciever());
reciever.start();
}
private class PipeEndpointReciever implements Runnable {
#Override
public void run() {
try {
while (true) {
System.out.print((char)pipe.read());
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
And with "can't read anything" I mean that it never returns from pipe.read().
Oh, and the Java application starts after the C++(Qt) LocalServer started listening and waits for a new connection.

Sending a file line by line, in 2 seconds intervals, using ticker behaviours

My question is: How to send a line of a file to another agent every 2 seconds using ticker behaviours?
More specifically, in the first iteration, the agent sends the first line. In the second, the agent sends the second line etc.
My code below:
package pack1;
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.LineNumberReader;
import jade.core.AID;
import jade.core.Agent;
import jade.core.behaviours.TickerBehaviour;
import jade.lang.acl.ACLMessage;
import jade.wrapper.ControllerException;
public class Agent2 extends Agent {
private static final long serialVersionUID = 1L;
int nombre_ligne = 0;
BufferedReader lecteurAvecBuffer = null;
#Override
protected void setup() {
FileInputStream fis;
try {
fis = new FileInputStream("/home/hduser/Bureau/word.txt");
#SuppressWarnings("resource")
LineNumberReader l = new LineNumberReader(new BufferedReader(
new InputStreamReader(fis)));
while ((l.readLine()) != null) {
nombre_ligne = l.getLineNumber();
}
lecteurAvecBuffer = new BufferedReader(new FileReader(
"/home/hduser/Bureau/esclave1/abc.txt"));
int a = 1;
while (a <= ((int) nombre_ligne) / 3) {
a++;
String word = lecteurAvecBuffer.readLine();
addBehaviour(new TickerBehaviour(this, 2000) {
private static final long serialVersionUID = 1L;
#Override
protected void onTick() {
ACLMessage message = new ACLMessage(ACLMessage.INFORM);
message.addReceiver(new AID("agent1", AID.ISLOCALNAME));
message.setContent(word);
send(message);
}
});
a++;
}
lecteurAvecBuffer.close();
} catch (FileNotFoundException exc) {
System.out.println("Erreur d'ouverture");
} catch (IOException e) {
e.printStackTrace();
}
}
protected void takeDown() {
System.out.println("Destruction de l'agent");
}
#Override
protected void afterMove() {
try {
System.out.println(" La Destination : "
+ this.getContainerController().getContainerName());
} catch (ControllerException e) {
e.printStackTrace();
}
}
}

You don't say nothing about what is the problem with your code. I guess you get at least a compiler message about:
message.setContent(word);
As you access a local variable from an inner class, you must declare the variable as final in the context, like:
final String word = lecteurAvecBuffer.readLine();

How to find very large file copying status in java

I have requirement for monitoring the files copying status in a directory and the files are placed continuously into the directory in java.
I am planing to use Executor framework to find out individual files copy status and I have written below code but it is not working as expected, file without copy completion I am getting notification as copying got completed.
private boolean isFileCopied(String filePath) {
File file = new File(filePath);
Scanner scanner;
boolean isCopied = true;
while (true) {
try {
scanner = new Scanner(file);
isCopied = false;
} catch (FileNotFoundException e) {
System.out.println(filePath + " File is in copy State. ");
sleepFile();
}
if (isCopied == false) {
break;
}
}
System.out.println(filePath + " copy completed");
return isCopied;
}
private static void sleepFile() {
System.out.println("sleeping for 10 seconds");
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
Please someone help me out how can I find the exact status of a file like file "copy in progress" or "copying done" and how can I monitor each and every file copying status If bunch of large files placed in a directory.
I have used watcher API but it is not solving my purpose. even the file without copying got completed I am getting notification as copying got completed. Below are my code changes.
Using watcher service:
import static java.nio.file.StandardWatchEventKinds.ENTRY_CREATE;
import static java.nio.file.StandardWatchEventKinds.ENTRY_DELETE;
import static java.nio.file.StandardWatchEventKinds.ENTRY_MODIFY;
import java.io.IOException;
import java.nio.file.FileSystems;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.WatchEvent;
import java.nio.file.WatchKey;
import java.nio.file.WatchService;
public class FolderWatchDemo {
public static void main(String[] args) {
final Path outputWatchFolderPath = Paths.get("/outputFolder/");
final Path sourceFolderPath = Paths.get("/sourceFolder/");
try {
//Registering outputWatchFolderPath
WatchService watcher = FileSystems.getDefault().newWatchService();
Path dir = Paths.get(outputWatchFolderPath.toString());
dir.register(watcher, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
System.out.println("Watch Service registered for dir: " + dir.getFileName());
//copy files from inputfolder to o
for (final Path path: Files.newDirectoryStream(sourceFolderPath))
Files.copy(path, outputWatchFolderPath.resolve(path.getFileName()));
while (true) {
WatchKey key;
try {
key = watcher.take();
} catch (InterruptedException ex) {
return;
}
for (WatchEvent<?> event : key.pollEvents()) {
WatchEvent.Kind<?> kind = event.kind();
WatchEvent<Path> ev = (WatchEvent<Path>) event;
if (kind == ENTRY_CREATE) {
System.out.println("file got created !!");
}
if (kind == ENTRY_MODIFY) {
System.out.println("copying got completed !!");
}
if (kind == ENTRY_DELETE) {
System.out.println("file deleted successfully !!");
}
}
boolean valid = key.reset();
if (!valid) {
break;
}
}
} catch (IOException ex) {
System.err.println(ex);
}
}
}
Thanks in Advance.

There is a very elegant way to create a Call Back system and implementing it via implementing ReadableByteChannel in order to monitor the progerss of copying files. Also the benefit is there is no need to monitor a directory. You can explicitly monitor the progress of the file which is being copied.
The main idea is proposed by this site, but changed it a little to fit in your problem:
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.ByteBuffer;
import java.nio.channels.Channels;
import java.nio.channels.FileChannel;
import java.nio.channels.ReadableByteChannel;
import java.nio.file.Files;
import java.nio.file.Paths;
interface ProgressCallBack {
public void callback(CallbackByteChannel rbc, double progress);
}
public class Main{
public static void main(String[] args) {
ProgressCallBack progressCallBack = new ProgressCallBack() {
#Override
public void callback(CallbackByteChannel rbc, double progress) {
System.out.println(rbc.getReadSoFar());
System.out.println(progress);
}
};
try {
copy("SOURCE FILE PATH", "DESTINATION FILE PATH", progressCallBack);
} catch (IOException e) {
e.printStackTrace();
}
}
public static void copy(String source, String destination, ProgressCallBack callBack) throws IOException {
FileOutputStream fos = null;
FileChannel sourceChannel = null;
try {
sourceChannel = new FileInputStream(new File(source)).getChannel();
ReadableByteChannel rbc = new CallbackByteChannel(sourceChannel, Files.size(Paths.get(source)), callBack);
fos = new FileOutputStream(destination);
fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
} catch (Exception e) {
e.printStackTrace();
} finally {
if(sourceChannel.isOpen()){
sourceChannel.close();
}
fos.close();
}
}
}
class CallbackByteChannel implements ReadableByteChannel {
ProgressCallBack delegate;
long size;
ReadableByteChannel rbc;
long sizeRead;
CallbackByteChannel(ReadableByteChannel rbc, long expectedSize, ProgressCallBack delegate) {
this.delegate = delegate;
this.size = expectedSize;
this.rbc = rbc;
}
public void close() throws IOException {
rbc.close();
}
public long getReadSoFar() {
return sizeRead;
}
public boolean isOpen() {
return rbc.isOpen();
}
public int read(ByteBuffer bb) throws IOException {
int n;
double progress;
if ((n = rbc.read(bb)) > 0) {
sizeRead += n;
progress = size > 0 ? (double) sizeRead / (double) size * 100.0 : -1.0;
delegate.callback(this, progress);
}
return n;
}
}
Hope this help.

package test;
import static java.nio.file.StandardWatchEventKinds.ENTRY_CREATE;
import static java.nio.file.StandardWatchEventKinds.ENTRY_DELETE;
import static java.nio.file.StandardWatchEventKinds.ENTRY_MODIFY;
import java.io.IOException;
import java.nio.file.FileSystems;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
import java.nio.file.WatchEvent;
import java.nio.file.WatchKey;
import java.nio.file.WatchService;
public class FolderWatchDemo {
public static void main(String[] args) {
final Path outputWatchFolderPath = Paths.get("/outputFolder/");
final Path sourceFolderPath = Paths.get("/sourceFolder/");
try {
// Registering outputWatchFolderPath
WatchService watcher = FileSystems.getDefault().newWatchService();
Path dir = Paths.get(outputWatchFolderPath.toString());
dir.register(watcher, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
System.out.println("Watch Service registered for dir: " + dir.getFileName());
// copy files from inputfolder to o
for (final Path path : Files.newDirectoryStream(sourceFolderPath))
Files.copy(path, outputWatchFolderPath.resolve(path.getFileName()),
StandardCopyOption.REPLACE_EXISTING);
String fileName = null;
while (true) {
WatchKey key;
try {
key = watcher.take();
} catch (InterruptedException ex) {
return;
}
for (WatchEvent<?> event : key.pollEvents()) {
WatchEvent.Kind<?> kind = event.kind();
WatchEvent<Path> ev = (WatchEvent<Path>) event;
if (kind == ENTRY_CREATE) {
System.out.println("file got created !!" + ev.context());
}
if (kind == ENTRY_MODIFY) {
String fName = ev.context().getFileName().toString();
if (fileName != null && !fName.equals(fileName)) {
System.out.println("file copying completed !!" + fileName);
}
fileName = fName;
System.out.println("copying " + fName);
}
if (kind == ENTRY_DELETE) {
System.out.println("file deleted successfully !!");
}
}
boolean valid = key.reset();
if (!valid) {
break;
}
}
} catch (IOException ex) {
System.err.println(ex);
}
}
}

Get different results using main and junit in java

When I tested a simple producer/consumer example, I got a very strange result as below.
If I used main() to test the following code, I'll get the correct and expected result.
But I only can get the 1st directory correctly, the remaining works were dropped by the JUnit.
What is the exact reason?
Working code:
import java.io.File;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import org.junit.Test;
public class TestProducerAndConsumer {
public static void main(String[] args) {
BlockingQueue<File> queue = new LinkedBlockingQueue<File>(1000);
new Thread(new FileCrawler(queue, new File("C:\\"))).start();
new Thread(new Indexer(queue)).start();
}
}
Bad Code:
import java.io.File;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import org.junit.Test;
public class TestProducerAndConsumer {
#Test
public void start2() {
BlockingQueue<File> queue = new LinkedBlockingQueue<File>(1000);
new Thread(new FileCrawler(queue, new File("C:\\"))).start();
new Thread(new Indexer(queue)).start();
}
}
Other function code:
import java.io.File;
import java.util.Arrays;
import java.util.concurrent.BlockingQueue;
public class FileCrawler implements Runnable {
private final BlockingQueue<File> fileQueue;
private final File root;
private int i = 0;
public FileCrawler(BlockingQueue<File> fileQueue, File root) {
this.fileQueue = fileQueue;
this.root = root;
}
#Override
public void run() {
try {
craw(root);
} catch (InterruptedException e) {
System.out.println("shit!");
e.printStackTrace();
Thread.currentThread().interrupt();
}
}
private void craw(File file) throws InterruptedException {
File[] entries = file.listFiles();
//System.out.println(Arrays.toString(entries));
if (entries != null && entries.length > 0) {
for (File entry : entries) {
if (entry.isDirectory()) {
craw(entry);
} else {
fileQueue.offer(entry);
i++;
System.out.println(entry);
System.out.println(i);
}
}
}
}
public static void main(String[] args) throws InterruptedException {
FileCrawler fc = new FileCrawler(null, null);
fc.craw(new File("C:\\"));
System.out.println(fc.i);
}
}
import java.io.File;
import java.util.concurrent.BlockingQueue;
public class Indexer implements Runnable {
private BlockingQueue<File> queue;
public Indexer(BlockingQueue<File> queue) {
this.queue = queue;
}
#Override
public void run() {
try {
while (true) {
indexFile(queue.take());
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
private void indexFile(File file) {
System.out.println("Indexing ... " + file);
}
}

Junit's presumably allowing the JVM & threads to terminate, once the test is finished -- thus your threads do not complete working.
Try waiting for the threads to 'join':
Thread crawlerThread = new Thread(new FileCrawler(queue, new File("C:\\")));
Thread indexerThread = new Thread(new Indexer(queue));
crawlerThread.start();
indexerThread.start();
//
// wait for them to finish.
crawlerThread.join();
indexerThread.join();
This should help.
.. The other thing that can go wrong, is that log output (via Log4J) can sometimes be truncated at the end of execution; flushing & pausing can help. But I don't think that will affect you here.

Is there a way to have FileChannels close automatically?

I am currently developing an application that requires random access to many (60k-100k) relatively large files.
Since opening and closing streams is a rather costly operation, I'd prefer to keep the FileChannels for the largest files open until they are no longer needed.
The problem is that since this kind of behaviour is not covered by Java 7's try-with statement, I'm required to close all the FileChannels manually.
But that is becoming increasingly too complicated since the same files could be accessed concurrently throughout the software.
I have implemented a ChannelPool class that can keep track of opened FileChannel instances for each registered Path. The ChannelPool can then be issued to close those channels whose Path is only weakly referenced by the pool itself in certain intervals.
I would prefer an event-listener approach, but I'd also rather not have to listen to the GC.
The FileChannelPool from Apache Commons doesn't address my problem, because channels still need to be closed manually.
Is there a more elegant solution to this problem? And if not, how can my implementation be improved?
import java.io.IOException;
import java.lang.ref.WeakReference;
import java.nio.channels.FileChannel;
import java.nio.file.Path;
import java.nio.file.StandardOpenOption;
import java.util.Map;
import java.util.Timer;
import java.util.TimerTask;
import java.util.concurrent.ConcurrentHashMap;
public class ChannelPool {
private static final ChannelPool defaultInstance = new ChannelPool();
private final ConcurrentHashMap<String, ChannelRef> channels;
private final Timer timer;
private ChannelPool(){
channels = new ConcurrentHashMap<>();
timer = new Timer();
}
public static ChannelPool getDefault(){
return defaultInstance;
}
public void initCleanUp(){
// wait 2 seconds then repeat clean-up every 10 seconds.
timer.schedule(new CleanUpTask(this), 2000, 10000);
}
public void shutDown(){
// must be called manually.
timer.cancel();
closeAll();
}
public FileChannel getChannel(Path path){
ChannelRef cref = channels.get(path.toString());
System.out.println("getChannel called " + channels.size());
if (cref == null){
cref = ChannelRef.newInstance(path);
if (cref == null){
// failed to open channel
return null;
}
ChannelRef oldRef = channels.putIfAbsent(path.toString(), cref);
if (oldRef != null){
try{
// close new channel and let GC dispose of it
cref.channel().close();
System.out.println("redundant channel closed");
}
catch (IOException ex) {}
cref = oldRef;
}
}
return cref.channel();
}
private void remove(String str) {
ChannelRef ref = channels.remove(str);
if (ref != null){
try {
ref.channel().close();
System.out.println("old channel closed");
}
catch (IOException ex) {}
}
}
private void closeAll() {
for (Map.Entry<String, ChannelRef> e : channels.entrySet()){
remove(e.getKey());
}
}
private void maintain() {
// close channels for derefenced paths
for (Map.Entry<String, ChannelRef> e : channels.entrySet()){
ChannelRef ref = e.getValue();
if (ref != null){
Path p = ref.pathRef().get();
if (p == null){
// gc'd
remove(e.getKey());
}
}
}
}
private static class ChannelRef{
private FileChannel channel;
private WeakReference<Path> ref;
private ChannelRef(FileChannel channel, WeakReference<Path> ref) {
this.channel = channel;
this.ref = ref;
}
private static ChannelRef newInstance(Path path) {
FileChannel fc;
try {
fc = FileChannel.open(path, StandardOpenOption.READ);
}
catch (IOException ex) {
return null;
}
return new ChannelRef(fc, new WeakReference<>(path));
}
private FileChannel channel() {
return channel;
}
private WeakReference<Path> pathRef() {
return ref;
}
}
private static class CleanUpTask extends TimerTask {
private ChannelPool pool;
private CleanUpTask(ChannelPool pool){
super();
this.pool = pool;
}
#Override
public void run() {
pool.maintain();
pool.printState();
}
}
private void printState(){
System.out.println("Clean up performed. " + channels.size() + " channels remain. -- " + System.currentTimeMillis());
for (Map.Entry<String, ChannelRef> e : channels.entrySet()){
ChannelRef cref = e.getValue();
String out = "open: " + cref.channel().isOpen() + " - " + cref.channel().toString();
System.out.println(out);
}
}
}
EDIT:
Thanks to fge's answer I have now exactly what I needed. Thanks!
import com.google.common.cache.CacheBuilder;
import com.google.common.cache.CacheLoader;
import com.google.common.cache.LoadingCache;
import com.google.common.cache.RemovalListener;
import com.google.common.cache.RemovalNotification;
import java.io.IOException;
import java.nio.channels.FileChannel;
import java.nio.file.Path;
import java.nio.file.StandardOpenOption;
import java.util.concurrent.ExecutionException;
public class Channels {
private static final LoadingCache<Path, FileChannel> channelCache =
CacheBuilder.newBuilder()
.weakKeys()
.removalListener(
new RemovalListener<Path, FileChannel>(){
#Override
public void onRemoval(RemovalNotification<Path, FileChannel> removal) {
FileChannel fc = removal.getValue();
try {
fc.close();
}
catch (IOException ex) {}
}
}
)
.build(
new CacheLoader<Path, FileChannel>() {
#Override
public FileChannel load(Path path) throws IOException {
return FileChannel.open(path, StandardOpenOption.READ);
}
}
);
public static FileChannel get(Path path){
try {
return channelCache.get(path);
}
catch (ExecutionException ex){}
return null;
}
}

Have a look here:
http://code.google.com/p/guava-libraries/wiki/CachesExplained
You can use a LoadingCache with a removal listener which would close the channel for you when it expires, and you can specify expiry after access or write.