We Keep Coding

Java: Reading and Reading/Writing to a file via multiple threads?

I am a newcomer to Java threads and I have a situation where I must continuously read from a file and also in a separate operation read and edit the same file(not continuously but at any random time). I understand this is a situation that can benefit from the use of multiple threads. How do I set up this situation so that both processes can take place together. I have the following code which is not thread-enabled :
Continuously reading segment(assume that readFile method has the code to read a file line-by-line):
public void init() throws ServletException
{
AlertsMonitor monitor = new AlertsMonitor();
try {
while(true)
monitor.readFile();
} catch (Exception e) {
e.printStackTrace();
}
}
And the following method may execute at any time:
public void searchUser(String user)
{
new FileSearcher().searchUser(user); //Search for the supplied string
//inside the same file
}
Both methods work on the same file. Thanks for any help !

Workaround for Java bug which causes crash dump

A program that I've developed is crashing the JVM occasionally due to this bug: http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8029516. Unfortunately the bug has not been resolved by Oracle and the bug report says that there are no known workarounds.
I've tried to modify the example code from the bug report by calling .register(sWatchService, eventKinds) in the KeyWatcher thread instead, by adding all pending register request to a list that I loop through in the KeyWatcher thread but it's still crashing. I'm guessing this just had the same effect as synchronizing on sWatchService (like the submitter of the bug report tried).
Can you think of any way to get around this?

From comments:
It appears that we have an issue with I/O cancellation when there is a pending ReadDirectoryChangesW outstanding.
The statement and example code indicate that the bug is triggered when:
There is a pending event that has not been consumed (it may or may not be visible to WatchService.poll() or WatchService.take())
WatchKey.cancel() is called on the key
This is a nasty bug with no universal workaround. The approach depends on the specifics of your application. Consider pooling watches to a single place so you don't need to call WatchKey.cancel(). If at one point the pool becomes too large, close the entire WatchService and start over. Something similar to.
public class FileWatcerService {
static Kind<?>[] allEvents = new Kind<?>[] {
StandardWatchEventKinds.ENTRY_CREATE,
StandardWatchEventKinds.ENTRY_DELETE,
StandardWatchEventKinds.ENTRY_MODIFY
};
WatchService ws;
// Keep track of paths and registered listeners
Map<String, List<FileChangeListener>> listeners = new ConcurrentHashMap<String, List<FileChangeListener>>();
Map<WatchKey, String> keys = new ConcurrentHashMap<WatchKey, String>();
boolean toStop = false;
public interface FileChangeListener {
void onChange();
}
public void addFileChangeListener(String path, FileChangeListener l) {
if(!listeners.containsKey(path)) {
listeners.put(path, new ArrayList<FileChangeListener>());
keys.put(Paths.get(path).register(ws, allEvents), path);
}
listeners.get(path).add(l);
}
public void removeFileChangeListener(String path, FileChangeListener l) {
if(listeners.containsKey(path))
listeners.get(path).remove(l);
}
public void start() {
ws = FileSystems.getDefault().newWatchService();
new Thread(new Runnable() {
public void run() {
while(!toStop) {
WatchKey key = ws.take();
for(FileChangeListener l: listeners.get(keys.get(key)))
l.onChange();
}
}
}).start();
}
public void stop() {
toStop = true;
ws.close();
}
}

I've managed to create a workaround though it's somewhat ugly.
The bug is in JDK method WindowsWatchKey.invalidate() that releases native buffer while the subsequent calls may still access it. This one-liner fixes the problem by delaying buffer clean-up until GC.
Here is a compiled patch to JDK. In order to apply it add the following Java command-line flag:
-Xbootclasspath/p:jdk-8029516-patch.jar
If patching JDK is not an option in your case, there is still a workaround on the application level. It relies on the knowledge of Windows WatchService internal implementation.
public class JDK_8029516 {
private static final Field bufferField = getField("sun.nio.fs.WindowsWatchService$WindowsWatchKey", "buffer");
private static final Field cleanerField = getField("sun.nio.fs.NativeBuffer", "cleaner");
private static final Cleaner dummyCleaner = Cleaner.create(Thread.class, new Thread());
private static Field getField(String className, String fieldName) {
try {
Field f = Class.forName(className).getDeclaredField(fieldName);
f.setAccessible(true);
return f;
} catch (Exception e) {
throw new IllegalStateException(e);
}
}
public static void patch(WatchKey key) {
try {
cleanerField.set(bufferField.get(key), dummyCleaner);
} catch (IllegalAccessException e) {
throw new IllegalStateException(e);
}
}
}
Call JDK_8029516.patch(watchKey) right after the key is registred, and it will prevent watchKey.cancel() from releasing the native buffer prematurely.

You might not be able to work around the problem itself but you could deal with the error and handle it. I don't know your specific situation but I could imagine the biggest issue is the crash of the whole JVM. Putting all in a try block does not work because you cannot catch a JVM crash.
Not knowing more about your project makes it difficult to suggest a good/acceptable solution, but maybe this could be an option: Do all the file watching stuff in a separate JVM process. From your main process start a new JVM (e.g. using ProcessBuilder.start()). When the process terminates (i.e. the newly started JVM crashes), restart it. Obviously you need to be able to recover, i.e. you need to keep track of what files to watch and you need to keep this data in your main process too.
Now the biggest remaining part is to implement some communication between the main process and the file watching process. This could be done using standard input/output of the file watching process or using a Socket/ServerSocket or some other mechanism.

Program displays filenames in a JTextArea as it walks the directory tree but I don't know how to stop it via a keypress

There are two windows: a GUI for user input and Output window for list of filenames found. Execution must be user-stoppable via a keypress and must leave both windows open because the program processes subdirectories, so it can run a long time, possibly stepping thru 100_000 files, either producing tons of output or none at all, depending on how user's filename pattern matches files encountered in the selected starting node.
Here's my question:
How do I look for a keypress (e.g., ESC or CTRL-C) to allow user to terminate? (Clicking red X isn't an option since that closes windows; user needs to see what's been found before termination. Doing so does not close either window anyway since all buttons are disabled once tree walk begins.)
I've tried putting keyListeners in several places, but once the "Start" button is clicked, all the swing components are disabled.
This seems like such a common situation that I'm surprised I can't find any textbook, thread, or Google info that directly answers the question. So I'm afraid it's not gonna be at all easy. That would be no surprise. I may have found a clue here but I can't get it to compile and the link contained there doesn't lead to that code snippet.
The search begins when the Search button is clicked:
private void jbSearchActionPerformed(ActionEvent evt) {
SearchyGUI.doIt();
}
The doIt() method walks the directory tree by an extension of SimplefileVisitor:
public class OverriddenFileVisitor extends SimpleFileVisitor<Path> {
...
}
public static void doIt(){
try {
visitor = new OverriddenFileVisitor();
info.setVisible(true);
Files.walkFileTree(SearchyGUI.p , visitor);
}
catch (Exception e) { }
}
}
Output is written to jTextArea1 via the report() method:
public static void report(String s){
Output.jTextArea1.append(s + "\n");
}
This is done primarily in the visitFile() method of SimpleFileVisitor:
public FileVisitResult visitFile(Path f, BasicFileAttributes a) throws IOException {
report(foundkt + "--" + f.getFileName().toString());
return FileVisitResult.CONTINUE;
}
Here's the main class:
public class SearchyGUI {
static Output info;
static Path p ;
static FileVisitor visitor ;
static GUI gui
public static void main(String args[]) {
java.awt.EventQueue.invokeLater(new Runnable() {
public void run() {
gui = new GUI();
gui.setVisible(true);
}
});
java.awt.EventQueue.invokeLater(new Runnable() {
public void run() {
info = new Output();
}
});
}

The problem is you are hogging the GUI thread, so the GUI thread can't process any events originating from the user.
You need to create a new Thread and do the work in there. Then, to display output from that thread, you can use SwingUtilities.invokeLater or something like that.

The Key Bindings API is probably the best choice for monitoring key strokes.
I would also add a [Cancel] button to the UI, which shared the same action...
public class CancelAction extends AbstractAction {
public CancelAction() {
putValue(NAME, "Cancel");
}
public void actionPerformed(ActionEvent evt) {
// Perform the cancel operation...
}
}
Then some where else in your code...
CancelAction cancelAction = new CancelAction();
JButton cancelButton = new JButton(cancelAction);
InputMap im = getInputMap(WHEN_IN_FOCUSED_WINDOW);
ActionMap am = getActionMap();
im.put(KeyStroke.getKeyStroke(KeyEvent.VK_ESCAPE, 0), "Cancel");
am.put("Cancel", am);
Now the other problem you're going to have is the fact that you look like you are running a long running task within the context of the Event Dispatching Thread. This is going to prevent your program from being able to update the UI or allow the user to interact with the UI.
If you need to make changes to the UI (ie, show the output of the file processing), you should try a SwingWorker.
The main reason being is that it allows you to execute the long running task in another thread, but provides the mechanism for re-syncing updates back to the EDT, where it is safe to make changes to the UI.
Take a look at Concurrency in Swing for more details.
Regardless of which direction you take, you're going to need to supply a reference to the object that is carrying out the task and provide some kind of "cancel" flag, which the "task" object will need to monitor

The way I had left this program last night was unsatisfactory since Exit resulted in user not being able to see the output so far displayed (it could be useful). So I established window listeners and used the close event to set a boolean aborted to true to prevent further output to the window, but the thread kept running, which led to intermittent problems if another search was started before the thread ended.
Here's how I fixed it.
The FileVisitor interface has 4 methods to implement to walk the tree--two for each file visited, two for each directory. Each returns a FileVisitResult which is normally FileVisitResult.CONTINUE. By changing the return value to FileVisitResult.TERMINATE in the file visitor thread, it terminates appropriately! That is, I set a flag that the thread could check and take appropriate action, which is exactly what #MadProgrammer suggested.
public static FileVisitResult disposition = FileVisitResult.CONTINUE;
...
private static void report(String s){
if (! aborted)
try{
Output.jTextArea1.append(s + "\n");
}
catch (Exception e){
aborted = true ;
disposition = FileVisitResult.TERMINATE;
}
}
...
#Override
public FileVisitResult visitFile(Path f, BasicFileAttributes a) throws IOException {
f1 = new File(f.getParent().toString() + "\\" + f.getFileName().toString());
long filesize = f1.length();
report(f.getFileName().toString() + "\t found in " + f.getParent().toString());
return disposition;
}
I am one happy camper! Thank you BOTH for your ideas and input.

Well, I made it stop. I guess if you wander the woods long enough you'll find a gnome. I read Robin's hint last week and sort of gave up. Then I read some more and more. And then more. But Robin assured me that gnomes DO exist in these here woods!
The code I used was a modification of some I found for a MatLab/Java app. (Why'd I even look at it?? Best apparent Google hint.)
I made the "file visitor" (directory tree walker component) startable as a thread as Robin advised:
public class OverriddenFileVisitor extends SimpleFileVisitor<Path> implements Runnable{
// ................................................................^^^^^^^^^^^^^^^^^^^
In doIt() I made a couple of changes, moving the lines that process the directory to the now-runnable class and started the file visitor as its own thread in doIt():
public static void doIt(){
try {
new OverriddenFileVisitor().startTh();
//^^^^^^^^^^
//(moved) Files.walkFileTree(SearchyGUI.p , visitor);
...
I added the new method in the previous line to OverriddenFileVisitor class: (This is the main part of the MatLab/Java code that made sense to me so I used and modified it.)
public void startTh() {
Thread t = new Thread(this);
t.start();
}
And I inserted the overridden run() method for the class:
public void run() {
try {
Files.walkFileTree(SearchyGUI.p , this); // Used to be in doIt().
}
catch (IOException ex) { }
}
It ran and gave correct results and stopped when I hit Exit button, which "became" enabled after revising the file visitor to run in its own thread, which is what #Robin Green was saying. I almost feel like I know what I've done.
P.S. Note that I already was able to get my output via invokeLater()--last several lines of original question.
It's not finished but it's much more satisfactory.

Unable to lock files on Linux using java.nio.channels.FileLock

I was creating an application in Java for which I want only one instance running. For this purpose I created a file and got a lock while my application is running.
I have following code which works on Windows, but failed on Linux: once I acquire a lock without unlocking it I can get another lock on it.
import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.channels.FileChannel;
import java.nio.channels.FileLock;
public class MyApp {
private static File f;
private static FileChannel channel;
private static FileLock lock;
public static void main(String[] args) {
try {
f = new File("RingOnRequest.lock");
// Check if the lock exist
if (f.exists()) {
// if exist try to delete it
f.delete();
}
// Try to get the lock
channel = new RandomAccessFile(f, "rw").getChannel();
lock = channel.tryLock();
if(lock == null)
{
// File is lock by other application
channel.close();
throw new RuntimeException("Only 1 instance of MyApp can run.");
}
// Add shutdown hook to release lock when application shutdown
ShutdownHook shutdownHook = new ShutdownHook();
Runtime.getRuntime().addShutdownHook(shutdownHook);
//Your application tasks here..
System.out.println("Running");
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
catch(IOException e)
{
throw new RuntimeException("Could not start process.", e);
}
}
public static void unlockFile() {
// release and delete file lock
try {
if(lock != null) {
lock.release();
channel.close();
f.delete();
}
} catch(IOException e) {
e.printStackTrace();
}
}
static class ShutdownHook extends Thread {
public void run() {
unlockFile();
}
}
}

You are deleting the lock file every time you run, so only one process can have a lock on it.
When you use FileLock, it is purely advisory—acquiring a lock on a file may not stop you from doing anything…reading, writing, and deleting a file may all be possible even when another process has acquired a lock. Sometimes, a lock might do more than this on a particular platform, but this behavior is unspecified, and relying on more than is guaranteed in the class documentation is a recipe for failure.
An "advisory lock" is only a signal that is visible to other processes that bother to look for it. If you depend on it for more than that, your program will break when run on some other platform.
Why would you delete the lock file anyway? The lock file is just like a boolean flag that is visible to every process on the system. Design your protocol to use it that way, and you'll have a reliable, cross platform locking mechanism.

Why don't you save the PID into a file, and instead of locking the file, verify if there's a process with that ID. If there is, and it's an instance of your application, you know it's already running.
A socket might be a good idea as well, since you can use it to communicate to the running instance something.
EDIT:
Also, from FileLock's javadoc:
Whether or not a lock actually prevents another program from accessing
the content of the locked region is system-dependent and therefore
unspecified.

Use mkdir. On unix systems this is an atomic operation – it will succeed if a new directory is successfully created, otherwise it will fail.
Example:
File lockFile = new File("/path/to/lockdir");
boolean hasLock = lockFile.mkdir();
if (!hasLock) {
throw new IOException("could not get lock");
}
// do stuff
lockFile.delete();

I used same sample as you and got same problem on Mac OS X. It seems that file lock does not prevent file deletion on POSIX systems . Your app wil still have some kind of handle to that file until you unlock it. So consider using lock file with PID in it's name( or inside file).

I tested it on both Windows and Linux. Works fine. The lock file gets deleted automatically when the application closes normally. So you don't have to worry about the lock file staying there when you restart the application. Just comment out the following lines:
if (f.exists()) {
// if exist try to delete it
f.delete();
}
However, you may want to consider what happens if your application crashes and does not close in a normal fashion.

Recently i encountered the same kind of problem, but in my case i had an advantage: my application polled some directory only after some timeout. As my application did not immediately poll for directory i wrote special class that creates lock file with his own PID inside in init method, after that before it tries to work with directory it needs to call ownedLock() - if it returns true then we can work otherwise exit(code is in Kotlin but you will get the main idea):
import java.io.File
import java.lang.management.ManagementFactory
class DirectoryLocker(private val directory: String, private val lockName: String) {
private val lockFile by lazy { File("$directory/$lockName.lock") }
// Will try to acquire lock to directory, whoever last writes its pid to file owns the directory
fun acquireLock() = with(lockFile) {
createNewFile()
writeText(currentPID())
}
fun ownedLock(): Boolean = lockFilePid() == currentPID()
fun releaseOwnedLock() {
if(lockFilePid() == currentPID()) lockFile.delete()
}
private fun currentPID(): String {
val processName = ManagementFactory.getRuntimeMXBean().name
return processName.split("#".toRegex()).first()
}
private fun lockFilePid(): String? {
return if(lockFile.exists()) lockFile.readLines().first() else null
}
}

IDE-Style program running

My Goal
I am attempting to make a Java program in which a user can select any .class or .jar file from their computer. My program will then pop up a JInternalFrame with a JEditorPane in it as the console, capturing any console output from the user's program. When the user's program closes (calls System.exit(int status);), my program must not close along with it. My program might also have such features as a button to immediately stop the user's program and others an IDE would. My program need not compile Java code, only run .class and .jar files.
My Experience
I have made a small test version of this program wherein I got two specific files from a package and had the user click one of two buttons, each representing one of the two programs. A press of a button calls the following method:
private void run(Class runnable)
{
java.lang.reflect.Method[] m = runnable.getMethods();
boolean hasMain = false;
for (int i = 0; i < m.length; i++)
{
if (m[i].getName().equals("main") && m[i].getParameterTypes()[0].isArray() && m[i].getParameterTypes()[0].getName().contains("java.lang.String"))
try
{
Object invoke = m[i].invoke(null, (Object)globalArgs);
hasMain = true;
hub.setExtendedState(Hub.ICONIFIED);
numPrograms++;
}
catch (Throwable t)
{
java.util.logging.Logger.getLogger(Hub.class.getName()).log(java.util.logging.Level.SEVERE, null, t);
javax.swing.JOptionPane.showMessageDialog(null, "Could not run " + runnable.getName(), "Error in invocation", javax.swing.JOptionPane.ERROR_MESSAGE);
}
finally
{
break;
}
}
if (!hasMain)
javax.swing.JOptionPane.showMessageDialog(null, runnable.getName()
+ " does not have a public static main method that\nreturns void and takes in an array of Strings",
"No main method", javax.swing.JOptionPane.ERROR_MESSAGE);
}
This method successfully calls either program's main method and runs a copy of said program. However, when any of the programs this hub has started calls the System.exit(int status) command, the hub closes, too. Also, I haven't the slightest clue as to how to capture console output.
My Questions
Does anyone have any experience or advice they would be willing to share to help me make a fully-functional program that can...
Open and run a compiled Java file (remember that .jar files may have more than one class with main(String[] args) method)
Catch System.exit(int status); so that the hub program handles the internal program's exiting
Catch new java.io.PrintStream().println(Object o) and similar calls and place their output in a JEditorPane
Make a button that, when pressed, stops the internal program from running
Possibly make all JFrames the internal program uses into JInternalFrames and place them in a JDesktopPane

If you don't want the other program (which you call through it's main method) to be able to shut down the JVM you're running in, you have, as I see it, three options:
1. Using a SecurityManager
Set up the SecurityManager so that it prevents the System.exit call:
public class Test {
public static void main(String args[]) {
SecurityManager sm = System.getSecurityManager();
System.setSecurityManager(new SecurityManager() {
#Override
public void checkExit(int status) {
throw new SecurityException("Client program exited.");
}
});
try {
System.out.println("hello");
System.exit(0);
System.out.println("world");
} catch (SecurityException se) {
System.out.println(se.getMessage());
}
}
}
Prints:
hello
Client program exited.
This is probably the nicest solution. This is the way application servers prevent an arbitrary servlet from terminating the entire server.
2. Separate JVM
Run the other program in a separate JVM, using for instance ProcessBuilder
import java.io.*;
public class Test {
public static void main(String args[]) throws IOException {
ProcessBuilder pb = new ProcessBuilder("java", "other.Program");
pb.redirectErrorStream();
Process p = pb.start();
InputStream is = p.getInputStream();
int ch;
while ((ch = is.read()) != -1)
System.out.print((char) ch);
is.close();
System.out.println("Client program done.");
}
}
3. Use shutdown hooks instead
Don't disallow the termination of the JVM, but instead add shutdown-hooks that cleans up the "hub" and exits gracefully. (This option probably only makes sense if your running one "external" program at a time.)
import java.io.*;
public class Test {
public static void main(String args[]) throws IOException {
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run() {
System.out.println("Uninitializing hub...");
System.out.println("Exiting gracefully.");
}
});
// Run client program
System.out.println("Running... running... running...");
System.exit(0);
}
}
Prints:
Running... running... running...
Uninitializing hub...
Exiting gracefully.