I'm trying to setup a WatchService on an MacOS (High Sierra) and it seems to be working ok except when I rename dir. As an example I rename "test" to "hello" and here's what I get:
ENTRY_CREATE: /Users/david/Desktop/watchme/hello
update: /Users/david/Desktop/watchme/test -> /Users/david/Desktop/watchme/hello
ENTRY_DELETE: /Users/david/Desktop/watchme/test
I would expect to see:
register: /Users/david/Desktop/watchme/hello
But that isn't happening so when I modify anything inside of "hello", the only response I get is:
ENTRY_MODIFY: /Users/david/Desktop/watchme/hello
Nothing about the files or folders inside "hello"
I copied the following code right from Oracle with the exception of hard coding the attrs so I can run it in Idea:
import java.nio.file.*;
import static java.nio.file.StandardWatchEventKinds.*;
import static java.nio.file.LinkOption.*;
import java.nio.file.attribute.*;
import java.io.*;
import java.util.*;
/**
* Example to watch a directory (or tree) for changes to files.
*/
public class WatchDir {
private final WatchService watcher;
private final Map<WatchKey,Path> keys;
private final boolean recursive;
private boolean trace = false;
#SuppressWarnings("unchecked")
private static <T> WatchEvent<T> cast(WatchEvent<?> event) {
return (WatchEvent<T>)event;
}
/**
* Register the given directory with the WatchService
*/
private void register(Path dir) throws IOException {
WatchKey key = dir.register(watcher, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
if (trace) {
Path prev = keys.get(key);
if (prev == null) {
System.out.format("register: %s\n", dir);
} else {
if (!dir.equals(prev)) {
System.out.format("update: %s -> %s\n", prev, dir);
}
}
}
keys.put(key, dir);
}
/**
* Register the given directory, and all its sub-directories, with the
* WatchService.
*/
private void registerAll(final Path start) throws IOException {
// register directory and sub-directories
Files.walkFileTree(start, new SimpleFileVisitor<Path>() {
#Override
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs)
throws IOException
{
register(dir);
return FileVisitResult.CONTINUE;
}
});
}
/**
* Creates a WatchService and registers the given directory
*/
private WatchDir(Path dir, boolean recursive) throws IOException {
this.watcher = FileSystems.getDefault().newWatchService();
this.keys = new HashMap<>();
this.recursive = recursive;
if (recursive) {
System.out.format("Scanning %s ...\n", dir);
registerAll(dir);
System.out.println("Done.");
} else {
register(dir);
}
// enable trace after initial registration
this.trace = true;
}
/**
* Process all events for keys queued to the watcher
*/
private void processEvents() {
for (;;) {
// wait for key to be signalled
WatchKey key;
try {
key = watcher.take();
} catch (InterruptedException x) {
return;
}
Path dir = keys.get(key);
if (dir == null) {
System.err.println("WatchKey not recognized!!");
continue;
}
for (WatchEvent<?> event: key.pollEvents()) {
WatchEvent.Kind kind = event.kind();
// TBD - provide example of how OVERFLOW event is handled
if (kind == OVERFLOW) {
continue;
}
// Context for directory entry event is the file name of entry
WatchEvent<Path> ev = cast(event);
Path name = ev.context();
Path child = dir.resolve(name);
// print out event
System.out.format("%s: %s\n", event.kind().name(), child);
if(child.toString().contains("testdir")) {
System.out.println("This is an update");
}
// if directory is created, and watching recursively, then
// register it and its sub-directories
if (recursive && (kind == ENTRY_CREATE)) {
try {
if (Files.isDirectory(child, NOFOLLOW_LINKS)) {
registerAll(child);
}
} catch (IOException x) {
// ignore to keep sample readbale
}
}
}
// reset key and remove from set if directory no longer accessible
boolean valid = key.reset();
if (!valid) {
keys.remove(key);
// all directories are inaccessible
if (keys.isEmpty()) {
break;
}
}
}
}
private static void usage() {
System.err.println("usage: java WatchDir [-r] dir");
System.exit(-1);
}
#SuppressWarnings("ParameterCanBeLocal")
public static void main(String[] args) throws IOException {
// parse arguments
args = new String[2]; // Added for debug
args[0] = "-r"; // Added for debug
args[1] = "/Users/david/Desktop/watchme"; // Added for debug
if (args.length == 0 || args.length > 2)
usage();
boolean recursive = false;
int dirArg = 0;
if (args[0].equals("-r")) {
if (args.length < 2)
usage();
recursive = true;
dirArg++;
}
// register directory and process its events
Path dir = Paths.get(args[dirArg]);
new WatchDir(dir, recursive).processEvents();
}
}
It's an interesting question. I tried on my own Mac and get the same result.
When we rename test to hello, OS first create hello and then delete test, both of watchme and test will get event. WatchKey of watchme folder gets ENTRY_CREATE and ENTRY_DELETE events.
In this step, though new dir name is hello, but after this code
WatchKey key = dir.register(watcher, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
the key we get is still test's watch key, that's why we see update log in console.
ENTRY_CREATE: /Users/david/Desktop/watchme/hello
update: /Users/david/Desktop/watchme/test -> /Users/david/Desktop/watchme/hello
ENTRY_DELETE: /Users/david/Desktop/watchme/test
WatchKey of test also recieved an event. But this event is an invalid event, the key(Infact this is hello's watchKey now) will be removed from keys map. So the change of hello folder will show nothing in console.
Here is the the interesting thing. Why we get test's watchKey when input dir is hello. When I debug code, stop it at
WatchKey key = dir.register(watcher, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
for a while when receive hello's ENTRY_CREATE event, and I get hello's key not test's key and register log not update log. It seems we can't see the hello folder at once when call dir.register method. I don't know more about what OS do when we rename a folder, hope others' reply.
This is my test code added logs in processEvents, you may try it yourself.
private void processEvents() {
for (; ; ) {
// wait for key to be signalled
WatchKey key;
try {
key = watcher.take();
} catch (InterruptedException x) {
return;
}
Path dir = keys.get(key);
if (dir == null) {
System.err.println("WatchKey not recognized!!");
continue;
} else {
System.out.println("CurrentDir=" + dir.getFileName() + " keySize=" + keys.size() + " valid=" + key.isValid());
}
for (WatchEvent<?> event : key.pollEvents()) {
WatchEvent.Kind kind = event.kind();
// TBD - provide example of how OVERFLOW event is handled
if (kind == OVERFLOW) {
continue;
}
// Context for directory entry event is the file name of entry
WatchEvent<Path> ev = cast(event);
Path name = ev.context();
Path child = dir.resolve(name);
// print out event
System.out.format("%s: %s\n", event.kind().name(), child);
// if directory is created, and watching recursively, then
// register it and its sub-directories
if (recursive && (kind == ENTRY_CREATE)) {
try {
if (Files.isDirectory(child, NOFOLLOW_LINKS)) {
registerAll(child);
}
} catch (IOException x) {
// ignore to keep sample readbale
}
}
}
// reset key and remove from set if directory no longer accessible
boolean valid = key.reset();
if (!valid) {
System.out.println("Remove From keys");
keys.remove(key);
// all directories are inaccessible
if (keys.isEmpty()) {
break;
}
}
}
}
Related
I am trying to write the facility to process media files using a watch folder. The Oracle example WatchDir demonstrates how to know when there are changes in a folder. However the issue with this is that I don't know when all the media has finished uploading. So for example when an SD card containing media which contains multiple files in different folders is dragged onto the watch folder, I need to be able process the media once all the files and sub folders are present. Media is not always just stored in single files but may have sidecar files so both sets of files need to be present in order to process the files correctly. Can anyone suggest how I can know that all files and sub folders have finished being copied into the watch folder?
This is my slightly modified version of the WatchDir to include logging:
public class WatchDir {
private final WatchService watcher;
private final Map<WatchKey,Path> keys;
private final boolean recursive;
private boolean trace = false;
#SuppressWarnings("unchecked")
static <T> WatchEvent<T> cast(WatchEvent<?> event) {
return (WatchEvent<T>)event;
}
/**
* Register the given directory with the WatchService
*/
private void register(Path dir) throws IOException {
WatchKey key = dir.register(watcher, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
if (trace) {
Path prev = keys.get(key);
if (prev == null) {
System.out.format("register: %s%n", dir);
} else {
if (!dir.equals(prev)) {
System.out.format("update: %s -> %s%n", prev, dir);
}
}
}
keys.put(key, dir);
}
/**
* Register the given directory, and all its sub-directories, with the
* WatchService.
*/
private void registerAll(final Path start) throws IOException {
// register directory and sub-directories
Files.walkFileTree(start, new SimpleFileVisitor<Path>() {
#Override
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs)
throws IOException
{
register(dir);
return FileVisitResult.CONTINUE;
}
});
}
/**
* Creates a WatchService and registers the given directory
*/
WatchDir(Path dir, boolean recursive) throws IOException {
this.watcher = FileSystems.getDefault().newWatchService();
this.keys = new HashMap<WatchKey,Path>();
this.recursive = recursive;
if (recursive) {
System.out.format("Scanning %s ...\n", dir);
registerAll(dir);
System.out.println("Done.");
} else {
register(dir);
}
// enable trace after initial registration
this.trace = true;
}
/**
* Process all events for keys queued to the watcher
*/
void processEvents() {
System.out.println("process event");
boolean processing = false;
for (;;) {
System.out.println("loop");
// wait for key to be signalled
WatchKey key;
try {
processing = false;
System.out.println("about to take");
key = watcher.take();
processing = true;
} catch (InterruptedException x) {
System.out.println("take interrupted");
return;
}
Path dir = keys.get(key);
if (dir == null) {
System.err.println("WatchKey not recognized!!");
continue;
}
for (WatchEvent<?> event: key.pollEvents()) {
System.out.println("poll");
WatchEvent.Kind kind = event.kind();
// TBD - provide example of how OVERFLOW event is handled
if (kind == OVERFLOW) {
System.out.println("Overflow");
continue;
}
// Context for directory entry event is the file name of entry
WatchEvent<Path> ev = cast(event);
Path name = ev.context();
Path child = dir.resolve(name);
// print out event
System.out.format("%s: %s\n", event.kind().name(), child);
// if directory is created, and watching recursively, then
// register it and its sub-directories
if (recursive && (kind == ENTRY_CREATE)) {
try {
if (Files.isDirectory(child, NOFOLLOW_LINKS)) {
registerAll(child);
}
} catch (IOException x) {
// ignore to keep sample readable
System.out.println("ex: " + x.getMessage());
}
}
}
// reset key and remove from set if directory no longer accessible
boolean valid = key.reset();
if (!valid) {
keys.remove(key);
System.out.println("finished this set of files");
// all directories are inaccessible
if (keys.isEmpty()) {
break;
}
}
if (processing) {
System.out.println("processing files...");
} else {
System.out.println("not processing files");
}
System.out.println("End of loop\n\n");
}
}
static void usage() {
System.err.println("usage: java WatchDir [-r] dir");
System.exit(-1);
}
public static void main(String[] args) throws IOException {
// parse arguments
if (args.length == 0 || args.length > 2)
usage();
boolean recursive = false;
int dirArg = 0;
if (args[0].equals("-r")) {
if (args.length < 2)
usage();
recursive = true;
dirArg++;
}
// register directory and process its events
Path dir = Paths.get(args[dirArg]);
new WatchDir(dir, recursive).processEvents();
}
}
Maybe you could not do that. According to the doc, The WatchService only provided these events:
static WatchEvent.Kind<Path> ENTRY_DELETE Directory entry deleted.
static WatchEvent.Kind<Path> ENTRY_MODIFY Directory entry modified.
static WatchEvent.Kind<Object> OVERFLOW A special event to indicate that events may have been lost or discarded. ```
So you won't know whether there are new files to be create/copy to your directory.
Maybe there are some workarounds you could consider:
Set a timeout, for that time, if no new file is created, consider the file transfer is finished and start to do the work.
Let your application to handle the copy, so it could know the progress and trigger work after all file is copied.
I am trying to watch a folder for changes. This folder contains subfolders which I do not want to watch. Unfortunately the WatchService notifies me of changes in these subfolders. I guess this happens because the last change date of these folders updates.
So I tried to exlude them:
WatchService service = FileSystems.getDefault().newWatchService();
workPath.register(watchService, StandardWatchEventKinds.ENTRY_CREATE,
StandardWatchEventKinds.ENTRY_DELETE, StandardWatchEventKinds.ENTRY_MODIFY);
try {
WatchKey watchKey = watchService.take();
for (WatchEvent<?> event : watchKey.pollEvents()) {
#SuppressWarnings("unchecked")
WatchEvent<Path> ev = (WatchEvent<Path>) event;
Path fileName = ev.context();
if (!Files.isDirectory(fileName)) {
logger.log(LogLevel.DEBUG, this.getClass().getSimpleName(),
"Change registered in " + fileName + " directory. Checking configurations.");
/* do stuff */
}
}
if (!watchKey.reset()) {
break;
}
} catch (InterruptedException e) {
return;
}
This doesn't work though. The resulting path of the context is relative and Files.isDirectory() cannot determine whether it's a directory or a file.
Is there a way to exclude the subfolders?
You can try the below code snippet. In order to get the full path, you need to call resolve() function
Map<WatchKey, Path> keys = new HashMap<>();
try {
Path path = Paths.get("<directory u want to watch>");
FileSystem fileSystem = path.getFileSystem();
WatchService service = fileSystem.newWatchService();
Files.walkFileTree(path, new SimpleFileVisitor<Path>() {
#Override
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {
if (<directory you want to exclude>) {
return FileVisitResult.SKIP_SUBTREE;
}
WatchKey key = dir.register(service, ENTRY_CREATE, ENTRY_MODIFY, ENTRY_DELETE);
keys.put(key, dir);
return FileVisitResult.CONTINUE;
}
});
WatchKey key = null;
while (true) {
key = service.take();
while (key != null) {
WatchEvent.Kind<?> kind;
for (WatchEvent<?> watchEvent : key.pollEvents()) {
kind = watchEvent.kind();
if (OVERFLOW == kind) {
continue;
}
Path filePath = ((WatchEvent<Path>) watchEvent).context();
Path absolutePath = keys.get(key).resolve(filePath);
if (kind == ENTRY_CREATE) {
if (Files.isDirectory(absolutePath, LinkOption.NOFOLLOW_LINKS)) {
WatchKey newDirKey = absolutePath.register(service, ENTRY_CREATE, ENTRY_MODIFY, ENTRY_DELETE);
keys.put(newDirKey, absolutePath);
}
}
}
if (!key.reset()) {
break; // loop
}
}
}
} catch (Exception ex) {
}
I am using this WatchService example from Oracle:
import java.nio.file.*;
import static java.nio.file.StandardWatchEventKinds.*;
import static java.nio.file.LinkOption.*;
import java.nio.file.attribute.*;
import java.io.*;
import java.util.*;
public class WatchDir {
private final WatchService watcher;
private final Map<WatchKey,Path> keys;
private final boolean recursive;
private boolean trace = false;
#SuppressWarnings("unchecked")
static <T> WatchEvent<T> cast(WatchEvent<?> event) {
return (WatchEvent<T>)event;
}
/**
* Register the given directory with the WatchService
*/
private void register(Path dir) throws IOException {
WatchKey key = dir.register(watcher, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
if (trace) {
Path prev = keys.get(key);
if (prev == null) {
System.out.format("register: %s\n", dir);
} else {
if (!dir.equals(prev)) {
System.out.format("update: %s -> %s\n", prev, dir);
}
}
}
keys.put(key, dir);
}
/**
* Register the given directory, and all its sub-directories, with the
* WatchService.
*/
private void registerAll(final Path start) throws IOException {
// register directory and sub-directories
Files.walkFileTree(start, new SimpleFileVisitor<Path>() {
#Override
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs)
throws IOException
{
register(dir);
return FileVisitResult.CONTINUE;
}
});
}
/**
* Creates a WatchService and registers the given directory
*/
WatchDir(Path dir, boolean recursive) throws IOException {
this.watcher = FileSystems.getDefault().newWatchService();
this.keys = new HashMap<WatchKey,Path>();
this.recursive = recursive;
if (recursive) {
System.out.format("Scanning %s ...\n", dir);
registerAll(dir);
System.out.println("Done.");
} else {
register(dir);
}
// enable trace after initial registration
this.trace = true;
}
/**
* Process all events for keys queued to the watcher
*/
void processEvents() {
for (;;) {
// wait for key to be signalled
WatchKey key;
try {
key = watcher.take();
} catch (InterruptedException x) {
return;
}
Path dir = keys.get(key);
if (dir == null) {
System.err.println("WatchKey not recognized!!");
continue;
}
for (WatchEvent<?> event: key.pollEvents()) {
WatchEvent.Kind kind = event.kind();
// TBD - provide example of how OVERFLOW event is handled
if (kind == OVERFLOW) {
continue;
}
// Context for directory entry event is the file name of entry
WatchEvent<Path> ev = cast(event);
Path name = ev.context();
Path child = dir.resolve(name);
// print out event
System.out.format("%s: %s\n", event.kind().name(), child);
// if directory is created, and watching recursively, then
// register it and its sub-directories
if (recursive && (kind == ENTRY_CREATE)) {
try {
if (Files.isDirectory(child, NOFOLLOW_LINKS)) {
registerAll(child);
}
} catch (IOException x) {
// ignore to keep sample readbale
}
}
}
// reset key and remove from set if directory no longer accessible
boolean valid = key.reset();
if (!valid) {
keys.remove(key);
// all directories are inaccessible
if (keys.isEmpty()) {
break;
}
}
}
}
static void usage() {
System.err.println("usage: java WatchDir [-r] dir");
System.exit(-1);
}
public static void main(String[] args) throws IOException {
// parse arguments
if (args.length == 0 || args.length > 2)
usage();
boolean recursive = false;
int dirArg = 0;
if (args[0].equals("-r")) {
if (args.length < 2)
usage();
recursive = true;
dirArg++;
}
// register directory and process its events
Path dir = Paths.get(args[dirArg]);
new WatchDir(dir, recursive).processEvents();
}
}
I am developing an app in Windows 7, and deployment environment is rhel 7.2. At first, in both OS'es, whenever I copied a file, it fired one ENTRY_CREATED and then two ENTRY_MODIFY. The first ENTRY_MODIFY was at the beginning of copying, and the second ENTRY_MODIFY was at the end of copying. So I was able to understand the copying process was over. However, It only fires one ENTRY_MODIFY in rhel 7.2 now. It still fires two ENTRY_MODIFY events in Windows 7 though.
I have found this in stackoverflow. That question asks why two ENTRY_MODIFY are fired. It is not exactly my question, but one of its answers disputes what I'm asking. Sadly, there is no solution to my question in that dispute though.
Because there is no ENTRY_MODIFY fired at the end but only in the beginning of copying, I can not understand when the copying is finished. What do you think might be the cause for this? Can it be fixed, how can I understand the copying is finished? I can't change rhel 7.2, but anything other than that I would gladly accept. Thanks in advance.
I just check if the file length is zero or not.
Here an example of
for (WatchEvent<?> event : key.pollEvents()) {
Path fileName = (Path) event.context();
if("tomcat-users.xml".equals(fileName.toString())) {
Path tomcatUsersXml = tomcatConf.resolve(fileName);
if(tomcatUsersXml.toFile().length() > 0) {
load(tomcatUsersXml);
}
}
}
You can use a DelayQueue to deduplicate the events.
I am using java watch service in my application to get notified of any file changes.
But the problem is watch service holds the file handle of the parent directory.
For e.g. if my hierarchy is
F
-F1
-F2
When i register watch service on F, F1 and F2. Then if i try to rename or delete the parent folder F, there comes a message of file is opened by another program which is the watch service.
I have found this issue mentioned here http://bugs.java.com/bugdatabase/view_bug.do;jsessionid=76a42b61021a94ffffffffa049f7587fd4149?bug_id=6972833
Also have tried the File_Tree modifier to circumvent this issue but it did not help. Probably i have not used the File_Tree modifier properly.
Below is the sample code which i am testing on. Please check if the usage of FILE_TREE modifier is proper in my code.
/**
* Example to watch a directory (or tree) for changes to files.
* This code is direct copy from https://docs.oracle.com/javase/tutorial/essential/io/examples/WatchDir.java
*/
public class FileWatcher {
private final WatchService watcher;
private final Map<WatchKey,Path> keys;
private final boolean recursive;
private boolean trace = false;
#SuppressWarnings("unchecked")
static <T> WatchEvent<T> cast(WatchEvent<?> event) {
return (WatchEvent<T>)event;
}
/**
* Register the given directory with the WatchService
*/
private void register(Path dir) throws IOException {
WatchKey key = dir.register(watcher, new WatchEvent.Kind[] { ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY, OVERFLOW },com.sun.nio.file.ExtendedWatchEventModifier.FILE_TREE);
if (trace) {
Path prev = keys.get(key);
if (prev == null) {
System.out.format("register: %s\n", dir);
} else {
if (!dir.equals(prev)) {
System.out.format("update: %s -> %s\n", prev, dir);
}
}
}
keys.put(key, dir);
}
/**
* Register the given directory, and all its sub-directories, with the
* WatchService.
*/
private void registerAll(final Path start) throws IOException {
// register directory and sub-directories
Files.walkFileTree(start, new SimpleFileVisitor<Path>() {
#Override
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs)
throws IOException
{
register(dir);
return FileVisitResult.CONTINUE;
}
});
}
/**
* Creates a WatchService and registers the given directory
*/
FileWatcher(Path dir, boolean recursive) throws IOException {
this.watcher = FileSystems.getDefault().newWatchService();
this.keys = new HashMap<WatchKey,Path>();
this.recursive = recursive;
if (recursive) {
System.out.format("Scanning %s ...\n", dir);
registerAll(dir);
System.out.println("Done.");
} else {
register(dir);
}
// enable trace after initial registration
this.trace = true;
}
/**
* Process all events for keys queued to the watcher
*/
void processEvents() {
for (;;) {
// wait for key to be signalled
WatchKey key;
try {
key = watcher.take();
} catch (InterruptedException x) {
return;
}
Path dir = keys.get(key);
if (dir == null) {
System.err.println("WatchKey not recognized!!");
continue;
}
for (WatchEvent<?> event: key.pollEvents()) {
WatchEvent.Kind kind = event.kind();
// TBD - provide example of how OVERFLOW event is handled
if (kind == OVERFLOW) {
continue;
}
// Context for directory entry event is the file name of entry
WatchEvent<Path> ev = cast(event);
Path name = ev.context();
Path child = dir.resolve(name);
// print out event
System.out.format("%s: %s\n", event.kind().name(), child);
// if directory is created, and watching recursively, then
// register it and its sub-directories
if (recursive && kind == ENTRY_CREATE) {
try {
if (Files.isDirectory(child, NOFOLLOW_LINKS)) {
registerAll(child);
}
} catch (IOException x) {
// ignore to keep sample readbale
}
}
}
// reset key and remove from set if directory no longer accessible
boolean valid = key.reset();
if (!valid) {
keys.remove(key);
// all directories are inaccessible
if (keys.isEmpty()) {
break;
}
}
}
}
static void usage() {
System.err.println("usage: java WatchDir [-r] dir");
System.exit(-1);
}
public static void main(String[] args) throws IOException {
// parse arguments
if (args.length == 0 || args.length > 2) {
usage();
}
boolean recursive = false;
int dirArg = 0;
if (args[0].equals("-r")) {
if (args.length < 2) {
usage();
}
recursive = false;
dirArg++;
}
// register directory and process its events
Path dir = Paths.get(args[dirArg]);
new FileWatcher(dir, recursive).processEvents();
}
}
Dug a little bit into the docs and find out a fix for myself.
With the File_Tree modifier a single key is assigned to all the folders/dirs. So we need to work with a single key.
Hence i am trying to reregister with the same root path and then reusing the key on every new creation of an item under the watched directory.
major take aways are
Recursive way of visting every folder in the directory is not required anymore.
Re registering on the same path is required whenever a new folder is created under the root path because only key is required every time
I am not sure on the performance part since everytime i am registering the whole path. probably i will do some performance analysis.
Below is the modified version of my code. Let me know if the code is not clear
package com.sap.mcm;
import static java.nio.file.StandardWatchEventKinds.ENTRY_CREATE;
import static java.nio.file.StandardWatchEventKinds.ENTRY_DELETE;
import static java.nio.file.StandardWatchEventKinds.ENTRY_MODIFY;
import static java.nio.file.StandardWatchEventKinds.OVERFLOW;
import java.io.IOException;
import java.nio.file.FileSystems;
import java.nio.file.FileVisitResult;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.SimpleFileVisitor;
import java.nio.file.WatchEvent;
import java.nio.file.WatchKey;
import java.nio.file.WatchService;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.HashMap;
import java.util.Map;
import com.sun.nio.file.ExtendedWatchEventModifier;
/**
* Example to watch a directory (or tree) for changes to files.
* This code is direct copy from https://docs.oracle.com/javase/tutorial/essential/io/examples/WatchDir.java
*/
public class FileWatcher {
private final WatchService watcher;
private final Map<WatchKey,Path> keys;
private final boolean recursive;
private boolean trace = true;
private Path parentDir;
private WatchKey key;
#SuppressWarnings("unchecked")
static <T> WatchEvent<T> cast(WatchEvent<?> event) {
return (WatchEvent<T>)event;
}
/**
* Register the given directory with the WatchService
*/
private void register(Path dir) throws IOException {
try {
parentDir = dir;
key = parentDir.register(watcher, new WatchEvent.Kind[] { ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY, OVERFLOW }, ExtendedWatchEventModifier.FILE_TREE);
} catch (UnsupportedOperationException usoe) {
usoe.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
}
public void reregister() throws IOException {
// WatchKey key = keys.get(path);
key.cancel();
register(parentDir);
}
/**
* Register the given directory, and all its sub-directories, with the
* WatchService.
*/
private void registerAll(final Path start) throws IOException {
// register directory and sub-directories
Files.walkFileTree(start, new SimpleFileVisitor<Path>() {
#Override
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs)
throws IOException {
register(dir);
return FileVisitResult.CONTINUE;
}
});
}
/**
* Creates a WatchService and registers the given directory
*/
FileWatcher(Path dir, boolean recursive) throws IOException {
this.watcher = FileSystems.getDefault().newWatchService();
this.keys = new HashMap<WatchKey,Path>();
this.recursive = recursive;
if (recursive) {
System.out.format("Scanning %s ...\n", dir);
registerAll(dir);
System.out.println("Done.");
} else {
register(dir);
}
// enable trace after initial registration
this.trace = true;
}
/**
* Process all events for keys queued to the watcher
*/
void processEvents() {
for (;;) {
// wait for key to be signalled
WatchKey key;
try {
key = watcher.take();
} catch (InterruptedException x) {
return;
}
// Path dir = keys.get(key);
// if (dir == null) {
// System.err.println("WatchKey not recognized!!");
// continue;
// }
if (this.key != key) {
return;
}
for (WatchEvent<?> event: key.pollEvents()) {
WatchEvent.Kind kind = event.kind();
// TBD - provide example of how OVERFLOW event is handled
if (kind == OVERFLOW) {
continue;
}
// Context for directory entry event is the file name of entry
WatchEvent<Path> ev = cast(event);
ev.context();
// print out event
System.out.format("%s: %s \n", event.kind().name(), event.context());
// if directory is created, and watching recursively, then
// register it and its sub-directories
if (kind == ENTRY_CREATE) {
try {
// if (Files.isDirectory(child, NOFOLLOW_LINKS)) {
reregister();
// }
} catch (IOException x) {
// ignore to keep sample readbale
}
}
}
// reset key and remove from set if directory no longer accessible
boolean valid = this.key.reset();
if (!valid) {
keys.remove(key);
// all directories are inaccessible
if (keys.isEmpty()) {
break;
}
}
}
}
static void usage() {
System.err.println("usage: java WatchDir [-r] dir");
System.exit(-1);
}
public static void main(String[] args) throws IOException {
// parse arguments
if (args.length == 0 || args.length > 2) {
usage();
}
boolean recursive = false;
int dirArg = 0;
if (args[0].equals("-r")) {
if (args.length < 2) {
usage();
}
recursive = false;
dirArg++;
}
// register directory and process its events
Path dir = Paths.get(args[dirArg]);
new FileWatcher(dir, recursive).processEvents();
}
}
I'm using WatchService to monitor a directory. Another third party will upload large CSV files to that directory via SFTP. I have to wait until all the files are finished to start processing the files.
My trouble right now is that SFTP creates the file as soon as the uploading starts I get ENTRY_CREATE and continuously get ENTRY_MODIFY until the file is done. Is there anyway to tell if the file is actually done.
This is the code I use which I got it from Java Documentation
public class WatchDir {
private final WatchService watcher;
private final Map<WatchKey, Path> keys;
private final boolean recursive;
private boolean trace = false;
#SuppressWarnings("unchecked")
static <T> WatchEvent<T> cast(WatchEvent<?> event) {
return (WatchEvent<T>) event;
}
/**
* Register the given directory with the WatchService
*/
private void register(Path dir) throws IOException {
WatchKey key = dir.register(watcher, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
if (trace) {
Path prev = keys.get(key);
if (prev == null) {
System.out.format("register: %s\n", dir);
} else {
if (!dir.equals(prev)) {
System.out.format("update: %s -> %s\n", prev, dir);
}
}
}
keys.put(key, dir);
}
/**
* Register the given directory, and all its sub-directories, with the
* WatchService.
*/
private void registerAll(final Path start) throws IOException {
// register directory and sub-directories
Files.walkFileTree(start, new SimpleFileVisitor<Path>() {
#Override
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs)
throws IOException {
register(dir);
return FileVisitResult.CONTINUE;
}
});
}
/**
* Creates a WatchService and registers the given directory
*/
WatchDir(Path dir, boolean recursive) throws IOException {
this.watcher = FileSystems.getDefault().newWatchService();
this.keys = new HashMap<WatchKey, Path>();
this.recursive = recursive;
if (recursive) {
System.out.format("Scanning %s ...\n", dir);
registerAll(dir);
System.out.println("Done.");
} else {
register(dir);
}
// enable trace after initial registration
this.trace = true;
}
/**
* Process all events for keys queued to the watcher
*/
void processEvents() {
for (; ; ) {
// wait for key to be signalled
WatchKey key;
try {
key = watcher.take();
} catch (InterruptedException x) {
return;
}
Path dir = keys.get(key);
if (dir == null) {
System.err.println("WatchKey not recognized!!");
continue;
}
for (WatchEvent<?> event : key.pollEvents()) {
WatchEvent.Kind kind = event.kind();
// TBD - provide example of how OVERFLOW event is handled
if (kind == OVERFLOW) {
continue;
}
// Context for directory entry event is the file name of entry
WatchEvent<Path> ev = cast(event);
Path name = ev.context();
Path child = dir.resolve(name);
// print out event
System.out.format("%s: %s\n", event.kind().name(), child);
// if directory is created, and watching recursively, then
// register it and its sub-directories
if (recursive && (kind == ENTRY_CREATE)) {
try {
if (Files.isDirectory(child, NOFOLLOW_LINKS)) {
registerAll(child);
}
} catch (IOException x) {
// ignore to keep sample readbale
}
}
}
// reset key and remove from set if directory no longer accessible
boolean valid = key.reset();
if (!valid) {
keys.remove(key);
// all directories are inaccessible
if (keys.isEmpty()) {
break;
}
}
}
}
static void usage() {
System.err.println("usage: java WatchDir [-r] dir");
System.exit(-1);
}
public static void main(String[] args) throws IOException {
// parse arguments
if (args.length == 0 || args.length > 2)
usage();
boolean recursive = false;
int dirArg = 0;
if (args[0].equals("-r")) {
if (args.length < 2)
usage();
recursive = true;
dirArg++;
}
// register directory and process its events
Path dir = Paths.get(args[dirArg]);
new WatchDir(dir, recursive).processEvents();
}
}
Under Linux, you can use the "inotify" tools. they probably arrive with all major destributions. here is wiki for it : wiki - Inotify
Note in the supported events list you have:
IN_CLOSE_WRITE - sent when a file opened for writing is closed
IN_CLOSE_NOWRITE - sent when a file opened not for writing is closed
these are what you look for. I did not manage to see something similar on windows.
Now as for using them, there can be various ways. I used a java library jnotify
Note that the library is cross platform, so you don't want to use the main class as windows does not support events for close file. you will want to use the linux API which exposes the full linux capabilities. just read the description page and you know its what you request : jnotify - linux
Note that in my case, I had to download the library source because I needed compile the shared object file "libjnotify.so" for 64 bit. the one provided worked only under 32 bit. Maybe they provide it now you can check.
check the examples for code and how to add and remove watches. just remember to use the "JNotify_linux" class instead of "JNotify" and then you can use a mask with your operation, for example.
private final int MASK = JNotify_linux.IN_CLOSE_WRITE;
I hope it will work for you.