How to identify which link modified the target file when multiple symbolic link exists for a single file using Java? I was unable to find which file modified the target file.
Example: D:\sample.txt,D:\folder1\sample.txt; these two are links. The target file is located in E:\sample.txt.
Now how to identify whether D:\sample.txt or D:\folder1\sample.txt modified the E:\sample.txt?
How to identify which link modified the target file when multiple symbolic link exists for a single file using java?
It is not possible.
It is not possible in any programming language.
This functionality would have to be supported by the operating system, and no operating system I've ever come across does.
There are heuristics (using timestamps) that will probably work "most of the time", but in each case there are circumstances under which the heuristic will give no answer or even the wrong answer. Here are some of the confounding issues:
With simple timestamp heuristics:
it won't if either of the symlinks is on a read-only file system, or a file system where access times are not recorded (e.g. depending on mount options), and
it won't work if a file read occurs on the symlink after the last file write.
When you add a watcher:
it won't work if you weren't "watching" at the time (duh!), and
it won't work if you have too many watcher events ... and you can't keep up.
(Besides, I don't think you can get events on the use of a symlink. So you would still need to check the symlink access timestamps anyway. And that means that read-only file systems, etc are a problem here too.)
And then there are scenarios like:
both symlinks are used to write the file,
you don't know about all of the symlinks, or
the symlink used for writing has been deleted or "touched".
These are probably beyond of the scope of the OP's use-case. But they are relevant to the general question as set out by the OP's first sentence.
Maybe you can do that using Files.readAttributes(). The below works with Linux, since when you "use" a symlink under Linux, its last access time is modified. No idea under Windows, you'll have to test.
If symlink1 is a Path to your first symlink and symlink2 a Path to your second symlink, and realFile a Path to your real file, then you can retrieve FileTime objects of the last access time for both symlinks and last modification time of the file using:
Files.readAttributes(symlink1, BasicFileAttributes.class).lastAccessTime();
Files.readAttributes(symlink2, BasicFileAttributes.class).lastAccessTime();
Files.readAttributes(realFile, BasicFileAttributes.class).lastModifiedTime();
Since FileTime is Comparable to itself, you may spot which symlink is modified but this is NOT a guarantee.
Explanation: if someone uses symlink1 to modify realFile, then the access time of symlink1 will be modified and the modification time of realFile will be modified. If the last access time of symlink1 is GREATER than the last access time of symlink2, then there is a possibility that symlink1 was used for this operation; on the other hand, if the last access time of symlink2 is greater and the last modification time of realFile is lesser, then you are sure that symlink2 was not used for this purpose.
But again there is no REAL guarantee. Those are only heuristics!
You should also have a look at using a WatchService in order to watch for modifications on the real file; this would make the heuristics above more precise even. But again, no guarantee.
Related
I have a special use case where I need to verify if a file is a special block file (only if my java program is run on Linux). My code can already detect if it's ran on Windows or Linux, so if it's windows I simply invalidate the verification.
For Linux, I was reading about the different FileAttributeView interfaces; and the first thing that popped up was to use the PosixFileAttributeView (since block and char device files are Linux-specific), but after looking further into its Javadoc, it only gives access to the different RWX (read, write, execute permissions relative to the owner, group and others) but nothing about special block files or other file attributes. On the other hand, the BasicFileAttributeView is supported by all filesystems and has an instance method called isOther(), this could work but there has to be some other way that's more specific than just knowing that a file is not a regular file, directory or symlink. Knowing that it's something other is just too vague.. Any help is deeply appreciated.
Wow, this is actually harder than I imagined!
To ground this in Linux, what you're looking for is the st_mode field from stat() system call (in C-world). Its different values indicate whether the file is a character device, block device, directory, etc... The first character on each line of the return from ls -la represent that field: b for block device, d for directory, - for regular file, and on. For example, if you run ls -la /dev/sda you'll probably get something like brw-rw---- 1..., because sda is a block device. The first line of ls -la /opt will be dr-xr-xrx... since opt is a directory.
(https://linux.die.net/man/2/stat, for reference)
Poking around the Javadocs, I didn't find a file class that allows direct access to this st_mode field. Like you mentioned, there are methods to determine if the file is a directory, file, or other, but this is not sufficient to distinguish between all the options and figure out if it's a block device.
I did find this project which looks like it lets you get at that stat() struct in Java. It is a fair bit of overhead. Perhaps you could extract the bits you need for this application, but it might not be trivial.
The specific method you'd need to call is isBlockDev()
Hope that helps a little bit, curious to see if you figure something out, or if there's a better answer out there!
I hope you're doing well in these complicated times, good containment all around :D
So I come to my problem, currently I'm working on a game in Java and I know that it's possible to modify the jar file of my game which annoys me a little bit because it can give cheating possibilities for malicious players... I had thought of a solution even if it's not infallible, it would be to make sure to check if the jar file has not been modified. But the problem is that I don't know how to check this, I had thought of a system that would check the point of the file even if I doubt it would be the best solution.
If you have any other ideas to secure my game I would be interested :D
It is possible to check if a JAR file on your machine has been modified. Simply compare a cryptographic hash of the current JAR file with a previously recorded hash for a pristine copy. (Or just do a byte-by-byte comparison with the pristine copy.)
But you can't do this for JAR file on the user's machine:
You can't login to the user's machine and access their file system to look at the JAR file. (Even if you could, there is no guarantee that you would see the file that the cheater is actually using.)
If your application (running on the user's machine) tries to report on the integrity of its JAR files, this can be defeated by the cheater. All they need to do is to modify the JAR file containing the reporting code to report a fake hash.
Basically, there is no reliable way to detect that a cheater is running a modified JAR ... apart from detecting the anomalous behavior of the cheat itself.
But think of it this way. If there was a good (reliable, no circumvention) mechanism for detect that a cheater is running a modified client, then cheats would not be a problem in the many online games that exist out there. And (by extension) there would be no way to defeat software license enforcement schemes ... because software vendors would use a similar mechanism.
So, my advice would be not waste too much time on this approach. It only works against people with limited technical expertise or limited motivation.
The only way to completely prevent cheats is to control the platform on which the client runs. That is usually impractical.
In regards to your question about other ideas, the best thing you can do is validate everything from the client. One thing you should always remember is that the client cannot be trusted because you cannot verify anything from it. All you can do is store the state on a remote server and when the client tells you something, validate it, and give a response if necessary or prevent the action if necessary.
You will need to somehow find out where the jar file is: Java - Search for files in a directory
Then you can check for the last modified date of the file: https://www.boraji.com/java-get-last-modified-date-of-a-file
However, I would not consider this a very powerful defense against cheating, because one can modify the jar file and remove the validation of the file date.
You will need to think about the kind of cheating that can occur and to come up with other security measures as well.
EDIT
As Dave Thompson pointed out, the modified timestamp can be changed as well, which makes the modification of the jar file unnecessary, even though reverse engineering is still needed by the hackers, because that's how they find out what the rules of the application are.
Currently I am using file.delete() but it is showing a security risk for this as files deleted like this can be recovered by different means. So please provide me a correct way to delete a file. The security risk depicted here is provided by a testing tool called Quixxi and it checks for any vulnerability in app.
The reason a "deleted" file is recoverable is because a delete operation simply unlinks the file in the filesystem, so the directory no longer considers that file part of it. The contents on disk (or whatever storage) still exist on that device.
If you want to guarantee the contents can never be recovered, you have to overwrite the contents first. There are no built-in functions to do this - you'd have to find a library or write the code yourself. Typically you'd write something like all 0s over the file (make sure to flush to media), write all 1s, write a pattern of 01 repeating, 10 repeating, something like that. After you've written with garbage patterns to media (flush) a few times, then you issue the delete.
Not possible in JRE, unfortunately. The JVM is not designed for that, and you need OS-dependent utilities.
The answer by user1676075 contains a mistake. Let's go by steps.
As pointed out already, Java's File.delete method only unlinks the file leaving its contents on disk. It actually invokes the underlying OS APIs to perform this unlink operation.
The problem occurs when you want to overwrite contents in Java.
Java can open a file for overwrite, but will leverage OS utils to do so. And the OS will likely:
Unlink the allocated space on disk
Link the file to a new free area of disk
The result is that you are now writing tons of zeroes... somewhere else!!!
And even if you managed to write zeroes on the same sectors used by the original file, Gutmann method exists for a reason. Gutmann utilities require root/Administrator (Super User) permissions and direct DMA access to precisely control where the writes have to occur.
And with SSDs, things changes. Actually, it might get easier! At this point, I should provide source for SSDs having a CLEAR instructions to replace a sector with zeroes and that privacy-savy disk controllers do that. But maybe pretend you have read nothing.
This will be a sufficient answer for now, because we have demonstrated that there is no out-of-the-box and straightforward way to securely clear a file in Java.
What Java allows, and is called Java Native Interfaces (please also see Java Native Access), is to call native code from Java. So, you got your Gutmann tool in C++ ready? Are you running root? You can write code to invoke Gutmann-ish erasure from Java, but that's a whole other point.
Never tried, but surely feasible
Problem description
I would like to watch a complete file system for changes. I'm talking about watching changes in a directory recursively. So, when watching a directory (or a whole file system) all changes in sub-directories need to be captured too. The application needs to be able to track all changes by getting notified.
Java's WatchService isn't suitable
Java already has a WatchService feature, which allows you to monitor a directory for changes. The problem is however, that this isn't a recursive process as far as I know, thus you can't use this to monitor all changes in the root directory of a file system.
Watching all sub-directories explicitly
A solution I've thought of would be to register each directory inside the specified root directory explicitly. The problem with this is however, that walking through, and registering these directories is very resource expensive on a system with more than a million sub-directories. This is because the system would need to go through the whole file system recursively to only register all directories in the first place. The performance impact of this feature would be too big, if it's even possible without crashing the application.
Logical solution
I would assume an operating system would fire/call some sort of event when anything is changed on the file system, that an application is able to listen to. I did however, not find anything like this yet. This would allow the application to listen to all changes without the need to register all sub-directories explicitly. Thus the performance impact with such a method would be minimal.
Question
Is watching a whole file system, or watching a directory recursively possible in Java, and how would this be achieved?
The question should be split into several:
How to track file events across the disk on certain OS
How to use this mechanism in Java
The answer to the first question is that the approaches are different. On Windows there exist Windows API functions that let you do this (and famous FileSystemWatcher class in .NET Framework is a kind of wrapper around this API function set). The more robust method on windows is to create or use a pre-created file system filter driver. On Linux there exists inotify. On MacOS X there exist several approaches (there was a question on this topic somewhere around), none of them being universal or always available.
Also all approaches except a filesystem filter driver are good only for being notified after the event happens, but they don't let you intercept and deny the request (AFAIK, I can be mistaken here).
As for the second question, there seems to be no universal solution that would cover all or most variants that I mentioned above. You would need to first choose the mechanism for each OS, then find some wrappers for Java to use those mechanisms.
Here is an example to to watch a directory (or tree) for changes to file
Please find example https://github.com/syncany/syncany/blob/59cf87c72de4322c737f0073ce8a7ddd992fd898/syncany-lib/src/main/java/org/syncany/operations/watch/RecursiveWatcher.java
Even you can filtered our directory that you don't want to watch
I am looking for the most efficient way to move a directory recursively in Java. At the moment, I am using Apache commons-io as shown in the code below. (If the destDir exists and contains part of the files, I would like those to be overwritten and the nested directory structures to be merged).
FileUtils.copyDirectoryToDirectory(srcDir, destDir);
FileUtils.deleteDirectory(srcDir);
While this does the trick, in my opinion, it isn't efficient enough. There are at least two issues that come to mind:
You will need to have twice as much space.
If this is an SSD, copying the data over to another part of the drive and then erasing the old data will eventually have an impact on the on the hardware, as it will in effect shorten the hard disk's life.
What is the best approach to do this?
As per my understanding commons-io doesn't seem to be using the new Java 7/8 features available in Files. On the other hand, I was unable to get Files.move(...) to work, if the destDir exists (by "get it to work" I mean have it merge the directory structures -- it complains that the destDir exists).
Regarding failures to move (please correct me, if I am wrong):
As far as I understand, an atomic move is one that only succeeds, if all files are moved at once. If I understand this correctly, this means that this is copying first and then deleting. I don't think this is what I'm looking for.
If a certain path/file cannot be moved, then the operation should cease and throw an exception, preserving the current source path it reached.
Please, note that I am not limiting myself to using the commons-io library. I am open to suggestions. I am using Java 8.
This is just an answer to the "what needs to happen to the filesystem" part of the question, not how to do it with Java.
Even if you did want to call out to an external tool, Unix mv is not like Windows Explorer. Same-name directories don't merge. So you will need to implement that yourself, or find a library function that does. There is no single Unix system call that does the whole recursive operation (let alone atomically), so it's something either your code or a library function has to do.
If you need to atomically cut from one version of a tree to another, you need to build a new tree. The files can be hard-links to the old version. i.e. do the equivalent of
cp -al dir/ new
rsync -a /path/to/some/stuff/ new/
# or maybe something smarter / custom that renames instead of copies files.
# your sanity check here
mv dir old &&
mv new dir && # see below for how to make this properly atomic
rm -rf old
This leaves a window where dir doesn't exist. To solve this, add a level of indirection, by making dir a symlink. Symlinks can be replaced atomically with mv (but not ln -sf). So in Java, you want something that will end up doing a rename system call, not an unlink / rename.
Unless you have a boatload of extremely small files (under 100 bytes), the directory metadata operations of building a hardlink farm are much cheaper than a full copy of a directory tree. The file data will stay put (and never even be read), the directory data will be a fresh copy. The file metadata (inodes) will be written for all files (to update the ctime and link count, when creating the hardlink farm, and again when removing the old tree, leaving files with the original link count.
If you're running on a recent Linux kernel, there is a new(2013) system call (called renameat2) available that can exchange two paths atomically. This avoids the symlink level of indirection. Using a Linux-only system call from Java is going to be more trouble than it's worth, though, since symlinks are easy.
I am answering my own question, as I ended up writing my own implementation.
What I didn't like about the implementations of:
Apache Commons IO
Guava
Springframework
for moving files was that all of them first copy the directories and files and then delete them. (As far as I checked, September 2015) They all seem to be stuck with methods from JDK 1.6.
My solution isn't atomic. It handles the moving by walking the directory structure and performing the moves file by file. I am using the new methods from JDK 1.7. It does the job for me and I'm sure other people would like to do the same and wonder how to do it and then waste time. I have therefore created a small Github project, which contains an illustration here:
FileUtils#moveDirectory((Path srcPath, Path destPath)
An illustration of how to use it can be seen here.
If anybody has suggestions on how to improve it, or would like to add features, please feel free to open a pull request.
Traverse the source directory tree:
When meeting a directory, ensure the same directory exist in the target tree (and has the right permissions etc).
When meeting a file, rename it to the same name in the corresponding directory in the target tree.
When leaving a directory, ensure it is empty and delete it.
Consider carefully how any error should be handled.
Note that you might also simply call out to "rsync" if it is available on your system.