I have a simple JUnit test which checks two files have the same content. It works perfectly fine in my Unix laptop.
Here it is the test:
boolean response = false;
try {
File got = File.createTempFile("got-", ".csv");
String outputPath = got.getAbsolutePath();
testedObject.createCsvFile(outputPath);
got = new File(outputPath);
String expectedFilePath = getClass().getClassLoader().getResource("expected.csv").getFile();
File expected = new File(expectedFilePath);
response = FileUtils.contentEquals(got, expected); // Here it is the key
} catch (IOException e) {
// Nothing to do Yay!
}
Assert.assertTrue(response);
It works because if I compare both files manually, example via diff command, are exactly the same. Now.
My teem-mate codes with a Windows laptop, when he ran the test it brokes down! and we started debugging.
Visually, both files are the same; I mean in a human revision you cannot realize any difference. But If in a Cwin terminal we executed:
diff expected.csv got.csv and windows thought each line was different
And the test falls.
What is the problem, is the operative system? If that is true, Is there any way to compare file content not dependent on operative system
My guess is that this is most likely this is due to the \n value, which in unix like software is \r\n.
Anyway, the correct way to see if two files have the same content, is to hash both of them (ie via sha1) and check if the hashes matches!
This behaviour can be attributed to the Line Feed being different on both operating systems.
If you want it to be platform independent , you should pick up the value from the runtime using
System.getProperty("line.separator");
Also you might want to have a look at the char encoding for both the files
This answer can help you: Java Apache FileUtils readFileToString and writeStringToFile problems. The question's author is talking about PDF file, but this answer makes sense for your question.
Related
I am creating a JUnitTest test that compares a file that is created with a benchmark file, present in the resources folder in the src folder in Eclipse.
Code
public class CompareFileTest
{
private static final String TEST_FILENAME = "/resources/CompareFile_Test_Output.xls";
#Test
public void testCompare()
{
InputStream outputFileInputStream = null;
BufferedInputStream bufferedInputStream = null;
File excelOne = new File(StandingsCreationHelper.directoryPath + "CompareFile_Test_Input1.xls");
File excelTwo = new File(StandingsCreationHelper.directoryPath + "CompareFile_Test_Input1.xls");
File excelThree = new File(StandingsCreationHelper.directoryPath + "CompareFile_Test_Output.xls");
CompareFile compareFile = new CompareFile(excelOne, excelTwo, excelThree);
// The result of the comparison is stored in the excelThree file
compareFile.compare();
try
{
outputFileInputStream = new FileInputStream(excelThree);
bufferedInputStream = new BufferedInputStream(outputFileInputStream);
assertTrue(IOUtils.contentEquals(CompareFileTest.class.getResourceAsStream(TEST_FILENAME), bufferedInputStream));
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
However, I get an Assertion Error message, without any details. Since I just created the benchmark file from the compare file operation, both files should be identical.
Thanks in advance!
EDIT: After slim's comments, I used a file diff tool and found that both files are different, although, since they are copies, I am not sure how that happened. Maybe there is a timestamp or something?
IOUtils.contentEquals() does not claim to give you any more information than a boolean "matches or does not match", so you cannot hope to get extra information from that.
If your aim is just to get to the bottom of why these two files are different, you might step away from Java and use other tools to compare the files. For example https://superuser.com/questions/125376/how-do-i-compare-binary-files-in-linux
If your aim is for your jUnit tests to give you more information when the files do not match (for example, the exception could say Expected files to match, but byte 5678 differs [0xAE] vs [0xAF]), you will need to use something other than IOUtils.contentEquals() -- by rolling your own, or by hunting for something appropriate in Comparing text files w/ Junit
I had a similar issue.
I was using JUNIT assertion library Assertions and got the memory address being compared rather than the actual file it seemed.
Instead of comparing the InputStream objects I converted them to byte arrays and compared those. Not an absolute specials, but I dare to claim that if the byte array is identical, then the underlying InputStream and its file have a large chance of being equal.
like this:
Assertions.assertEquals(
this.getClass().getResourceAsStream("some_image_or_other_file.ext").readAllBytes(),
someObject.getSomeObjectInputStream().readAllBytes());
Not sure that this will scale though for larger files. Certainly not OK for complex diffs, but it does the trick for an assertion.
This is probably ridiculously simple for gun Java programmers, yet the fact that I (a relative newbie to Java) couldn't find a simple, straightforward example of how to do it means that I'm going to use the self-answer option to hopefully prevent others going through similar frustration.
I needed to output error information to a simple text file. These actions would be infrequent and small (and sometimes not needed at all) so there is no point keeping a stream open for the file; the file is opened, written to and closed in the one action.
Unlike other "append" questions that I've come across, this one requires that the file be created on the first call to the method in that run of the Java application. The file will not exist before that.
The original code was:
Path pathOfLog = Paths.get(gsOutputPathUsed + gsOutputFileName);
Charset charSetOfLog = Charset.forName("US-ASCII");
bwOfLog = Files.newBufferedWriter(pathOfLog, charSetOfLog);
bwOfLog.append(stringToWrite, 0, stringToWrite.length());
iReturn = stringToWrite.length();
bwOfLog.newLine();
bwOfLog.close();
The variables starting with gs are pre-populated string variables showing the output location, and stringToWrite is an argument which is passed in.
So the .append method should be enough to show that I wanted to append content, right?
But it isn't; each time the procedure was called the file was left containing only the string of the most recent call.
The answer is that you also need to specify open options when calling the newBufferedWriter method. What gets you is the default arguments as specified in the documentation:
If no options are present then this method works as if the CREATE,
TRUNCATE_EXISTING, and WRITE options are present.
Specifically, it's TRUNCATE_EXISTING that causes the problem:
If the file already exists and it is opened for WRITE access, then its
length is truncated to 0.
The solution, then, is to change
bwOfLog = Files.newBufferedWriter(pathOfLog, charSetOfLog);
to
bwOfLog = Files.newBufferedWriter(pathOfLog, charSetOfLog,StandardOpenOption.CREATE, StandardOpenOption.APPEND);
Probably obvious to long time Java coders, less so to new ones. Hopefully this will help someone avoid a bit of head banging.
You can also try this :
Path path = Paths.get("C:\\Users", "textfile.txt");
String text = "\nHello how are you ?";
try (BufferedWriter writer = Files.newBufferedWriter(path, StandardCharsets.UTF_8, StandardOpenOption.APPEND,StandardOpenOption.CREATE)) {
writer.write(text);
} catch (IOException e) {
e.printStackTrace();
}
I know this has been probably answered a million times on here but everything I have looked at has not helped me. Here is my code:
for(File g: f.listFiles()){
for(File h : g.listFiles()){
try{
Scanner s = new Scanner(h);
String timestamp = s.next().split("[?]")[4];
File z = new File(h.getAbsolutePath().split("[.]")[0] + timestamp + h.getAbsolutePath().split("[.]")[1]);
boolean q = h.renameTo(z);
}catch(Exception e){
}
}
}
I have checked to see if File z exists and it doesnt. I have checked if File h exists and it does. I have doublechecked that h is an absolute path. If I print the absolute path of z, I get the correct path. None of the directories in f or files in g are open. The files denoted by h are not open. Could there be some flag set or something on the file where windows is not allowing my program to rename it?
My guess is that you are having a similar problem to one I had here File deletion/moving failing
Try using FileinputStreams for the Scanner
FileInputStream fin = new FileInputStream(h);
fin.open()
Scanner s = new Scanner(fin);
//do work
fin.close()
and closing the stream before renaming
The behavior of renameTo varies from platform to platform. Operations that succeed on one platform may fail on another. For example, on my local development workstation (OS X), everything worked as expected. On a production system (Solaris), renameTo failed consistently. I finally determined that it failed when the files were located on different partitions. Obviously that is not the case here, but it illustrates that the method can behave in unexpected ways.
To get consistent behavior, copy the data to a new file, then delete the original.
I had a almost same issue. Some of rename cases succeeded, some failed. For those failed cases, I found, the source file path and destination file path are not on in same file system. In my cases, the NTFS mounted another file system which the destination file would be moved to. Since the rename function's original purpose simply rename a name, not to move the data of the concerned file. If both source file path and destination file path are in different file system, some version of JVM will fail on certain platforms. Actually, it is a bug in java.io and Solaris has fixed this bug in new versions.
Good Luck!
HappyForever,
I'm looking for an efficient way to detect whether two java.io.Files refer to the same physical file. According to the docs, File.equals() should do the job:
Tests this abstract pathname for
equality with the given object.
Returns true if and only if the
argument is not null and is an
abstract pathname that denotes the
same file or directory as this
abstract pathname.
However, given a FAT32 partition (actually a TrueCrypt container) which is mounted at /media/truecrypt1:
new File("/media/truecrypt1/File").equals(new File("/media/truecrypt1/file")) == false
Would you say that this conforms to the specification? And in this case, how to work around that problem?
Update: Thanks to commenters, for Java 7 I've found java.io.Files.isSameFile() which works for me.
The answer in #Joachim's comment is normally correct. The way to determine if two File object refer to the same OS file is to use getCanonicalFile() or getCanonicalPath(). The javadoc says this:
"A canonical pathname is both absolute and unique. [...] Every pathname that denotes an existing file or directory has a unique canonical form."
So the following should work:
File f1 = new File("/media/truecrypt1/File"); // different capitalization ...
File f2 = new File("/media/truecrypt1/file"); // ... but same OS file (on Windows)
if (f1.getCanonicalPath().equals(f2.getCanonicalPath())) {
System.out.println("Files are equal ... no kittens need to die.");
}
However, it would appear that you are viewing a FAT32 file system mounted on UNIX / Linux. AFAIK, Java does not know that this is happening, and is just applying the generic UNIX / Linux rules for file names ... which give the wrong answer in this scenario.
If this is what is really happening, I don't think there is a reliable solution in pure Java 6. However,
You could do some hairy JNI stuff; e.g. get the file descriptor numbers and then in native code, use the fstat(2) system call to get hold of the two files' device and inode numbers and comparing those.
Java 7 java.nio.file.Path.equals(Object) looks like it might give the right answer if you call resolve() on the paths first to resolve symlinks. (It is a little unclear from the javadoc whether each mounted filesystem on Linux will correspond to a distinct FileSystem object.)
The Java 7 tutorials have this section on seeing if two Path objects are for the same file ... which recommends using java.nio.file.Files.isSameFile(Path, Path)
Would you say that this conforms to the specification?
No and yes.
No in the sense that the getCanonicalPath() method is not returning the same value for each existing OS file ... which is what you'd expect from reading the javadoc.
Yes in the technical sense that the Java codebase (not the javadoc) is the ultimate specification ... both in theory and in practice.
you could try to obtain an exclusive write lock on the file, and see if that fails:
boolean isSame;
try {
FileOutputStream file1 = new FileOutputStream (file1);
FileOutputStream file2 = new FileOutputStream (file2);
FileChannel channel1 = file1.getChannel();
FileChannel channel2 = file2.getChannel();
FileLock fileLock1 = channel1.tryLock();
FileLock fileLock2 = channel2.tryLock();
isSame = fileLock2 != null;
} catch(/*appropriate exceptions*/) {
isSame = false;
} finally {
fileLock1.unlock();
fileLock2.unlock();
file1.close();
file2.close();
///cleanup etc...
}
System.out.println(file1 + " and " + file2 + " are " + (isSame?"":"not") + " the same");
This is not always guaranteed to be correct tho - because another process could potentially have obtained the lock, and thus fail for you. But at least this doesn't require you to shell out to an external process.
There's a chance the same file has two paths (e.g. over the network \\localhost\file and \\127.0.0.1\file would refer to the same file with a different path).
I would go with comparing digests of both files to determine whether they are identical or not. Something like this
public static void main(String args[]) {
try {
File f1 = new File("\\\\79.129.94.116\\share\\bots\\triplon_bots.jar");
File f2 = new File("\\\\triplon\\share\\bots\\triplon_bots.jar");
System.out.println(f1.getCanonicalPath().equals(f2.getCanonicalPath()));
System.out.println(computeDigestOfFile(f1).equals(computeDigestOfFile(f2)));
}
catch(Exception e) {
e.printStackTrace();
}
}
private static String computeDigestOfFile(File f) throws Exception {
MessageDigest md = MessageDigest.getInstance("MD5");
InputStream is = new FileInputStream(f);
try {
is = new DigestInputStream(is, md);
byte[] buffer = new byte[1024];
while(is.read(buffer) != -1) {
md.update(buffer);
}
}
finally {
is.close();
}
return new BigInteger(1,md.digest()).toString(16);
}
It outputs
false
true
This approach is of course much slower than any sort of path comparison, it also depends on the size of files. Another possible side effect is that two files will be considered equals equal indifferently from their locations.
The Files.isSameFile method was added for exactly this kind of usage - that is, you want to check if two non-equal paths locate the same file.
On *nix systems, casing does have an importance. file is not the same as File or fiLe.
The API doc of equals() says (right after your quote):
On UNIX systems, alphabetic case is
significant in comparing pathnames; on
Microsoft Windows systems it is not.
You can try Runtime.exec() of
ls -i /fullpath/File # extract the inode number.
df /fullpath/File # extract the "Mounted on" field.
If the mount point and the "inode" number is the same, they are the same file whether you have symbolic links or case-insensitive file systems.
Or even
bash test "file1" -ef "file2"
FILE1 and FILE2 have the same device and inode numbers
The traditional Unix way to test whether two filenames refer to the same underlying filesystem object is to stat them and test whether they have the same [dev,ino] pair.
That does assume no redundant mounts, however. If those are allowed, you have to go about it differently.
Java's File.renameTo() is problematic, especially on Windows, it seems.
As the API documentation says,
Many aspects of the behavior of this
method are inherently
platform-dependent: The rename
operation might not be able to move a
file from one filesystem to another,
it might not be atomic, and it might
not succeed if a file with the
destination abstract pathname already
exists. The return value should always
be checked to make sure that the
rename operation was successful.
In my case, as part of an upgrade procedure, I need to move (rename) a directory that may contain gigabytes of data (lots of subdirectories and files of varying sizes). The move is always done within the same partition/drive, so there's no real need to physically move all the files on disk.
There shouldn't be any file locks to the contents of the dir to be moved, but still, quite often, renameTo() fails to do its job and returns false. (I'm just guessing that perhaps some file locks expire somewhat arbitrarily on Windows.)
Currently I have a fallback method that uses copying & deleting, but this sucks because it may take a lot of time, depending on the size of the folder. I'm also considering simply documenting the fact that the user can move the folder manually to avoid waiting for hours, potentially. But the Right Way would obviously be something automatic and quick.
So my question is, do you know an alternative, reliable approach to do a quick move/rename with Java on Windows, either with plain JDK or some external library. Or if you know an easy way to detect and release any file locks for a given folder and all of its contents (possibly thousands of individual files), that would be fine too.
Edit: In this particular case, it seems we got away using just renameTo() by taking a few more things into account; see this answer.
See also the Files.move() method in JDK 7.
An example:
String fileName = "MyFile.txt";
try {
Files.move(new File(fileName).toPath(), new File(fileName).toPath(), java.nio.file.StandardCopyOption.REPLACE_EXISTING);
} catch (IOException ex) {
Logger.getLogger(SomeClass.class.getName()).log(Level.SEVERE, null, ex);
}
For what it's worth, some further notions:
On Windows, renameTo() seems to fail if the target directory exists, even if it's empty. This surprised me, as I had tried on Linux, where renameTo() succeeded if target existed, as long as it was empty.
(Obviously I shouldn't have assumed this kind of thing works the same across platforms; this is exactly what the Javadoc warns about.)
If you suspect there may be some lingering file locks, waiting a little before the move/rename might help. (In one point in our installer/upgrader we added a "sleep" action and an indeterminate progress bar for some 10 seconds, because there might be a service hanging on to some files). Perhaps even do a simple retry mechanism that tries renameTo(), and then waits for a period (which maybe increases gradually), until the operation succeeds or some timeout is reached.
In my case, most problems seem to have been solved by taking both of the above into account, so we won't need to do a native kernel call, or some such thing, after all.
The original post requested "an alternative, reliable approach to do a quick move/rename with Java on Windows, either with plain JDK or some external library."
Another option not mentioned yet here is v1.3.2 or later of the apache.commons.io library, which includes FileUtils.moveFile().
It throws an IOException instead of returning boolean false upon error.
See also big lep's response in this other thread.
On windows i use Runtime.getRuntime().exec("cmd \\c ") and then use commandline rename function to actually rename files. It is much more flexible, e.g if you want to rename extension of all txt files in a dir to bak just write this to output stream:
rename *.txt *.bak
I know it is not a good solution but apparently it has always worked for me, much better then Java inline support.
In my case it seemed to be a dead object within my own application, which kept a handle to that file. So that solution worked for me:
for (int i = 0; i < 20; i++) {
if (sourceFile.renameTo(backupFile))
break;
System.gc();
Thread.yield();
}
Advantage: it is pretty quick, as there is no Thread.sleep() with a specific hardcoded time.
Disadvantage: that limit of 20 is some hardcoded number. In all my tests, i=1 is enough. But to be sure I left it at 20.
I know this seems a little hacky, but for what I've been needing it for, it seems buffered readers and writers have no issue making the files.
void renameFiles(String oldName, String newName)
{
String sCurrentLine = "";
try
{
BufferedReader br = new BufferedReader(new FileReader(oldName));
BufferedWriter bw = new BufferedWriter(new FileWriter(newName));
while ((sCurrentLine = br.readLine()) != null)
{
bw.write(sCurrentLine);
bw.newLine();
}
br.close();
bw.close();
File org = new File(oldName);
org.delete();
}
catch (FileNotFoundException e)
{
e.printStackTrace();
}
catch (IOException e)
{
e.printStackTrace();
}
}
Works well for small text files as part of a parser, just make sure oldName and newName are full paths to the file locations.
Cheers
Kactus
The following piece of code is NOT an 'alternative' but has reliably worked for me on both Windows and Linux environments:
public static void renameFile(String oldName, String newName) throws IOException {
File srcFile = new File(oldName);
boolean bSucceeded = false;
try {
File destFile = new File(newName);
if (destFile.exists()) {
if (!destFile.delete()) {
throw new IOException(oldName + " was not successfully renamed to " + newName);
}
}
if (!srcFile.renameTo(destFile)) {
throw new IOException(oldName + " was not successfully renamed to " + newName);
} else {
bSucceeded = true;
}
} finally {
if (bSucceeded) {
srcFile.delete();
}
}
}
Why not....
import com.sun.jna.Native;
import com.sun.jna.Library;
public class RenamerByJna {
/* Requires jna.jar to be in your path */
public interface Kernel32 extends Library {
public boolean MoveFileA(String existingFileName, String newFileName);
}
public static void main(String[] args) {
String path = "C:/yourchosenpath/";
String existingFileName = path + "test.txt";
String newFileName = path + "renamed.txt";
Kernel32 kernel32 = (Kernel32) Native.loadLibrary("kernel32", Kernel32.class);
kernel32.MoveFileA(existingFileName, newFileName);
}
}
works on nwindows 7, does nothing if existingFile does not exist, but obviously could be better instrumented to fix this.
I had a similar issue. File was copied rather moving on Windows but worked well on Linux. I fixed the issue by closing the opened fileInputStream before calling renameTo(). Tested on Windows XP.
fis = new FileInputStream(originalFile);
..
..
..
fis.close();// <<<---- Fixed by adding this
originalFile.renameTo(newDesitnationForOriginalFile);
In my case, the error was in the path of the parent directory. Maybe a bug, I had to use the substring to get a correct path.
try {
String n = f.getAbsolutePath();
**n = n.substring(0, n.lastIndexOf("\\"));**
File dest = new File(**n**, newName);
f.renameTo(dest);
} catch (Exception ex) {
...
Well I have found a pretty straight forward solution to this problem -
boolean retVal = targetFile.renameTo(new File("abcd.xyz"));
while(!retVal) {
retVal= targetFile.renameTo(new File("abcd.xyz"));
}
As suggested by Argeman, you can place a counter and limit the number of times the while loop will run so that it doesn't get into an infinite loop in case of some file are being used by another windows process.
int counter = 0;
boolean retVal = targetFile.renameTo(new File("abcd.xyz"));
while(!retVal && counter <= 10) {
retVal = targetFile.renameTo(new File("abcd.xyz"));
counter = counter + 1;
}
I know it sucks, but an alternative is to create a bat script which outputs something simple like "SUCCESS" or "ERROR", invoke it, wait for it to be executed and then check its results.
Runtime.getRuntime().exec("cmd /c start test.bat");
This thread may be interesting. Check also the Process class on how to read the console output of a different process.
You may try robocopy. This is not exactly "renaming", but it's very reliable.
Robocopy is designed for reliable mirroring of directories or directory trees. It has features to ensure all NTFS attributes and properties are copied, and includes additional restart code for network connections subject to disruption.
To move/rename a file you can use this function:
BOOL WINAPI MoveFile(
__in LPCTSTR lpExistingFileName,
__in LPCTSTR lpNewFileName
);
It is defined in kernel32.dll.
File srcFile = new File(origFilename);
File destFile = new File(newFilename);
srcFile.renameTo(destFile);
The above is the simple code. I have tested on windows 7 and works perfectly fine.