Is there a Matcher for recursively comparing directories? - java

I'm writing unit tests for IM- and exporting files. I need to test the resulting directory recursively byte by byte. I implemented a routine for flat directories by myself and know how to do this recursively also. But I don't want to reinvent the wheel.
So is there something like the following examples?
Matchers.matches(Path actual, equalsRecursive(Path value));
or
FileAssertions.equalsRecursive(Path actual, Path value);

I am not aware of such a Matcher. So, IMO you will have to do it yourself.
2 options I could think of are as follows:
Use Apache Commons FileUtils in order to
1.1. make sure that each and every file/sub-directory(recursive call goes here) exists inside the directory currently being tested. For each subdir get a collection of files, iterate and use directoryContains method for the corresponding subdir.
1.2 make sure that the content of 2 corresponding files are equal using contentEquals method. I am not sure what happens if you pass 2 directories to this method.
Second option: if you run your tests on Linux box you can run a Linux command from Java using Runtime.exec()docs are here. The single command you need to run is diff -r <directory1> <directory2>
Good luck!

Didn't found anything. So i programmed it by myself. It's not very sophisticated and slow for large files, but seems to work.
import java.io.IOException;
import java.nio.file.FileVisitResult;
import java.nio.file.FileVisitor;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.attribute.BasicFileAttributes;
import org.junit.Assert;
/**
* Assertion for recursively testing directories.
*
* #author andreas
*/
public class AssertFile {
private AssertFile() {
throw new RuntimeException("This class should not be instantiated");
}
/**
* Asserts that two directories are recursively equal. If they are not, an {#link AssertionError} is thrown with the
* given message.<br/>
* There will be a binary comparison of all files under expected with all files under actual. File attributes will
* not be considered.<br/>
* Missing or additional files are considered an error.<br/>
*
* #param expected
* Path expected directory
* #param actual
* Path actual directory
*/
public static final void assertPathEqualsRecursively(final Path expected, final Path actual) {
Assert.assertNotNull(expected);
Assert.assertNotNull(actual);
final Path absoluteExpected = expected.toAbsolutePath();
final Path absoluteActual = actual.toAbsolutePath();
try {
Files.walkFileTree(expected, new FileVisitor<Path>() {
#Override
public FileVisitResult preVisitDirectory(Path expectedDir, BasicFileAttributes attrs)
throws IOException {
Path relativeExpectedDir = absoluteExpected.relativize(expectedDir.toAbsolutePath());
Path actualDir = absoluteActual.resolve(relativeExpectedDir);
if (!Files.exists(actualDir)) {
Assert.fail(String.format("Directory \'%s\' missing in target.", expectedDir.getFileName()));
}
Assert.assertEquals(String.format("Directory size of \'%s\' differ. ", relativeExpectedDir),
expectedDir.toFile().list().length, actualDir.toFile().list().length);
return FileVisitResult.CONTINUE;
}
#Override
public FileVisitResult visitFile(Path expectedFile, BasicFileAttributes attrs) throws IOException {
Path relativeExpectedFile = absoluteExpected.relativize(expectedFile.toAbsolutePath());
Path actualFile = absoluteActual.resolve(relativeExpectedFile);
if (!Files.exists(actualFile)) {
Assert.fail(String.format("File \'%s\' missing in target.", expectedFile.getFileName()));
}
Assert.assertEquals(String.format("File size of \'%s\' differ. ", relativeExpectedFile),
Files.size(expectedFile), Files.size(actualFile));
Assert.assertArrayEquals(String.format("File content of \'%s\' differ. ", relativeExpectedFile),
Files.readAllBytes(expectedFile), Files.readAllBytes(actualFile));
return FileVisitResult.CONTINUE;
}
#Override
public FileVisitResult visitFileFailed(Path file, IOException exc) throws IOException {
Assert.fail(exc.getMessage());
return FileVisitResult.TERMINATE;
}
#Override
public FileVisitResult postVisitDirectory(Path dir, IOException exc) throws IOException {
return FileVisitResult.CONTINUE;
}
});
} catch (IOException e) {
Assert.fail(e.getMessage());
}
}
}

Related

virtualbox - cannot move files within shared folder from java

Alright, so I tried moving some files with java ...
Exception in thread "main" java.nio.file.FileSystemException: baz -> ./foo/bar/baz: Operation not permitted
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:451)
at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262)
at java.nio.file.Files.move(Files.java:1347)
It's a kubuntu guest on a win 10 host.
I CAN move the folder by terminal (mv baz ./foo/bar/baz). It just seems the java application didn't get the notice it can do that.
How do I resolve this (without moving the content out of the shared folder)?
CODE
String targetDir ="./foo/bar/"
Path bazDir = Paths.get(MyConstants.BAZ_DIR);
DirectoryRemover.remove(Paths.get(targetDir).resolve(MyConstants.BAZ_DIR)); //just to be sure
Files.move(bazDir,Paths.get(targetDir).resolve(MyConstants.BAZ_DIR), StandardCopyOption.REPLACE_EXISTING);
where
import java.io.IOException;
import java.nio.file.FileVisitResult;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.SimpleFileVisitor;
import java.nio.file.attribute.BasicFileAttributes;
public class DirectoryRemover {
public static void remove(Path obstacle){
try {
Files.walkFileTree(obstacle, new FileWalker());
}catch(Exception e){
//didn't exist / already deleted, etc.
}
}
static class FileWalker extends SimpleFileVisitor<Path>{
#Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
Files.delete(file);
return FileVisitResult.CONTINUE;
}
#Override
public FileVisitResult postVisitDirectory(Path dir, IOException exc) throws IOException {
Files.delete(dir);
return FileVisitResult.CONTINUE;
}
}
}
From all I can tell this is an actual issue in NIO. I observe similar issue with e.g. using Eclipse/jgit on a Ubuntu VirtualBox VM running on macOS.
The only suggestion I have is to use a local folder within the VM instead.

Java - Differentiate ZIP file from CSV file

I'm using a webservice that is always sending me a plain/text file. However, that file can either be a zip or a csv but I'm not being informed of its type beforehand.
Is there a way to know the file type by looking through its content programmatically wise of course. As one is in byte code and the other one an actually readeable text.
I've already thought of looking for lots of commas in the file content but that seems inaccurate.
You can use java.util.zip.ZipFile, if the constructor throws a ZipException, it's not a zip file...
try(ZipFile zip = new ZipFile(filename)) {
// It's a zip file
}
catch(ZipException e) {
// Not a valid zip
}
You could make use of the ZIP file structure.
As per the file header, each file should start with the bytes: 0x04 0x03 0x4b 0x50.
You could also use a MIME detection library such as Apache Tika import org.apache.tika.Tika;
import org.apache.tika.mime.MediaType;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
public class Detect {
/**
* Resolves the MediaType using Tika and prints it to the standard output.
* #param file the path of the file to probe.
* #throws IOException whenever an I/O exception occurs.
*/
private void detect(Path file) throws IOException {
Tika tika = new Tika();
try(InputStream is = Files.newInputStream(file)){
MediaType mediaType = MediaType.parse(tika.detect(is));
System.out.println(mediaType);
}
}
public static void main(String[] args) throws IOException {
Detect d = new Detect();
d.detect(Paths.get("zip_file"));
d.detect(Paths.get("csv_file"));
}
}

Moving large files in java

I have to move files from one directory to other directory.
Am using property file. So the source and destination path is stored in property file.
Am haivng property reader class also.
In my source directory am having lots of files. One file should move to other directory if its complete the operation.
File size is more than 500MB.
import java.io.File;
import java.nio.file.Files;
import java.nio.file.StandardCopyOption;
import static java.nio.file.StandardCopyOption.*;
public class Main1
{
public static String primarydir="";
public static String secondarydir="";
public static void main(String[] argv)
throws Exception
{
primarydir=PropertyReader.getProperty("primarydir");
System.out.println(primarydir);
secondarydir=PropertyReader.getProperty("secondarydir");
File dir = new File(primarydir);
secondarydir=PropertyReader.getProperty("secondarydir");
String[] children = dir.list();
if (children == null)
{
System.out.println("does not exist or is not a directory");
}
else
{
for (int i = 0; i < children.length; i++)
{
String filename = children[i];
System.out.println(filename);
try
{
File oldFile = new File(primarydir,children[i]);
System.out.println( "Before Moving"+oldFile.getName());
if (oldFile.renameTo(new File(secondarydir+oldFile.getName())))
{
System.out.println("The file was moved successfully to the new folder");
}
else
{
System.out.println("The File was not moved.");
}
}
catch (Exception e)
{
e.printStackTrace();
}
}
System.out.println("ok");
}
}
}
My code is not moving the file into the correct path.
This is my property file
primarydir=C:/Desktop/A
secondarydir=D:/B
enter code here
Files should be in B drive. How to do? Any one can help me..!!
Change this:
oldFile.renameTo(new File(secondarydir+oldFile.getName()))
To this:
oldFile.renameTo(new File(secondarydir, oldFile.getName()))
It's best not to use string concatenation to join path segments, as the proper way to do it may be platform-dependent.
Edit: If you can use JDK 1.7 APIs, you can use Files.move() instead of File.renameTo()
Code - a java method:
/**
* copy by transfer, use this for cross partition copy,
* #param sFile source file,
* #param tFile target file,
* #throws IOException
*/
public static void copyByTransfer(File sFile, File tFile) throws IOException {
FileInputStream fInput = new FileInputStream(sFile);
FileOutputStream fOutput = new FileOutputStream(tFile);
FileChannel fReadChannel = fInput.getChannel();
FileChannel fWriteChannel = fOutput.getChannel();
fReadChannel.transferTo(0, fReadChannel.size(), fWriteChannel);
fReadChannel.close();
fWriteChannel.close();
fInput.close();
fOutput.close();
}
The method use nio, it make use os underling operation to improve performance.
Here is the import code:
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
If you are in eclipse, just use ctrl + shift + o.

How do I find/list a file or directory with specific name inside a directory and its subdirectories in Java?

What my file/directory tree on Linux (Redhat) looks like.
/search
├───sub1
│ └───data (directory)
└───sub2
└───data (file)
What I already tried is listed below.
Example I (I assumed directories are files in Linux and Java knows this)
FileUtils.listFiles(new File("/search"),
new NameFileFilter("data"), TrueFileFilter.INSTANCE)
.forEach(System.out::println);
Exampe II
FileUtils.listFilesAndDirectories(new File("/search"),
new NameFileFilter("data"), TrueFileFilter.INSTANCE)
.forEach(System.out::println);
The above code examples does not print "/search/sub1/data" and /search/sub2/data" like I want them to. FileUtils is from http://commons.apache.org/proper/commons-io/javadocs/api-release/org/apache/commons/io/FileUtils.html.
Does anyone have a solution?
You can achieve the same using Java 7 Files.walkFileTree, PathMatcher and FileVisitor like this
import java.io.IOException;
import java.nio.file.FileSystems;
import java.nio.file.FileVisitResult;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.PathMatcher;
import java.nio.file.Paths;
import java.nio.file.SimpleFileVisitor;
import java.nio.file.attribute.BasicFileAttributes;
public class MyVisitor extends SimpleFileVisitor<Path> {
PathMatcher matcher = FileSystems.getDefault().getPathMatcher("glob:data");
#Override
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {
if (matcher.matches(dir.getFileName())) {
System.out.println("dir found " + dir);
}
return FileVisitResult.CONTINUE;
}
#Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
if (matcher.matches(file.getFileName())) {
System.out.println("file found " + file);
}
return FileVisitResult.CONTINUE;
}
public static void main(String[] args) throws IOException {
Files.walkFileTree(Paths.get("full/path/to/your/search"), new MyVisitor());
}
}

Java.nio.files - Copying files

Can anyone tell me what I've done wrong with the following code. I receive no errors - it just goes straight to the catch.
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
public class Main {
public static void main(String[] args) {
Path source = Paths.get("C:\\Users\\Public\\Pictures\\SamplePictures");
Path nwdir = Paths.get("D:\\NetbeansProjects\\CopyingFiles\\copiedImages");
try{
Files.copy(source, nwdir);
}catch (IOException e){
System.out.println("Unsucessful. What a surprise!");
}
}
}
If you take a look at the Javadocs of Files.copy, you'll notice this line (emphasis added):
If the file is a directory then it creates an empty directory in the target location (entries in the directory are not copied). This method can be used with the walkFileTree method to copy a directory and all entries in the directory, or an entire file-tree where required.
So it looks like you need to use that walkFileTree method.
(And as the commenters said, print out exceptions and they'll often tell you what's wrong!)
Came across here looking for a NIO Java7 approach to recursively copy a directory to another location. This can be done with Files.walkFileTree as Jon7 mentioned in the other anwer. This code I got for a simple directory copy:
final Path srcDir, final Path dstDir;
Files.walkFileTree(srcDir, new SimpleFileVisitor<Path>() {
public FileVisitResult visitFile( Path file, BasicFileAttributes attrs ) throws IOException {
return copy(file);
}
public FileVisitResult preVisitDirectory( Path dir, BasicFileAttributes attrs ) throws IOException {
return copy(dir);
}
private FileVisitResult copy( Path fileOrDir ) throws IOException {
Files.copy( fileOrDir, dstDir.resolve( srcDir.relativize( fileOrDir ) ) );
return FileVisitResult.CONTINUE;
}
});
For a more detailed example which also handles file attributes and overwriting of existing files, see http://docs.oracle.com/javase/tutorial/essential/io/examples/Copy.java .
This is how I have managed to copy a file from one location to another:
import java.io.IOException;
import static java.nio.file.StandardCopyOption.*;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
public class App {
public static void main(String[] args)
{
Path source = Paths.get("E:/myFile.pdf");
Path nwdir = Paths.get("F:");
try
{
Files.copy(source, nwdir.resolve(source.getFileName()), REPLACE_EXISTING);
System.out.println("File Copied");
}
catch(IOException e)
{
e.printStackTrace();
}
}
}

Categories