We Keep Coding

Issue with listing files recursivelly in Java

I've decided to write a recursive program that writes all the files in my C drive into a .txt file, however it is very slow.
I've read online that recursion is slow, but i can't think of any other way. Is there any way i can optimize this ?
EDIT : changed the deepInspect method to use a Stack instead of recursion, which slightly improved performance.
Here is the code
public class FileCount {
static long fCount = 0;
public static void main(String[] args) {
System.out.println("Start....");
long start = System.currentTimeMillis();
File cDir = new File("C:\\");
inspect(cDir);
System.out.println("Operation took : " + (System.currentTimeMillis() - start) + " ms");
}
private static void inspect(File cDir) {
for (File f : cDir.listFiles()) {
deepInspect(f);
}
}
private static void deepInspect(File f) {
Stack<File> stack = new Stack<File>();
stack.push(f);
while (!stack.isEmpty()) {
File current = stack.pop();
if (current.listFiles() != null) {
for (File file : current.listFiles()) {
stack.push(file);
}
}
writeData(current.getAbsolutePath());
}
}
static FileWriter writer = null;
private static void writeData(String absolutePath) {
if (writer == null)
try {
writer = new FileWriter("C:\\Collected\\data.txt");
} catch (IOException e) {}
try {
writer.write(absolutePath);
writer.write("\r\n");//nwline
writer.write("Files : " + fCount);
writer.write("\r\n");
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}

Java 8 provides a stream to process all files.
Files.walk(Paths.get("/"))
.filter(Files::isRegularFile)
.forEach(System.out::println);
You could add "parallel" processing for improved performance
Files.walk(Paths.get("/"))
.parallel()
.filter(Files::isRegularFile)
.forEach(System.out::println);
I tried this under linux, so you would need to replace "/" with "C:" and try it. Besides in my case stops when I try to read I don't have access, so you would need to check that too if you are not running as admin.
Check this out

I don't think the recursion is an issue here. The main issue in your code is the File IO which you are doing at every level. The disk access is extremely costly w.r.t the memory access. If you profile your code you should definitely see huge spike in the disk IO.
So, essentially you want to reduce the disk I/O. To do so you could have a in memory finite size Buffer where you can write the output and when the buffer is full flush the data to the file.
This however considerable more amount of work.

Error - Deleting files in java returning false

I am trying to delete a file from a directory using Java with the following code:
static String deleta ="C:\\res\\in\\CANteste2.xml";
.
.
.
Boolean file = new File (deleta) .delete();
System.out.println ("file:" + file);
Permissions: http://imgur.com/a/dVPPW
But it returns always false with no errors

You should use try and catch code for deleting the files
if using java 7 then use Files API
or try to check the File exists() method before deleting file
try {
Files.delete(path);
} catch (NoSuchFileException x) {
System.err.format("%s: no such" + " file or directory%n", path);
} catch (DirectoryNotEmptyException x) {
System.err.format("%s not empty%n", path);
} catch (IOException x) {
// File permission problems are caught here.
System.err.println(x);
}

I see some compilation errors in the code snippet.
System.out.println (file:" + file);
This statement will not compile. Probably you meant:
System.out.println ("file: "+file);
The three lines will not be present in a single method. Reason is that you have declared the deleta variable as static. This means that you have to make it Class level. Thus, if you make deleta and file as Class level variables, you cannot have the System.out.println ("file: "+file); in the Class Level. The Syso statement should be in a method.
Finally your code should look like:
public class Test {
static String deleta = "C:\\res\\in\\CANteste2.xml";
Boolean file = new File(deleta).delete();
public static void main(String[] args) {
Test test = new Test();
test.print();
}
public void print() {
System.out.println("file: " + file);
}
}
This code will return TRUE if the file was PRESENT and is now DELETED. It will return FALSE if the file is not found in the directory.
So the first instance you will run the code, you will get the output as TRUE (if the file is present). From the second instance onwards, you will get the output as FALSE, as it has already been deleted!

Writing to a file from a void method

I am working on a program that uses Dijkstra's algorithm and records the results to a text file. The bit of code I have that writes to the file looks like this:
try (PrintWriter pr = new PrintWriter(filename + "Out.txt")) {
pr.println("Adjacency Matrix: " + (endTime - startTime) + " ms ");
pr.println("Min-Heap: ");
pr.println("Fibonnaci Heap:");
pr.println("Dijkstra Adjacency Matrix");
pr.println(g.printPath(END));
}
} catch (Exception e) {
}
I have no problems with this bit of code, except for the line g.printPath(END). The error I receive is "void type not allowed here". I fully understand what this means. It occurs because the printPath method is void. It looks like this:
public void printPath(String end) {
if (!graph.containsKey(end)) {
System.err.printf("End vertex is not contained within graph \"%s\"\n", end);
return;
}
graph.get(end).printPath();
System.out.println();
}
Since I need access to the variable it would print, I tried to modify it to have a return type that I could write to the text file. What I came up with was this:
public String printPath(String end) {
if (!graph.containsKey(end)) {
System.err.printf("End vertex is not contained within graph \"%s\"\n", end);
return null;
}
graph.get(end).printPath();
System.out.println();
return graph.get(end).printPath();
}
This again has errors, since the method is of type string but graph.get(end).printPath() is void (the get method is also void). I have attempted returning other variables such as graph and graph.get(end) but they do not return the actual variables inside the graph. I know that graph.get(end).printPath() prints out the correct values I want. I am just struggling to find a way to store them. Is there an easy way I can write this to the text file that I am overlooking without having to go back and edit all of my methods to make them not void? Thanks!

There is a way to do it by redirectSystem.out.print:
public String printPath(Graph graph, String end) {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
PrintStream printStream = new PrintStream(bos);
//set output stream to bos to capture output
System.setOut(printStream);
graph.get(end).printPath(); //your output
System.out.println();
//reset output stream to file descriptor
System.setOut(new PrintStream(new FileOutputStream(FileDescriptor.out)));
return bos.toString();
}
Redirect the System.out to ByteArrayOutputStream,
Start print
Reset the System.out to FileDescriptor
Finally, Really don't suggestion do it, it's dirty code and important it's not thread-safe, and it's confusing. There is a suggestion about how to deal this:
Create a method to formatt graph.get(end) and return correct String type path

With your current usage, printPath shouldn't be printing anything: maybe you could even rename it to getPath. You need to build a string with the correct value and return it, so that the returned value can be passed to println.
public String printPath(String end) {
if (!graph.containsKey(end)) {
return "End vertex is not contained within graph \"%s\"\n", end);
}
// Also rework this to return a string instead of printlning stuff.
return graph.get(end).printPath();
}
Alternatively, don't pass the value to println and just call g.printPath(END); directly.
try (PrintWriter pr = new PrintWriter(filename + "Out.txt")) {
pr.println("Adjacency Matrix: " + (endTime - startTime) + " ms ");
pr.println("Min-Heap: ");
pr.println("Fibonnaci Heap:");
pr.println("Dijkstra Adjacency Matrix");
g.printPath(END);
} catch (Exception e) {
}

Traversing and enumerating a directory with multi-threads

I am running a thread to traverse my local directory (no sub directory) and as soon as I am getting a text file, I am starting a new thread which will search a word in that file.
What is wrong in the below code?
Searching and traversing are working fine, separately. But when I am putting it together, some thing is going wrong, it is skipping some files (Not exactly, due to multithreading object sunchronization is not happening properly).
Please help me out.
Traverse.java
public void executeTraversing() {
Path dir = null;
if(dirPath.startsWith("file://")) {
dir = Paths.get(URI.create(dirPath));
} else {
dir = Paths.get(dirPath);
}
listFiles(dir);
}
private synchronized void listFiles(Path dir) {
ExecutorService executor = Executors.newFixedThreadPool(1);
try (DirectoryStream<Path> stream = Files.newDirectoryStream(dir)) {
for (Path file : stream) {
if (Files.isDirectory(file)) {
listFiles(file);
} else {
search.setFileNameToSearch(file);
executor.submit(search);
}
}
} catch (IOException | DirectoryIteratorException x) {
// IOException can never be thrown by the iteration.
// In this snippet, it can only be thrown by
// newDirectoryStream.
System.err.println(x);
}
}
Search.java
/**
* #param wordToSearch
*/
public Search(String wordToSearch) {
super();
this.wordToSearch = wordToSearch;
}
public void run() {
this.search();
}
private synchronized void search() {
counter = 0;
Charset charset = Charset.defaultCharset();
try (BufferedReader reader = Files.newBufferedReader(fileNameToSearch.toAbsolutePath(), charset)) {
// do you have permission to read this directory?
if (Files.isReadable(fileNameToSearch)) {
String line = null;
while ((line = reader.readLine()) != null) {
counter++;
//System.out.println(wordToSearch +" "+ fileNameToSearch);
if (line.contains(wordToSearch)) {
System.out.println("Word '" + wordToSearch
+ "' found at "
+ counter
+ " in "
+ fileNameToSearch);
}
}
} else {
System.out.println(fileNameToSearch
+ " is not readable.");
}
} catch (IOException x) {
System.err.format("IOException: %s%n", x);
}
}

this Search instance that you keep reusing here:
search.setFileNameToSearch(file);
executor.submit(search);
while its actual search() method is synchronized, it appears like by the time it actually gets to searching something setFileNameToSearch() would have been called several times, which would explain the skipping.
create a new instance of Search each time, then you wouldnt need to sync the actual search() function.

You are creating the ExecutorService inside your listFiles method, this is probably not a good idea: because of that you're probably creating too many threads.
On top of that you're not monitoring the state of all these ExecutorServices, some of them might not be started when you application stops
Instead you should create the ExecutorService only once, before starting the recursion. When the recursion is over, call shutdown() on your ExecutorService to wait for all tasks completion
Furthermore you are reusing a Search object and passing it to mutliple tasks while modifying it, you should create a Search for each file you're processing

Redundant use of .delete()

I am trying to create a method that renames a log file in a data collection program, but there is an IOException that is thrown (infrequently) when trying to access the log file shortly after accessing it once (for a second trace of data).
I do not know if it is because of the call to .delete() or if I have to make sure the file is closed before accessing it. I am really stuck on this one. Here is the method I am working on.
public static void renameFile(String oldName, String newName) throws IOException
{
File srcFile = new File(oldName).getAbsoluteFile();
boolean bSucceeded = false;
try
{
File destFile = new File(newName).getAbsoluteFile();
if (destFile.exists())
{
if (!destFile.delete())
{
throw new IOException(oldName + " was not successfully renamed to " + newName + ", could not perform !destFile.delete()");
}
}
if (!srcFile.renameTo(destFile))
{
throw new IOException(oldName + " was not successfully renamed to " + newName + ", could not rename source file");
}
else
{
bSucceeded = true;
}
}
finally
{
if (bSucceeded)
{
srcFile.delete();
}
}
}

The code seems Ok, but as you mentioned, you must make sure to close the file before calling this method. On Unix, this may work, but on Windows you definetly cannot rename or delete a file which has an open file descriptor.