We Keep Coding

How to apply multithreading in java to find a word in a directory(main directory and the word given) and recursively its subdirectories

I am quite new on Stack Overflow and a beginner in Java so please forgive me if I have asked this question in an improper way.
PROBLEM
I have an assignment which tells me to make use of multi-threading to search files for a given word, which might be present in any file of type .txt and .html, on any-level in the given directory (So basically the entire directory). The absolute file path of the file has to be displayed on the console if the file contains the given word.
WHAT HAVE I TRIED
So I thought of dividing the task into 2 sections, Searching and Multithreading respectively,
I was able to get the Searching part( File_search.java ). This file has given satisfactory results by searching through the directory and finding all the files in it for the given word.
File_search.java
public class File_search{
String fin_output = "";
public String searchInTextFiles(File dir,String search_word) {
File[] a = dir.listFiles();
for(File f : a){
if(f.isDirectory()) {
searchInTextFiles(f,search_word);
}
else if(f.getName().endsWith(".txt") || f.getName().endsWith(".html") || f.getName().endsWith(".htm") ) {
try {
searchInFile(f,search_word);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
}
}
return fin_output;
}
public void searchInFile(File f,String search_word) throws FileNotFoundException {
final Scanner sc = new Scanner(f);
while(sc.hasNextLine()) {
final String lineFromFile = sc.nextLine();
if(lineFromFile.contains(search_word)) {
fin_output += "FILE : "+f.getAbsolutePath().toString()+"\n";
}
}
}
Now, I want to be able to use multiple threads to execute the task File_search.java using ThreadPoolExecuter service. I'm not sure If I can do it using Runnable ,Callable or by using a Thread class or by any other method?
Can you please help me with the code to do the multi-threading part? Thanks :)

I agree to the comment of #chrylis -cautiouslyoptimistic, but for the purpose of understanding below will help you.
One simpler approach could be to do the traversal of directories in the main Thread, I mean the logic which you have added in function searchInTextFiles and do the searching logic as you did in function searchInFile in a Threadpool of size let's say 10.
Below sample code will help you to understand it better.
public class Traverser {
private List<Future<String>> futureList = new ArrayList<Future<String>>();
private ExecutorService executorService;
public Traverser() {
executorService = Executors.newFixedThreadPool(10);
}
public static void main(String[] args) throws InterruptedException, ExecutionException {
System.out.println("Started");
long start = System.currentTimeMillis();
Traverser traverser = new Traverser();
traverser.searchInTextFiles(new File("Some Directory Path"), "Some Text");
for (Future<String> future : traverser.futureList) {
System.out.println(future.get());
}
traverser.executorService.shutdown();
while(!traverser.executorService.isTerminated()) {
System.out.println("Not terminated yet, sleeping");
Thread.sleep(1000);
}
long end = System.currentTimeMillis();
System.out.println("Time taken :" + (end - start));
}
public void searchInTextFiles(File dir,String searchWord) {
File[] filesList = dir.listFiles();
for(File file : filesList){
if(file.isDirectory()) {
searchInTextFiles(file,searchWord);
}
else if(file.getName().endsWith(".txt") || file.getName().endsWith(".html") || file.getName().endsWith(".htm") ) {
try {
futureList.add(executorService.submit(new SearcherTask(file,searchWord)));
} catch (Exception e) {
e.printStackTrace();
}
}
}
}}
public class SearcherTask implements Callable<String> {
private File inputFile;
private String searchWord;
public SearcherTask(File inputFile, String searchWord) {
this.inputFile = inputFile;
this.searchWord = searchWord;
}
#Override
public String call() throws Exception {
StringBuilder result = new StringBuilder();
Scanner sc = null;
try {
sc = new Scanner(inputFile);
while (sc.hasNextLine()) {
final String lineFromFile = sc.nextLine();
if (lineFromFile.contains(searchWord)) {
result.append("FILE : " + inputFile.getAbsolutePath().toString() + "\n");
}
}
} catch (Exception e) {
//log error
throw e;
} finally {
sc.close();
}
return result.toString();
}}

Why does Scanner nextLine() read one file but not another in the same directory?

I have two files in a directory. Both of them have lines of text in them. According to Notepad++, each line has a CRLF. My ReadFile class reads one file and not the other and not the other.
private FileRead fileRead = new FileRead();
private JobSiteData jobSiteData = new JobSiteData();
private String path = "C:/Users/Antonio/IdeaProjects/JobSearch/websiteTest/";
If I uncomment out the block and comment out the other block it works fine.
#Test
void setAllText_ItReturnsAllLinesFromSite_arrayListOfLinesFromSite() throws IOException {
// ArrayList<String> thisWorks = new ArrayList<>();
// thisWorks = fileRead.getLinesFromFile(path + "allTextTest.txt");
// for (String s : thisWorks) {
// System.out.println(s);
// }
ArrayList<String> thisDoesntWork = new ArrayList<>();
thisDoesntWork = fileRead.getLinesFromFile(path + "goDaddySplitTestAllLines.txt");
for (String s : thisDoesntWork) {
System.out.println(s); // prints "File is empty."
}
}
Here is my FileRead Class.
public class FileRead {
private String fileName;
private ArrayList<String> linesFromFile = new ArrayList<>();
private Scanner fileScanner;
public ArrayList<String> getLinesFromFile(String fileName) {
this.fileName = fileName;
setLinesFromFile();
if (isFileEmpty()) {
linesFromFile.add("File is empty.");
}
return linesFromFile;
}
private void setLinesFromFile() {
File file = new File(fileName);
if (doesFileExist(file)) {
readFile();
closeFileReader();
}
}
private Boolean doesFileExist(File file) {
try {
fileScanner = new Scanner(file);
} catch (FileNotFoundException e) {
linesFromFile.add("File does not exist.");
return false;
}
return true;
}
private void readFile() {
if (fileScanner != null) {
if (!linesFromFile.isEmpty()) {
linesFromFile.clear();
}
while (fileScanner.hasNextLine()) {
linesFromFile.add(fileScanner.nextLine());
}
}
}
private void closeFileReader() {
if (fileScanner != null) {
fileScanner.close();
}
}
private boolean isFileEmpty() {
return linesFromFile.size() == 0;
}
}
UPDATE: I found the issue. For some reason this line in the file is causing the scanner not to read the lines.
"GoDaddy’s Productivity suite of products help millions of small businesses run and grow their ventures. We are looking for an engineer that is passionate about software development to help accelerate our growth. If you’re into building world-class experiences on public cloud infrastructure using JavaScript and React, learning new things, and working with a fun team, we want to meet you!"
This line is the forth line down. If I delete it and everything after it, the scanner works fine. If I delete everything in the file and only have that line in the file as the first line it won't work either. Does anyone know why this is?

How to use a Path object as a String

I'm looking to try and create a Java trivia application that reads the trivia from separate question files in a given folder. My idea was to use the run() method in the FileHandler class to set every text file in the folder into a dictionary and give them integer keys so that I could easily randomize the order at which they appear in the game. I found a simple chunk of code that is able to step through the folder and get the paths of every single file, but in the form a Path class. I need the paths (or just the names) in the form a String class. Because I need to later turn them into a file class (which excepts a String Constructor, not a Path). Here is the chunk of code that walks through the folder:
public class FileHandler implements Runnable{
static Map<Integer, Path> TriviaFiles; //idealy Map<Integer, String>
private int keyChoices = 0;
public FileHandler(){
TriviaFiles = new HashMap<Integer, Path>();
}
public void run(){
try {
Files.walk(Paths.get("/home/chris/JavaWorkspace/GameSpace/bin/TriviaQuestions")).forEach(filePath -> {
if (Files.isRegularFile(filePath)) {
TriviaFiles.put(keyChoices, filePath);
keyChoices++;
System.out.println(filePath);
}
});
} catch (FileNotFoundException e) {
System.out.println("File not found for FileHandler");
} catch (IOException e ){
e.printStackTrace();
}
}
public static synchronized Path getNextValue(){
return TriviaFiles.get(2);
}
}
There is another class named TextHandler() which reads the individual txt files and turns them into questions. Here it is:
public class TextHandler {
private String A1, A2, A3, A4, question, answer;
//line = null;
public void determineQuestion(){
readFile("Question2.txt" /* in file que*/);
WindowComp.setQuestion(question);
WindowComp.setAnswers(A1,A2,A3,A4);
}
public void readFile(String toRead){
try{
File file = new File("/home/chris/JavaWorkspace/GameSpace/bin/TriviaQuestions",toRead);
System.out.println(file.getCanonicalPath());
FileReader fr = new FileReader(file);
BufferedReader br = new BufferedReader(fr);
question = br.readLine();
A1 = br.readLine();
A2 = br.readLine();
A3 = br.readLine();
A4 = br.readLine();
answer = br.readLine();
br.close();
}
catch(FileNotFoundException e){
System.out.println("file not found");
}
catch(IOException e){
System.out.println("error reading file");
}
}
}
There is stuff I didn't include in this TextHandler sample which is unimportant.
My idea was to use the determineQuestion() method to readFile(FileHandler.getNextQuestion).
I am just having trouble working around the Path to String discrepancy
Thanks a bunch.

You can simply use Path.toString() which returns full path as a String. But kindly note that if path is null this method can cause NullPointerException. To avoid this exception you can use String#valueOf instead.
public class Test {
public static void main(String[] args) throws NoSuchFieldException, SecurityException {
Path path = Paths.get("/my/test/folder/", "text.txt");
String str = path.toString();
// String str = String.valueOf(path); //This is Null Safe
System.out.println(str);
}
}
Output
\my\test\folder\text.txt

how to input txt file in eclipse if i already have skeleton code to read/iterate though these files?

My teacher already gave us a skeleton code to iterate through .txt files, but I am not quite sure how to tell Eclipse (my IDE) which one to use?
I moved the sample .txt file into the same src folder, and I made sure that the names were exactly the same, but I am getting an error message saying that the file I specified can't be found.
I also tried copying and pasting all the text into the run configuration (in the main argument box)... I think it's the equivalent of the command line in MS Windows.
Here's the piece of code I'm referring to:
public class FileCharIterator implements Iterator<String> {
protected FileInputStream input;
private String inputFileName;
private int nextChar;
public FileCharIterator(String inputFileName) {
try {
input = new FileInputStream(inputFileName);
nextChar = input.read();
this.inputFileName = inputFileName;
} catch (FileNotFoundException e) {
System.err.printf("No such file: %s\n", inputFileName);
System.exit(1);
} catch (IOException e) {
System.err.printf("IOException while reading from file %s\n",
inputFileName);
System.exit(1);
}
}
#Override
public boolean hasNext() {
return nextChar != -1;
}
#Override
public String next() {
if (this.nextChar == -1) {
return "";
} else {
Byte b = (byte) this.nextChar;
String toRtn = String.format("%8s",
Integer.toBinaryString(b & 0xFF)).replace(' ', '0');
try {
this.nextChar = this.input.read();
} catch (IOException e) {
System.err.printf(
"IOException while reading in from file %s\n",
this.inputFileName);
}
return toRtn;
}
}
#Override
public void remove() {
throw new UnsupportedOperationException(
"FileCharIterator does not delete from files.");
}
}
I am very sure that the skeleton code works, and that the error happens because I am not sending commands correctly. If you could explain to me what I am doing that wrong I will be very grateful.
Thank you very much in advance!

Copy text File into class Path.(into the project folder , not to src folder)

How to display the contents of a directory

I need to write a recursive algorithm to display the contents of a directory in a computer's file system but I am very new to Java. Does anyone have any code or a good tutorial on how to access a directory in a file system with Java??

You can use the JFileChooser class, check this example.
Optionally you can also execute native commands like DIR , lsusing java , here is an example

This took me way too long to write and test, but here's something that should work.
Note: You can pass in either a string or file.
Note 2: This is a naive implementation. Not only is it single-threaded, but it does not check to see if files are links, and could get stuck in an endless loop due to this.
Note 3: The lines immediately after comments can be replaced with your own implementation.
import java.io.*;
public class DirectoryRecurser {
public static void parseFile(String filePath) throws FileNotFoundException {
File file = new File(filePath);
if (file.exists()) {
parseFile(file);
} else {
throw new FileNotFoundException(file.getPath());
}
}
public static void parseFile(File file) throws FileNotFoundException {
if (file.isDirectory()) {
for(File child : file.listFiles()) {
parseFile(child);
}
} else if (file.exists()) {
// Process file here
System.out.println(file.getPath());
} else {
throw new FileNotFoundException(file.getPath());
}
}
}
Which could then be called something like this (using a Windows path, because this Workstation is using Windows):
public static void main(String[] args) {
try {
DirectoryRecurser.parseFile("D:\\raisin");
} catch (FileNotFoundException e) {
// Error handling here
System.out.println("File not found: " + e.getMessage());
}
}
In my case, this prints out:
File not found: D:\raisin
because said directory is just one I made up. Otherwise, it prints out the path to each file.

Check out Apache Commons VFS: http://commons.apache.org/vfs/
Sample:
// Locate the Jar file
FileSystemManager fsManager = VFS.getManager();
FileObject jarFile = fsManager.resolveFile( "jar:lib/aJarFile.jar" );
// List the children of the Jar file
FileObject[] children = jarFile.getChildren();
System.out.println( "Children of " + jarFile.getName().getURI() );
for ( int i = 0; i < children.length; i++ )
{
System.out.println( children[ i ].getName().getBaseName() );
}
If you need to access files on a network drive, check out JCIFS: http://jcifs.samba.org/

check this out buddy
http://java2s.com/Code/Java/File-Input-Output/Traversingallfilesanddirectoriesunderdir.htm
public class Main {
public static void main(String[] argv) throws Exception {
}
public static void visitAllDirsAndFiles(File dir) {
System.out.println(dir);
if (dir.isDirectory()) {
String[] children = dir.list();
for (int i = 0; i < children.length; i++) {
visitAllDirsAndFiles(new File(dir, children[i]));
}
}
}
}

For each file you need to check if it is a directory. If it is, you need to recurse. Here is some untested code, which should help:
public void listFiles(File f){
System.out.println(f.getAbsolutePath());
if(f.isDirectory()){
for (File i : f.listFiles()){
listFiles(i);
}
}
}