We Keep Coding

How to apply multithreading in java to find a word in a directory(main directory and the word given) and recursively its subdirectories

I am quite new on Stack Overflow and a beginner in Java so please forgive me if I have asked this question in an improper way.
PROBLEM
I have an assignment which tells me to make use of multi-threading to search files for a given word, which might be present in any file of type .txt and .html, on any-level in the given directory (So basically the entire directory). The absolute file path of the file has to be displayed on the console if the file contains the given word.
WHAT HAVE I TRIED
So I thought of dividing the task into 2 sections, Searching and Multithreading respectively,
I was able to get the Searching part( File_search.java ). This file has given satisfactory results by searching through the directory and finding all the files in it for the given word.
File_search.java
public class File_search{
String fin_output = "";
public String searchInTextFiles(File dir,String search_word) {
File[] a = dir.listFiles();
for(File f : a){
if(f.isDirectory()) {
searchInTextFiles(f,search_word);
}
else if(f.getName().endsWith(".txt") || f.getName().endsWith(".html") || f.getName().endsWith(".htm") ) {
try {
searchInFile(f,search_word);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
}
}
return fin_output;
}
public void searchInFile(File f,String search_word) throws FileNotFoundException {
final Scanner sc = new Scanner(f);
while(sc.hasNextLine()) {
final String lineFromFile = sc.nextLine();
if(lineFromFile.contains(search_word)) {
fin_output += "FILE : "+f.getAbsolutePath().toString()+"\n";
}
}
}
Now, I want to be able to use multiple threads to execute the task File_search.java using ThreadPoolExecuter service. I'm not sure If I can do it using Runnable ,Callable or by using a Thread class or by any other method?
Can you please help me with the code to do the multi-threading part? Thanks :)

I agree to the comment of #chrylis -cautiouslyoptimistic, but for the purpose of understanding below will help you.
One simpler approach could be to do the traversal of directories in the main Thread, I mean the logic which you have added in function searchInTextFiles and do the searching logic as you did in function searchInFile in a Threadpool of size let's say 10.
Below sample code will help you to understand it better.
public class Traverser {
private List<Future<String>> futureList = new ArrayList<Future<String>>();
private ExecutorService executorService;
public Traverser() {
executorService = Executors.newFixedThreadPool(10);
}
public static void main(String[] args) throws InterruptedException, ExecutionException {
System.out.println("Started");
long start = System.currentTimeMillis();
Traverser traverser = new Traverser();
traverser.searchInTextFiles(new File("Some Directory Path"), "Some Text");
for (Future<String> future : traverser.futureList) {
System.out.println(future.get());
}
traverser.executorService.shutdown();
while(!traverser.executorService.isTerminated()) {
System.out.println("Not terminated yet, sleeping");
Thread.sleep(1000);
}
long end = System.currentTimeMillis();
System.out.println("Time taken :" + (end - start));
}
public void searchInTextFiles(File dir,String searchWord) {
File[] filesList = dir.listFiles();
for(File file : filesList){
if(file.isDirectory()) {
searchInTextFiles(file,searchWord);
}
else if(file.getName().endsWith(".txt") || file.getName().endsWith(".html") || file.getName().endsWith(".htm") ) {
try {
futureList.add(executorService.submit(new SearcherTask(file,searchWord)));
} catch (Exception e) {
e.printStackTrace();
}
}
}
}}
public class SearcherTask implements Callable<String> {
private File inputFile;
private String searchWord;
public SearcherTask(File inputFile, String searchWord) {
this.inputFile = inputFile;
this.searchWord = searchWord;
}
#Override
public String call() throws Exception {
StringBuilder result = new StringBuilder();
Scanner sc = null;
try {
sc = new Scanner(inputFile);
while (sc.hasNextLine()) {
final String lineFromFile = sc.nextLine();
if (lineFromFile.contains(searchWord)) {
result.append("FILE : " + inputFile.getAbsolutePath().toString() + "\n");
}
}
} catch (Exception e) {
//log error
throw e;
} finally {
sc.close();
}
return result.toString();
}}

Why does Scanner nextLine() read one file but not another in the same directory?

I have two files in a directory. Both of them have lines of text in them. According to Notepad++, each line has a CRLF. My ReadFile class reads one file and not the other and not the other.
private FileRead fileRead = new FileRead();
private JobSiteData jobSiteData = new JobSiteData();
private String path = "C:/Users/Antonio/IdeaProjects/JobSearch/websiteTest/";
If I uncomment out the block and comment out the other block it works fine.
#Test
void setAllText_ItReturnsAllLinesFromSite_arrayListOfLinesFromSite() throws IOException {
// ArrayList<String> thisWorks = new ArrayList<>();
// thisWorks = fileRead.getLinesFromFile(path + "allTextTest.txt");
// for (String s : thisWorks) {
// System.out.println(s);
// }
ArrayList<String> thisDoesntWork = new ArrayList<>();
thisDoesntWork = fileRead.getLinesFromFile(path + "goDaddySplitTestAllLines.txt");
for (String s : thisDoesntWork) {
System.out.println(s); // prints "File is empty."
}
}
Here is my FileRead Class.
public class FileRead {
private String fileName;
private ArrayList<String> linesFromFile = new ArrayList<>();
private Scanner fileScanner;
public ArrayList<String> getLinesFromFile(String fileName) {
this.fileName = fileName;
setLinesFromFile();
if (isFileEmpty()) {
linesFromFile.add("File is empty.");
}
return linesFromFile;
}
private void setLinesFromFile() {
File file = new File(fileName);
if (doesFileExist(file)) {
readFile();
closeFileReader();
}
}
private Boolean doesFileExist(File file) {
try {
fileScanner = new Scanner(file);
} catch (FileNotFoundException e) {
linesFromFile.add("File does not exist.");
return false;
}
return true;
}
private void readFile() {
if (fileScanner != null) {
if (!linesFromFile.isEmpty()) {
linesFromFile.clear();
}
while (fileScanner.hasNextLine()) {
linesFromFile.add(fileScanner.nextLine());
}
}
}
private void closeFileReader() {
if (fileScanner != null) {
fileScanner.close();
}
}
private boolean isFileEmpty() {
return linesFromFile.size() == 0;
}
}
UPDATE: I found the issue. For some reason this line in the file is causing the scanner not to read the lines.
"GoDaddy’s Productivity suite of products help millions of small businesses run and grow their ventures. We are looking for an engineer that is passionate about software development to help accelerate our growth. If you’re into building world-class experiences on public cloud infrastructure using JavaScript and React, learning new things, and working with a fun team, we want to meet you!"
This line is the forth line down. If I delete it and everything after it, the scanner works fine. If I delete everything in the file and only have that line in the file as the first line it won't work either. Does anyone know why this is?

How to use a Path object as a String

I'm looking to try and create a Java trivia application that reads the trivia from separate question files in a given folder. My idea was to use the run() method in the FileHandler class to set every text file in the folder into a dictionary and give them integer keys so that I could easily randomize the order at which they appear in the game. I found a simple chunk of code that is able to step through the folder and get the paths of every single file, but in the form a Path class. I need the paths (or just the names) in the form a String class. Because I need to later turn them into a file class (which excepts a String Constructor, not a Path). Here is the chunk of code that walks through the folder:
public class FileHandler implements Runnable{
static Map<Integer, Path> TriviaFiles; //idealy Map<Integer, String>
private int keyChoices = 0;
public FileHandler(){
TriviaFiles = new HashMap<Integer, Path>();
}
public void run(){
try {
Files.walk(Paths.get("/home/chris/JavaWorkspace/GameSpace/bin/TriviaQuestions")).forEach(filePath -> {
if (Files.isRegularFile(filePath)) {
TriviaFiles.put(keyChoices, filePath);
keyChoices++;
System.out.println(filePath);
}
});
} catch (FileNotFoundException e) {
System.out.println("File not found for FileHandler");
} catch (IOException e ){
e.printStackTrace();
}
}
public static synchronized Path getNextValue(){
return TriviaFiles.get(2);
}
}
There is another class named TextHandler() which reads the individual txt files and turns them into questions. Here it is:
public class TextHandler {
private String A1, A2, A3, A4, question, answer;
//line = null;
public void determineQuestion(){
readFile("Question2.txt" /* in file que*/);
WindowComp.setQuestion(question);
WindowComp.setAnswers(A1,A2,A3,A4);
}
public void readFile(String toRead){
try{
File file = new File("/home/chris/JavaWorkspace/GameSpace/bin/TriviaQuestions",toRead);
System.out.println(file.getCanonicalPath());
FileReader fr = new FileReader(file);
BufferedReader br = new BufferedReader(fr);
question = br.readLine();
A1 = br.readLine();
A2 = br.readLine();
A3 = br.readLine();
A4 = br.readLine();
answer = br.readLine();
br.close();
}
catch(FileNotFoundException e){
System.out.println("file not found");
}
catch(IOException e){
System.out.println("error reading file");
}
}
}
There is stuff I didn't include in this TextHandler sample which is unimportant.
My idea was to use the determineQuestion() method to readFile(FileHandler.getNextQuestion).
I am just having trouble working around the Path to String discrepancy
Thanks a bunch.

You can simply use Path.toString() which returns full path as a String. But kindly note that if path is null this method can cause NullPointerException. To avoid this exception you can use String#valueOf instead.
public class Test {
public static void main(String[] args) throws NoSuchFieldException, SecurityException {
Path path = Paths.get("/my/test/folder/", "text.txt");
String str = path.toString();
// String str = String.valueOf(path); //This is Null Safe
System.out.println(str);
}
}
Output
\my\test\folder\text.txt

fill a file inside a inner class

I'm quiet new to java and I'm not even sure if the title makes any sense.
Anyway how do I gain access to a file inside of onStatus?
I'm just trying to write into a file in that function so in the code below "T_file.length()" the "T_file" cannot be resolved.
public static void main (){
foo("file_name");
}
foo(String fileName)
{
try{
final File T_file = new File (fileName);
final FileWriter fw = new FileWriter(fileName);
final BufferedWriter bw = new BufferedWriter (fw);
}catch (IOException e){}
StatusListener listener = new StatusListener(){
public void onStatus(Status status){
if (T_file.length() > 100){
System.out.println ("I have access here");
}
}
}
}

"T_file.length()" the "T_file" cannot be resolved
This is because T_file is not in the scope of onStatus. You declare T_file in the foo method, but then create a new StatusListener in which you redefine the onStatus, but the compiler still see StatusListener as an other class.
A workaround would be to declare the variable globally in your class, then access it specifying the name of your Class.this.T_file. Here an example :
public class Foo {
File T_file;
public static void main (){
new Foo("file_name");
}
Foo(String fileName)
{
T_file = new File (fileName);
StatusListener listener = new StatusListener(){
public void onStatus(Status status){
if (Foo.this.T_file.length() > 100){
System.out.println ("I have access here");
}
}
};
}
}
This example would compile.
Notice that I added a ; at the end of the StatusListener, else it would not compile.

File Sorter - fileNotFoundException

I am trying to create a program that reads a File, saves the content (words) in an ArrayList, sorts the ArrayList, and then writes the content of the sorted ArrayList back to the File.
I have no idea why, it keeps on giving me a FileNotFoundException or a NullPointerException (both are occurring, it's a bit wierd)...
Here's my code, if anyone can help that would be great.
Thanks.
By the way, the code contains four classes:
DriverClass, View(the GUI), ReadFile, and WriteFile.
You can ignore the comments, I just wrote those for myself - they're pretty obvious. For "field.getText();" let's say the user enters C:\Users\Corecase\Desktop\test.txt I've tried doing C:\\Users\\Corecase\\Desktop\\test.txt) and that doesn't work either.
Thanks again!
public class DriverClass
{
public static void main(String[] args)
{
View open= new View();
}
}
//View
import javax.swing.*;
import java.awt.event.*;
import java.awt.*;
public class View implements ActionListener
{
private JFrame frame = new JFrame("File Sorter");
private JPanel mainPane = new JPanel();
private JPanel textPane = new JPanel();
private JPanel buttonPane = new JPanel();
private JButton sortButton = new JButton("Sort");
private JLabel label = new JLabel("Enter file path: ");
public JTextField field = new JTextField(25);
private Font f = new Font("Trebuchet MS", Font.PLAIN, 20);
public View()
{
frame.setSize(500,500);
frame.setVisible(true);
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
frame.add(mainPane);
mainPane.setLayout(new GridLayout(2,1));
mainPane.setBackground(Color.gray);
mainPane.add(textPane);
mainPane.add(buttonPane);
textPane.setLayout(new GridLayout(2,1));
textPane.add(label);
textPane.add(field);
buttonPane.add(sortButton);
field.setFont(f);
sortButton.setFont(f);
label.setFont(f);
sortButton.addActionListener(this);
}
public void actionPerformed(ActionEvent e)
{
if(e.getSource() == sortButton)
{
ReadFile r = new ReadFile(field.getText());
WriteFile w = new WriteFile(field.getText());
r.openFile();
r.readAndSortFile();
r.closeFile();
w.openFile();
w.writeFile(r.getList());
w.closeFile();
}
}
}
//ReadFile
import java.io.*;
import java.util.*;
public class ReadFile extends View
{
private ArrayList<String> words = new ArrayList<String>();
private String fileName = new String();
private Scanner x;
public ReadFile(String address)
{
fileName = address;
}
public void openFile()
{
try
{
x = new Scanner(new File(fileName));
}
catch(Exception e)
{
field.setText("Could not read file.");
}
}
public void readAndSortFile()
{
while(x.hasNext())
words.add(x.next());
sort();
}
public void closeFile()
{
x.close();
}
public ArrayList<String> sort()
{
String temp = "";
for(int index = 0; index < words.size(); index++)
{
for(int inner = 0; inner < words.size(); inner++)
{
if((words.get(inner)).compareTo(words.get(inner+1)) > 0)
{
temp = words.get(inner);
words.set(inner, words.get(inner + 1));
words.set(inner + 1, temp);
}
}
}
return words;
}
public ArrayList<String> getList()
{
return words;
}
}
//WriteFile
import java.io.*;
import java.util.*;
import java.lang.*;
public class WriteFile extends View
{
private Formatter x;
private String fileName = new String();
public WriteFile(String address)
{
fileName = address;
}
public void openFile()
{
try
{
x = new Formatter(fileName);
}
catch(Exception e)
{
field.setText("Could not write to file.");
}
}
public void writeFile(ArrayList<String> myWords)
{
for(int index = 0; index < myWords.size(); index++)
x.format("%s", myWords.get(index), "\n");//%s means string - in this case ONE string
}
public void closeFile()
{
x.close();
}
}

You have several issues in your code. You are extending View to obtain a reference of the textField 'field'! and that is not the way to go. You should use Exception handling to do this simple task. Also you cannot read and write from/to a file simultaneously! so you need to separate the duties here. When you finish reading the file, close it and open it again with whatever writer you want to use. Final note: You could use FileChooser to obtain the path, and that saves you the need of checking valid inputs! If you want to do the hard way and force the user to input the path manually, you have to add the escape character '/', In your case, a valid path would be C:\\Users\\Corecase\\Desktop\\test.txt
Change the following code in 'View.java'
if (e.getSource() == sortButton)
{
ReadFile r;
try
{
r = new ReadFile(field.getText());
r.readAndSortFile();
WriteFile w = new WriteFile(field.getText());
w.writeFile(r.getList());
}
catch (FileNotFoundException e1)
{
field.setText(e1.getMessage());
e1.printStackTrace();
}
}
and change 'WriteFile.java' to
public class WriteFile
{
private Formatter x;
private String fileName;
public WriteFile(String address) throws FileNotFoundException
{
fileName = address;
try
{
x = new Formatter(fileName);
} catch (Exception e)
{
throw new FileNotFoundException("Could not write to file.");
}
}
public void writeFile(ArrayList<String> myWords)
{
for (int index = 0; index < myWords.size(); index++)
x.format("%s%s", myWords.get(index), System.lineSeparator());
// now you are done writing so close the file.
x.close();
}
}
Change 'ReadFile.java' to
public class ReadFile
{
private ArrayList<String> words;
private String fileName;
private Scanner x;
public ReadFile(String path) throws FileNotFoundException
{
fileName = path;
words = new ArrayList<String>();
try
{
x = new Scanner(new File(fileName));
} catch (Exception e)
{
throw new FileNotFoundException("File Doesn't exist in the place you specified.");
}
}
public void readAndSortFile()
{
while (x.hasNext())
words.add(x.next());
Collections.sort(words);
// Now you are done reading and sorting, so close the file.
x.close();
}
.
.
.
}

There are couple of issue in your code:
ReadFile and WriteFile are extending View, as per the constructor you will have multiple instances of JFrame opened, as you are making the frame visible in your constructor frame.setVisible(true);, ReadFile and WriteFile just need a reference of the JTextField which should be updated, just pass this as a parameter.
Your sort will definitely throw IndexOutOfBoundsException for this line
if ((words.get(inner)).compareTo(words.get(inner + 1)) > 0) {
This line won't work when it reaches the last index, why not use simple Collections.sort(words);
You don't have a check if user has entered path or not, if nothing is entered, you will get NullPointerException in your ReadFile, ideally if file is not found i.e. your scanner is null, don't proceed further. Currently you are showing an error message but your code doesn't stop there, it still tries to read and sort file which is wrong.

i have tried your example and debugged it. I used c:\\dir\\file.text as parameter to the GUI and the file is read properly, so it is not there your problem. The exception i was getting was coming from this code:
for (int index = 0; index < words.size(); index++) {
for (int inner = 0; inner < words.size(); inner++) {
if ((words.get(inner)).compareTo(words.get(inner + 1)) > 0) {
temp = words.get(inner);
words.set(inner, words.get(inner + 1));
words.set(inner + 1, temp);
}
}
}

FileNotFoundException means that the file was not found.
NullPointerException means you tried to dereference a null pointer.
If that doesn't answer your question I don't know what does. StackOverflow is not a debugging service.