We Keep Coding

Reading all files in a package and copying the contents to a List or a map

I am currently working on a machine learning project. I have a package/directory of java files and i want to read their contents. Later, i will apply other methods to achieve results.
The problem is that the given code reads the txt files, however, when i pass the directory containing java files it doesn't work properly. Following is what I did
I read the names of all files in a directory.
As every directory has different number of files and different structure of files and folders inside it. I am looking for a generic solution.
Next, I read the contents of every file and put it in a list or MAP or whatever
The given code is as follows. I have written 3 methods.
This method list all files in a directory and make a set
// it will list all files in a directory.
public Collection<File> listFileTree(File dir) {
Set<File> fileTree = new HashSet<File>();
for (File entry : dir.listFiles()) {
if (entry.isFile())
fileTree.add(entry);
else
fileTree.addAll(listFileTree(entry));
}
return fileTree;
}
Here using the above method i have tried to read the contents of each file.
File file = new File("C:\\txt_sentoken");// c\\japa..if i use it code only show directory files
Iterator<File> i = Util.listFileTree(file).iterator();
String temp = null;
while(i.hasNext()){
temp = Util.readFile(i.next().getAbsolutePath().toString());
System.out.println(temp);
}
}
This is the readFile method
// using scanner class for reading file contents
public String readFile(String pathname) throws IOException {
File file = new File(pathname);
StringBuilder fileContents = new StringBuilder((int)file.length());
Scanner scanner = new Scanner(file);
String lineSeparator = System.getProperty("line.separator");
try {
while(scanner.hasNextLine()) {
fileContents.append(scanner.nextLine() + lineSeparator);
}
return fileContents.toString();
} finally {
scanner.close();
}
}
If i pass a directory (in File file = new File("C:\\txt_sentoken");) containing txt files this code works but for java or c++ or other code directories or packages it doesn't.
Can anyone guide me in refining this code? Also if there is any API or generic solution available please share.

Use Java NIO.2 to achieve your goal.
If you need any filtering you can put checks in the FileVisitor.
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.FileVisitResult;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.SimpleFileVisitor;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.TreeMap;
public class Test {
public static void main(String... args) {
try {
System.out.println(readAllFiles("")); // <----- Fill in path
} catch (IOException e) {
e.printStackTrace();
}
}
public static Map<Path, List<String>> readAllFiles(String path) throws IOException {
final Map<Path, List<String>> readFiles = new TreeMap<>();
Files.walkFileTree(Paths.get(path), new SimpleFileVisitor<Path>() {
#Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
Objects.requireNonNull(file);
readFiles.put(file, Files.readAllLines(file, StandardCharsets.UTF_8));
return FileVisitResult.CONTINUE;
}
});
return readFiles;
}
}
A Java 8 - also sorted - solution would be:
public static Map<Path, List<String>> readAllFiles(String path) throws IOException {
return Files.walk(Paths.get(path)).filter(p -> !Files.isDirectory(p)).collect(Collectors.toMap(k -> k, k -> {
try {
return Files.readAllLines(k);
} catch (IOException e) {
throw new RuntimeException(e);
}
} , (u, v) -> {
throw new IllegalStateException(String.format("Duplicate key %s", u));
} , TreeMap::new));
}

Alternative to FileUtils.listFilesAndDirs that does not list children as separate files

I want to pack a directory of other directories into a zip file. These other directories contain a text file and a PNG file each. However, FileUtils.listFilesAndDirs() lists these directories AND their children, and upon packing them into a zip, it packs the directories and their children, but also it packs a copy of the children into the root of the zip.
The zip looks like this:
Class1
-Image1
-Text1
Class2
-Image2
-Text2
Class3
-Image3
-Text3
Image1
Text1
Image2
Text2
Image3
Text3
It should not have the files after the empty line. Are there any alternative functions that could list just the directories and their children, and not list the children separately?

Sounds to me like some misunderstanding when working with file/directory filters and ZipOutputStream. How about something like this?
import java.io.*;
import java.util.Collection;
import java.util.zip.*;
import javax.swing.JPanel;
import org.apache.commons.io.*;
import org.apache.commons.io.filefilter.*;
public class Demo extends JPanel {
private static final long serialVersionUID = 1L;
public static void main(String s[]) throws Exception {
Demo demo = new Demo();
demo.launch();
}
private void launch() throws Exception {
Collection<File> fs = FileUtils.listFilesAndDirs(new File("/tmp/fakerepo"),
FileFileFilter.FILE, DirectoryFileFilter.DIRECTORY);
ZipOutputStream zos = null;
try {
zos = new ZipOutputStream(new FileOutputStream("/tmp/test.zip"));
for (File f : fs) {
System.out.println(f.getAbsolutePath());
if (!f.isFile())
continue;
ZipEntry ze = new ZipEntry(f.getAbsolutePath());
zos.putNextEntry(ze);
InputStream is = null;
try {
is = new FileInputStream(f);
IOUtils.copy(is, zos);
} finally {
IOUtils.closeQuietly(is);
}
}
zos.flush();
} finally {
IOUtils.closeQuietly(zos);
}
}
}

Trying to copy files in specified path with specified extension and replace them with new extension

I have most of it down but when I try to make the copy, no copy is made.
It finds the files in the specified directory like it is supposed to do and I think the copy function executes but there aren't any more files in the specified directory. Any help is appreciated. I made a printf function that isn't shown here. Thanks!
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.Scanner;
import org.apache.commons.io.FilenameUtils;
import org.apache.commons.io.FileUtils;
import static java.nio.file.StandardCopyOption.*;
public class Stuff {
static String path, oldExtn, newExtn;
static Boolean delOrig = false;
private static void getPathStuff() {
printf("Please enter the desired path\n");
Scanner in = new Scanner(System.in);
path = in.next();
printf("Now enter the file extension to replace\n");
oldExtn = in.next();
printf("Now enter the file extension to replace with\n");
newExtn = in.next();
in.close();
}
public static void main(String[] args) {
getPathStuff();
File folder = new File(path);
printf("folder = %s\n", folder.getPath());
for (final File fileEntry : folder.listFiles()) {
if (fileEntry.getName().endsWith(oldExtn)) {
printf(fileEntry.getName() + "\n");
File newFile = new File(FilenameUtils.getBaseName(fileEntry
.getName() + newExtn));
try {
printf("fileEntry = %s\n", fileEntry.toPath().toString());
Files.copy(fileEntry.toPath(), newFile.toPath(),
REPLACE_EXISTING);
} catch (IOException e) {
System.err.printf("Exception");
}
}
}
}
}`

The problem is that the new file is created without a full path (only the file name). So your new file is created - only not where you expect...
You can see that it'll work if you'll replace:
File newFile = new File(FilenameUtils.getBaseName(fileEntry
.getName() + newExtn));
with:
File newFile = new File(fileEntry.getAbsolutePath()
.substring(0,
fileEntry.getAbsolutePath()
.lastIndexOf(".")+1) + newExtn);

Read from file inside Eclipse Project

I want to make an application that splits a big text file inside inputfolder into several small XML files to be put inside outputfolder.
This is project outline:
The following code works fine when it comes to getting a file from an outside folder, but when I modified it to read from a folder inside the project, it gave me this error:
Exception in thread "main" java.lang.NullPointerException
at com.zakaria.cut.XmlCutter.cut(XmlCutter.java:45)
at com.zakaria.cut.Main.main(Main.java:8)
[XmlCutter.java]
package com.zakaria.cut;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.text.MessageFormat;
import java.util.logging.ConsoleHandler;
import java.util.logging.Handler;
import java.util.logging.Level;
import java.util.logging.Logger;
public class XmlCutter {
private static final String OUTPUT_FILE_NAME = "/file";
//private static String USER_HOME = System.getProperty("user.home");
private static final String INPUT_FOLDER = "../inputfolder";
private static String OUTPUT_FOLDER = "../outputfolder";
private static Logger LOG = Logger.getLogger("XmlCutter");
private static long COUNTER = 0;
public XmlCutter() {
super();
// TODO Auto-generated constructor stub
}
public void cut() {
Handler h = new ConsoleHandler();
h.setLevel(Level.FINE);
LOG.addHandler(h);
LOG.setLevel(Level.FINE);
File inputDir = new File(INPUT_FOLDER);
File[] filesInInputDir = inputDir.listFiles();
for (File f : filesInInputDir) {
if ((f.getName()).endsWith(".txt")) {
LOG.fine((MessageFormat.format(
"Found a text file {0}. Processing docs...",
f.getName())));
processFile(f);
}
}
}
private static void processFile(File f) {
StringBuilder out = new StringBuilder();
char prev = '#';
try {
BufferedReader br = new BufferedReader(new InputStreamReader(
new FileInputStream(f), "UTF8"));
char[] buf = new char[1];
while (br.read(buf) >= 0) {
out.append(buf[0]);
if (prev == '<' && buf[0] == '?') {
LOG.finest((MessageFormat.format(
"Start of XML PI Found: {0}{1}", prev, buf[0])));
if (out.length() > 2) {
flushToFile(out.substring(0, out.length() - 2));
}
out.setLength(2);
}
prev = buf[0];
}
LOG.finest("Writing final file");
flushToFile(out.toString());
br.close();
} catch (IOException e) {
LOG.fine(e.getMessage());
}
LOG.fine(MessageFormat.format("Generated {0} XML Documents", COUNTER));
}
private static void flushToFile(String s) {
File f = new File(OUTPUT_FOLDER + OUTPUT_FILE_NAME + (++COUNTER)
+ ".xml");
LOG.finest(MessageFormat.format("Writing file: {0}", f.getName()));
try {
FileOutputStream fos = new FileOutputStream(f);
OutputStreamWriter osw = new OutputStreamWriter(fos, "UTF8");
osw.write(s);
osw.flush();
} catch (IOException e) {
LOG.fine(e.getMessage());
}
}
}
[Main.java]
package com.zakaria.cut;
public class Main {
public static void main(String[] args) {
XmlCutter cutter = new XmlCutter();
cutter.cut();
}
}
The problem, I guess, is definitely here:
private static final String INPUT_FOLDER = "../inputfolder";
private static String OUTPUT_FOLDER = "../outputfolder";
How can I fix it?

Do you know what folder the program is executing from? My guess is the relative links are pointing to the wrong spot? Have you tried hard coding the paths and see if they work? If they do you might have to look at the your execution folder and then change the relative paths accordingly?

ArrayList Menu Java

I need to make a menu that list the .txt files in a directory. For example, if i have jonsmith12.txt , lenovo123.txt , dell123.txt in the directory how would I make an arraylist menu of:
Please choose one of the following:
jonsmith12
lenovo123
dell123
Please enter your choice:
I need an arraylist menu is because I don't know how many .txt files are in the directory at any given time.
import java.io.File;
public class ListFiles
{
public static void listRecord() {
// Directory path here
String path = ".";
String files;
File folder = new File(path);
File[] listOfFiles = folder.listFiles();
for (int i = 0; i < listOfFiles.length; i++)
{
if (listOfFiles[i].isFile())
{
files = listOfFiles[i].getName();
if (files.endsWith(".txt") || files.endsWith(".TXT"))
{
System.out.println(files);
}
}
}
}
}
Here is the class that will display the information in the .txt file onto the console. It stills need some modifying too but I could probably figure that out.
import java.io.BufferedInputStream;
import java.io.DataInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
/**
* This program reads a text file line by line and print to the console. It uses
* FileOutputStream to read the file.
*
*/
public class DisplayRec {
public static void displayRecord() throws IOException {
File file = new File("williamguo5.txt");
FileInputStream fis = null;
BufferedInputStream bis = null;
DataInputStream dis = null;
try {
fis = new FileInputStream(file);
// Here BufferedInputStream is added for fast reading.
bis = new BufferedInputStream(fis);
dis = new DataInputStream(bis);
// dis.available() returns 0 if the file does not have more lines.
while (dis.available() != 0) {
// this statement reads the line from the file and print it to
// the console.
System.out.println(dis.readLine());
}
// dispose all the resources after using them.
fis.close();
bis.close();
dis.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
So the question is: How do I implement a ArrayList menu into my ListFiles class so it will display the .txt files.

You can use the alternate method signature for File.listFiles(FilenameFilter filter) to simplify your code:
File[] files = dir.listFiles(new FilenameFilter() {
#Override
public boolean accept(File dir, String name) {
return name.toLowerCase().endsWith(".txt");
}
});
And unless you really enjoy writing loops, you don't even need to manually loop over the array to convert it to a List:
List<File> lstRecords = Arrays.asList(files);
Your displayRecord method was pretty close; you just needed to pass the file as an argument and use that instead of a hard-coded filename, and you needed to initialize dis.
Putting it all together:
package com.example.file;
import java.io.BufferedInputStream;
import java.io.DataInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FilenameFilter;
import java.io.IOException;
import java.util.Arrays;
import java.util.List;
public class FileExample {
public static List<File> listRecords(File dir) {
File[] files = dir.listFiles(new FilenameFilter() {
#Override
public boolean accept(File dir, String name) {
return name.toLowerCase().endsWith(".txt");
}
});
return Arrays.asList(files);
}
public static void displayRecord(File file) {
FileInputStream fis = null;
BufferedInputStream bis = null;
DataInputStream dis = null;
try {
fis = new FileInputStream(file);
// Here BufferedInputStream is added for fast reading.
bis = new BufferedInputStream(fis);
dis = new DataInputStream(bis);
String line = dis.readLine();
while (line != null) {
// this statement reads the line from the file and print it to
// the console.
System.out.println(line);
line = dis.readLine();
}
// dispose all the resources after using them.
fis.close();
bis.close();
dis.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
/**
* #param args
*/
public static void main(String[] args) {
List<File> lstRecords = listRecords(new File("."));
for (File record : lstRecords) {
displayRecord(record);
}
}
}
It's also better to use Reader/Writer instead of InputStream/OutputStream if you're working with text files, and you should close your files in the finally block to avoid a potential resource leak.
You'll also notice I didn't explicitly use an ArrayList. In most cases, it's better to program against the interface (in this case, List) as much as possible, and only declare variables using the implementing class when you need to use a method that's only available to that class.

It looks like your sticking point above is the array. If you just need to iterate over the files in a directory, something as simple as the following will do the trick.
import java.io.File;
public class TxtEnumerator {
public static void main(String[] args) {
TxtEnumerator te = new TxtEnumerator();
te.listFiles();
}
public void listFiles() {
String filepath = "." + File.separator + "textDirectory";
File file = new File(filepath);
if (file.isDirectory()) {
for (File f : file.listFiles()) {
if (f.getName().endsWith(".txt")) {
System.out.println(f.getName());
}
}
}
}
}