We Keep Coding

How to read a text file into an array list of objects in Java

I'm currently working on a project and I'm running into a couple of issues. This project involves working with 2 classes, Subject and TestSubject. Basically, I need my program (in TestSubject class) to read details (subject code and subject name) from a text file and create subject objects using this information, then add those to an array list. The text file looks like this:
ITC105: Communication and Information Management
ITC106: Programming Principles
ITC114: Introduction to Database Systems
ITC161: Computer Systems
ITC204: Human Computer Interaction
ITC205: Professional Programming Practice
the first part is the subject code i.e. ITC105 and the second part is the name (Communication and Information Management)
I have created the subject object with the code and name as strings with getters and setters to allow access (in the subject class):
private static String subjectCode;
private static String subjectName;
public Subject(String newSubjectCode, String newSubjectName) {
newSubjectCode = subjectCode;
newSubjectName = subjectName;
}
public String getSubjectCode() {
return subjectCode;
}
public String getSubjectName() {
return subjectName;
}
public void setSubjectCode(String newSubjectCode) {
subjectCode= newSubjectCode;
}
public void setSubjectName(String newSubjectName) {
subjectName = newSubjectName;
}
The code I have so far for reading the file and creating the array list is:
public class TestSubject {
#SuppressWarnings({ "null", "resource" })
public static void main(String[] args) throws IOException {
File subjectFile = new File ("A:\\Assessment 3 Task 1\\src\\subjects.txt");
Scanner scanFile = new Scanner(subjectFile);
System.out.println("The current subjects are as follows: ");
System.out.println(" ");
while (scanFile.hasNextLine()) {
System.out.println(scanFile.nextLine());
}
//This array will store the list of subject objects.
ArrayList <Object> subjectList = new ArrayList <>();
//Subjects split into code and name and added to a new subject object.
String [] token = new String[3];
while (scanFile.hasNextLine()) {
token = scanFile.nextLine().split(": ");
String code = token [0] + ": ";
String name = token [1];
Subject addSubjects = new Subject (code, name);
//Each subject is then added to the subject list array list.
subjectList.add(addSubjects);
}
//Check if the array list is being filled by printing it to the console.
System.out.println(subjectList.toString());
This code isn't working, the array list is just printing as blank. I have tried doing this several ways including a buffered reader but I can't get it to work so far. The next section of code allows a user to enter a subject code and name, which is then added to the array list as well. That section of code works perfectly, I'm just stuck on the above part. Any advice on how to fix it to make it work would be amazing.
Another small thing:
File subjectFile = new File ("A:\\Assessment 3 Task 1\\src\\subjects.txt"); //this file path
Scanner scanFile = new Scanner(subjectFile);
I'd like to know how I can change the file path so that it will still work if the folder is moved or the files are opened on another computer. The .txt file is in the source folder with the java files. I have tried:
File subjectFile = new File ("subjects.txt");
But that doesn't work and just throws errors.

That is because you have already read through the file
while (scanFile.hasNextLine()) {
System.out.println(scanFile.nextLine());
}
The contents are exhausted. So when you do
while (scanFile.hasNextLine()) {
token = scanFile.nextLine().split(": ");
there is no data left.
Remove the first loop or re-open the file.
Or as #UsagiMiyamoto mentions
Or read the line to a String variable, print it, then split it... All in one loop.

I assume you are just beginning with learning Java and hence the below code is probably way too advanced, but it may help others who are trying to do something similar to you and also give you a glimpse of what you will probably learn in future.
The below code uses the following (in no particular order):
Streams
Accessing resources
Records
try-with-resources
Multi-catch
Method references
NIO.2
More notes after the code.
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.stream.Collectors;
import java.util.stream.Stream;
public record Subject(String subjectCode, String subjectName) {
private static final String DELIMITER = ": ";
private static Path getPath(String filename) throws URISyntaxException {
URL url = Subject.class.getResource(filename);
URI uri = url.toURI(); // throws java.net.URISyntaxException
return Paths.get(uri);
}
private static Subject makeSubject(String line) {
String[] parts = line.split(DELIMITER);
return new Subject(parts[0].trim(), parts[1].trim());
}
/**
* Reads contents of a text file and converts its contents to a list of
* instances of this record and displays that list.
*
* #param args - not used.
*/
public static void main(String[] args) {
try {
Path path = getPath("subjects.txt");
try (Stream<String> lines = Files.lines(path)) { // throws java.io.IOException
lines.map(Subject::makeSubject)
.collect(Collectors.toList())
.forEach(System.out::println);
}
}
catch (IOException | URISyntaxException x) {
x.printStackTrace();
}
}
}
A Java record is applicable for an immutable object and it simply saves you from writing code for methods including getters as well as equals, hashCode and toString. (There are no setters since a record is immutable.) It's a bit like Project Lombok. I would say that a Subject is immutable since I don't think the code or name would need to be changed and that's why I thought making Subject a record was applicable.
Running the above code produces the following output:
Subject[subjectCode=ITC105, subjectName=Communication and Information Management]
Subject[subjectCode=ITC106, subjectName=Programming Principles]
Subject[subjectCode=ITC114, subjectName=Introduction to Database Systems]
Subject[subjectCode=ITC161, subjectName=Computer Systems]
Subject[subjectCode=ITC204, subjectName=Human Computer Interaction]
Subject[subjectCode=ITC205, subjectName=Professional Programming Practice]
Regarding
I'd like to know how I can change the file path so that it will still work if the folder is moved
I placed file subjects.txt in the same folder as file Subject.class, which allowed me to use method getResource. Refer to the Accessing resources link, above. Note that this can't be used if
the files are opened on another computer
Alternatively, there are several directories whose paths are stored in System properties including
java.home
java.io.tmpdir
user.home
user.dir

what did your debug console said about the exception?
your code works very well in my editor.
code result
and you should code like below if you want to read file through relative path
before ->
new File ("A:\Assessment 3 Task 1\src\subjects.txt");
after ->
new File (".\\subjects.txt");

Reading data from Excel using Java

I have 2 columns in an excel sheet, A and B, and I need to divide column A (randomly) into 2 and calculate the average of the first and second halves of column B separately. Then I need to divide column A again but each time moving with one cell downwards and calculate the averages etc. Can this be done using Java only? Since I am finding you have to use software like Apache and I have never used them.
Thanks

Yes, you can save the Excel data as a text file and use classes like java.io.File, java.util.Scanner to read the text file.

A simple approach would be to save the Excel file as a comma-separated values (.CSV) file.
Since you have never used a framework before, I will show you how to use the OpenCSV framework.
First you need to add this framework to your Build Path. Drag the downloaded .jar files to your workspace in Eclipse. Open the project's properties, Select Java Build Path, Select Libraries tab, and then Add a Library from Workspace and select that jar file.
To use it, your class will look something like this,
import java.io.FileReader;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import au.com.bytecode.opencsv.CSVReader;
public class CSV {
private List<String[]> fileContent;
public CSV(String filePath) {
try {
readCSV(filePath);
} catch (IOException e) {
System.out.print("There was an error reading this CSV file. Error follows:");
e.printStackTrace();
}
}
public void readCSV(String csvPath) throws IOException {
// Store coupon objects in array list
CSVReader csvReader = new CSVReader(new FileReader(csvPath));
// Store each row into a String list
List<String[]> csvData = csvReader.readAll();
csvReader.close();
fileContent = csvData;
}
public void printFileContent() {
System.out.println("");
for (String[] row : fileContent.subList(1, fileContent.size())) {
for (String s: row) {
System.out.print(s + " ");
}
System.out.println();
}
System.out.println("");
}
}
Inside the main method, you can use a simple
CSV csvFile = new CSV("csv/Path/Here.csv");
// Verify Results
csvFile.printFileContent();
As always, please use my code as a reference. It is always better to learn how to write code yourself, especially when this question pertains to a homework assignment.

File I/O bottleneck found via VisualVM

I've found a bottleneck in my app that keeps growing as data in my files grow (see attached screenshot of VisualVM below).
Below is the getFileContentsAsList code. How can this be made better performance-wise? I've read several posts on efficient File I/O and some have suggested Scanner as a way to efficiently read from a file. I've also tried Apache Commons readFileToString but that's not running fast as well.
The data file that's causing the app to run slower is 8 KB...that doesn't seem too big to me.
I could convert to an embedded database like Apache Derby if that seems like a better route. Ultimately looking for what will help the application run faster (It's a Java 1.7 Swing app BTW).
Here's the code for getFileContentsAsList:
public static List<String> getFileContentsAsList(String filePath) throws IOException {
if (ReceiptPrinterStringUtils.isNullOrEmpty(filePath)) throw new IllegalArgumentException("File path must not be null or empty");
Scanner s = null;
List<String> records = new ArrayList<String>();
try {
s = new Scanner(new BufferedReader(new FileReader(filePath)));
s.useDelimiter(FileDelimiters.RECORD);
while (s.hasNext()) {
records.add(s.next());
}
} finally {
if (s != null) {
s.close();
}
}
return records;
}

The size of an ArrayList is multiplied by 1.5 when necessary. This is O(log(N)). (Doubling was used in Vector.) I would certainly use an O(1) LinkedList here, and BufferedReader.readLine() rather than a Scanner if I was trying to speed it up. It's hard to believe that the time to read one 8k file is seriously a concern. You can read millions of lines in a second.

So, file.io gets to be REAL expensive if you do it a lot...as seen in my screen shot and original code, getFileContentsAsList, which contains file.io calls, gets invoked quite a bit (18.425 times). VisualVM is a real gem of a tool to point out bottlenecks like these!
After contemplating over various ways to improve performance, it dawned on me that possibly the best way is to do file.io calls as little as possible. So, I decided to use private static variables to hold the file contents and to only do file.io in the static initializer and when a file is written to. As my application is (fortunately) not doing excessive writing (but excessive reading), this makes for a much better performing application.
Here's the source for the entire class that contains the getFileContentsAsList method. I took a snapshot of that method and it now runs in 57.2 ms (down from 3116 ms). Also, it was my longest running method and is now my 4th longest running method. The top 5 longest running methods run for a total of 498.8 ms now as opposed to the ones in the original screenshot that ran for a total of 3812.9 ms. That's a percentage decrease of about 85%
[100 * (498.8 - 3812.9) / 3812.9].
package com.mbc.receiptprinter.util;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.logging.Level;
import org.apache.commons.io.FileUtils;
import com.mbc.receiptprinter.constant.FileDelimiters;
import com.mbc.receiptprinter.constant.FilePaths;
/*
* Various File utility functions. This class uses the Apache Commons FileUtils class.
*/
public class ReceiptPrinterFileUtils {
private static Map<String, String> fileContents = new HashMap<String, String>();
private static Map<String, Boolean> fileHasBeenUpdated = new HashMap<String, Boolean>();
static {
for (FilePaths fp : FilePaths.values()) {
File f = new File(fp.getPath());
try {
FileUtils.touch(f);
fileHasBeenUpdated.put(fp.getPath(), false);
fileContents.put(fp.getPath(), FileUtils.readFileToString(f));
} catch (IOException e) {
ReceiptPrinterLogger.logMessage(ReceiptPrinterFileUtils.class,
Level.SEVERE,
"IOException while performing FileUtils.touch in static block of ReceiptPrinterFileUtils", e);
}
}
}
public static String getFileContents(String filePath) throws IOException {
if (ReceiptPrinterStringUtils.isNullOrEmpty(filePath)) throw new IllegalArgumentException("File path must not be null or empty");
File f = new File(filePath);
if (fileHasBeenUpdated.get(filePath)) {
fileContents.put(filePath, FileUtils.readFileToString(f));
fileHasBeenUpdated.put(filePath, false);
}
return fileContents.get(filePath);
}
public static List<String> convertFileContentsToList(String fileContents) {
List<String> records = new ArrayList<String>();
if (fileContents.contains(FileDelimiters.RECORD)) {
records = Arrays.asList(fileContents.split(FileDelimiters.RECORD));
}
return records;
}
public static void writeStringToFile(String filePath, String data) throws IOException {
fileHasBeenUpdated.put(filePath, true);
FileUtils.writeStringToFile(new File(filePath), data);
}
public static void writeStringToFile(String filePath, String data, boolean append) throws IOException {
fileHasBeenUpdated.put(filePath, true);
FileUtils.writeStringToFile(new File(filePath), data, append);
}
}

ArrayLists have a good performance at reading and also on writing IF the lenth does not change very often. In your application the length changes very often (size is doubled, when it is full and an element is added) and your application needs to copy your array into an new, longer array.
You could use a LinkedList, where new elements are appended and no copy actions are needed.
List<String> records = new LinkedList<String>();
Or you could initialize the ArrayList with the approximated finished Number of Words. This will reduce the number of copy actions.
List<String> records = new ArrayList<String>(2000);

Make PrintWriter or OutputStreamWriter print immediately

Is there a way to make PrintWriter or OutputStreamWriter print immediately after .write method is invoked? I have autoFlush turned on for PrintWriter. Yet for both these classes, the contents get printed only when the writer is closed.
For what it's worth, I am using Writers because I need to abstract over console output, file output, and string output.
Thanks!
Edit:
An SSCCE that shows the problem:
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.io.PrintWriter;
import java.util.Scanner;
public class Main {
public static void main(String[] args) {
PrintWriter p = new PrintWriter(System.out, true);
Scanner read = new Scanner(System.in);
while (read.hasNextLine()) {
String input = read.nextLine();
if (input.equals("end"))
break;
p.write(input);
}
p.close();
}
}
/* Sample run: (first four lines are input)
cow
mow
pow
end
cowmowpow
*/

The documentation for the autoFlush constructor parameter says:
A boolean; if true, the println, printf, or format methods will flush the output buffer
Given that you're not using any of those methods, it's not entirely surprising that it's not helping.
The simplest approach would be to just call flush() manually after every write - that's what you're trying to achieve, after all. I don't know of anything which will make a writer flush by default after every write.
Of course, you could write your own wrapper class - the equivalent of BufferedWriter, but with the opposite effect. It could delegate all methods to the wrapped writer, but then immediately call flush() afterwards.

which of the two is a better way of creating and destroying objects?

i have a question on lines 26 & 27:
String dumb = input.nextLine();
output.println(dumb.replaceAll(REMOVE, ADD));
i was hoping that i'd be able to shrink this down to a single line and be able to save space, so i did:
output.println(new String(input.nextLine()).replaceAll(REMOVE, ADD));
but now i'm wondering about performance. i understand that this program is quiet basic and doesn't need optimization, but i'd like to learn this.
the way i look at it, in the first scenario i'm creating a string object dumb, but once i leave the loop the object is abandoned and the JVM should clean it up, right? but does the JVM clean up the abandoned object faster than the program goes through the loop? or will there be several string objects waiting for garbage collection once the program is done?
and is my logic correct that in the second scenario the String object is created on the fly and destroyed once the program has passed through that line? and is this in fact a performance gain?
i'd appreciate it if you could clear this up for me.
thank you,
p.s. in case you are wondering about the program (i assumed it was straight forward) it takes in an input file, and output file, and two words, the program takes the input file, replaces the first word with the second and writes it into the second file. if you've actually read this far and would like to suggest ways i could make my code better, PLEASE DO SO. i'd be very grateful.
import java.io.File;
import java.util.Scanner;
import java.io.PrintWriter;
public class RW {
public static void main(String[] args) throws Exception{
String INPUT_FILE = args[0];
String OUTPUT_FILE = args[1];
String REMOVE = args[2];
String ADD = args[3];
File ifile = new File(INPUT_FILE);
File ofile = new File(OUTPUT_FILE);
if (ifile.exists() == false) {
System.out.println("the input file does not exists in the current folder");
System.out.println("please provide the input file");
System.exit(0);
}
Scanner input = new Scanner(ifile);
PrintWriter output = new PrintWriter(ofile);
while(input.hasNextLine()) {
String dumb = input.nextLine();
output.println(dumb.replaceAll(REMOVE, ADD));
}
input.close();
output.close();
}
}

The very, very first thing I'm going to say is this:
Don't worry about optimizing performance prematurely. The Java compiler is smart, it'll optimize a lot of this stuff for you, and even if it didn't you're optimizing out incredibly tiny amounts of time. The stream IO you've got going there is already running for orders of magnitude longer than the amount of time you're talking about.
What is most important is how easy the code is to understand. You've got a nice code style, going from your example, so keep that up. Which of the two code snippets is easier for someone other than you to read? That is the best option. :)
That said, here are some more specific answers to your questions:
Garbage collection will absolutely pick up objects which are instantiated inside the scope of a loop. The fact that it's instantiated inside the loop means that Java will already have marked it for clean up as soon as it fell out of scope. The next time GC runs, it will clean up all of those things which have been marked for clean up.
Creating an object inline will still create an object. The constructor is still called, memory is still allocated... Under the hood, they are really, really similar. It's just that in one case that object has a name, and in the other it doesn't. You're not going to save any real resources by combining two lines of code into one.
"input.nextLine()" already returns a String, so you don't need to wrap it in a new String(). (So yes, removing that actually will result in one less object being instantiated!)

Local Objects are eligible for GC once they go out of scope. That does not mean that GC cleans them that very moment. The eligible objects undergone a lifecycle. GC may or may not collect them immediately.
As far your program is concerned, there is nothing much to optimize except a line or two. Below is a restructured program.
import java.io.File;
import java.util.Scanner;
import java.io.PrintWriter;
public class Test {
public static void main(String[] args) throws Exception {
String INPUT_FILE = args[0];
String OUTPUT_FILE = args[1];
String REMOVE = args[2];
String ADD = args[3];
File ifile = new File(INPUT_FILE);
File ofile = new File(OUTPUT_FILE);
if (ifile.exists() == false) {
System.out.println("the input file does not exists in the current folder\nplease provide the input file");
System.exit(0);
}
Scanner input = null;
PrintWriter output = null;
try {
input = new Scanner(ifile);
output = new PrintWriter(ofile);
while (input.hasNextLine()) {
output.println(input.nextLine().replaceAll(REMOVE, ADD));
}
} finally {
if (input != null)
input.close();
if(output != null)
output.close();
}
}
}

If you arew concerned about obejct creation and performance, use a profiler to mesure your code. And keep in mind that doing new String(input.nextLine()) is totally pointless since input.nextLine() returns an immutable instance of String. So just do output.println(input.nextLine().replaceAll(REMOVE, ADD));.