Reading from csv files - java

This is a project i'm working on at college, everything seems good except in the game class which initializes the game. Here is a snippet
public class Game{
private Player player;
private World world;
private ArrayList<NonPlayableFighter> weakFoes;
private ArrayList<NonPlayableFighter> strongFoes;
private ArrayList<Attack> attacks;
private ArrayList<Dragon> dragons;
public Game() throws IOException{
player = new Player("");
world = new World();
weakFoes = new ArrayList<NonPlayableFighter>();
strongFoes = new ArrayList<NonPlayableFighter>();
attacks = new ArrayList<Attack>();
dragons = new ArrayList<Dragon>();
loadAttacks ("Database-Attacks_20309.csv");
loadFoes ("Database-Foes_20311.csv");
loadDragons ("Database-Dragons_20310.csv");
}
after that follows some getters and the 4 method i am supposed to implement.
These methods are loadCSV(String filePath),loadAttacks(String filePath),loadFoes(String filePath),loadDragons(String filePath)
I have created loadCSV(String filePath) such that it returns an ArrayList of String[] here:
private ArrayList<String[]> loadCSV(String filePath) throws IOException{
String currentLine = "";
ArrayList<String[]> result = new ArrayList<String[]>();
FileReader fileReader = new FileReader(filePath);
BufferedReader br = new BufferedReader(fileReader);
currentLine = br.readLine();
while (currentLine != null){
String[] split = currentLine.split(",");
result.add(split);
}
br.close();
return result;
}
Then i would like to load some attacks, foes, and dragons and inserting them in the appropriate ArrayList.
I applied loadAttacks(String filePath) here:
private void loadAttacks(String filePath) throws IOException{
ArrayList<String[]> allAttacks = loadCSV(filePath);
for(int i = 0; i < allAttacks.size(); i++){
String[] current = allAttacks.get(i);
Attack temp = null;
switch(current[0]){
case "SA": temp = new SuperAttack(current[1],
Integer.parseInt(current[2]));
break;
case "UA": temp = new UltimateAttack(current[1],
Integer.parseInt(current[2]));
break;
case "MC": temp = new MaximumCharge();
break;
case "SS": temp = new SuperSaiyan();
break;
}
attacks.add(temp);
}
}
I wrote it such that it takes the ArrayList returned from loadCSV(String filePath) and searches in each String[] within the ArrayList on the first String using a switch thus creating the appropriate attack and adding it to attacks.
Then i would like to read another CSV for the Foes and the CSV file is structured such that in the first line there are some attributes the second line some attacks of type SuperAttack and the third line holds some attacks of type Ultimate attack. Also within each foe there is a boolean attribute that determines if it is a Strong or Weak Foe thus putting it in the right Arraylist. Here is the code for loadFoes(String filePath):
private void loadFoes(String filePath) throws IOException{
ArrayList<String[]> allFoes = loadCSV(filePath);
for(int i = 0; i < allFoes.size(); i += 3){
String[] current = allFoes.get(i);
String[] supers = allFoes.get(i+1);
String[] ultimates = allFoes.get(i+2);
ArrayList<SuperAttack> superAttacks = new ArrayList<SuperAttack>();
ArrayList<UltimateAttack> ultimateAttacks = new ArrayList<UltimateAttack>();
NonPlayableFighter temp = null;
for(int j = 0; i < supers.length; j++){
int index = attacks.indexOf(supers[j]);
if(index != -1){
superAttacks.add((SuperAttack)attacks.get(index));
}
else break;
}
for(int j = 0; i < ultimates.length; j++){
int index = attacks.indexOf(ultimates[j]);
if(index != -1){
ultimateAttacks.add((UltimateAttack)attacks.get(index));
}
else break;
}
if(current[7].equalsIgnoreCase("True")){
temp = new NonPlayableFighter(current[0], Integer.parseInt(current[1]),
Integer.parseInt(current[2]), Integer.parseInt(current[3]),
Integer.parseInt(current[4]), Integer.parseInt(current[5]),
Integer.parseInt(current[6]), true, superAttacks, ultimateAttacks);
strongFoes.add(temp);
}
else{
temp = new NonPlayableFighter(current[0], Integer.parseInt(current[1]),
Integer.parseInt(current[2]), Integer.parseInt(current[3]),
Integer.parseInt(current[4]), Integer.parseInt(current[5]),
Integer.parseInt(current[6]), false, superAttacks, ultimateAttacks);
weakFoes.add(temp);
}
}
}
First i get the first three String[] in the ArrayList returned from loadCSV(String filePath and made 2 loops to check if the attacks are within the previously loaded attacks CSV then i check for the attribute that determines if it is a strong or weak and accordingly creating a new NonPlayableFighter and adding it to the appropriate list.
Running the jUnit4 tests for this assignment it gives me a Compilation Error: Unhandled exception type IOException. And generally speaking does the code have any notable problems ?

It's better to reuse already exist CSV file readers for Java (e.g. CVSReader) if isn't a part of you task.

That makes a lot of code. I'll answer to your Compilation Error.
While reading a file you have to pu your code in a try catch in order to avoid this kind of error. In your loadCSV method you have to set up a try catch block.
Please refer to this site for complete tutorial.
try (BufferedReader br = new BufferedReader(new FileReader("C:\\testing.txt")))
{
String sCurrentLine;
while ((sCurrentLine = br.readLine()) != null) {
String[] split = currentLine.split(",");
result.add(split);
}
} catch (IOException e) {
e.printStackTrace();
}
To make it short, code that access to files have to be in a try catch to avoid IO Exception, or be in a method that throws the exception (but then it has to be catched elsewhere).
In that code you have a good example of a try-with-resource, very good way to manage your ressource and memory.

loadCSV(String filePath) is a infinite loop isn't it? And as for the IOException it as #RPresle suggested a try/catch would do the trick around the BufferedReader.

Related

Read Tab-Separated-Columns into Lists - Java

Tab-Separated File:
2019-06-06 10:00:00 1.0
2019-06-06 11:00:00 2.0
I'd like to iterate over the file once and add the value of each column to a list.
My working approach would be:
import java.util.*;
import java.io.*;
public class Program {
public static void main(String[] args)
{
ArrayList<Double> List_1 = new ArrayList<Double>();
ArrayList<Double> List_2 = new ArrayList<Double>();
String[] values = null;
String fileName = "File.txt";
File file = new File(fileName);
try
{
Scanner inputStream = new Scanner(file);
while (inputStream.hasNextLine()){
try {
String data = inputStream.nextLine();
values = data.split("\\t");
if (values[1] != null && !values[1].isEmpty() == true) {
double val_1 = Double.parseDouble(values[1]);
List_1.add(val_1);
}
if (values[2] != null && !values[2].isEmpty() == true) {
double val_2 = Double.parseDouble(values[2]);
List_2.add(val_2);
}
}
catch (ArrayIndexOutOfBoundsException exception){
}
}
inputStream.close();
}
catch (FileNotFoundException e) {
e.printStackTrace();
}
System.out.println(List_1);
System.out.println(List_2);
}
}
I get:
[1.0]
[2.0]
It doesn't work without the checks for null, ìsEmpty and the ArrayIndexOutOfBoundsException.
I would appreciate any hints on how to save a few lines while keeping the scanner approach.
One option is to create a Map of Lists using column number as a key. This approach gives you "unlimited" number of columns and exactly the same output than one in the question.
public class Program {
public static void main(String[] args) throws Exception
{
Map<Integer, List<Double>> listMap = new TreeMap<Integer, List<Double>>();
String[] values = null;
String fileName = "File.csv";
File file = new File(fileName);
Scanner inputStream = new Scanner(file);
while (inputStream.hasNextLine()){
String data = inputStream.nextLine();
values = data.split("\\t");
for (int column = 1; column < values.length; column++) {
List<Double> list = listMap.get(column);
if (list == null) {
listMap.put(column, list = new ArrayList<Double>());
}
if (!values[column].isEmpty()) {
list.add(Double.parseDouble(values[column]));
}
}
}
inputStream.close();
for(List<Double> list : listMap.values()) {
System.out.println(list);
}
}
}
You can clean up your code some by using try-with resources to open and close the Scanner for you:
try (Scanner inputStream = new Scanner(file))
{
//your code...
}
This is useful because the inputStream will be closed automatically once the try block is left and you will not need to close it manually with inputStream.close();.
Additionally if you really want to "save lines" you can also combine these steps:
double val_2 = Double.parseDouble(values[2]);
List_2.add(val_2);
Into a single step each, since you do not actually use the val_2 anywhere else:
List_2.add(Double.parseDouble(values[2]));
Finally you are also using !values[1].isEmpty() == true which is comparing a boolean value to true. This is typically bad practice and you can reduce it to !values[1].isEmpty() instead which will have the same functionality. Try not to use == with booleans as there is no need.
you can do it like below:
BufferedReader bfr = Files.newBufferedReader(Paths.get("inputFileDir.tsv"));
String line = null;
List<List<String>> listOfLists = new ArrayList<>(100);
while((line = bfr.readLine()) != null) {
String[] cols = line.split("\\t");
List<String> outputList = new ArrayList<>(cols);
//at this line your expected list of cols of each line is ready to use.
listOfLists.add(outputList);
}
As a matter of fact, it is a simple code in java. But because it seems that you are a beginner in java and code like a python programmer, I decided to write a sample code to let you have a good start point. good luck

Reading, comparing and merging multiple files in Java

Given there are some files Customer-1.txt, Customer-2.txt and Customer-3.txt and these files have the following content:
Customer-1.txt
1|1|MARY|SMITH
2|1|PATRICIA|JOHNSON
4|2|BARBARA|JONES
Customer-2.txt
1|1|MARY|SMITH
2|1|PATRICIA|JOHNSON
3|1|LINDA|WILLIAMS
4|2|BARBARA|JONES
Customer-3.txt
2|1|PATRICIA|JOHNSON
3|1|LINDA|WILLIAMS
5|2|ALEXANDER|ANDERSON
These files have a lot of duplicate data, but it is possible that each file contains some data that is unique.
And given that the actual files are sorted, big (a few GB each file) and there are many files...
Then what is the:
a) memory cheapest
b) cpu cheapest
c) fastest
way in Java to create one file out of these three files that will contain all the unique data of each file sorted and concatenated like such:
Customer-final.txt
1|1|MARY|SMITH
2|1|PATRICIA|JOHNSON
3|1|LINDA|WILLIAMS
4|2|BARBARA|JONES
5|2|ALEXANDER|ANDERSON
I looked into the following solution https://github.com/upcrob/spring-batch-sort-merge , but I would like to know if its possible to perhaps do it with the FileInputStream and/or a non spring batch solution.
A solution to use an in memory or real database to join them is not viable for my use case due to the size of the files and the absence of an actual database.
Since the input files are already sorted, a simple parallel iteration of the files, merging their content, is the memory cheapest, cpu cheapest, and fastest way to do it.
This is a multi-way merge join, i.e. a sort-merge join without the "sort", with elimination of duplicates, similar to a SQL DISTINCT.
Here is a version that can do unlimited number of input files (well, as many as you can have open files anyway). It uses a helper class to stage the next line from each input file, so the leading ID value only has to be parsed once per line.
private static void merge(StringWriter out, BufferedReader ... in) throws IOException {
CustomerReader[] customerReader = new CustomerReader[in.length];
for (int i = 0; i < in.length; i++)
customerReader[i] = new CustomerReader(in[i]);
merge(out, customerReader);
}
private static void merge(StringWriter out, CustomerReader ... in) throws IOException {
List<CustomerReader> min = new ArrayList<>(in.length);
for (;;) {
min.clear();
for (CustomerReader reader : in)
if (reader.hasData()) {
int cmp = (min.isEmpty() ? 0 : reader.compareTo(min.get(0)));
if (cmp < 0)
min.clear();
if (cmp <= 0)
min.add(reader);
}
if (min.isEmpty())
break; // all done
// optional: Verify that lines that compared equal by ID are entirely equal
out.write(min.get(0).getCustomerLine());
out.write(System.lineSeparator());
for (CustomerReader reader : min)
reader.readNext();
}
}
private static final class CustomerReader implements Comparable<CustomerReader> {
private BufferedReader in;
private String customerLine;
private int customerId;
CustomerReader(BufferedReader in) throws IOException {
this.in = in;
readNext();
}
void readNext() throws IOException {
if ((this.customerLine = this.in.readLine()) == null)
this.customerId = Integer.MAX_VALUE;
else
this.customerId = Integer.parseInt(this.customerLine.substring(0, this.customerLine.indexOf('|')));
}
boolean hasData() {
return (this.customerLine != null);
}
String getCustomerLine() {
return this.customerLine;
}
#Override
public int compareTo(CustomerReader that) {
// Order by customerId only. Inconsistent with equals()
return Integer.compare(this.customerId, that.customerId);
}
}
TEST
String file1data = "1|1|MARY|SMITH\n" +
"2|1|PATRICIA|JOHNSON\n" +
"4|2|BARBARA|JONES\n";
String file2data = "1|1|MARY|SMITH\n" +
"2|1|PATRICIA|JOHNSON\n" +
"3|1|LINDA|WILLIAMS\n" +
"4|2|BARBARA|JONES\n";
String file3data = "2|1|PATRICIA|JOHNSON\n" +
"3|1|LINDA|WILLIAMS\n" +
"5|2|ALEXANDER|ANDERSON\n";
try (
BufferedReader in1 = new BufferedReader(new StringReader(file1data));
BufferedReader in2 = new BufferedReader(new StringReader(file2data));
BufferedReader in3 = new BufferedReader(new StringReader(file3data));
StringWriter out = new StringWriter();
) {
merge(out, in1, in2, in3);
System.out.print(out);
}
OUTPUT
1|1|MARY|SMITH
2|1|PATRICIA|JOHNSON
3|1|LINDA|WILLIAMS
4|2|BARBARA|JONES
5|2|ALEXANDER|ANDERSON
The code merges purely by ID value, and doesn't verify that rest of line is actually equal. Insert code at the optional comment to check for that, if needed.
This might help:
public static void main(String[] args) {
String files[] = {"Customer-1.txt", "Customer-2.txt", "Customer-3.txt"};
HashMap<Integer, String> customers = new HashMap<Integer, String>();
try {
String line;
for(int i = 0; i < files.length; i++) {
BufferedReader reader = new BufferedReader(new FileReader("data/" + files[i]));
while((line = reader.readLine()) != null) {
Integer uuid = Integer.valueOf(line.split("|")[0]);
customers.put(uuid, line);
}
reader.close();
}
BufferedWriter writer = new BufferedWriter(new FileWriter("data/Customer-final.txt"));
Iterator<String> it = customers.values().iterator();
while(it.hasNext()) writer.write(it.next() + "\n");
writer.close();
} catch (Exception e) {
e.printStackTrace();
}
}
If you have any cquestions ask me.

How can i read the same file two times in Java?

I want to counter the lines of the file and in the second pass i want to take every single line and manipulating it. It doesn't have a compilation error but it can't go inside the second while ((line = br.readLine()) != null) .
Is there a different way to get the lines(movies) of the file and storing in an array ?
BufferedReader br = null;
try { // try to read the file
br = new BufferedReader(new FileReader("movies.txt"));
String line;
int numberOfMovies = 0;
while ((line = br.readLine()) != null) {
numberOfMovies++;
}
Movie[] movies = new Movie[numberOfMovies]; // store in a Movie
// array every movie of
// the file
String title = "";
int id = 0;
int likes = 0;
int icounter = 0; // count to create new movie for each line
while ((line = br.readLine()) != null) {
line = line.trim();
line = line.replaceAll("/t", "");
line = line.toLowerCase();
String[] tokens = line.split(" "); // store every token in a
// string array
id = Integer.parseInt(tokens[0]);
likes = Integer.parseInt(tokens[tokens.length]);
for (int i = 1; i < tokens.length; i++) {
title = title + " " + tokens[i];
}
movies[icounter] = new Movie(id, title, likes);
icounter++;
}
} catch (IOException e) {
e.printStackTrace();
}
Simplest way would be to reset br again.
try { // try to read the file
br = new BufferedReader(new FileReader("movies.txt"));
String line; int numberOfMovies = 0;
while (br.hasNextLine()){
numberOfMovies++;
}
br.close();
Movie[] movies = new Movie[numberOfMovies];
// store in a Movie
// array every movie of
// the file
String title = "";
int id = 0;
int likes = 0;
int icounter = 0;
// count to create new movie for each line
br = new BufferedReader(new FileReader("movies.txt"));
while ((br.hasNextLine()) {
line = line.trim();
line = line.replaceAll("/t", "");
line = line.toLowerCase();
String[] tokens = line.split(" ");
// store every token in a
// string array
id = Integer.parseInt(tokens[0]);
likes = Integer.parseInt(tokens[tokens.length]);
for (int i = 1; i < tokens.length; i++) {
title = title + " " + tokens[i];
}
movies[icounter] = new Movie(id, title, likes);
icounter++;
}
} catch (IOException e) { e.printStackTrace(); }
I changed br.nextLine() != null to br.hasNextLine() because it's shorter and more appropriate in this case. Plus it won't consume a line.
There are two things here:
InputStreams and Readers are one-shot structures: once you've read them to the end, you either need to explicitly rewind them (if they support rewinding), or you need to close them (always close your streams and readers!) and open a new one.
However in this case the two passes are completely unnecessary, just use a dynamically growing structure to collect your Movie objects instead of arrays: an ArrayList for example.
Firstly, there is no need to read the file twice.
Secondly, why don't you use the java.nio.file.Files class to read your file.
It has a method readAllLines(Path path, Charset cs) that gives you back a List<String>.
Then if you want to know how many lines just call the size() method on the list and you can use the list to construct the Movie objects.
List<Movie> movieList = new ArrayList<>();
for (String line : Files.readAllLines(Paths.get("movies.txt"), Charset.defaultCharset())) {
// Construct your Movie object from each individual line and add to the list of Movies
movieList.add(new Movie(id, title, likes));
}
The use of the Files class also reduces your boilerplate code as it will handle closing the resource when it has completed reading meaning you will not need a finally block to close anything.
If you use the same Reader, everything is already read once you reach the second loop.
Close the first Reader, then create another one to read a second time.
You are running through the file with the BufferedReader, until the nextline points towards null. As your BufferedReader IS null, it won't even enter the second while((line = br.readline) != null), as the first read line is null.
Try getting a new BufferedReader. something like this:
...
int id = 0;
int likes = 0;
int icounter = 0;
br = new BufferedReader(new FileReader("movies.txt")) //Re-initialize the br to point
//onto the first line again
while ((line = br.readLine()) != null)
...
EDIT:
Close the reader first..
This is a combination of a couple of other answers already on this post, but this is how I would go about rewriting your code to populate a List. This doubly solves the problem of 1) needing to read the file twice 2) removing the boilerplate around using BufferedReader while using Java8 Streams to make the initializing of your List as concise as possible:
private static class Movie {
private Movie(int id, String title, int likes) {
//TODO: set your instance state here
}
}
private static Movie movieFromFileLine(String line) {
line = line.trim();
line = line.replaceAll("/t", "");
line = line.toLowerCase();
String[] tokens = line.split(" "); // store every token in a
String title = "";
int id = Integer.parseInt(tokens[0]);
int likes = Integer.parseInt(tokens[tokens.length]);
for (int i = 1; i < tokens.length; i++) {
title = title + " " + tokens[i];
}
return new Movie(id, title, likes);
}
public static void main(String[] args) throws IOException {
List<Movie> movies = Files.readAllLines(Paths.get("movies.txt"), Charset.defaultCharset()).stream().map
(App::movieFromFileLine).collect(Collectors.toList());
//TODO: Make some magic with your list of Movies
}
For cases where you absolutely need to read a source (file, URL, or other) twice, then you need to be aware that it is quite possible for the contents to change between the first and second readings and be prepared to handle those differences.
If you can make a reasonable assumption that the content of the source will fit in to memory and your code fully expects to work on multiple instances of Readers/InputStreams, you may first consider using an appropriate IOUtils.copy method from commons-io to read the contents of the source and copy it to a ByteArrayOutputStream to create a byte[] that can be re-read over and over again.

Read a text file to an array Java

I know there are many questions about reading text files here but I have gone through all of them and I think I'm having some difficulty with syntax or SOMETHING because nothing that I've been trying has been working at all.
What I'm attempting to do is this:
1) read a text file inputed by user
2) copy each individual line into an array, so each line is its own element in the array
I feel like I am very close but for some reason I can't figure out exactly how to get it to work!
Here is the relevant code I have right now:
I keep getting out of bounds exceptions in three locations which I've marked off.
Been working on this for quite a while not sure what to do next! Any ideas?
import java.io.IOException;
import java.util.Scanner;
public class FindWords {
public static void main (String args[]) throws IOException{
FindWords d = new Dictionary();
((Dictionary) d).dictionary(); //********* out of bounds here
}
/**
* Validates and returns the dictionary inputed by the user.
*
* #param
* #return the location of the dictionary
*/
public static String getDict(){
///////////////////ASK FOR DICTIONARY////////////////////
System.out.println("Please input your dictionary file");
//initiate input scanner
Scanner in = new Scanner(System.in);
// input by user
String dictionary = in.nextLine();
System.out.println("Sys.print: " + dictionary);
//make sure there is a dictionary file
if (dictionary.length() == 0){
throw new IllegalArgumentException("You must enter a dictionary");
}
else return dictionary;
}
}
which calls on the class Dictionary:
import java.io.*;
public class Dictionary extends FindWords{
public void dictionary () throws IOException{
String dict = getDict();
String[] a = readFile(dict); //********** out of bounds here
int i = 0;
while(a[i] != null){
System.out.println(a[i]);
i++;
}
}
public static String[] readFile(String input) throws IOException{
//read file
BufferedReader br = new BufferedReader(new InputStreamReader(new FileInputStream(input)));
System.out.println ();
int count = 0;
String[] array = new String[count];
try{
while (br.readLine() != null){
array[count] = br.readLine(); //********out of bounds here
count++;
}
br.close();
}
catch (IOException e){
}
return array;
}
}
Thank you for looking!
Edit: Just fyi: I have my .txt file in the parent project folder.
Have you tried this?:
List<String> lines = Files.readAllLines(Paths.get("/path/to/my/file.txt"));
and then transform your list to an array if you want:
String[] myLines = lines.toArray(new String[lines.size()]);
You start with an array size of zero...
int count = 0;
String[] array = new String[count];
Several issues here :
In Java, you can't expand arrays, i.e you have to know their length in advance when you instantiate them. Hence the ArrayOutOfBoundException. To make this easy, I suggest that you use an ArrayList instead.
In your while loop, you're making 2 calls to br.readLine(), so basically you're skipping one line out of 2.
You are initializing a zero-length array, hence the exception on the first iteration:
int count = 0;
String[] array = new String[count];
Since you probably don't know the expected size, work with a List instead:
List<String> list = new ArrayList<>();
String thisLine = null;
try{
while ((thisLine = br.readLine()) != null) {
list.add(thisLine);
}
}
You can get the total size afterwards by:
list.size();
Or even better, go with morganos solution and use Files.readAllLines().

How do I read from a File to an array

I am trying to read from a file to an array. I tried two different styles and both aren't working. Below are the two styles.
Style 1
public class FileRead {
int i;
String a[] = new String[2];
public void read() throws FileNotFoundException {
//Z means: "The end of the input but for the final terminator, if any"
a[i] = new Scanner(new File("C:\\Users\\nnanna\\Documents\\login.txt")).useDelimiter("\\n").next();
for(i=0; i<=a.length; i++){
System.out.println("" + a[i]);
}
}
public static void main(String args[]) throws FileNotFoundException{
new FileRead().read();
}
}
Style 2
public class FileReadExample {
private int j = 0;
String path = null;
public void fileRead(File file){
StringBuilder attachPhoneNumber = new StringBuilder();
try{
FileReader read = new FileReader(file);
BufferedReader bufferedReader = new BufferedReader(read);
while((path = bufferedReader.readLine()) != null){
String a[] = new String[3];
a[j] = path;
j++;
System.out.println(path);
System.out.println(a[j]);
}
bufferedReader.close();
}catch(IOException exception){
exception.printStackTrace();
}
}
I need it to read each line of string and store each line in an array. But neither works. How do I go about it?
Do yourself a favor and use a library that provides this functionality for you, e.g.
Guava:
// one String per File
String data = Files.toString(file, Charsets.UTF_8);
// or one String per Line
List<String> data = Files.readLines(file, Charsets.UTF_8);
Commons / IO:
// one String per File
String data = FileUtils.readFileToString(file, "UTF-8");
// or one String per Line
List<String> data = FileUtils.readLines(file, "UTF-8");
It's not really clear exactly what you're trying to do (partly with quite a lot of code commented out, leaving other code which won't even compile), but I'd recommend you look at using Guava:
List<String> lines = Files.readLines(file, Charsets.UTF_8);
That way you don't need to mess around with the file handling yourself at all.

Categories