Using opencsv - java.lang.OutOfMemoryError: Java heap space - java

I am using opencsv to parse two csv files. I only copy some values from the two files.
I have a seperate function which processes the CDax.csv. Which looks like that:
public HashMap<String,String> readCDax() throws Exception {
String csvDaxFile = "C:\\Users\\CDAX.csv";
CSVReader reader = new CSVReader(new FileReader(csvDaxFile), ';');
String [] line;
HashMap<String, String> cdaxMap = new HashMap<String, String>();
while ((line = reader.readNext()) != null) {
cdaxMap.put(line[0], line[7]);
}
System.out.println("Process CDax File!");
reader.close();
return cdaxMap;
}
My main method is run() which I execute in my main method:
public void run() throws Exception {
while ((firstLine = reader.readNext()) != null && (secondLine = reader.readNext()) != null && i<10) {
//fileName of the String
fileName = firstLine[0];
writerPath = "C:\\Users\\" + fileName + ".csv";
//write csv file
CSVWriter writer = new CSVWriter(new FileWriter(writerPath), ';');
//write Header
//String[] entries = "Name;Date;TotalReturn;Currency".split(";");
String [] entries = {"Name","Date", "TotalReturn", "Currency"};
writer.writeNext(entries);
//create Content
//companyName of the String
companyName = secondLine[1];
//currency
currency = secondLine[2];
//dates
dateList = new ArrayList<String>();
for(int p = 3; p < firstLine.length; p++) {
dateList.add(firstLine[p]);
}
//total returns
returnList = new ArrayList<String>();
for(int j = 3; j < secondLine.length; j++) {
returnList.add(secondLine[j]);
}
// cDaxList
cDaxList = new ArrayList<String>();
for(int j = 1; j <dateList.size(); j++) {
if(cDaxMethodValuesMap.containsKey(dateList.get(j))){
cDaxList.add(cDaxMethodValuesMap.get(dateList.get(j)));
} else{
dateList.add("na"); // I get the error here!
}
}
if(dateList.size()!=returnList.size()) {
System.out.println("Dates and Returns do not have the same length!");
}
int minSize = Math.min(dateList.size(), returnList.size());
//"Name;Date;TotalReturn;Currency"
List<String[]> data = new ArrayList<String[]>();
for(int m = 0; m < minSize; m++) {
data.add(new String[] {companyName, dateList.get(m), returnList.get(m), currency, cDaxList.get(m)});
}
writer.writeAll(data);
//close Writer
writer.close();
i++;
System.out.println(fileName + " parsed successfully!");
}
System.out.println("Done");
}
However when I run my program I get:
Process CDax File!
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2760)
at java.util.Arrays.copyOf(Arrays.java:2734)
at java.util.ArrayList.ensureCapacity(ArrayList.java:167)
at java.util.ArrayList.add(ArrayList.java:351)
at com.TransformCSV.main.ParseCSV.run(ParseCSV.java:109)
at com.TransformCSV.main.ParseCSV.main(ParseCSV.java:21)
I am getting the error in this method:
cDaxList = new ArrayList<String>();
for(int j = 1; j <dateList.size(); j++) {
if(cDaxMethodValuesMap.containsKey(dateList.get(j))){
cDaxList.add(cDaxMethodValuesMap.get(dateList.get(j)));
} else{
dateList.add("na"); //I get the error here!!!
}
}
I tried to put up the heapsize via the vm settings, however I do not think that this should be done because I only read in both csv files only 3000 values.
I appreciate your reply!

Your loop:
for(int j = 1; j <dateList.size(); j++) {
is looping through dateList but in that loop you are adding to dateList:
dateList.add("na"); //I get the error here!!!
so dateList will get bigger and bigger until you run out of memory. dateList.size() is evaluated every time through the loop, not once at the beginning.

Related

How to append only one line/row for each iteration in appending values to CSV file in Java

I am having some issues in terms of appending data into my CSV file. The problem is that whenever I try to append data into my CSV file on a second time, the second value which is appended to the CSV file comes with the first appended value. It's like it brings the existing value with it when appending to the CSV file. Thus, because of this issue, it results into an array index out of bounds exception in this statement: cust[read2DStringIndex][newVarIndexer] = fromfile[g]; , the data of the CSV file repeats the existing values along with the latest appended values and also the first value is only displayed on my GUI table.
CSV File:
Table:
Here's my source code in writing and reading the CSV:
public void writeCustomerCSV(){ // this creates a CSV file which stores the inputs of the user
try {
BufferedWriter bw = new BufferedWriter(new FileWriter("C:\\Users\\RALPH\\Documents\\Database Java CSV\\customers.csv",true)); // when I set append mode to true, cust[read2DStringIndex][newVarIndexer] = fromfile[g] results to index array out of bounds to 10
StringBuilder sb = new StringBuilder();
int y;
for(int x = 0; x < itemTo2D.length; x++){
if(itemTo2D[x][0] != null){
for(y = 0; y < itemTo2D[0].length; y++){
sb.append(itemTo2D[x][y]);
sb.append(",");
}
}
sb.append("-"); //separation for rows
sb.append(","); // separation for columns
}
bw.write(sb.toString());
bw.close();
}
catch (Exception ex){
}
}
public void readCustomerCSV(){ // reads the contents of the CSV file
String[][] twoDArray = new String[10][7];
int read2DStringIndex = 0;
int newVarIndexer = 0;
DefaultTableModel tblmodelll = (DefaultTableModel) mainTable.getModel(); // table
String[] fromfile = {}; // 1d string for getting the columns(7 columns) of the CSV file
int ak = 0;
int sk = 0;
try{
BufferedReader br = new BufferedReader(new FileReader("C:\\Users\\RALPH\\Documents\\Database Java CSV\\customers.csv"));
String line;
while ((line = br.readLine()) != null){
fromfile = line.split(","); //separates the columns by a comma
for(int c = 0; c < fromfile.length; c++){
if(fromfile[c].equals("-")){
sk = 0;
ak++;
if(c > 0){
if(!fromfile[c-1].equals("-")){
id = id + 1;
}
}
} else{
twoDArray[ak][sk] = fromfile[c];
sk++;
}
}
}
} catch (Exception ex){
}
for(int g = 0; g < fromfile.length; g++){
if(fromfile[g].equals("-")){ //if there is a presence of a dash, it increments the read2DStringINdex (row index) of the 2D array
read2DStringIndex++;
newVarIndexer = 0;
}
else{
cust[read2DStringIndex][newVarIndexer] = fromfile[g]; //cust is the 2D array(declared universal) which is going to display the values to the table
newVarIndexer++;
}
}
for(int h = 0; h < cust.length; h++){ //prints cust (2D array) , just to check what data is being stored
for(int p = 0; p < cust[0].length; p++){
System.out.println(cust[h][p] + ",");
}
}
setrowcount = 0;
for(int r = 0; r < cust.length; r++){
if(setrowcount == 0){
tblmodelll.setRowCount(0);
}
try{
if(cust[r][0].equals("null") == false){
tblmodelll.addRow(cust[r]); //displays the cust(2D array) data to table
}
} catch(Exception e){
}
setrowcount++;
}
}
Is there something missing in my structure of the codes or is my logic in appending the values not right?
Your responses would indeed help me in resolving this issue.
Thank you very much.

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space while using util Packages

My problem statement:
There will be given set of words in a file (>5000 words). We need return a list of anagrams separated by comma(,) in each string(set of anagrams)
Eg: [alter,later, part,trap, elbow,below, listen,silent , tensil ]
Exception:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3210)
at java.util.Arrays.copyOf(Arrays.java:3181)
at java.util.ArrayList.grow(ArrayList.java:265)
at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:239)
at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:231)
at java.util.ArrayList.add(ArrayList.java:462)
at FindAnagrams01.anagramsList(FindAnagrams01.java:25)
at FindAnagrams01.main(FindAnagrams01.java:7)
My code is :
public static List<String> anagramsList(String filePath) throws IOException {
BufferedReader br = new BufferedReader(new FileReader(filePath));
String str = br.readLine();
List<List<String>> result = new ArrayList<List<String>>(10000);
HashMap<String, ArrayList<String>> map = new HashMap<String,ArrayList<String>>(10000);
while(str != null) {
char[] arr = new char[26];
for(int i = 0; i < str.length(); i++) {
arr[str.charAt(i) - 'a']++;
}
String ns = new String(arr);
if(map.containsKey(ns)){
map.get(ns).add(str);
} else {
ArrayList<String> al = new ArrayList<String>(10000);
al.add(str);
map.put(ns, al);
}
}
br.close();
result.addAll(map.values());
String res[] = new String[10000];
for(int i = 0; i < result.size(); i++) {
int isIntial = 0;
for(String j : result.get(i)) {
if ((result.get(i).size()) > 1) {
if (isIntial == 0) {
res[i] = j;
isIntial = 1;
}
else
res[i] += "," + j;
}
}
}
List<String> angrms = new ArrayList<String>(10000);
for (int i = 0; i < res.length; i++) {
if (res[i] != null)
angrms.add(res[i]);
}
return angrms;
}
The problem is that this is an infinite loop:
while(str != null) {
char[] arr = new char[26];
for(int i = 0; i < str.length(); i++) {
arr[str.charAt(i) - 'a']++;
}
String ns = new String(arr);
if(map.containsKey(ns)){
map.get(ns).add(str);
} else {
ArrayList<String> al = new ArrayList<String>(10000);
al.add(str);
map.put(ns, al);
}
}
because str is not altered within the loop body. As a result, you end up repeatedly adding strings to an array list until you eventually run out of memory.
There are other problems with your code ... but this explains your OOME.

Searching in arraylist<String[]> from user input

I want to search in an arraylist from a user input but my if condition doesn't seem to work. Using boolean and .contains() doesn't work for my programme either. This is the coding:
String phone;
phone=this.text1.getText();
System.out.println("this is the phone: " + phone);
BufferedReader line = new BufferedReader(new FileReader(new File("C:\\Users\\Laura Sutardja\\Documents\\IB DP\\Computer Science HL\\cs\\data.txt")));
String indata;
ArrayList<String[]> dataArr = new ArrayList<String[]>();
while ((indata = line.readLine()) != null) {
String[] club = new String[2];
String[] value = indata.split(",", 2);
//for (int i = 0; i < 2; i++) {
int n = Math.min(value.length, club.length);
for (int i = 0; i < n; i++) {
club[i] = value[i];
}
boolean aa = dataArr.contains(this.text1.getText());
if(aa==true)
text2.setText("The data is found.");
else
text2.setText("The data is not found.");
dataArr.add(club);
}
for (int i = 0; i < dataArr.size(); i++) {
for (int x = 0; x < dataArr.get(i).length; x++) {
System.out.printf("dataArr[%d][%d]: ", i, x);
System.out.println(dataArr.get(i)[x]);
}
}
}
catch ( IOException iox )
{
System.out.println("Error");
}
Your dataArr is a list of String[], and you are searching for a String. The two are different kind of objects.
I don't really know how the content of the club array looks like, but you should either change dataArr in order to hold plain String, or to write a method which looks iteratively in dataArr for a String[] containing the output of this.text1.getText().
There is a lot wrong with the program. I assume you want to read a textfile and store each line in the arraylist. To do this you have to split each line of the textfile and store that array in the arrayList.
String[] value;
while ((indata = line.readLine()) != null) {
value = indata.split(",");
dataArr.add(value);
}
Now you have the contents of the file in the arrayList.
Next you want to compare the userinput with each element of the arraylist.
int j = 0;
for (int i = 0; i < dataArr.size(); i++) {
String[] phoneData = dataArr.get(i);
if (phoneData[1].equals(phone)) { // i am assuming here that the phone number is the 2nd element of the String[] array, since i dont know how the textfile looks.
System.out.println("Found number.");
club[j++] = phoneData[1];
} else if (i == dataArr.size()-1) {
System.out.println("Didn't find number.");
}
}
Edit:
As requested:
String phone;
phone = "38495";
System.out.println("this is the phone: " + phone);
BufferedReader line = new BufferedReader(new FileReader(new File("list.txt")));
String indata;
ArrayList<String[]> dataArr = new ArrayList<>();
String[] club = new String[2];
String[] value;// = indata.split(",", 2);
while ((indata = line.readLine()) != null) {
value = indata.split(",");
dataArr.add(value);
}
int j = 0;
for (int i = 0; i < dataArr.size(); i++) {
String[] phoneData = dataArr.get(i);
if (phoneData[1].equals(phone)) {
System.out.println("Found number.");
club[j++] = phoneData[1];
break;
} else if (i == dataArr.size()-1) {
System.out.println("Didn't find number.");
}
}
I hope this makes sense now.

How to write only the 5th value from a list into a csv

I am reading several csv sheet with the opencsv libary. I am storing the "read" csv file in hugeList = reader.readAll(); Now I only want to only write the 5th value of this huge csv sheet columnwise in another sheet.
At the moment I am standing at:
//reading all entries in a huge list
//-740 as it would take to much time to run it in "dev mode"
for (int j = 0; j < (fileList.size() - 740); j++) {
String csvFile = "C:\\Users\\" + fileList.get(j);
reader = new CSVReader(new FileReader(csvFile), ';');
hugeList = reader.readAll();
List<String[]> data = new ArrayList<String[]>();
// for(int m = 0; m < hugeList.size(); m++) {
// String[] value = hugeList.get(m);
data.add(hugeList.get(5));
// }
writer.writeAll(data);
}
However I have no idea how to write them into another column and just take the 5th value?
I really appreciate your answer!
UPDATE 1
Goal: I have saved all values from one sheet at the time in hugeList. Now I want to
write only the 5th column into a new sheet.
Problem: The logic behind this goal.
UPDATE 2
This is what my code looks like right now:
//reading all entries in a huge list
for (int j = 0; j < (fileList.size() - 740); j++) {
String csvFile = "C:\\" + fileList.get(j);
reader = new CSVReader(new FileReader(csvFile), ';');
hugeList = reader.readAll();
List<String[]> data = new ArrayList<String[]>();
List<String> tmp= new ArrayList<String>();
for(int m = 0; m < hugeList.size(); m++) {
String[] values = hugeList.get(m);
tmp.add(values[0]);
}
data.add(tmp.toArray(new String[0]));
writer.writeAll(data);
}
As you can see I get my data still vertically... Why?
I thing you are already on the right way.
You have 2 possibilities.
1. data.add(hugeList.get(4));
2. for(int m = 0; m < hugeList.size(); m++) {
if(m==4){
String[] values = hugeList.get(m);
data.add(values);
break;
}
}
UPDATE
Goal: I have saved all values from one sheet at the time in hugeList. Now I want to write only the 5th column into a new sheet. Problem: The logic behind this goal.
My Approach for a result in rows
List<String[]> data = new ArrayList<String[]>();
List<String> tmp= new ArrayList<String>();
for(int m = 0; m < hugeList.size(); m++) {
String[] values = hugeList.get(m);
tmp.add(values[4]);
}
data.add(tmp.toArray(new String[0]));
}
My Approach for a result in Column
for (int j = 0; j < fileList.size(); j++) {
String csvFile = readPath + fileList.get(j);
System.out.println("Read: " + csvFile);
reader = new CSVReader(new FileReader(csvFile), ';');
hugeList = reader.readAll();
String[] data = new String[1];
for (int m = 0; m < hugeList.size(); m++) {
String[] values = hugeList.get(m);
data[0] = values[0];
writer.writeNext(data);
}
}

Reading from file/array. No values. No errors

Any ideas what I am missing here? I am reading from a file array. The values in the text file don't get stored and there is no output. All I get is "names and totals" but no values.
I don't know.
private int[] totals;
private String[] names;
private String[] list;
private int count;
public void readData() throws IOException {
BufferedReader input = new BufferedReader(new FileReader("cookies.txt"));
//create the arrays
totals = new int[count];
names = new String[count];
list = new String[count];
//read in each pair of values
String quantityString = input.readLine();
for (int i = 0; i < count; i++) {
names[i] = input.readLine();
list[i] = input.readLine();
quantityString = input.readLine();
totals[i] = Integer.parseInt(quantityString);
}
}
public void display() {
System.out.println("names totals")
for (int i = 0; i < count; i++)
System.out.println(list[i] + " \t " + names[i] + " \t" + totals[i]);
}
//called to compute and print the result
public void printResults() {
//find the best teacher
int maxIndex = 0;
int maxValue = 0;
//for each record stores
for (int i = 0; i < count; i++) {
//if we have a new MAX value so far, update variables
if (maxValue < totals[i]) {
maxValue = totals[i];
maxIndex = i;
}
}
}
You never give the variable count a value, so it initialized to 0 by Java. This means that your arrays are of size 0 also.
So since count is zero, you never read anything from the file, which is why nothing is stored in your arrays and also why nothing is printed out.
Example: Reading a File line-by-line
// create temporary variable to hold what is being read from the file
String line = "";
// when you don't know how many things you have to read in use a List
// which will dynamically grow in size for you
List<String> names = new ArrayList<String>();
List<Integer> values = new ArrayList<Integer>();
// create a Reader, to read from a file
BufferedReader input = new BufferedReader(new FileReader("cookies.txt"));
// read a full line, this means if you line is 'Smith 36'
// you read both of these values together
while((line = input.readLine()) != null)
{
// break 'Smith 36' into an array ['Smith', '36']
String[] nameAndValue = line.split("\\s+");
names.add(nameAndValue[0]); // names.add('Smith')
values.add(Integer.parseInt(nameAndValue[1]); // values.add(36);
}

Categories