I'm trying to get the print output and store it into an array of Strings in order to write the values in a CSV File. The problem that comes up is that when a run the code, from 100 lines printed, only the last line gets stored:
public void organize(String line) throws IOException {
CSVWriter writer = new CSVWriter(new FileWriter("out.csv"), '\t');
String[] token = line.split(",");
String[] DependencyItems = token[1].split(" ");
List<String> entriesList;
String[] entries;
String row = "";
for (int i = 0; i < DependencyItems.length; i++) {
row = token[0] + "," + DependencyItems[i] + "," + token[2] + "," + token[3] + "," + token[5]+ "," + token[6]
+ "," + token[7]+ "," + token[8]+ "," + token[9] + "," + token[10] + "," + token[11] + "," + token[12]
+ "," + token[13] + "," + token[14] + "," + token[15] + "," + token[16]+ "," + token[17]
+ "," + token[18]+ "," + token[19]+ "," + token[20]+ "," + token[21] + "," + token[22]
+ "," + token[23]+ "," + token[24]+ "," + token[25]+ "," + token[26]+ "," + token[27]
+ "," + token[28]+ "," + token[29];
}
System.out.println(row);
entriesList = Arrays.asList(row);
entries = entriesList.toArray(new String[0]);
writer.writeNext(entries);
writer.close();
}
What I am doing wrong?
row = <big long string>
You are always resetting row then all of the stuff after the loop is just using the last value of row. If you want to use all the values, perhaps the code after the loop should be in the loop? At the very least it looks like these 2 lines belong in the loop.
System.out.println(row);
entriesList = Arrays.asList(row);
Note that if the scope of row was limited to just be inside the loop, then this may help to identify this type of issue as it would not be valid outside the loop.
Related
Currently I am working on a solution to read a 454 character/line based huge file (minimum 50000 rows) via Java.As per the requirement it is a positioned based file, we first need to read the file , then parse the position based values and need to insert into a table. (minimum 96 positions will be inserted into 96 columns of the table).
I took this concept after the parsing.
[ INSERT ALL INTO<TABLE NAME> [COL1,COL2,COL3] Values [VAL1,VAL2,VAL3]
INTO<TABLE NAME> [COL1,COL2,COL3] Values [VAL1,VAL2,VAL3]
SELECT * FROM DUAL;]
Here is my code:
try{
char[] line = new char[456];
while(br.read(line) > 0){
StringBuilder input = new StringBuilder(new String(line));
if(batchCounter>0){
int detailFileId = interfaceFileSequence();
sql.append(initSql+"(" +
detailFileId + "," + interfaceHeaderId + ", SYSDATE," +
interfaceRunId + "," + isSpace(input.substring(0, 2).trim()) + "," + "TO_DATE("+isSpace(input.substring(2, 12).trim())+",'YYYY-MM-DD')" +","+isSpace(input.substring(12, 22).trim()) + "," +
Double.parseDouble(input.substring(22, 35)+ "." + input.substring(35, 37)) + ", " +
Double.parseDouble(input.substring(22, 35)+ "." + input.substring(35, 37)) + ", " +
isSpace(input.substring(38, 44).trim()) + ","+isSpace(input.substring(38, 44).trim())+"," +isSpace(input.substring(38, 44).trim())+"," + isSpace(input.substring(44, 54).trim())+","+
isSpace(input.substring(54, 60).trim()) + "," + isSpace(input.substring(60, 68).trim()) + "," + isSpace(input.substring(68, 83).trim()) + "," +
isSpace(input.substring(83, 89).trim()) + "," + isSpace(input.substring(89, 94).trim()) + "," +
isSpace(input.substring(94, 102).trim()) + "," +
isSpace(input.substring(102, 103).trim()) + ","+"TO_DATE("+isSpace(input.substring(103,113).trim())+",'YYYY-MM-DD')"+"," +isSpace(input.substring(113, 125).trim()) + "," + isSpace(input.substring(125, 128).trim()) + "," +
isSpace(input.substring(131, 133).trim()) + "," + isSpace(input.substring(133, 135).trim()) + "," + isSpace(input.substring(135, 136).trim()) + "," +
isSpace(input.substring(136, 137).trim()) + "," + isSpace(input.substring(137, 142).trim()) + "," + isSpace(input.substring(142, 147).trim()) + "," +
isSpace(input.substring(147, 148).trim()) + "," + isSpace(input.substring(149, 159).trim()) + "," +
isSpace(input.substring(159, 160).trim()) + "," + isSpace(input.substring(160, 175).trim()) + "," + isSpace(input.substring(160, 175).trim()) + "," +
isSpace(input.substring(190, 220).trim()) +"," +"TO_DATE("+isSpace(input.substring(216, 220)+"-"+input.substring(220, 222)+"-"+input.substring(222, 224))+",'YYYY-MM-DD')"+","+
"TO_DATE("+isSpace(input.substring(216, 220)+"-"+input.substring(220, 222)+"-"+input.substring(222, 224))+",'YYYY-MM-DD')"+","+ isSpace(input.substring(226,227).trim()) + "," + isSpace(input.substring(231,236).trim()) + "," +
isSpace(input.substring(242, 245).trim()) + "," + isSpace(input.substring(245,275).trim()) + "," + isSpace(input.substring(275,280).trim()) + "," +
isSpace(input.substring(280, 290).trim()) + "," + isSpace(input.substring(290,293).trim()) + "," + isSpace(input.substring(293,303).trim()) + "," +
isSpace(input.substring(303, 314).trim()) + "," +
isSpace(input.substring(313,316).trim()) + //need check
"," + isSpace(input.substring(317,337).trim()) + "," +
isSpace(input.substring(337, 422).trim()) +
"," + isSpace(input.substring(422,433).trim()) + "," + isSpace(input.substring(433,443).trim())+","+isSpace(input.substring(22, 39).trim())+
")");
sql.append('\n');
}
/*if (batchCounter % 500 == 0) {
System.out.println("sql:::::::::::"+sql);
int executeUpdate = em.createNativeQuery(sql.toString()).executeUpdate();
System.out.println("executeUpdateexecu:::"+executeUpdate);
em.flush();
insertionCounter += executeUpdate;
System.out.println("insertionCounter::::"+insertionCounter);
sql.setLength(0);
System.out.println("SQL");
System.out.println("initSql"+initSql);
sql.append(initSql);
}*/
batchCounter++;
}
sql.append(" SELECT * FROM dual");
int executeUpdate = em.createNativeQuery(sql.toString()).executeUpdate();
em.flush();
insertionCounter += executeUpdate;
System.out.println("Check Rows in file::"+(batchCounter-1)+" Insertion counter::"+insertionCounter);
if((batchCounter-1)==insertionCounter){
detailFileObj = new DetailFileObject(FileName, "DETAIL", (batchCounter-1), "SUCCESS");
}
else {
detailFileObj = new DetailFileObject(FileName, "DETAIL", (batchCounter-1), "FAILED");
}
} catch (IOException e) {
e.printStackTrace();
}
But once I am running the code, if there are 40-50 lines, it is working fine.But it is more than that I am getting exception. Can anyone share me proper approach along with running code, so that I can use it.
Thanks
First, DO NOT use string concatenation to build a SQL statement with text values from outside. It leaves you susceptible to SQL Injection attacks, and may cause SQL syntax errors. Use a PreparedStatement with ? parameter markers instead.
Second, if you're inserting a lot of records, use JDBC batching.
Here is an example of how you would use it:
String sql = "INSERT INTO MyTable" +
" ( Col1, Col2, Col3, Col4, Col5, Col6, Col7 )" +
" VALUES (?,?,SYSDATE,?,?,?,?)";
try (PreparedStatement stmt = conn.prepareStatement(sql)) {
char[] line = new char[456];
int batchSize = 0;
while (br.read(line) > 0) {
String input = new String(line);
int detailFileId = interfaceFileSequence();
stmt.setInt (1, detailFileId);
stmt.setInt (2, interfaceHeaderId);
stmt.setInt (3, interfaceRunId);
stmt.setString(4, toString(input.substring(0, 2)));
stmt.setDate (5, toDate(input.substring(2, 12)));
stmt.setString(6, toString(input.substring(12, 22)));
// ...
stmt.addBatch();
if (++batchSize == 1000) {
stmt.executeBatch();
batchSize = 0;
}
}
if (batchSize != 0) {
stmt.executeBatch();
}
} catch (IOException e) {
e.printStackTrace();
}
The above code uses the following helper methods to keep code DRY:
private static String toString(String text) {
String trimmed = text.trim();
return (trimmed.isEmpty() ? null : trimmed);
}
private static java.sql.Date toDate(String text) {
String trimmed = text.trim();
return (trimmed.isEmpty() ? null : java.sql.Date.valueOf(LocalDate.parse(trimmed)));
}
I am trying to find a way to "collect all the classes from a selected date to another, managed by a specific teacher", and so far, its going okay. I have used to uppercase to match with a capital letter as the first one in the name... but since some teachers (McGonnagal) for example is having capital letters in her lastname, is there a way to, say , get the input to lowercase, and also transfer the data in the database (McGonnagal) all to lowercase?
So it would always be a match even if the user inserted McGoNNaGal in the program.... anyone got any ideas?`
try {
String lararesNamn = txtLoCLarare.getText();
String LFN = lararesNamn.toUpperCase().charAt(0)+lararesNamn.substring(1);
String datumFrom = txtLoCfom.getText();
String datumTom = txtLoCtom.getText();
String lararesEN = txtLarareEN.getText();
String LEN = lararesEN.toUpperCase().charAt(0)+lararesEN.substring(1);
ArrayList<HashMap<String,String>> listOfClasses = databas.fetchRows("SELECT KURSNAMN FROM KURS JOIN LARARE ON KURSLARARE = LARAR_ID WHERE KURSSTART >= " + "'" + datumFrom + "'" + " AND KURSSLUT <= " + "'" + datumTom + "'" + " AND(LARARE.FORNAMN = " + "'" + LFN + "'" + " AND LARARE.EFTERNAMN = " + "'" + LEN + "'" + ")");
System.out.println(listOfClasses);
JOptionPane.showMessageDialog(null, listOfClasses);
} catch(InfException ex){
System.out.println(ex.getMessage());
}
Best regards!
You can convert the fetched name from SQL DB to lower or upper case. For LOWER case modify your query to.
SELECT KURSNAMN FROM KURS JOIN LARARE ON KURSLARARE = LARAR_ID WHERE KURSSTART >= " + "'" + datumFrom + "'" + " AND KURSSLUT <= " + "'" + datumTom + "'" + " AND(LOWER(LARARE.FORNAMN) = " + "'" + LFN + "'" + " AND LOWER(LARARE.EFTERNAMN) = " + "'" + LEN + "'" + ")"
You can use UPPER function for upper case
I have the following code:
String insert = "INSERT INTO " + tableName +
"(" + COLUMNS.TILE_ID + "," + COLUMNS.TILE_DATA + "," +
COLUMNS.TILE_LEVEL + "," + COLUMNS.TILE_COLUMN +
"," + COLUMNS.TILE_ROW +
"," + COLUMNS.TILE_IMAGE_FORMAT + "," + COLUMNS.TILE_SOURCE +
")";
String values = id + ",?" + "," +
tile.getLevel() + "," + tile.computeColumn() + "," +
tile.computeRow() + ",\'" + tile.getFileType().toUpperCase() +
"\'," + "\'" +
tile.getSource() + "\');";
String query = insert + " VALUES (" + values;
System.out.println(query);
PreparedStatement statement = conn.prepareStatement(query);
statement.setBytes(2, tile.getData());
return this.conn.createStatement().executeUpdate(query);
The query value:
INSERT INTO level1 (TILE_ID,TILE_DATA,TILE_LEVEL,TILE_COLUMN,TILE_ROW,TILE_IMAGE_FORMAT,TILE_SOURCE) VALUES (0,?,1,0,0,'JPG','null');
The error I get:
org.postgresql.util.PSQLException: The column index is out of range: 2, number of columns: 1.
My Table:
tile_id bigint NOT NULL,
tile_data bytea,
tile_level smallint,
tile_row integer,
tile_column integer,
tile_image_format image_format,
tile_source character varying(30),
CONSTRAINT level10_pkey PRIMARY KEY (tile_id)
Any ideas?
You haven't posted your stacktrace but your error seems to appear from this:
statement.setBytes(2, tile.getData());
Which is because you have only one parameter to bind to:
INSERT INTO level1 (TILE_ID,TILE_DATA,TILE_LEVEL,TILE_COLUMN,TILE_ROW,TILE_IMAGE_FORMAT,TILE_SOURCE) VALUES (0,?,1,0,0,'JPG','null');
The fact that the parameter is at position 2 in the values list is not what counts. It's the fact that it's the first place holder that matters. So your code should be,
statement.setBytes(1, tile.getData());
Thanks to e4c5 answer, I fixed my code as following:
String insert = "INSERT INTO " + tableName +
"(" + COLUMNS.TILE_ID + "," + COLUMNS.TILE_DATA + "," +
COLUMNS.TILE_LEVEL + "," + COLUMNS.TILE_COLUMN +
"," + COLUMNS.TILE_ROW +
"," + COLUMNS.TILE_IMAGE_FORMAT + "," + COLUMNS.TILE_SOURCE +
")";
String values = id + ",?" + "," +
tile.getLevel() + "," + tile.computeColumn() + "," +
tile.computeRow() + ",\'" + tile.getFileType().toUpperCase() +
"\'," + "\'" +
tile.getSource() + "\');";
String query = insert + " VALUES (" + values;
PreparedStatement statement = conn.prepareStatement(query);
statement.setBytes(1, tile.getData());
int result = statement.executeUpdate();
statement.close();
return result;
The fact that the parameter is at position 2 in the values list is not what counts. It's the fact that it's the first place holder that matters.
I am taking one csv comparing each line with every line of another csv to find matches.
I then need to add some elements from the second csv with some from the first and write it to a new file.
It works for the first lines of the csv then gets the ArrayIndexOutOfBoundsException.
I understand how arrays work and I've checked my csv and as far as I can see I'm not going out of bounds.
The first csv has 8 fields and contains all the customer info. The second has 15 fields and holds sales info on customers. the first 2 fields [0] and [1] are the same in both csv's if there is a record of sales.
If anyone could have a quick look I may be missing something stupid.
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 3
at excel.parse.ExcelParse.main(ExcelParse.java:61)
package excel.parse;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.File;
import java.io.IOException;
public class ExcelParse {
/**
* #param args the command line arguments
*/
public static void main(String[] args) {
String csvFile2 = "\\\\SBS2011\\RedirectedFolders\\Josh.Hickinbotham\\My Documents\\Customer_Sales_Trends_Summary_by_Sales_Order_114641390.csv";
String csvFile1 = "\\\\SBS2011\\RedirectedFolders\\Josh.Hickinbotham\\My Documents\\All_Customers_Listing_for_Rep_114337469.csv";
BufferedReader br = null;
BufferedReader br2 = null;
BufferedWriter bw = null;
String line = "";
String line2 = "";
String csvSplitBy = ",";
Boolean match = false;
try {
br = new BufferedReader(new FileReader(csvFile1));
bw = new BufferedWriter(new FileWriter("\\\\SBS2011\\RedirectedFolders\\Josh.Hickinbotham\\My Documents\\newcsv.txt"));
while ((line = br.readLine()) != null) {
// use comma as separator
String[] customer = line.split(csvSplitBy);
System.out.println(customer[1]);
br2 = new BufferedReader(new FileReader(csvFile2));
while ((line2 = br2.readLine()) != null) {
String[] file2 = line2.split(csvSplitBy);
if (customer[1].equals(file2[1])) {
match = true;
bw.write(customer[0] + "," + customer[1] + "," + customer[2] + "," + customer[3] + "," + customer[4] + ","
+ customer[5] + "," + customer[6] + "," + customer[7] + ","+ file2[2] +","+ file2[3] + "," + file2[4] + "," + file2[5] + ","
+ file2[6] + "," + file2[7] + "," + file2[8] + "," + file2[9] + "," + file2[10] + "," + file2[11] + "," + file2[12] + ","
+ file2[13] + "," + file2[14]+"\r\n");
System.out.println(":::MATCH " +customer[1]+" : "+file2[1]+" :::");
} else {
match = false;
}
}
if (match == false) {
bw.write(customer[0] + "," + customer[1] + "," + customer[2] + "," + customer[3] + "," + customer[4] + ","
+ customer[5] + "," + customer[6] + "," + customer[7] + "," + "," + "," + "," + "," + "," + "," + "," + "," + "," + ","
+ "," + ","+"\r\n");
}
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (br != null) {
try {
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
if (br2 != null) {
try {
br2.close();
} catch (IOException e) {
e.printStackTrace();
}
}
if (bw != null) {
try {
bw.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
System.out.println("Done");
}
}
Try displaying what customer is before it goes into the if match == false.
May help to see what exactly you are dealing with. :)
What about a simple check on the bounds of the result of "split" ? if you dump your line at the point where the bounds are not OK, you will see the faulty input data....
I think the problem is in your file containing the csv. Maybe debug it and stop at the line where you get the exception to check what data customer contains.
But what I wanted to note:
I'm not sure your program is doing what it is supposed to do. When it finds a match it will print it out, but then keep going in the inner while loop and so at the end match will be false again and you print out the non-match.
So I think you want to put a break:
if (customer[1].equals(file2[1])) {
match = true;
bw.write(customer[0] + "," + customer[1] + "," + customer[2] + "," + customer[3] + "," + customer[4] + ","
+ customer[5] + "," + customer[6] + "," + customer[7] + ","+ file2[2] +","+ file2[3] + "," + file2[4] + "," + file2[5] + ","
+ file2[6] + "," + file2[7] + "," + file2[8] + "," + file2[9] + "," + file2[10] + "," + file2[11] + "," + file2[12] + ","
+ file2[13] + "," + file2[14]+"\r\n");
break;
}
I want to put what the use puts in for x to be stored in the yourNumbers array, how would I do that? Edit: How would I clean up the println parts at the bottom, using loops.
import java.util.Scanner;
public class array {
public class SS_Un8As1 {
public static void main(String[] args) {
Scanner scan = new Scanner(System.in);
int[] yourNumbers = new int[10];
for (int counter=0; counter < yourNumbers.length; counter++){
System.out.print("Enter your number: ");
yourNumbers[counter] = scan.nextInt();
}
System.out.println("Original numbers: " + yourNumbers[0] + "," + yourNumbers[1] + "," + yourNumbers[2] + "," + yourNumbers[3] + "," + yourNumbers[4] + "," + yourNumbers[5] + "," + yourNumbers[6] + "," + yourNumbers[7] + "," + yourNumbers[8] + "," + yourNumbers[9]);
System.out.println("Original numbers multiplied by five: " + yourNumbers[0]*5 + "," + yourNumbers[1]*5 + "," + yourNumbers[2]*5 + "," + yourNumbers[3]*5 + "," + yourNumbers[4]*5 + "," + yourNumbers[5]*5 + "," + yourNumbers[6]*5 + "," + yourNumbers[7]*5 + "," + yourNumbers[8]*5 + "," + yourNumbers[9]*5);
System.out.println("Original numbers minus the next number: " + (yourNumbers[0]-yourNumbers[1]) + "," + (yourNumbers[1]-yourNumbers[2]) + "," + (yourNumbers[2]-yourNumbers[3]) + "," + (yourNumbers[3]-yourNumbers[4]) + "," + (yourNumbers[4]-yourNumbers[5]) + "," + (yourNumbers[5]-yourNumbers[6]) + "," + (yourNumbers[6]-yourNumbers[7]) + "," + (yourNumbers[7]-yourNumbers[8]) + "," + (yourNumbers[8]-yourNumbers[9]) + "," + (yourNumbers[9]-yourNumbers[0]));
}
}
Like this:
yourNumbers[counter] = x;
The above code is stating: store the value of x in the array yourNumbers in the index (position) counter. Because counter is the iteration variable of a for loop, each time the loop advances the counter the next value of x will be stored in the next available position in the array.
You must make sure that the length of the array is the same as the value of counter. In your code, do this:
int[] yourNumbers = new int[10];
Why? because counter goes from 0 to 9, so the array must be of length 10.
replace this
int x = scan.nextInt();
with
yourNumbers[counter] = scan.nextInt();
also, the condition should be counter<9, or better counter<yourNumbers.length
Simply pass the input into the next element of the array, while inside the for loop:
yourNumbers[counter] = x;
I should also point out, however, that you should validate that your input is actually an integer before assigning x the value. If the value isn't an integer, your program will crash.