We Keep Coding

How to change convert different csv files into standard format?

I have 2-3 .csv files with fields like Date, Amount, Transaction Description etc and all the csv files contains these fields but in shuffled order. I want a output file to with a standard order (like if I input the sample .csv file, then I can get the things in order in output file).
I tried to do it for one file by taking substrings from the .csv file (at that time I didn't know that other files have shuffled order fields).
I am kind of new, tell me if I am asking question in a good format!
Can I put a link for the sample input and output .csv file for the reference?
--> https://drive.google.com/drive/folders/1-NZi5OTMTbOWXAfCTsc-ahNYm1N5DG2g (just because it would be very hard to explain that how file looks like)
What I have done?
I have just tried to extract data from the fields using the BufferReader using split but it can only work for one type of file, I cant have a standard format using this!
Sorry for posting such a long code but what I have done is selected field from the file and copied them into output file corresponding to the standard fields in the output file.
Suggest me if there is any other method with which I can proceed.
File file = new File("C:\\Users\\R\\Desktop\\CSVDemo.csv");
try {
// create FileWriter object with file as parameter
FileWriter outputfile = new FileWriter(file);
CSVWriter writer = new CSVWriter(outputfile, ',',
CSVWriter.NO_QUOTE_CHARACTER,
CSVWriter.DEFAULT_ESCAPE_CHARACTER,
CSVWriter.DEFAULT_LINE_END);
// create a List which contains String array
String[] header = { "Date", "Transaction Description", "Debit","Credit","Currency","CardName","Transaction","Location" };
writer.writeNext(header);
String splitBy = ",";
BufferedReader br = new BufferedReader(new FileReader("G:\\US\\HDFC-Input-Case1.csv"));
String line;
String transaction = "",name = "";
while ((line = br.readLine()) != null) {
// use comma as separator
String[] cols = line.split(splitBy);
if(cols.length == 2 && cols[1].equals("Domestic Transactions")) {
transaction = "Domestic";
continue;
}
else if(cols.length == 2 && cols[1].equals("International Transactions")) {
transaction = "International";
continue;
}
else if(cols.length == 2) {
name = cols[1];
continue;
}
else if(cols.length<1){
continue;
}
else if(cols.length>2) {
if(cols[0].contains("Date")){
continue;
}
String[] data1 = new String[header.length];
data1[0] = cols[0];
String curr ;
if(cols[1].substring(cols[1].length()-3).equals("USD") || cols[1].substring(cols[1].length()-3).equals("EUR")) {
data1[4] = cols[1].substring(cols[1].length()-3);
curr = cols[1].substring(0,cols[1].length()-4);
data1[1] = curr;
}
else {
data1[4] = "INR";
data1[1] = cols[1];
}
if(cols[2].contains("cr")){
data1[3] = cols[2].substring(0,cols[2].length()-2);
data1[2] = "0";
}
else {
data1[2] = cols[2];
data1[3] = "0";
}
data1[5] = name;
data1[6] = transaction;
writer.writeNext(data1);
}
System.out.println();
}
// closing writer connection
writer.close();
}

You can read the header of your input csv files first and find the indexes of required field in this given csv file.
Once you have required indexes for every header, read those fields using indexes in the standard order you want for your output csv file.
sample codes:
`CSVReader reader = new CSVReader(new FileReader(fileName ));
String[] header = reader.readNext();
List<String> list= Arrays.asList(header);
int indexOfFieldTransaction=list.indexOf("transaction");`
Now make a List and insert the field in order you want to write in output file.you will get -1 if the field you are trying to get index of is not present in the input file.

Parsing txt file which has delimiters as part of column value using opencsv reader

I have a tab delimited text file which I want to parse using openscsv and upload to a database. I used CSVReader() to parse the file. The problem is, some column values have tabs within. For instance, a column ends with a tab, and then it has another tab which is used for separating it from the next column.
I'm having trouble in parsing this file. How do I avoid delimiters which are as part of the value?
This is the file I'm trying to parse. Each line has 2 columns and there are 5 rows in total. The first row is the header. However, when I parse it using the following code, I get only 3 rows:
CSVReader reader = new CSVReader(new FileReader("input.txt"), '\t');
String[] nextLine;
int cnt = 0;
while ((nextLine = reader.readNext()) != null) {
if (nextLine != null) {
cnt++;
System.out.println("Length of row "+cnt+" = "+nextLine.length);
System.out.println(Arrays.toString(nextLine));
}
}
******** Update ********
Doing a normal readline such as below prints 5 lines:
BufferedReader br = new BufferedReader(new FileReader("input.txt"));
int lines = 0;
while(br.readLine() != null){
lines++;
}
System.out.println(lines);

Put quotes on your data - here is a modified unit test from CSVReaderTest that shows quotes will work:
#Test
public void testSkippingLinesWithDifferentEscape() throws IOException
{
StringBuilder sb = new StringBuilder(CSVParser.INITIAL_READ_SIZE);
sb.append("Skip this line?t with tab").append("\n"); // should skip this
sb.append("And this line too").append("\n"); // and this
sb.append("a\t'b\tb\tb'\t'c'").append("\n"); // single quoted elements
CSVReader c = new CSVReader(new StringReader(sb.toString()), '\t', '\'', '?', 2);
String[] nextLine = c.readNext();
assertEquals(3, nextLine.length);
assertEquals("a", nextLine[0]);
assertEquals("b\tb\tb", nextLine[1]);
assertEquals("c", nextLine[2]);
}
If that does not work please post some of the lines from your input.txt. When I click on the link it takes me to some website trying to sell me a dropbox clone.

How to read a ;-separated CSV in Java that can countain an unknown number of elements

I know there exist a lot questions about reading CSV files, but I simply can't find one that fits my needs.
I try to get keywords from a keywords.csv that can be in a form like this. The delimeter is always the ";".
SAP;BI; Business Intelligence;
ERP;
SOA;
SomethingElse;
I already looked into openCSV and so on, but I can't find a functioning example how to do that (simple) task.
I tried this:
public void getKeywords()
{
try {
int rowCount = 0;
CSVReader reader = new CSVReader(new FileReader(csvFilename), ';');
String[] row = null;
while((row = reader.readNext()) != null) {
System.out.println(row[rowCount]);
rowCount++;
}
//...
reader.close();
}
catch (IOException e) {
System.out.println("File Read Error");
}
But it will just return the first element. I don't know what I do wrong. Im new to coding as you may have noticed :)
EDIT: Got what I wanted, thanks for your help!
while((row = reader.readNext()) != null) {
for (int i=0; i< row.length; i++ )
{
System.out.println(row[i]);
}
Please help an old man out.
Thank you!

Using openCSV, you could use this code:
CSVReader reader = new CSVReader(new FileReader("yourfile.csv"), ';');
That will open the .csv file, read it in, and use a ; as the delimiter. A similar example can be found on the openCSV home page.
Once you have the file read in, you can use the data with something like the following:
String [] nextLine;
// Read from the csv sequentially until all the lines have been read.
while ((nextLine = reader.readNext()) != null) {
// nextLine[] is an array of values from the line
System.out.println(nextLine[0] + nextLine[1] + "etc...");
}
Where nextLine is a line from the file, and nextLine[0] will be the first element of the line, nextLine[1] will be the second, etc.
Edit:
In your comment below, you mentioned that you don't know how many elements will be in each row. You can handle that by using nextLine.length and figuring out how many elements are in that row.
For example, change the above code to something like:
String [] nextLine;
while ((nextLine = reader.readNext()) != null) {
if(nextLine.length == 1) {
// Do something with the first element, nextLine[0]
System.out.println(nextLine[0]);
}
else if(nextLine.length == 2) {
// Do something with both nextLine[0] and nextLine[1]
System.out.println(nextLine[0] + ", " + nextLine[1]);
}
// Continue depending on how you want to handle the different rows.
}

You can read the file using the readLine() method from the Scanner class. The output of this method is one line of the input file. You can then use the String.split(";") method to get the individual elements. You can then move to the next line using the methods in the Scanner class and then continue from thereon.
You will get a number of arrays - one corresponding to each line from the input file. You can just combine them to get what you want.

getting the content of a .csv using open csv

How to get and display some lines of a csv using openCSV.
I currently have the following code :
CSVReader reader1 = new CSVReader(new FileReader(mydata_csv.getpath()));
List myDatas = reader1.readAll();
How to display one specific line ?
Maybe can I use a better way to store my datas (the csv contains lines of hundreds variables). any suggestion would be welcome.

The documentation for opencsv http://opencsv.sourceforge.net/#how-to-read seems to say that your code returns a list of String[]
in which case I would write it like so:
CSVReader reader1 = new CSVReader(new FileReader(mydata_csv.getpath()));
List<String[]> myDatas = reader1.readAll();
String[] lineI = myDatas.get(i);
for (String[] line : myDatas) {
for (String value : line) {
//do stuff with value
}
}

You should use the following code:
CSVReader reader = new CSVReader(new FileReader(mydata_csv.getpath()));
String [] nextLine;
while ((nextLine = reader.readNext()) != null) {
// nextLine[] is an array of values from the line
System.out.println(nextLine[0] + nextLine[1] + "etc...");
}

How to read a String (file) to array in java

Suppose there is a file named as SUN.txt
File contains : a,b,dd,ss,
I want to make dynamic array depending upon the number of attributes in file.
If ther is a char after comma then array will be of 0-4 i.e of length 5.
In the above mentioned case there is no Char which returns 0-3 Array of length 4. I want to read the NULL after comma too.
How do i do that?
Sundhas

You should think about
Reading the file into a String
Splitting the file by separator ','
Using a list for adding the characters and convert the list to an array, when the list is filled

As Markus said, you want to do something like this..
//Create a buffred reader so that you can read in the file
BufferedReader reader = new BufferedReader(new FileReader(new File(
"\\SUN.txt")));
//The StringBuffer will be used to create a string if your file has multiple lines
StringBuffer sb = new StringBuffer();
String line;
while((line = reader.readLine())!= null)
{
sb.append(line);
}
//We now split the line on the "," to get a string array of the values
String [] store = sb.toString().split(",");
I do not quite understand why you would want the NULL after the comma? I am assuming that you mean after the last comma you would like that to be null in your array? I do not quite see the point in that but that is not what the question is.
If that is the case you wont read in a NULL, if after the comma there was a space, you could read that in.
If you would like a NULL you would have to add it in yourself at the end so you could do something like
//Create a buffred reader so that you can read in the file
BufferedReader reader = new BufferedReader(new FileReader(new File(
"\\SUN.txt")));
//Use an arraylist to store the values including nulls
ArrayList<String> store = new ArrayList<String>();
String line;
while((line = reader.readLine())!= null)
{
String [] splitLine = line.split(",");
for(String x : splitLine)
{
store.add(line);
}
//This tests to see if the last character of the line is , and will add a null into the array list
if(line.endsWith(","))
store.add(null);
}
String [] storeWithNull = store.toArray();

Well if you want want to simply open the file and store the content in a array of string then
1) open the file into a string
2) split the string using a regex "," http://download.oracle.com/javase/1.5.0/docs/api/java/lang/String.html#split(java.lang.String)
but I'm curious why you can't use a String file directly ?

For your datatructure, use a list of arrays. Each list entry is a line of your textfile, each entry is an array that holds the comma separated values:
List<String[]> data = new ArrayList<String[]>();
String line = readNextLine(); // custom method, to be implemented
while (line != null) {
data.add(line.split(","));
line = readNextLine();
}
(assuming, your file contains 1..n lines of comma separated values)
You may want to have it like this:
"a,b,c,d," -> {"a", "b", "c", "d", null}
Here's a suggestion how to solve that problem:
List<String[]> data = new ArrayList<String[]>();
String line = readNextLine(); // custom method, to be implemented
while (line != null) {
String[] values = new String[5];
String[] pieces = line.split(",");
for (int i = 0; i<pieces.length; i++)
values[i] = pieces[i];
data.add(values);
line = readNextLine();
}

its seems like a CSV file something like this will work assuming it has 5 lines and 5 values
String [][] value = new String [5][5];
File file = new File("SUN.txt");
BufferedReader br = new BufferedReader(new FileReader(file));
String line = null;
int row = 0;
int col = 0;
while((line = br.readLine()) != null ){
StringTokenizer s = new StringTokenizer(line,",");
while (s.hasMoreTokens()){
value[row][col] = s.nextToken();
col++;
}
col = 0;
row++;
}
i havent tested this code

Read the file, using BufferedReader, one line at the time.
Use split(",", -1) to convert to an array of String[] including also empty strings beyond the last comma as part of your array.
Load the String[] parts into a List.