Selecting second column (FileReader) - java

I'm looping csv. I have two question:
1) I'm selecting second column by name like
if(tab[1].equals("Col2")
I don't want to put the name of column. I want to select just second column.
2) how to skip first line (header)
Here is sample of code to looping csv:
String csvFile = "C:\\test.csv";
BufferedReader br = null;
String line = "";
String cvsSplitBy = ";";
try{
br = new BufferedReader(new FileReader(csvFile));
while ((line = br.readLine()) != null) {
String[] tab=line.split(cvsSplitBy);
int tmp;
if(tab[1].equals("Col2")){
tmp = Integer.parseInt(tab[2]);
for(int i=0;i<tmp;i++){
// TO DO
}
}
}
}


Better to make use of CSVReader for this, which provides lot of APIs for processing your csv files. Here is a complete working code, ofcourse, without exception handling.
String csvFile = "C:\\test.csv";
CSVReader reader;
String[] nextRow;
char cvsSplitBy = ';';
try {
//Last argument will determine how many lines to skip. 1 means skip header
reader = new CSVReader(new FileReader(csvFile), cvsSplitBy, CSVParser.DEFAULT_QUOTE_CHARACTER, 1);
while ((nextRow = reader.readNext()) != null) {
if(nextRow.length > 2){
//nextRow[1] will always give second column value
int tmp = Integer.parseInt(nextRow[1]);
for (int i = 0; i < tmp; i++) {
// TO DO
}
}
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}


Here is an example using Apache Commons CSV, and its CSVParser.
The first line is considered to be the header and is skipped (withFirstRecordAsHeader()) , the "columns" of each record can be accessed with their index (get(int)) .The indexes are 0-based .
Just adapt the charset and CSVFormat to your needs.
CSVParser parser = null;
try {
parser = CSVParser.parse(new File(csvFile), Charset.forName("UTF-8"),
CSVFormat.RFC4180.withFirstRecordAsHeader());
List<CSVRecord> records = parser.getRecords();
for (CSVRecord record : records) {
int tmp = Integer.parseInt(record.get(1));
for (int i = 0; i < tmp; i++) {
// TO DO
}
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
parser.close();
} catch (IOException e) {
}
}


With univocity-parsers this becomes a piece of cake:
CsvParserSettings parserSettings = new CsvParserSettings(); //many options here, check the tutorial.
parserSettings.setHeaderExtractionEnabled(true); //header is extracted and not part of the result
parserSettings.selectIndexes(1); //select 2nd column (indexes are 0-based)
CsvParser parser = new CsvParser(parserSettings);
List<String[]> allRows = parser.parseAll(csvFile);
Note that this will work even if some of the rows are empty of have just one column, whereas all other solutions posted here will fail unless you handle such situations by yourself.
Not only this involves WAY less code (and complexity), the parser is also ~4 times faster than Commons CSV, and ~3 times faster than OpenCSV.
Disclaimer: I'm the author of this library, it's open-source and free (Apache v2.0 License)

Related

Java ignore a specific symbol after another specific symbol

I have a .csv file. Data is divided by commas and I need to extract information out this file. Thing is if i just write this it works but partially:
String file = "FinalProject/src/Data.csv";
BufferedReader rd = null;
String line = "";
HashSet<String> platforms = new HashSet<String>();
try
{
rd = new BufferedReader(new FileReader(file));
rd.readLine();
while ((line = rd.readLine())!=null)
{
String [] arr = line.split("\"");
var words = new ArrayList<String>();
for(int i =0; i < arr.length;i++)
{
if(i % 2 == 0)
{
words.addAll(Arrays.asList(arr[i].split(",")));
}
else
{
words.add(arr[i]);
}
platforms.add(words.get(2));
}
}
}
catch (Exception e)
{
System.out.println("");
}
finally
{
try
{
rd.close();
}
catch (IOException e)
{
throw new RuntimeException(e);
}
}
When I check the contents of Set and extract the same data out of the database created from this .csv file it shows difference. For example - my set has 38 values, when the database has 40, all of them are unique( nothing is repeated). I think the problem is caused by separation of data in .csv file with comma signs. Because some of these signs are inside of quotes and this probably causes a loss of the potential values that i need. Is there any solution to that problem? Or perhaps there is a more efficient way to deal with the comma sings inside of the quotes so that they are ignored?

How to remove special characters from SQLite DB while importing from a CSV file in Android Studio?

I have a CSV file named barcode, I have successfully imported that file into my SQLite DB but the problem is the data is saving as it is as what we have in CSV file.
The data im getting: 8.90103E+12, The data i wanted: 8901030382253.
Example:
ITEM - EAN_CODE
100047253 - 8.90103E+12
100047252 - 8.90103E+12
I have two columns, the data in Ean_code is not getting proper into my db.
I have used the trim function but the o/p remains the same.
My code:
if (exportDir.exists()) {
FileReader file = null;
try {
file = new FileReader(exportDir);
BufferedReader buffer = new BufferedReader(file);
String line = "";
int iteration = 0;
ArrayList<MasterDataModel2> arrayList_stock2 = new ArrayList<>();
while ((line = buffer.readLine()) != null) {
if (iteration == 0) {
iteration++;
continue;
}
//StringBuilder sb = new StringBuilder();
String[] str = line.split(",");
arrayList_stock2.add(new MasterDataModel2(str[0].replace("\"", ""), str[1].replace("\"", "")));
//arrayList_stock2.add(new MasterDataModel2(str[0],str[1]));
Log.d("insertTotal", "Msg:" + lastId);
}
Log.e("size",String.valueOf(arrayList_stock2.size()));
db.addAllMasterData2(arrayList_stock2);
} catch (FileNotFoundException e) {
e.printStackTrace();
//CallingImportantMethod.showToast(this, "File is not available");
} catch (IOException e) {
e.printStackTrace();
//CallingImportantMethod.showToast(this, "Something wents wrong");
}
}

This below code is incorrect.
arrayList_stock2.add(new MasterDataModel2(str[0].replace("\"", ""), str[1].replace("\"", "")));
\ is special escape character, so you need \\ to escape it. Even though unrelated to the original problem suggest you to read this to have a clear idea about usage of backslashes in java.
So back to the problem, i guess you really don't need to call the str[].replace(). Correct me if i am wrong.
arrayList_stock2.add(new MasterDataModel2(str[0].trim(), str[1].trim()));
I think above code segment will be enough.
Edit: for your actual problem you can use below. To avoid null issues you may need to check that also.
if(str[1] != null) {
Double eanCodeDouble = Double.parseDouble(str[1]);
NumberFormat nf = NumberFormat.getInstance();
String eanCodeString = nf.format(eanCodeDouble);
System.out.println(resultedString);
} else {
String eanCodeString = '';
}
if(str[0] == null) {
str[0] = '';
}
arrayList_stock2.add(new MasterDataModel2(str[0].trim(), eanCodeString));

Retrieve data from txt file and replace the data into a new txt file using java

I am trying to read in a text file and then manipulate a little and update the records into a new text file.
Here is what I have so far:
ArrayList<String> linesList = new ArrayList<>();
BufferedReader br;
String empid, email;
String[] data;
try {
String line;
br = new BufferedReader(new FileReader("file.txt"));
while ((line = br.readLine()) !=null) {
linesList.add(line);
}
br.close();
}
catch (IOException e) { e.printStackTrace(); }
for (int i = 0; i < linesList.size(); i++) {
data = linesList.get(i).split(",");
empid = data[0];
ccode = data[3];
}
File tempFile = new File("File2.txt");
BufferedWriter bw = new BufferedWriter(new FileWriter(tempFile));
for (int i = 0; i < linesList.size(); i++) {
if(i==0){
bw.write(linesList.get(i));
bw.newLine();
}
else{
data = linesList.get(i).split(",");
String empid1 = data[0];
if(data[13].equals("IND")) {
String replace = data[3].replaceAll("IND", "IN");
ccode1 = replace;
System.out.println(ccode1);
}
else if(data[13].equals("USA")) {
String replace = data[3].replaceAll("USA", "US");
ccode1 = replace;
}
else {
ccode1 = replace; //This does not work as replace is not defined here, but how can I get it to work here.
}
String newData=empid1+","+ccode1;
bw.write(newData);
bw.newLine();
}
}
Here is what is inside the text file:
EID,First,Last,Country
1,John,Smith,USA
2,Jane,Smith,IND
3,John,Adams,USA
So, what I need help with is editing the three letter country code and replacing it with a 2 letter country code. For example: USA would become US, and IND would become IN. I am able to read in the country code, but am having trouble in changing the value and then replacing the changed value back into a different text file. Any help is appreciated. Thanks in advance.

Open file in text editor, Search and Replace, ,USA with ,US, ,IND with ,IN and so on.
As such, to automate it, on the same while loop you read a line do:
//while(read){ line.replaceAll(",USA",",US");
That will be the easiest way to complete your objective.
To save, open a BufferedWriter bw; just like you opened a reader and use bw.write(). You would probably prefer to open both at the same time, the reader on your source file, and the writer on a new file, with _out suffix. That way you dont need to keep the file data in memory, you can read and write as you loop.
For harder ways, read the csv specs: https://www.rfc-editor.org/rfc/rfc4180#section-2
Notice that you have to account for the possibility of fields being enclosed in quotes, like: "1","John","Smith","USA", which means you also have to replace ,\"USA with ,\"US.
The delimiter may or may not be a comma, you have to make sure yur input will always use the same delimiter, or that you can detect and switch at runtime.
You have to account for the case where a delimiter may be part of a field, or where quotes are part of a field.
Now you know/can solve these issues you can, instead of using replace, parse the lines character by character using while( (/*int*/ c = br.read()) != -1), and do this replacement manually with an if gate.
/*while(read)*/
if( c == delimiter ){
if not field, start next field, else add to field value
} else if( c == quote ){
if field value empty, ignore and expect closing quote, else if quote escape not marked, mark it, else, add quote to field value
}
(...)
} else if( c == 13 or c == 10 ){
finished line, check last field of row read and replace data
}
To make it better/harder, define a parsing state machine, put the states into an Enum, and write the if gates with them in mind (this will make your code be more like a compiler parser).
You can find parsing code at different stages here: https://www.mkyong.com/java/how-to-read-and-parse-csv-file-in-java/

You need to change a little bit in your concept. If you want to edit a file then,
create a new file and write content in new file and delete old file and rename new file
with old name.
ArrayList<String> linesList = new ArrayList<>();
BufferedReader br;
String[] data;
File original=new File("D:\\abc\\file.txt");
try {
String line;
br = new BufferedReader(new FileReader(original));
while ((line = br.readLine()) !=null) {
linesList.add(line);
}
br.close();
}
catch (IOException e) { e.printStackTrace(); }
File tempFile = new File("D:\\abc\\tempfile.txt");
BufferedWriter bw = new BufferedWriter(new FileWriter(tempFile));
for (int i = 0; i < linesList.size(); i++) {
if(i==0){
bw.write(linesList.get(i));
bw.newLine();
}
else{
data = linesList.get(i).split(",");
String empid = data[0];
String name=data[1];
String lname=data[2];
String ccode = data[3].substring(0, 2);
String newData=empid+","+name+","+lname+","+ccode+"\n";
bw.write(newData);
bw.newLine();
}
}
bw.close();
if (!original.delete()) {
System.out.println("Could not delete file");
return;
}
// Rename the new file to the filename the original file had.
if (!tempFile.renameTo(original))
System.out.println("Could not rename file");

Split file into multiple files

I want to cut a text file.
I want to cut the file 50 lines by 50 lines.
For example, If the file is 1010 lines, I would recover 21 files.
I know how to count the number of files, the number of lines but as soon as I write, it's doesn't work.
I use the Camel Simple (Talend) but it's Java code.
private void ExtractOrderFromBAC02(ProducerTemplate producerTemplate, InputStream content, String endpoint, String fileName, HashMap<String, Object> headers){
ArrayList<String> list = new ArrayList<String>();
BufferedReader br = new BufferedReader(new InputStreamReader(content));
String line;
long numSplits = 50;
int sourcesize=0;
int nof=0;
int number = 800;
try {
while((line = br.readLine()) != null){
sourcesize++;
list.add(line);
}
System.out.println("Lines in the file: " + sourcesize);
double numberFiles = (sourcesize/numSplits);
int numberFiles1=(int)numberFiles;
if(sourcesize<=50) {
nof=1;
}
else {
nof=numberFiles1+1;
}
System.out.println("No. of files to be generated :"+nof);
for (int j=1;j<=nof;j++) {
number++;
String Filename = ""+ number;
System.out.println(Filename);
StringBuilder builder = new StringBuilder();
for (String value : list) {
builder.append("/n"+value);
}
producerTemplate.sendBodyAndHeader(endpoint, builder.toString(), "CamelFileName",Filename);
}
}
} catch (IOException e) {
e.printStackTrace();
}
finally{
try {
if(br != null)br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
For people who don't know Camel, this line is used to send the file:
producerTemplate.sendBodyAndHeader (endpoint, line.toString (), "CamelFileName" Filename);
endpoint ==> Destination (it's ok with another code)
line.toString () ==> Values
And then the file name (it's ok with another code)

you count the lines first
while((line = br.readLine()) != null){
sourcesize++; }
and then you're at the end of the file: you read nothing
for (int i=1;i<=numSplits;i++) {
while((line = br.readLine()) != null){
You have to seek back to the start of the file before reading again.
But that's a waste of time & power because you'll read the file twice
It's better to read the file once and for all, put it in a List<String> (resizable), and proceed with your split using the lines stored in memory.
EDIT: seems that you followed my advice and stumbled on the next issue. You should have maybe asked another question, well... this creates a buffer with all the lines.
for (String value : list) {
builder.append("/n"+value);
}
You have to use indexes on the list to build small files.
for (int k=0;k<numSplits;k++) {
builder.append("/n"+list[current_line++]);
current_line being the global line counter in your file. That way you create files of 50 different lines each time :)

reading CSV file till particular column numbers

I am reading csv file method as under -
public ArrayList<String> fileRead(File f) throws IOException {
FileReader fr = new FileReader(f);
BufferedReader br = new BufferedReader(fr);
ArrayList<String> CSVData = new ArrayList<String>();
String text;
try {
while ((text = br.readLine()) != null) {
CSVData.add(text);
log.debug(text);
}
log.info(f + ": Read successfully");
br.close();
fr.close();
} catch (IOException e) {
log.error("Error in reading file " + e.getMessage());
}
return CSVData;
}
but I want to read file till defined column number e.g. till 20th column,
but if in between I will found empty cell for some column then as above code it will exit on (text = br.readLine()) != null ,
so finally my question is how to read CSV file till particular columns either its empty cell or whatever it should read till those column and break point for moving next line should be that column example 20th column ,
Thank in advance for help and support

You should use uniVocity-parsers' column selection feature to process your file.
Here's an example:
CsvParserSettings settings = new CsvParserSettings();
parserSettings.selectFields("Foo", "Bar", "Blah");
// or if your file does not have a row with column headers, you can use indexes:
parserSettings.selectIndexes(4, 20, 2);
CsvParser parser = new CsvParser(settings);
List<String[]> allRows = parser.parseAll(new FileReader(f));
Disclosure: I am the author of this library. It's open-source and free (Apache V2.0 license).
Do NOT try to parse a CSV by yourself. There are many intricacies there. For example, using code such as the one you pasted:
while ((text = br.readLine()) != null) {
Will break as soon as your CSV has values that contain newline characters.

Categories