Reading Line of String in Text File is Not Consistent - java

Hi StackOverFlow People,
I have this issue in my development of System, where I have 4451 lines of record in a text file, and I am retrieving it using BufferedReader and split every line by pipe ( | ). I'm using Quartz also to run this reading of file every day. when I test it, I set the quartz job every minute so I can test It if it actually reading the file in every minute. It reads all of the line in the text file by checking it using this.
BufferedReader reader = new BufferedReader((newInputStreamReader(inputStream));
String line = null;
int counter = 0;
while((line = reader.readLine()) != null){
counter++;
}
System.out.println(counter);
But when I split the String, The result of retrieving 4451 records is inconsistent. Sometimes, It only retrieves 1000+ to 2000+ records, and Sometime it retrieves 4451, but not consistently. This is my code.
try {
BufferedReader reader = new BufferedReader((newInputStreamReader(inputStream));
String line = null;
int counter = 0;
String[] splitLine = null;
while((line = reader.readLine()) != null){
splitLine = line.split("\\|"); // Splitting the line using '|' Delimiter
for(String temp : splitLine) {
System.out.println(temp);
}
counter++;
}
System.out.println(counter);
} catch (IOException e) {
e.printStackTrace();
}
Is the splitting of String and Iterating of the readfile at the same time could be the cause?
EDIT:
There's no Exception Occured in the Situation. It Only print the length of by using the counter variable.
My Expected Output is I want to Retrieve all the records per line in the text file and split the string per line by pipe. counter is the count of lines retrieved.

I didn't find any error in your code but the code that I have written is working perfectly fine. Here is the code
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
class Test {
public static void main(String[] args) {
FileReader inputStream = null;
BufferedReader reader = null;
try {
inputStream = new FileReader("Input.txt");
reader = new BufferedReader(inputStream);
String line = null;
int counter = 0;
String[] splitLine = null;
while ((line = reader.readLine()) != null) {
splitLine = line.split("\\|");
for (String temp : splitLine) {
System.out.println(temp);
}
counter++;
}
System.out.println(counter);
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}

Shouldn't pipe delimiter be just "|" instead of "\\|"?
Try Changing your code to:
splitLine = line.split("|"); // Splitting the line using '|' Delimiter

Related

Java - Reading from csv file getting null value

I am getting a null value when im reading from my teachers. csv file. Column 1 which is att[0] works but att[1] returns 3 null values.
My csv looks like this:
1, Mr Murphy
2, Mr Davis
3, Ms Simpson
Each on separate lines ie line 1 -> 1, Mr Murphy etc
Here is my code:
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
public class ReadCSV
{
public static void main(String[] args)
{
//Input file which needs to be parsed
String readTeachers = "teacher.csv";
BufferedReader fileReader = null;
//Delimiter used in CSV file
final String DELIMITER = ",";
try
{
String line = "";
//String line = inFile.readLine();
//Create the file reader
fileReader = new BufferedReader(new FileReader(readTeachers));
int count=0;
String[] att = new String[10];
//Read the file line by line
while ((line = fileReader.readLine()) != null)
{
//Get all tokens available in line
String[] tokens = line.split(DELIMITER);
int i=0;
count++;
for(String token : tokens)
{
att[i] = token;
i++;
//Print all tokens
// System.out.println(token);
System.out.println(att[1]);
break;
}
}
//System.out.println(count);
//System.out.println(att[1]);
}
catch (Exception e) {
e.printStackTrace();
}
finally
{
try {
fileReader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
Your issue here is that you have a break statement inside the for loop that exits the loop at the end of the first iteration. Therefore, you are only putting a value in the first index of your array. Take out that statement and it should be fine.
for(String token : tokens)
{
att[i] = token;
i++;
//Print all tokens
// System.out.println(token);
System.out.println(att[1]);
break; // <---- ***take this out***
}

Using tokenizer to count number of words in a file?

I am making a program that makes an user choose a file then the program reads from the file. Now I've been told to make the program using bufferedreader and string tokenizer. So far I got program opening the file and counting the number of lines. But the number of words is not so easy.
This is my code so far:
int getWords() throws IOException
{
int count = 0;
BufferedReader BF = new BufferedReader(new FileReader(f));
try {
StringTokenizer words = new StringTokenizer(BF.readLine());
while(words.hasMoreTokens())
{
count++;
words.nextToken();
}
BF.close();
} catch(FileNotFoundException e) {
}
return count;
}
Buffered reader can only read a line at a time but I don't know how to make it read more lines.
to count words you can use countTokens() instead of loop
to read all lines use
String line = null;
while(null != (line = BF.readLine())) {
StringTokenizer words = new StringTokenizer(line);
words.countTokens();//use this value as number of words in line
}
As you said, buffered reader will read one line at a time. So you have to read lines until there are no more lines. readLine() returns null when the end of file is reached.
So do something like this
int getWords() throws IOException {
int count = 0;
BufferedReader BF = new BufferedReader(new FileReader(f));
String line;
try {
while ((line = BF.readLine()) != null) {
StringTokenizer words = new StringTokenizer(line);
while(words.hasMoreTokens()) {
count++;
words.nextToken();
}
}
return count;
} catch(FileNotFoundException e) {
} finally {
BF.close();
}
// Either rethrow the exception or return an error code like -1.
}

Importing data from a large file into a 2d Array

I am trying to import a large data file and insert the information into a 2D array. The file has around 19,000 lines, and consists of 5 columns. My code is absolutely correct, there are no run time errors nor exceptions. Though, the problem is that when I try to print out data[15000][0], it says null. but my line does have 15,000 lines and it should print out the element inside the array. But when I print out data[5000][0], it works. What could possibly be wrong? I have 19,000 cities in 19,000 different lines, but it seems like when It goes around 10,000+ nothing gets stored in the 2d array. Help please
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.FileNotFoundException;
import java.io.IOException;
public class Data1
{
public static void main(String[] args)
{
try{
FileReader file = new FileReader("/Users/admin/Desktop/population.csv");
BufferedReader in = new BufferedReader(file);
String title = in.readLine();
String[][] data = new String[20000][5];
int currentRow = 0;
String current;
int i = 0;
String temp;
while ((temp = in.readLine()) !=null)
{
String[] c = new String[5];
String line = in.readLine().replaceAll("\"", ""); //changing the format of the data input
c = line.split(",");
c[1] = c[1].replace(" ", "");
for (int j = 0; j <data[0].length; j++)
{
current = c[j];
data[i][j] = c[j];
}
i++;
}
System.out.println(data[15000][0]);
}
catch (FileNotFoundException ex)
{
ex.printStackTrace();
}
catch (IOException ex)
{
ex.printStackTrace();
}
catch (Exception ex)
{
ex.printStackTrace();
}
}
}
You're throwing away a line on each loop.
while (in.readLine() != null)
should be
String temp;
while ((temp = in.readLine()) != null)
And then no calls to .readLine() inside the loop but refer to "temp".
Read line only once...
String line=null;
while ((line=in.readLine()) !=null) // reading line once here
{
String[] c = new String[5];
line = line.replaceAll("\"", ""); //
c = line.split(",");
c[1] = c[1].replace(" ", "");
One of your errors are the loops
while (in.readLine() !=null)
{
String[] c = new String[5];
String line = in.readLine().replaceAll("\"", ""); //changing the format of the data input
c = line.split(",");
c[1] = c[1].replace(" ", "");
Each time you invoke in.readLine() it reads a line,so you are skipping one line each time since you are calling readline twice(thus reading two lines) but storing only the second line.
You should replace it with.
String line=in.readLine();
while (line !=null)
{
String[] c = new String[5];
line.replaceAll("\"", ""); //changing the format of the data input
c = line.split(",");
c[1] = c[1].replace(" ", "");
//whatever code you have
//last line of the loop
line=in.readLine();
Can you provide us with a couple of lines of your file? And are you sure that all the file is formatted correctly ?

Problems reading a CSV file in Java. Only the first line is read

For a Java homework assignment, I need to create a class that reads and writes CSV files. I'm currently having some problems reading the the CSV. The code below, only outputs the first line of the code and then generates the following error message: 'Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 1 at com.gc01.FileManager.CSVManager.main(CSVManager.java:27)".
I have looked at various examples, and I am aware of the 'opencsv' package, but I need to write this code myself. I have located the problem to the statement "System.out.print(data[i]);". However, when cross-referencing this code it all seems to be fine.
I am using the methods from the FileInput class, as specified by my teacher (http://www.devjavasoft.org/SecondEdition/SourceCode/Share/FileInput.java).
public class CSVManager {
public static void main(String [] args){
Scanner sc = new Scanner (System.in);
System.out.println("Please enter the file directory of the chosen CSV");
System.out.println("For Example: /Users/UserName/Downloads/FileName.csv");
///Users/ReeceAkhtar/Desktop/GeoIPCountryWhois.csv
final String fileName = sc.nextLine();
System.out.println("How many columns?");
final int columns = sc.nextInt();
BufferedReader br = null;
String line = "";
String splitBy = " , ";
try {
br = new BufferedReader(new FileReader(fileName));
while((line = br.readLine()) != null) {
for (int i = 0; i < columns; i++) {
String[] data = line.split(splitBy);
System.out.print(data[i]);
}
}
} catch (FileNotFoundException e){
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (br != null) {
try {
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
System.out.println("File Read");
}
}
Exception is very clear
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 1
means, you are trying to access 1st element in the array which doesn't exist
Since you are saying System.out.print(data[i]); is the line where the exception is occurring, then for the first line data must have populated with only single element
Debug the issue with IDE to find out why split method is resulting unexpected elements. I suspect usage of spaces around , is the cause in " , "
Try this one. If you take splitting out the for loop everything will be okay.
String[] data = line.split(splitBy);
while((line = br.readLine()) != null){
for (int i = 0; i < columns; i++){
System.out.print(data[i]);
}
}
while((line = br.readLine()) != null){
for (int i = 0; i < columns; i++){
String[] data = line.split(splitBy);
System.out.print(data[i]);
}
You are splitting one line multiple times inside the for loop without any reason.
You are using " , " for splitting (which might be the reason you are having ArrayIndexOfBound exception) Instead use ","; use trim() on data[i] to get rid of trailing/leading white space if you wish to.
After Splitting, put checking whither data.length is equal to columns for consistency.
We are now in the era of JDK 7 where we can use try-with-resource which close the declared resource inside try(){} context, allowing us to get rid of finally block
So your could should look like as follows:
try (BufferedReader br = new BufferedReader(new FileReader(fileName))){
while((line = br.readLine()) != null){
String[] data = line.split(splitBy);
if(data.length != columns)continue; // check for consistency,
//might throw an exception
for (int i = 0; i < columns; i++){
System.out.print(data[i].trim());
}
}catch(IoExection ex)
{
ex.printStackTrace();
}

Parsing Data from CSV to Array in Java

I'm trying to import a CSV file into an array that I can use within a Java program. The CSV file has successfully imported itself and the output appears on Terminal but it throws the error:
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 1
at CompareCSV.main(CompareCSV.java:19)
at the end. In addition, when I try to call up elements in the array, it also shows the same error. My code is below:
import java.io.*;
import java.util.*;
public class CompareCSV {
public static void main(String[] args) {
String fileName = "sampledata1.csv";
try {
BufferedReader br = new BufferedReader( new FileReader(fileName));
String strLine = null;
StringTokenizer st = null;
int lineNumber = 0, tokenNumber = 0;
while((fileName = br.readLine()) != null) {
lineNumber++;
String[] result = fileName.split(",");
for (int x=0; x<result.length; x++) {
System.out.println(result[x]);
}
}
}
catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
You are much better off using a proper CSV parser than hacking a faulty one up yourself: http://opencsv.sourceforge.net/
CSV is not the simple format one might be let to think (yes, a line can contain a , that does not separate two pieces of data).
This is the answer for above Question
public class Readline {
/**
* #param args
*/
public static void main(String[] args) {
String fileName = "C:/Users/karthikrao/Desktop/cvsFile.csv";
ArrayList<Integer> margins = new ArrayList<Integer>();
BufferedReader br;
String line, token;
int i;
try {
br = new BufferedReader(new FileReader(fileName));
try {
while ((line = br.readLine()) != null) {
StringTokenizer st = new StringTokenizer(line, ",\"");
i = 0;
while (st.hasMoreTokens()) {
token = st.nextToken();
if (margins.size() <= i) {
margins.add((Integer) token.length());
} else {
margins.set(
i,
Math.max(margins.get(i),
(Integer) token.length()));
}
i++;
}
}
br = new BufferedReader(new FileReader(fileName));
while ((line = br.readLine()) != null) {
StringTokenizer st = new StringTokenizer(line, ",\"");
i = 0;
while (st.hasMoreTokens()) {
token = st.nextToken();
System.out.print(token);
for (int j = 0; j < margins.get(i) - token.length(); j++) {
System.out.print(" ");
}
System.out.print("|");
i++;
}
System.out.println();
}
} catch (IOException e) {
e.printStackTrace();
}
} catch (FileNotFoundException e) {
e.printStackTrace();
}
}
}
I suggest you not re-inventing wheel when there are so many great libraries out there. Try the uniVocity-parsers with the following code snippt as reference:
public static void main(String[] args) throws FileNotFoundException {
/**
* ---------------------------------------
* Read CSV rows into 2-dimensional array
* ---------------------------------------
*/
// 1st, creates a CSV parser with the configs
CsvParser parser = new CsvParser(new CsvParserSettings());
// 2nd, parses all rows from the CSV file into a 2-dimensional array
List<String[]> resolvedData = parser.parseAll(new FileReader("/examples/example.csv"));
// 3rd, process the 2-dimensional array with business logic
// ......
}
As you can see, only 2 lines required to finish the task of parsing csv data into array. Additionally, the library provides full list of features in parsing CSV data with excellent performance.
Looks like your assumption, that a line in the file always has three columns isn't true for all lines. Replace the for loop statement with the following line to eliminate the exception and see, why it happend:
for (int x=0; x<result.length; x++)

Categories