I am trying to import a large data file and insert the information into a 2D array. The file has around 19,000 lines, and consists of 5 columns. My code is absolutely correct, there are no run time errors nor exceptions. Though, the problem is that when I try to print out data[15000][0], it says null. but my line does have 15,000 lines and it should print out the element inside the array. But when I print out data[5000][0], it works. What could possibly be wrong? I have 19,000 cities in 19,000 different lines, but it seems like when It goes around 10,000+ nothing gets stored in the 2d array. Help please
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.FileNotFoundException;
import java.io.IOException;
public class Data1
{
public static void main(String[] args)
{
try{
FileReader file = new FileReader("/Users/admin/Desktop/population.csv");
BufferedReader in = new BufferedReader(file);
String title = in.readLine();
String[][] data = new String[20000][5];
int currentRow = 0;
String current;
int i = 0;
String temp;
while ((temp = in.readLine()) !=null)
{
String[] c = new String[5];
String line = in.readLine().replaceAll("\"", ""); //changing the format of the data input
c = line.split(",");
c[1] = c[1].replace(" ", "");
for (int j = 0; j <data[0].length; j++)
{
current = c[j];
data[i][j] = c[j];
}
i++;
}
System.out.println(data[15000][0]);
}
catch (FileNotFoundException ex)
{
ex.printStackTrace();
}
catch (IOException ex)
{
ex.printStackTrace();
}
catch (Exception ex)
{
ex.printStackTrace();
}
}
}
You're throwing away a line on each loop.
while (in.readLine() != null)
should be
String temp;
while ((temp = in.readLine()) != null)
And then no calls to .readLine() inside the loop but refer to "temp".
Read line only once...
String line=null;
while ((line=in.readLine()) !=null) // reading line once here
{
String[] c = new String[5];
line = line.replaceAll("\"", ""); //
c = line.split(",");
c[1] = c[1].replace(" ", "");
One of your errors are the loops
while (in.readLine() !=null)
{
String[] c = new String[5];
String line = in.readLine().replaceAll("\"", ""); //changing the format of the data input
c = line.split(",");
c[1] = c[1].replace(" ", "");
Each time you invoke in.readLine() it reads a line,so you are skipping one line each time since you are calling readline twice(thus reading two lines) but storing only the second line.
You should replace it with.
String line=in.readLine();
while (line !=null)
{
String[] c = new String[5];
line.replaceAll("\"", ""); //changing the format of the data input
c = line.split(",");
c[1] = c[1].replace(" ", "");
//whatever code you have
//last line of the loop
line=in.readLine();
Can you provide us with a couple of lines of your file? And are you sure that all the file is formatted correctly ?
Related
i've got a propably simple question. I try to read the file and i want to add each single word to my array "phrase". The problem occures in for loop. I got the exception "index 0 out of bounds for length 0".
Can you please help me with that?
String [] tokens;
String line;
String hash = " ";
int n = 0;
String [] phrase = new String [n];
public void loadFile()
{
try
{
#SuppressWarnings("resource")
BufferedReader br = new BufferedReader(new FileReader("z3data1.txt"));
while((line = br.readLine()) != null)
{
tokens = line.split("[ ]");
n += tokens.length;
}
for(int j = 0; j<tokens.length; j++)
{
phrase[j] = tokens[j];
}
}
catch(IOException ex)
{
ex.printStackTrace();
}
}
A couple observations.
you are getting the error because your array is not large enough and the index j is exceeding its size.
you keep overwriting tokens in the while loop. The while loop needs to encompass the copying of the tokens to the phrase array.
So try the following:
while((line = br.readLine()) != null) {
tokens = line.split("[ ]");
n += tokens.length; // don't really need this.
//starting offset to write into phrase
int len = phrase.length;
phrase = Arrays.copyOf(phrase,phrase.length + tokens.length);
for(int j = 0; j<tokens.length; j++) {
phrase[j + len] = tokens[j];
}
}
This statement
phrase = Arrays.copyOf(phrase,phrase.length + tokens.length)
Copies the contents of phrase and increases the array size to handle the writing of tokens.
Another (and probably preferred) alternative is to use a List<String> which grows as you need it.
List<String> phrase = new ArrayList<>();
for(int j = 0; j<tokens.length; j++) {
phrase.add(tokens[j]);
}
// or skip the loop and just do
Collections.addAll(phrase,tokens);
One observation. I don't know what you are splitting on but your split statement looks suspicious.
You're setting n to 0, so phrase is also of length 0 when you say String[] phrase = String[n]. Therefore, you can't add anything to it.
If you want something of variable length, you can use an ArrayList. In the code below, you can directly use Collections.addAll to split up the line and put everything into the phrase ArrayList.
String line;
//Note that you can get rid of tokens here, since it's being inlined below
ArrayList<String> phrase = new ArrayList<>();
public void loadFile()
{
try
{
#SuppressWarnings("resource")
BufferedReader br = new BufferedReader(new FileReader("z3data1.txt"));
while((line = br.readLine()) != null)
{
//No need for a for-loop below, you can do everything in one line
Collections.addAll(phrase, line.split("[ ]"));
}
}
catch(IOException ex)
{
ex.printStackTrace();
}
}
Hi StackOverFlow People,
I have this issue in my development of System, where I have 4451 lines of record in a text file, and I am retrieving it using BufferedReader and split every line by pipe ( | ). I'm using Quartz also to run this reading of file every day. when I test it, I set the quartz job every minute so I can test It if it actually reading the file in every minute. It reads all of the line in the text file by checking it using this.
BufferedReader reader = new BufferedReader((newInputStreamReader(inputStream));
String line = null;
int counter = 0;
while((line = reader.readLine()) != null){
counter++;
}
System.out.println(counter);
But when I split the String, The result of retrieving 4451 records is inconsistent. Sometimes, It only retrieves 1000+ to 2000+ records, and Sometime it retrieves 4451, but not consistently. This is my code.
try {
BufferedReader reader = new BufferedReader((newInputStreamReader(inputStream));
String line = null;
int counter = 0;
String[] splitLine = null;
while((line = reader.readLine()) != null){
splitLine = line.split("\\|"); // Splitting the line using '|' Delimiter
for(String temp : splitLine) {
System.out.println(temp);
}
counter++;
}
System.out.println(counter);
} catch (IOException e) {
e.printStackTrace();
}
Is the splitting of String and Iterating of the readfile at the same time could be the cause?
EDIT:
There's no Exception Occured in the Situation. It Only print the length of by using the counter variable.
My Expected Output is I want to Retrieve all the records per line in the text file and split the string per line by pipe. counter is the count of lines retrieved.
I didn't find any error in your code but the code that I have written is working perfectly fine. Here is the code
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
class Test {
public static void main(String[] args) {
FileReader inputStream = null;
BufferedReader reader = null;
try {
inputStream = new FileReader("Input.txt");
reader = new BufferedReader(inputStream);
String line = null;
int counter = 0;
String[] splitLine = null;
while ((line = reader.readLine()) != null) {
splitLine = line.split("\\|");
for (String temp : splitLine) {
System.out.println(temp);
}
counter++;
}
System.out.println(counter);
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
Shouldn't pipe delimiter be just "|" instead of "\\|"?
Try Changing your code to:
splitLine = line.split("|"); // Splitting the line using '|' Delimiter
I'm trying to write some code that will take in a list of IDs (numbers and letters) from a .csv file and output them to a new file with the IDs in "natural order". My files are compiling, but I am getting the error:
java.lang.NumberFormatException: For input string: "Alpha"
I think the issue is I am not accounting for both number and letter values in the .csv file. What am I doing wrong?! Sorry if my variable Id's are confusing...
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
public class IdReader {
public static String CSV_FILE_PATH = "/Users/eringray/Desktop/idreader/idData.csv";
public static void main(String[] args){
try {
BufferedReader br = new BufferedReader(new FileReader(CSV_FILE_PATH));
BufferedWriter bw = new BufferedWriter(new FileWriter(CSV_FILE_PATH + ".tsv"));
ArrayList<String> textIds = new ArrayList<>();
ArrayList<Integer> numberIds = new ArrayList<>();
String line = "";
while((line = br.readLine()) != null) {
String[] values = line.split(" ");
if(values.length == 1) {
String idAsString = values[0];
try{
int id = Integer.parseInt(idAsString);
numberIds.add(id);
}
catch(NumberFormatException e){
textIds.add(idAsString);
}
}
}
Collections.sort(textIds);
Collections.sort(numberIds);
for(int i = 0; i < textIds.size(); i++){
String stu = textIds.get(i);
String lineText = stu.toString();
bw.write(lineText);
bw.newLine();
}
for(int i = 0; i < numberIds.size(); i++){
int numValues = numberIds.get(i);
bw.write(numValues);
bw.newLine();
}
br.close();
bw.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
The exception is coming at this line
int id = Integer.parseInt(idAsString);
Clearly alpha is not an integer, so it will throw NumberFormatException. In a case, where you encounter such Strings which cannot be converted into numbers, you can either skips them or throw an exception.
Update
//Use two seperate lists, one for maintaining numbers and other for text
ArrayList<String> textIds = new ArrayList<>();
ArrayList<Integer> numberIds = new ArrayList<>();
String line = "";
while((line = br.readLine()) != null) {
String[] values = line.split(" ");
if(values.length == 1) {
String idAsString = values[0];
try {
//Parse the value. If successful, it means it was a number. Add to integer array.
int id = Integer.parseInt(idAsString);
numberIds.add(id);
}catch (NumberFormatException e){
//If not successful, it means it was a string.
textIds.add(idAsString);
}
}
}
//In the end sort both the list
Collections.sort(textIds);
Collections.synchronizedList(numberIds);
for(int i = 0; i < textIds.size(); i++){
String stu = textIds.get(i);
bw.write(stu);
bw.newLine();
}
for(int i = 0; i < numberIds.size(); i++){
int numValues = numberIds.get(i);
bw.write(numValues+"");
bw.newLine();
}
br.close();
bw.close();
I am not putting code for writing this data to a new file. I hope you can do that.
Sample Input
4
6
33
2
5632
23454
Alpha
So after running my code
numberIds will have [ 2,4,6,33,5632,23454]
textIds will have ["Alpha"]
NumberFormatException occurs because of AlphaNumeric characters in the input.
Please use isNumeric(str) metod in https://commons.apache.org/proper/commons-lang/javadocs/api-2.6/org/apache/commons/lang/StringUtils.html api to verify whether the input is numeric or not and convert to int , only it is numeric
For a Java homework assignment, I need to create a class that reads and writes CSV files. I'm currently having some problems reading the the CSV. The code below, only outputs the first line of the code and then generates the following error message: 'Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 1 at com.gc01.FileManager.CSVManager.main(CSVManager.java:27)".
I have looked at various examples, and I am aware of the 'opencsv' package, but I need to write this code myself. I have located the problem to the statement "System.out.print(data[i]);". However, when cross-referencing this code it all seems to be fine.
I am using the methods from the FileInput class, as specified by my teacher (http://www.devjavasoft.org/SecondEdition/SourceCode/Share/FileInput.java).
public class CSVManager {
public static void main(String [] args){
Scanner sc = new Scanner (System.in);
System.out.println("Please enter the file directory of the chosen CSV");
System.out.println("For Example: /Users/UserName/Downloads/FileName.csv");
///Users/ReeceAkhtar/Desktop/GeoIPCountryWhois.csv
final String fileName = sc.nextLine();
System.out.println("How many columns?");
final int columns = sc.nextInt();
BufferedReader br = null;
String line = "";
String splitBy = " , ";
try {
br = new BufferedReader(new FileReader(fileName));
while((line = br.readLine()) != null) {
for (int i = 0; i < columns; i++) {
String[] data = line.split(splitBy);
System.out.print(data[i]);
}
}
} catch (FileNotFoundException e){
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (br != null) {
try {
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
System.out.println("File Read");
}
}
Exception is very clear
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 1
means, you are trying to access 1st element in the array which doesn't exist
Since you are saying System.out.print(data[i]); is the line where the exception is occurring, then for the first line data must have populated with only single element
Debug the issue with IDE to find out why split method is resulting unexpected elements. I suspect usage of spaces around , is the cause in " , "
Try this one. If you take splitting out the for loop everything will be okay.
String[] data = line.split(splitBy);
while((line = br.readLine()) != null){
for (int i = 0; i < columns; i++){
System.out.print(data[i]);
}
}
while((line = br.readLine()) != null){
for (int i = 0; i < columns; i++){
String[] data = line.split(splitBy);
System.out.print(data[i]);
}
You are splitting one line multiple times inside the for loop without any reason.
You are using " , " for splitting (which might be the reason you are having ArrayIndexOfBound exception) Instead use ","; use trim() on data[i] to get rid of trailing/leading white space if you wish to.
After Splitting, put checking whither data.length is equal to columns for consistency.
We are now in the era of JDK 7 where we can use try-with-resource which close the declared resource inside try(){} context, allowing us to get rid of finally block
So your could should look like as follows:
try (BufferedReader br = new BufferedReader(new FileReader(fileName))){
while((line = br.readLine()) != null){
String[] data = line.split(splitBy);
if(data.length != columns)continue; // check for consistency,
//might throw an exception
for (int i = 0; i < columns; i++){
System.out.print(data[i].trim());
}
}catch(IoExection ex)
{
ex.printStackTrace();
}
I have a problem in reading data from a text file and put it in 2 dimensional array. The sample of dataset is:
1,2,3,4,5,6
1.2,2.3,4.5,5.67,7.43,8
The problem of this code is that it just read the first line and does not read the next lines. Any suggestion is appreciated.
package test1;
import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
public class Test1{
public static void main(String args[])throws FileNotFoundException, IOException{
try{
double[][] X = new double[2][6];
BufferedReader input = new BufferedReader(new FileReader(file));
String [] temp;
String line = input.readLine();
String delims = ",";
temp = line.split(delims);
int rowCounter = 0;
while ((line = input.readLine())!= null) {
for(int i = 0; i<6; i++){
X[rowCounter][i] = Double.parseDouble(temp[i]);
}
rowCounter++;
}
}catch (Exception e){//Catch exception if any
System.err.println("Error: " + e.getMessage());
}finally{
}
}
}
Have you tried the Array utilities? Something like this:
while ((line = input.readLine())!= null) {
List<String> someList = Arrays.asList(line.split(","));
//do your conversion to double here
rowCounter++;
}
I think the blank line might be throwing your for loop off
The only place that your temp array is being assigned is before your while loop. You need to assign your temp array inside the loop, and don't read from the BufferedReader until the loop.
String[] temp;
String line;
String delims = ",";
int rowCounter = 0;
while ((line = input.readLine())!= null) {
temp = line.split(delims); // Moved inside the loop.
for(int i = 0; i<6; i++){
X[rowCounter][i] = Double.parseDouble(temp[i]);
}
Try:
int rowCounter = 0;
while ((line = input.readLine())!= null) {
String [] temp;
String line = input.readLine();
String delims = ",";
temp = line.split(delims);
for(int i = 0; i<6; i++){
X[rowCounter][i] = Double.parseDouble(temp[i]);
}
...
readLine expects a new line character at the end of the line. You should put a blank line to read the last line or use read instead.
I couldn't run the code, but one of your problems is that you were only splitting the first text line.
package Test1;
import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
public class Test1 {
public static void main(String args[]) {
try {
double[][] X = new double[2][];
BufferedReader input = new BufferedReader(new FileReader(file));
String line = null;
String delims = ",";
int rowCounter = 0;
while ((line = input.readLine()) != null) {
String[] temp = line.split(delims);
for (int i = 0; i < temp.length; i++) {
X[rowCounter][i] = Double.parseDouble(temp[i]);
}
rowCounter++;
}
} catch (Exception e) {// Catch exception if any
System.err.println("Error: " + e.getMessage());
e.printStackTrace();
} finally {
}
}
}
I formatted your code to make it more readable.
I deferred setting the size of the second element of the two dimensional array until I knew how many numbers were on a line.