My TAB-delimited input file has 1 million lines, it looks like this:
id name artist_name genre notoriete_fr notoriete_us notoriete_uk notoriete_it notoriete_sp notoriete_no notoriete_de notoriete_wd
1 10ème bougie 113 rap 0 -5 -5 -5 -5 -5 -5 -5
2 I'm not in love 10cc pop 1 1 1 1 1 1 1 1
5 Generation Black Rebel Motorcycle Club rock 0 0 0 0 0 0 0 0
I've coded a file format transformation, and the output file to looks like this:
id:ID;genre;notoriete_fr:int;notoriete_us:int;notoriete_uk:int;notoriete_sp:int;notoriete_de:int;notoriete_it:int;notoriete_no:int;notoriete_wd:int;:LABEL
t1;rap;0;-5;-5;-5;-5;-5;-5;-5;Track
t5;rock;0;0;0;0;0;0;0;0;Track
I have two problems:
the output file only has 50% of input file lines
the output file has missing lines, e.g. t2's line is missing
Here's my code, thanks in advance!
Note: I've also added a buffer size to new BufferedWriter()/Reader(), no impact.
public static void main(String[] args) throws Exception {
BufferedReader br = null;
BufferedWriter bw = null;
try{
// prepare input file
File inFile = new File(inputFile);
br = new BufferedReader(new FileReader(inFile));
String line = "";
String cvsSplitBy = "\t";
// prepare output file
File outFile = new File(outputFile);
bw = new BufferedWriter(new FileWriter(outFile));
// Write header
bw.write("id:ID;genre;notoriete_fr:int;notoriete_us:int;notoriete_uk:int;notoriete_sp:int;notoriete_de:int;notoriete_it:int;notoriete_no:int;notoriete_wd:int;:LABEL\n");
while ((line = br.readLine()) != null) {
// READING
line = br.readLine();
String[] features = line.split(cvsSplitBy);
// WRITING
bw.write("t"+features[0]+";"+features[3]+";"+features[4]+";"+features[5]+";"+features[6]+";"+features[7]+";"+features[8]+";"+features[9]+";"+features[10]+";"+features[11]+";Track\n");
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (br != null) {
try {
br.close();
bw.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
}
the output file only has 8.3% of input file lines
As far as you code is concerned, It should be 50% of the lines should be missing. You have the difference in size because the data that is in the parent file is of different format than that in the file you are creating. I am saying this because you code skips the alternate lines.
Let me explain, in your while loop condition you are using line = br.readLine() Which reads the line 1. now in the first line of the while loop you are again using line = br.readLine() this will read the line 2. the file. You are using it to write the data, so line 2 data gets written. Now in the second looping, in the while loop condition you are reading line 3 of the file and in the first line of while loop you are reading line 4 of the file and this line gets written. So you see you get 50% of the output.
Now you think you understand why you are getting lesser lines in the output file. so the simple solution is to get rid of preferable the first line of the while loop and let the condition remain the same.
this behavior can be attributed to the following two lines in the code.
while ((line = br.readLine()) != null) {
// READING
line = br.readLine();
you are reading two lines from the file one during while check and one during the line = br.readline() , causing skipped lines. you should read only at the while loop check.
while ((line = br.readLine()) != null) {
// use line variable value for printing
Related
Is there in Java some sort of equivalent to BufferedReader.skip(), that would take number of lines as parameter instead of number of characters?
I want to jump to a specific line in a text file and start reading from that point without the need going thru all the lines of the file and checking against the line number (tens of thousands of them - model obj file).
All the examples I saw were dealing with the checking of line number which is not what I want.
So, the solution is to use FileInputStream.skip().
UPDATE: manually adding system-specific new line separator bytes length to line bytes length at each line iteration solved the problem of erroneous bytes skipping, so now it finally works as expected!
Define some Long variable where you will store the number of bytes to skip. I did that in my main application class (App.class):
public static long lineByteOffset = 0;
Then, in your method/function where you read your lines with BufferedReder make it like this (all my files that I read from are encoded as UTF-8):
File objFile = new File(PATH_TO_YOUR_FILE_HERE);
FileInputStream fir = null;
try {
fir = new FileInputStream(objFile);
} catch (FileNotFoundException e) {
System.err.println("File not found!");
}
fir.skip(App.lineByteOffset);//<--- 1ST IMPORTANT PART: SET HOW MANY BYTES TO SKIP, YOU START WITH 0 FOR THE 1ST TIME
BufferedReader reader = new BufferedReader(new InputStreamReader(fir, "UTF-8"));
int nls = System.getProperty("line.separator").getBytes().length;
String line;
try {
while ((line = reader.readLine()) != null) {
App.lineByteOffset += (long) (line.getBytes().length + nls);//<--- 2ND IMPORTANT PART: INCREASE NUMBER OF BYTES TO SKIP FOR NEXT TIME
/*
DO YOUR STUFF HERE...
IN MY CASE IT RETURNS SPECIFIC BLOCK
WHICH IN EFFECT EXIT THE WHILE LOOP AS NEEDED
SO THAT THE NEXT TIME IT CONTINUE WHERE WE LEFT IT
WITHOUT NEED TO READ THE WHOLE FILE FROM THE START ONCE AGAIN
*/
}
reader.close();
} catch (IOException e) {
System.err.println("Error reading the file");
}
try {
FileInputStream fstream = new FileInputStream("C:\\Users\\xxx\\Desktop\\nginx.access.log");
BufferedReader br = new BufferedReader(new InputStreamReader(fstream));
String strLine;
List<String> list = new ArrayList<String>();
/* read log line by line */
while ((strLine = br.readLine()) != null)
{
/* parse strLine to obtain what you want */
list.add(strLine);
}
String[] stringArr = list.toArray(new String[0]);
for (int i=0;i<stringArr.length;i++)
{
System.out.println(stringArr[i]);
}
fstream.close();
} catch (Exception e) {
System.err.println("Error: " + e.getMessage());
}
When I try to print System.out.println(stringArr[0]);-->First line
of txt
Also,System.out.println(stringArr[1]);-->Second line of txt etc.
But when I use for loop to print all the lines of text,it starts with a random line.
I can not see any mistake on my for loop codes.I need your help.
For me your code works. All lines of my test files are printed in the correct order.
Could it be, that your console has a limited number of lines? With a limit of 100 lines you would loose the first 5 lines after printing 105 lines. This would then look as if your program started to print the 6th line first.
Windows' cmd has a limited number of lines. Eclipse has a setting to limit/unlimit the number of lines.
To check whether that's your problem, execute your program and redirect its output into a file, then compare that file to the original file:
java YourProgram > output
FC "C:\Users\xxx\Desktop\nginx.access.log" output
I have a text file that consists of several entries such as:
one
two
three
four
five
six
The text file contains 100 lines and I want to read each 5 lines in a once. I have this code but it give null values:
BufferedReader br = null;
String sCurrentLine;
int lines = 0;
try {
br = new BufferedReader(new
FileReader("/users/MoathIbrahem/Desktop/Questions.txt"));
while(br.readLine()!= null)lines++;
for(int i = 0;i < lines;i++)
System.out.println(br.readLine());
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
In this line
while(br.readLine()!= null)lines++;
you are going to read ALL of the text file.
Nothing more to read unless you re-open the File or use mark/reset
This is happening because when you read a line a reader pointer advances. When you try to read in the for loop this pointer has reached the end of the document.
I recommend you to not count the lines before. Like the user gotomanners says in his answer:
String line;
while ((line = fileReader.readLine()) != null) {
System.out.println(line);
}
I am currently building an application that is extracting values from a text file inside a project. Somehow managed extract data from specific lines but don't seem to get the right one.
Here is the code:
private String getInputsFromATextFile(int item) throws FileNotFoundException {
InputStream is = this.getResources().openRawResource(R.drawable.input);
StringBuilder builder = new StringBuilder();
int lineNo = 0;
try{
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
String line;
while((line = reader.readLine()) != null){
lineNo++;
if(lineNo == item){
builder.append(reader.readLine());
}
}
reader.close();
}
catch (IOException e){
e.printStackTrace();
}
return builder.toString();
}
And here are the text file contents:
20.45
21.65
1
225
4102
401
3
3
6
1
196.41
64.11
7
3
5
2
144.01
3
452.33
12
701.33
33
78.12
12
123.90
4
25.00
10
6.51
30.98
2.50
Spiderman
100.00
90
150.00
100
10
34
12
James
1267
Joshue
401
Christelle
3050
Ryan
888
Hanna
5
13
24
9
5
3
50
Suppose we assign a certain line number in a parameter. This method returns the exact next data from which the line number is assigned. Although, maybe I can adjust to the output that it always returns (lineNo + 1), but if in case I assigned '0' (zero) in the parameter, it instead returns null. Why is that so? I must be missing something really important.
That's because you're reading the line again in the statement builder.append(reader.readLine()).
Notice that you've already read it in while loop.
So, the correct statement would be:
builder.append(line);
Don't read it again when appending it. Use :
builder.append(line);
Also, if you want it to be 0 indexed, you should increment lineno after comparing it.
if(lineno == item)
{
}
lineno ++;
If you do it before comparision, it will never be 0 and hence returns a null .
Is that optimal?
If it just a one time retireval Yes.
If you use it again and again - No, everytime you want data from the line, you need to traverse the whole file till that line.
One way you can do is to store it in an ArrayList.
InputStream is = this.getResources().openRawResource(R.drawable.input);
ArrayList<String> list = new ArrayList();
try{
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
String line;
while((line = reader.readLine()) != null){
list.add(line);
}
reader.close();
}
catch (IOException e){
e.printStackTrace();
}
getLineFromFile(list, 10);
Store all the strings in an arrayList and then retrieve them.
public String getLineFromFile(ArrayList<String> list, int lineNo);
{
return list.get(lineNo - 1);
}
I have a csv file that currently has 20 lines of data.
The data contains employee info and is in the following format:
first name, last name, Employee ID
So one line would like this: Emma, Nolan, 2
I know how to write to the file in java and have all 20 lines print to the console, but what I'm not sure how to do is how to get Java to print one specific line to the console.
I also want to take the last employee id number in the last entry and have java add 1 to it one I add new employees. I thinking this needs to be done with a counter just not sure how.
You can do something like this:
BufferedReader reader = new BufferedReader(new FileReader(<<your file>>));
List<String> lines = new ArrayList<>();
String line = null;
while ((line = reader.readLine()) != null) {
lines.add(line);
}
System.out.println(lines.get(0));
With BufferedReader you are able to read lines directly. This example reads the file line by line and stores the lines in an array list. You can access the lines after that by using lines.get(lineNumber).
You can read text from a file one line at a time and then do whatever you want to with that line, print it, compare it, etc...
// Construct a BufferedReader object from the input file
BufferedReader r = new BufferedReader(new FileReader("employeeData.txt"));
int i = 1;
try {
// "Prime" the while loop
String line = r.readLine();
while (line != null) {
// Print a single line of input file to console
System.out.print("Line "+i+": "+line);
// Prepare for next loop iteration
line = r.readLine();
i++;
}
} finally {
// Free up file descriptor resources
r.close();
}
// Remember the next available employee number in a one-up scheme
int nextEmployeeId = i;
BufferedReader reader =new BufferedReader(new FileReader("yourfile.csv"));
String line = "";
while((line=reader.readLine())!=null){
String [] employee =line.trim().split(",");
// if you want to check either it contains some name
//index 0 is first name, index 1 is last name, index 2 is ID
}
Alternatively, If you want more control over read CSV files then u can think about CsvBeanReader that will give you more access over files contents..
Here is an algorithm which I use for reading csv files. The most effective way is to read all the data in the csv file into a 2D array first. It just makes it a lot more flexible to manipulate the data.
That way you can specify which line of the file to print to the console by specifying it in the index of the array and using a for. I.e: System.out.println(employee_Data[1][y]); for record 1. y is the index variable for fields. You would need to use a For Loop of course, to print every element for each line.
By the way, if you want to use the employee data in a larger program, in which it may for example store the data in a database or write to another file, I'd recommend encapsulating this entire code block into a function named Read_CSV_File(), which will return a 2D String array.
My Code
// The return type of this function is a String.
// The CSVFile_path can be for example "employeeData.csv".
public static String[][] Read_CSV_File(String CSVFile_path){
String employee_Data[][];
int x;
int y;
int noofFields;
try{
String line;
BufferedReader in = new BufferedReader(new FileReader(CSVFile_path));
// reading files in specified directory
// This assigns the data to the 2D array
// The program keeps looping through until the line read in by the console contains no data in it i.e. the end of the file.
while ( (( line = in.readLine()) != null ){
String[] current_Record = line.split(",");
if(x == 0) {
// Counts the number of fields in the csv file.
noofFields = current_Record.length();
}
for (String str : values) {
employee_Data[x][y] = str;
System.out.print(", "+employee_Data[x][y]);
// The field index variable, y is incremented in every loop.
y = y + 1;
}
// The record index variable, x is incremented in every loop.
x = x + 1;
}
// This frees up the BufferedReader file descriptor resources
in.close();
/* If an error occurs, it is caught by the catch statement and an error message
* is generated and displayed to the user.
*/
}catch( IOException ioException ) {
System.out.println("Exception: "+ioException);
}
// This prints to console the specific line of your choice
System.out.println(("Employee 1:);
for(y = 0; y < noofFields ; y++){
// Prints out all fields of record 1
System.out.print(employee_Data[1][y]+", ");
}
return employee_Data;
}
For reading large file,
log.debug("****************Start Reading CSV File*******");
copyFile(inputCSVFile);
StringBuilder stringBuilder = new StringBuilder();
String line= "";
BufferedReader brOldFile = null;
try {
String inputfile = inputCSVFile;
log.info("inputfile:" + inputfile);
brOldFile = new BufferedReader(new FileReader(inputfile));
while ((line = brOldFile.readLine()) != null) {
//line = replaceSpecialChar(line);
/*do your stuff here*/
stringBuilder.append(line);
stringBuilder.append("\n");
}
log.debug("****************End reading CSV File**************");
} catch (Exception e) {
log.error(" exception in readStaffInfoCSVFile ", e);
}finally {
if(null != brOldFile) {
try {
brOldFile.close();
} catch (IOException e) {
}
}
}
return stringBuilder.toString();