We Keep Coding

Unexpected amount of lines when writing to a csv file

A part of my application writes data to a .csv file in the following way:
public class ExampleWriter {
public static final int COUNT = 10_000;
public static final String FILE = "test.csv";
public static void main(String[] args) throws Exception {
try (OutputStream os = new FileOutputStream(FILE)){
os.write(239);
os.write(187);
os.write(191);
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(os, StandardCharsets.UTF_8));
for (int i = 0; i < COUNT; i++) {
writer.write(Integer.toString(i));
writer.newLine();
}
} catch (IOException e) {
e.printStackTrace();
}
System.out.println(checkLineCount(COUNT, new File(FILE)));
}
public static String checkLineCount(int expectedLineCount, File file) throws Exception {
BufferedReader expectedReader = new BufferedReader(new FileReader(file));
try {
int lineCount = 0;
while (expectedReader.readLine() != null) {
lineCount++;
}
if (expectedLineCount == lineCount) {
return "correct";
} else {
return "incorrect";
}
}
finally {
expectedReader.close();
}
}
}
The file will be opened in excel and all kind of languages are present in the data. The os.write parts are for prefixing the file with a byte order mark as to enable all kinds of characters.
Somehow the amount of lines in the file do not match the count in the loop and I can not figure out how. Any help on what I am doing wrong here would be greatly appreciated.

You simply need to flush and close your output stream (forcing fsync) before opening the file for input and counting. Try adding:
writer.flush();
writer.close();
inside your try-block. after the for-loop in the main method.

(As a side note).
Note that using a BOM is optional, and (in many cases) reduces the portability of your files (because not all consuming app's are able to handle it well). It does not guarantee that the file has the advertised character encoding. So i would recommend to remove the BOM. When using Excel, just select the file and and choose UTF-8 as encoding.

You are not flushing the stream,Refer oracle docs for more info
which says that
Flushes this output stream and forces any buffered output bytes to be
written out. The general contract of flush is that calling it is an
indication that, if any bytes previously written have been buffered by
the implementation of the output stream, such bytes should immediately
be written to their intended destination. If the intended destination
of this stream is an abstraction provided by the underlying operating
system, for example a file, then flushing the stream guarantees only
that bytes previously written to the stream are passed to the
operating system for writing; it does not guarantee that they are
actually written to a physical device such as a disk drive.
The flush method of OutputStream does nothing.
You need to flush as well as close the stream. There are 2 ways
manually call close() and flush().
use try with resource
As I can see from your code that you have already implemented try with resource and also BufferedReader class also implements Closeable, Flushable so use code as per below
public static void main(String[] args) throws Exception {
try (OutputStream os = new FileOutputStream(FILE); BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(os, StandardCharsets.UTF_8))){
os.write(239);
os.write(187);
os.write(191);
for (int i = 0; i < COUNT; i++) {
writer.write(Integer.toString(i));
writer.newLine();
}
} catch (IOException e) {
e.printStackTrace();
}
System.out.println(checkLineCount(COUNT, new File(FILE)));
}

When COUNT is 1, the code in main() will write a file with two lines, a line with data plus an empty line afterwards. Then you call checkLineCount(COUNT, file) expecting that it will return 1 but it returns 2 because the file has actually two lines.
Therefore if you want the counter to match you must not write a new line after the last line.

(As another side note).
Notice that writing CSV-files the way you are doing is really bad practice. CSV is not so easy as it may look at first sight! So, unless you really know what you are doing (so being aware of all CSV quirks), use a library!

Writing to Text File [duplicate]

This question already has answers here:
BufferedWriter not writing everything to its output file
(8 answers)
Closed 8 years ago.
So I'm using this function to write to text file, but the text file always ends up empty after executing. Can anyone see what the error might be? I've been stuck on this for a while.
public static void writeTextFile(String fileName, String s) {
FileWriter output = null;
try {
output = new FileWriter(fileName);
BufferedWriter writer = new BufferedWriter(output);
writer.write(s);
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
if (output != null) {
try {
output.close();
} catch (IOException e) {
// Ignore issues during closing
}
}
}
}

Just change your to include writer.close(); as given below
try {
output = new FileWriter(fileName);
BufferedWriter writer = new BufferedWriter(output);
writer.write(s);
writer.close();
}
//remaining code
The reason your data not saved in the file because , The Data is saved only if you call writer.flush(); And calling the writer.flush() method is enough to just save data. But you need to close the BufferedWriter() like writer.close(); to avoid resource leak. The close() calls flush() method for you before closing the stream.

After writing your output you should make sure to flush and close the socket, specially because you are using a buffered output.
writer.write(s);
writer.flush();
writer.close();
If you don't do that, the BufferedWriter will wait for additional data, but there does come none and the program execution is stopped suddenly. Using flush here is optional, as when closing it the flush is implicit, but personally I call it everytime I need to be sure that something goes out. Just like when on the toilet ;)

When you use a Buffer to write something, you must close him when you re end
writer.close();

Without closing bufferwriter you cannot see output on text file
try to add this code
writer.close()

Bufferedwriter works, but file empty?

I have the following code:
CSVmaker(LinkedList data) {
String [] myLines = makeStrings(data);
// for (int k = 0; k<myLines.length; k++)
// System.out.println(myLines[]);
this.file = new File("rawdata.csv");
try {
BufferedWriter buff = new BufferedWriter(new FileWriter(file));
for (int i = 0; i<myLines.length; i++){
buff.write(myLines[i]);
buff.newLine();
System.out.println("done");
}
} catch (IOException ex) {
System.out.println("except");
}
}
No, I checked for the contents of myLines, these are correct.
Also, I get the print which prints "done" just as often as I should.
The csv is created.
However, if I open it manually, it is empty.
What can be the reason for this?

You never flush the buffer, or close the BufferedWriter.
After the for loop, make the following calls:
buff.flush();
buff.close();
Even with other resources, closing them when done is a good idea.

You have to close() the stream after use.
Call buff.close() after write loop; BufferedWriter will flush data to file at close.

Though the question is answered . I would like to add how buffer works.
whenever you try to write to a file using buffer,whatever you write gets added to the buffer. When the buffer is full the contents are written to the file . This way we are reducing the number of hits to the hard-drive hence improving the efficency.
If we want to forcefully write to a file without the buffer getting full , we use flush() method.

Starting with Java 8, one would simply do it with a try with resources, which automatically closes the BufferedWriter. Also see the usage of the new class Files
try (BufferedWriter writer = Files.newBufferedWriter(somePath, yourCharset)){
writer.write(output);
}

How do I print to the file?

I am working through an assignment and have run into a few snags.
My program prints output to the screen, (not how I need it yet) but only prints the first entry to the file. Below is a snippet of the code. The file appears to be reading in the data from the input file, but the loop does not output to the file past the first entry.
Scanner in = new Scanner(System.in); //Scanner object to read input from the file
System.out.println("Enter filename to read "); //file name prompt
String inputFileName = in.nextLine(); //line input reads next line
/*
* TODO 2) Use an unbuffered file input stream to open listings.txt file
* and read in property listings.
*/
Scanner reader = null;
try {
reader = new Scanner(new File(inputFileName));
} catch (FileNotFoundException e) {
System.out.println("Try Again"); //error window if name is null
JOptionPane.showMessageDialog(null, "You must enter a filename", "File input error", JOptionPane.ERROR_MESSAGE);
return;
}
PrintWriter out = new PrintWriter("agentreport.txt"); //This method prints out the file readfile.txt a word at a time
while (reader.hasNextLine()) { //It needs to output to the text file. Currently a file is created, but it is empty?
Scanner s2 = new Scanner(reader.next());
#SuppressWarnings("unused")
boolean b;
while (b = s2.hasNext()) {
String output = s2.next();
String output2 = output.toUpperCase(); //converts output to upper case
System.out.println(output2);
out.print(output2); //only printing the first entry to the agentsreport.txt file. Not stepping thru the file for some reason?
}

Even if you are using automatic flushing, which you aren't in this case, the PrintWriter object would output anything in its internal buffer unless you do one of two things:
1) Use the println(), printf(), or format() to methods
2) Make a call to the flush() method every time you print, this way all of the data in the internal buffer gets written out.
Note: The print() method does not cause the PrintWriter object to flush() its buffer.
try adding a call to flush() after you call print()
Example of split()
PrintWriter out = new PrintWriter("agentreport.txt");
while (reader.hasNextLine()) {
String words = reader.nextLine().split();
#SuppressWarnings("unused")
boolean b;
for(String word : words) {
String output = word ;
String output2 = output.toUpperCase(); //converts output to upper case
System.out.println(output2);
out.print(output2);
}

One thing that immediately jumps out is that you aren't handling your resources properly.
Any time you use an IO resource such as a reader/database connection/etc., you should always close it using a finally block, using this sort of pattern:
Reader reader = /* construct it however */
try {
/* do something with the reader */
}
finally {
reader.close();
}
If you don't do this, there's no guarantee that the reader will actually be closed, and your application will leak file descriptors/connection pool connections/etc., until eventually it won't be able to get hold of any more and your app crashes. (This won't always have fatal consequences, but it's such a straightforward pattern you should use it every time until it becomes automatic).
In this case, you aren't closing your writer at all, which means that it's not guaranteed that it ever actually flushes its output to the file. It would be perfectly in accordance with the Writer interface for it to write everything or nothing - without the flush, you have no guarantees. Note that closing the writer will automatically call flush, so that's the best bet once you're done with it.
So the latter part of your code should look like:
PrintWriter out = new PrintWriter("agentreport.txt");
try {
// Existing code here
}
finally {
// This closes the file and frees the descriptor, but also flushes the buffers
out.close();
}
Also, how are you handling the IOExceptions that can be thrown by the reading and writing? Are you catching them and swallowing them somewhere? If so, it's possible that your code is throwing an exception telling you exactly why it can't write, and you're just ignoring it and then looking puzzled.
Not to put too fine a point on it, error handling is probably the most significant part of good software development. It's not too hard to write software that works when everything's fine; the most challenging part is handling things well when you run out of space on the hard drive, or the network is temporarily down, etc.
In this case the most pragmatic approach would be to just let the exception be thrown out of the top of your main method. In this case your application will "crash", and you'll get a stacktrace + error message on the console, which will make it immediately clear that something went wrong, and give you a very good idea of what it was.

try
out.println(output2);
http://docs.oracle.com/javase/6/docs/api/java/io/PrintWriter.html
also I'd use a var other than "out" as when system.out is imported to use the shortcode 'out.println()', this could cause variable confusion
edit: good point #Hunter McMillen, changed to println as append is for a CharSequence.

try (
Scanner reader = new Scanner(new File(inputFileName));
PrintWriter writer = new PrintWriter(new FileOutputStream("agentreport.txt"), true);
) {
while (reader.hasNextLine()) {
String output = reader.nextLine().toUpperCase();
System.out.println(output);
writer.println(output);
}
} catch (FileNotFoundException e) {
System.out.println("Try Again"); //error window if name is null
JOptionPane.showMessageDialog(null, "You must enter a filename", "File input error", JOptionPane.ERROR_MESSAGE);
}

Deleting a file using delete() - Java

My code makes use of BufferedReader to read from a file [main.txt] and PrintWriter to write to a another temp [main.temp] file. I close both the streams and yet I was not able to call delete() method on the File object associated with [main.txt]. Only after calling System.gc() after closing both the stream was I able to delete the File object.
public static boolean delete (String str1, String str2, File FileLoc)
{
File tempFile = null;
BufferedReader Reader = null;
PrintWriter Writer = null;
try
{
tempFile = new File (FileLoc.getAbsolutePath() + ".tmp");
Reader = new BufferedReader(new FileReader(FileLoc));
Writer = new PrintWriter(new FileWriter(tempFile));
String lsCurrLine = null;
while((lsCurrLine = Reader.readLine()) != null)
{
// ...
// ...
if (true)
{
Writer.println(lsCurrLine);
Writer.flush();
}
}
Reader.close();
Writer.close();
System.gc();
}
catch(FileNotFoundException loFileExp)
{
System.out.println("\n File not found . Exiting");
return false;
}
catch(IOException loFileExp)
{
System.out.println("\n IO Exception while deleting the record. Exiting");
return false;
}
}
Is this reliable? Or is there a better fix?

#user183717 - that code you posted is clearly not all of the relevant code. For instance, those "..."'s and the fact that File.delete() is not actually called in that code.
When a stream object is garbage collected, its finalizer closes the underlying file descriptor. So, the fact that the delete only works when you added the System.gc() call is strong evidence that your code is somehow failing to close some stream for the file. It may well be a different stream object to the one that is opened in the code that you posted.
Properly written stream handling code uses a finally block to make sure that streams get closed no matter what. For example:
Reader reader = new BufferedReader(new FileReader(file));
try {
// do stuff
} finally {
try {
reader.close();
} catch (IOException ex) {
// ...
}
}
If you don't follow that pattern or something similar, there's a good chance that there are scenarios where streams don't always get closed. In your code for example, if one of the read or write calls threw an exception you'd skip past the statements that closed the streams.
Is this [i.e. calling System.gc();] reliable?
No.
The JVM may be configured to ignore your application's gc() call.
There's no guarantee that the lost stream will be unreachable ... yet.
There's no guarantee that calling System.gc() will notice that the stream is unreachable. Hypothetically, the stream object might be tenured, and calling System.gc() might only collect the Eden space.
Even if the stream is found to be unreachable by the GC, there's no guarantee that the GC will run the finalizer immediately. Hypothetically, running the finalizers can be deferred ... indefinitely.
Or is there a better fix ?
Yes. Fix your application to close its streams properly.

try using java.io.File library. here the simple sample:
File f = new File("file path or file name");
f.delete();

When you say you "close both the streams" you mean the BufferedReader and the PrintWriter?
You should only need to close the BufferedReader before the delete will work, but you also need to close the underlying stream; normally calling BufferedReader.close() will do that. It sounds like you think you are closing the stream but you aren't actually succeeding.
One problem with your code: you don't close the streams if exceptions occur. It's usually best to close the streams in a finally block.
Also, the code you posted doesn't use File.delete() anywhere? And what exactly do the ... lines do - are they re-assinging Reader to a new stream by any chance?

try using the apache commons io
http://commons.apache.org/io/description.html