We Keep Coding

FileWriter output to csv file is blank

FileWriter outfile = new FileWriter("ouput.csv", true); //true = append
for(int len = 0; len < tempList.size(); len++) {
LineItem tempItem = tempList.get(len);
if ( len == 0 ) {
lastTime = tempItem.getTimeEnd();
tempItem.setStatus("OK");
//out
output( tempItem.toCSV(), outfile);
} else {
if ( tempItem.getTimeStart().compareTo(lastTime) <= 0 ) {
//WARN
if (!tempItem.getStatus().equals("OVERLAP")) {
tempItem.setStatus("WARN");
}
} else {
//OK
//System.out.println( "OK ;" + tempItem.toCSV());
if (!tempItem.getStatus().equals("OVERLAP")) {
tempItem.setStatus("OK");
}
}
// file out write
output( tempItem.toCSV(), outfile);
lastTime = tempItem.getTimeEnd();
}
}
}
private static void output(String line, FileWriter outfile) throws IOException {
System.out.println(line);
// Write each line to a new csv file
outfile.write(line + "\n");
}
Why is my output.csv file 0 kb and empty? But when I print to line I see each string in my console...

You aren't closing the FileWriter.
NB The suggestion to flush as well as close is redundant.

After output( tempItem.toCSV(), outfile); please add the below statement. You forgot to flush. Close automatically flush for you.
outfile.close();

When you flush(outfile) it will be written to the file.
When you close(outfile) it will be flushed too, automatically. Sometimes you want to flush() at other times, but often it's not necessary. You should always close files when you've finished with them.
Since Java 7, it's often a good idea to use try-with-resources:
try(FileWriter outfile = new FileWriter("output.csv", true)) {
// code that writes to outfile
}
Because FileWriter implements Closeable, it will call outfile.close() automatically when execution leaves this block.

Processing and splitting large files with Java 8

I'm new to Java 8 and I have just started using the NIO package for file-handling. I need help in how to process large files--varying from 100,000 lines to 1,000,000 lines per file--by transforming each line into a specific format and writing the formatted lines to new files. The new file(s) generated must only contain a maximum of 100,000 lines per file. So:
if I have a 500,000-line file for processing, I must transform those
lines and distribute and print them on 5 new files.
if I have a 745,000-line file for processing, I must transform those
lines and print them on 8 new files.
I'm having a hard time figuring out an approach that will efficiently utilize the new features of Java 8. I've started out with determining the number of new files to be generated based on the line count of the large file, and then creating those new empty files:
Path largFile = Path.get("path\to\file");
long recordCount = Files.lines(file).count();
int maxRecordOfNewFiles = 100000;
int numberOfNewFiles = 1;
if (recordCount > maxRecordOfNewFiles) {
numberOfNewFiles = Math.toIntExact(recordCount / maxRecordOfNewFiles);
if (Math.toIntExact(recordCount % maxRecordOfNewFiles) > 0) {
numberOfNewFiles ++;
}
}
IntStream.rangeClosed(1, numberOfNewFiles).forEach((i)
-> {
try {
Path newFile = Paths.get("path\to\newFiles\newFile1.txt");
Files.createFile(cdpFile);
} catch (IOException iOex) {
}
});
But as I go through the the lines of the largeFile through the Files.lines(largeFile).forEach(()) capability, I got lost on how to proceed with formatting the first 100,000 lines and then determining the first of the new files and printing them on that file, and then the second batch of 100,000 to the second new file, and so on.
Any help will be appreciated. :)

When you start conceiving batch processes, I think you should consider using a framework specialized in that. You may want to handle restarts, scheduling... Spring Batch is very good for that and already provides what you want: MultiResourceItemWriter that writes to multiple files with max lines per file and FlatFileItemReader to read data from a file.
In this case, what you want is to loop over each line of an input file and write a transformation of each line in multiple output files.
One way to do that would be to create a Stream over the lines of the input file, map each line and send it to a custom writer. This custom writer would implement the logic of switching writer when it has reached the maximum number of lines per file.
In the following code MyWriter opens a BufferedWriter to a file. When the maxLines is reached (a multiple of it), this writer is closed and another one is opened, incrementing currentFile. This way, it is transparent for the reader that we're writing to multiple files.
public static void main(String[] args) throws IOException {
try (
MyWriter writer = new MyWriter(10);
Stream<String> lines = Files.lines(Paths.get("path/to/file"));
) {
lines.map(l -> /* do transformation here */ l).forEach(writer::write);
}
}
private static class MyWriter implements AutoCloseable {
private long count = 0, currentFile = 1, maxLines = 0;
private BufferedWriter bw = null;
public MyWriter(long maxLines) {
this.maxLines = maxLines;
}
public void write(String line) {
try {
if (count % maxLines == 0) {
close();
bw = Files.newBufferedWriter(Paths.get("path/to/newFiles/newFile" + currentFile++ + ".txt"));
}
bw.write(line);
bw.newLine();
count++;
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}
#Override
public void close() throws IOException {
if (bw != null) bw.close();
}
}

From what I understand in question. A simple way can be:
BufferedReader buff = new BufferedReader(new FileReader(new File("H:\\Docs\\log.txt")));
Pair<Integer, BufferedWriter> ans = buff.lines().reduce(new Pair<Integer, BufferedWriter>(0, null), (count, line) -> {
try {
BufferedWriter w;
if (count.getKey() % 1000 == 0) {
if (count.getValue() != null) count.getValue().close();
w = new BufferedWriter(new FileWriter(new File("f" + count.getKey() + ".txt")));
} else w = count.getValue();
w.write(line + "\n"); //do something
return new Pair<>(count.getKey() + 1, w);
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}, (x, y) -> {
throw new RuntimeException("Not supproted");
});
ans.getValue().close();

Try Catch logic error with file i/o in Java

So, I've been trying to learn java from various sources, I've been learning for about 2 years now. So far everything has been going smoothly, i haven't had to post on stackoverflow for a while. Recently I've been trying to figure out how to create and read files with java. I can do both of those things in separate apps, but when i try to do both it doesn't always work.
What i want to happen:
I want my program to create data.txt, then I want it to read the data and produce an error log on error.txt.
What happens:
The data.txt file gets created as expected, but nothing is written to the error.txt file. I'm having trouble grasping the try/catch block and how exactly it works. Anyone got any ideas? even just some advice would be appreciated. Thanks!
import java.io.*;
import java.util.Scanner;
public class dataReader {
public static void main(String args[]) throws Exception {
File fileName;
fileName = new File("data.txt");
PrintWriter outputFile;
outputFile = new PrintWriter(fileName);
File errorFile;
errorFile = new File("errors.txt");
PrintWriter outputErrorFile;
outputErrorFile = new PrintWriter(errorFile);
Scanner inputFile;
int recordNumber = 0;
String inputData;
outputFile.println(77);
outputFile.println("Fred");
outputFile.println(92);
outputFile.println("Wilma");
outputFile.println(89.9);
outputFile.println("Barney");
outputFile.println(42);
outputFile.println("BettyS");
inputFile = new Scanner(fileName);
while (inputFile.hasNext()) {
recordNumber++;
try {
inputData = inputFile.nextLine();
if (Integer.parseInt(inputData) < 50) {
outputErrorFile.println(recordNumber + ", " + inputData + ", is less than 50.");
} else if (Integer.parseInt(inputData) > 90) {
outputErrorFile.println(recordNumber + ", " + inputData + ", is less than 50.");
}
} catch (Exception e) {
outputErrorFile.println(recordNumber + ", That's not an integer.");
}
}
outputFile.close();
outputErrorFile.close();
System.out.println("Program terminated.");
}
}

Move the outputFile.close(); line before inputFile = new Scanner(fileName);. Currently it's just cached in the memory and not written actually to the disk.

The documentation of PrintWriter says it all. The PrintWriter(Writer) constructor creates a writer which is not automatically flushed.
You have to call close or flush method to write your data to the file.
So you have to use outputFile.close(); method before starting reading.
and as a good practice you have to close all your PrintWriter instances to avoid memory leak.
just in this case please add inputFile.close(); at the end of your program.

Updating a single line on a text file with a Java method

I know previous questions LIKE this one have been asked, but this question has to do with the specifics of the code that I have written. I am trying to update a single line of code on a file that will be permanently updated even when the program terminates so that the data can be brought up again. The method that I am writing currently looks like this (no compile errors found with eclipse)
public static void editLine(String fileName, String name, int element,
String content) throws IOException {
try {
// Open the file specified in the fileName parameter.
FileInputStream fStream = new FileInputStream(fileName);
BufferedReader br = new BufferedReader(new InputStreamReader(
fStream));
String strLine;
StringBuilder fileContent = new StringBuilder();
// Read line by line.
while ((strLine = br.readLine()) != null) {
String tokens[] = strLine.split(" ");
if (tokens.length > 0) {
if (tokens[0].equals(name)) {
tokens[element] = content;
String newLine = tokens[0] + " " + tokens[1] + " "
+ tokens[2];
fileContent.append(newLine);
fileContent.append("\n");
} else {
fileContent.append(strLine);
fileContent.append("\n");
}
}
/*
* File Content now has updated content to be used to override
* content of the text file
*/
FileWriter fStreamWrite = new FileWriter(fileName);
BufferedWriter out = new BufferedWriter(fStreamWrite);
out.write(fileContent.toString());
out.close();
// Close InputStream.
br.close();
}
} catch (IOException e) {
System.out.println("COULD NOT UPDATE FILE!");
System.exit(0);
}
}
If you could look at the code and let me know what you would suggest, that would be wonderful, because currently I am only getting my catch message.

Okay. First off the bat, StringBuilder fileContent = new StringBuilder(); is bad practice as this file could well be larger than the user's available memory. You should not keep much of the file in memory at all. Do this by reading into a buffer, processing the buffer (adjusting it if necessary), and writing the buffer to a new file. When done, delete the old file and rename the secondary to the old one's name. Hope this helps.

First Java program (writing permutations to a file)

I'm trying to write combinations with repetitions to a text file, the problems is I'm trying to hack together some code without knowing the inner workings of java. I'm not really sure what I'm effecting when I'm rearranging the code.
import java.io.*;
public class Main {
public static void main(String args[]) {
brute("123", 3, new StringBuffer());
}
static void brute(String input, int depth, StringBuffer output) {
if (depth == 0) {
// System.out.println(output);
{
try{
// Create file
FileWriter fstream = new FileWriter("out.txt",true);
BufferedWriter out = new BufferedWriter(fstream);
out.write("blah" + output);}
else {
for (int i = 0; i < input.length(); i++) {
output.append(input.charAt(i));
brute(input, depth - 1, output);
output.deleteCharAt(output.length() - 1);
}
}
}
}
}
Any help is appreciated

I guess the problem is that you get an empty file at the end of running the application?
You should simplify the bit that writes the code out:
FileWriter fstream = new FileWriter("out.txt",true);
BufferedWriter out = new BufferedWriter(fstream);
out.write("blah" + output);
You're opening a file each time and writing it out. That's ok (best to write it to an already opened stream), but you don't need to create a BufferedWriter and you can simplify the code a bit more.
FileWriter fstream = new FileWriter("out.txt", true);
fstream.append(output);
If you run this code you'll still find that it doesn't work and it just produces an empty file on disk. It's important to close the after you've used it. Changing the above to:
FileWriter fstream = new FileWriter("out.txt", true);
fstream.append(output).append('\n');
fstream.close();
Seems to make the program work (there's a few syntax errors in the code, such as forgetting to catch/throw the checked exceptions, but I assume that's just because the code was copied in manually).
Suggestions for how to tidy this up more:
Write to a stream instead of opening and closing the file every time you write an item out
Use finally to ensure that your files are always closed, even in the event of an exception