I have source files in Cp1250 encoding. All of those file are in dirName directory or its subdirectories. I would like to merge them into one utf-8 file by adding their contents. Unfortunately I get empty line at the beginning of result file.
public static void processDir(String dirName, String resultFileName) {
try {
File resultFile = new File(resultFileName);
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(resultFile), "utf-8"));
Files.walk(Paths.get(dirName)).filter(Files::isRegularFile).forEach((path) -> {
try {
Files.readAllLines(path, Charset.forName("Windows-1250")).stream().forEach((line) -> {
try {
bw.newLine();
bw.write(line);
} catch (Exception e) {
e.printStackTrace();
}
});
} catch (Exception e) {
e.printStackTrace();
}
});
bw.close();
} catch (Exception e) {
e.printStackTrace();
}
}
The reason is that I don't know how to detect the first file in my stream.
I came up with extremely stupid solution which does not rely on streams so it is unsatisfactory:
public static void processDir(String dirName, String resultFileName) {
try {
File resultFile = new File(resultFileName);
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(resultFile), "utf-8"));
Files.walk(Paths.get(dirName)).filter(Files::isRegularFile).forEach((path) -> {
try {
Files.readAllLines(path, Charset.forName("Windows-1250")).stream().forEach((line) -> {
try {
if(resultFile.length() != 0){
bw.newLine();
}
bw.write(line);
if(resultFile.length() == 0){
bw.flush();
}
} catch (Exception e) {
e.printStackTrace();
}
});
} catch (Exception e) {
e.printStackTrace();
}
});
bw.close();
} catch (Exception e) {
e.printStackTrace();
}
}
Also I could use static boolean but that is total gibberish.
You can use the flatMap to create the stream of all lines of all files, then use flatMap again to interleave it with line separator, then use skip(1) to skip the leading separator like this:
public static void processDir(String dirName, String resultFileName) {
try(BufferedWriter bw = Files.newBufferedWriter(Paths.get(resultFileName))) {
Files.walk(Paths.get(dirName)).filter(Files::isRegularFile)
.flatMap(path -> {
try {
return Files.lines(path, Charset.forName("Windows-1250"));
} catch (IOException e) {
throw new UncheckedIOException(e);
}
})
.flatMap(line -> Stream.of(System.lineSeparator(), line))
.skip(1)
.forEach(line -> {
try {
bw.write(line);
} catch (IOException e) {
throw new UncheckedIOException(e);
}
});
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}
In general using flatMap+skip combination can help to solve many similar problems.
Also note the Files.newBufferedWriter method which is simpler way to create BufferedWriter. And don't forget about try-with-resources.
Rethink your strategy. If you want to join files and neither, remove nor convert, line terminators, there is no reason to process lines. It seems, the only reason for you to write code processing lines, is, that you have a demand to bail lambda expressions and streams into the solution and the only possibility offered by the current API is to process streams of lines. But obviously, they are not the right tool for the job:
public static void processDir(String dirName, String resultFileName) throws IOException {
Charset cp1250 = Charset.forName("Windows-1250");
CharBuffer buffer=CharBuffer.allocate(8192);
try(BufferedWriter bw
=Files.newBufferedWriter(Paths.get(resultFileName), CREATE, TRUNCATE_EXISTING)) {
Files.walkFileTree(Paths.get(dirName), new SimpleFileVisitor<Path>() {
#Override public FileVisitResult visitFile(
Path path, BasicFileAttributes attrs) throws IOException {
try(BufferedReader r=Files.newBufferedReader(path, cp1250)) {
while(r.read(buffer)>0) {
bw.write(buffer.array(), buffer.arrayOffset(), buffer.position());
buffer.clear();
}
}
return FileVisitResult.CONTINUE;
}
});
bw.close();
}
}
Note how this solution solves the problems of your first attempt. You don’t have to deal with line terminators here, this code doesn’t even waste resources in trying to find them in the input. All it does, is performing the charset conversion on chunks of input data and writing them to the target. The performance difference can be significant.
Further, the code isn’t cluttered with catching exceptions, that you can’t handle. If an IOException occurs at any place of the operation, all pending resources are properly closed and the exception is relayed to the caller.
Granted, it just uses a good old inner class instead of a lambda expression. But it doesn’t reduce the readability compared to your attempt. If it still really bothers you that there is no lambda expression involved, you may check this question & answer for a way to bring them in again.
Related
I've got kinda weird situation, I have methods:
public void generateRecords(Request request) {
String pathToFile = request.getPathFile();
String recordOne = generateRecordOne(request);
String recordTwo = generateRecordTwo(request);
fileService.writeToFile(pathToFile, recordOne);
fileService.writeToFile(pathToFile, recordTwo);
}
public void writeToFile(String path, String content) {
try {
FileWriter writer = new FileWriter(path, true);
writer.append(content);
writer.close();
} catch (IOException e) {
e.printStack();
}
}
generateRecords() is executing is rest endpoint. I am getting something like this:
id:1:record1
id:2:record1
id:1:record2
id:2:record2
While I would like to get something like this:
id:1:record1
id:1:record2
id:2:record1
id:2:record2
It is occuring sometimes, but still it is destroying my file. How can I avoid this?
Try using synchronized on writeToFile method.
Also, consider using the try-with-resources statement. In the code you have right now, an exception in your writer would lead to not closing the FileWriter.
public synchronized void writeToFile(String path, String content) {
try (FileWriter writer = new FileWriter(path, true)) {
writer.append(content);
} catch (IOException e) {
e.printStackTrace();
}
}
I am reading multiple files (1000 files of approx size 5mb) from folder. The code below is working fine to read, load and store the content of file.
public void readAllFiles(String path) {
try (Stream<Path> paths = Files.walk(Paths.get(path)).collect(toList()).parallelStream()) {
paths.forEach(filePath -> {
if (filePath.toFile().exists()) {
String fileName = filePath.getFileName().toString();
try {
List<String> loadedFile = readContent(filePath);
storeFiles(fileName, filePath, loadedFile);
} catch (Exception e) {
LOGGER.info("ERROR WHILE READING THE CONTENT OF FILE");
LOGGER.error(e.getMessage());
}
}
});
} catch (IOException e) {
LOGGER.info("ERROR WHILE READING THE FILES IN PARALLEL");
LOGGER.error(e.getMessage());
}
}
My problem is while reading the files I want to exclude some files, like exclude the file reading if for example the condition satisfies (filename contains "ABC" && flag is true)
Thanks in advance for any suggestions.
Files.walk() returns Stream<Path> so you don't need to convert it to list.
use the following code to use in parallel and filter it base
on conditions.
try (Stream<Path> paths = Files.walk(Paths.get(path)).parallel()
.filter(filePath->filePath.getFileName().toString().contains("ABC"))) {
paths.forEach(filePath -> {
//other staff...
});
} catch (IOException e) {
}
I would rewrite this using the filter function:
paths.filter(e -> e.toFile().exists()) //Make sure each file exists
.map(path -> path.getFileName().toString()) //Map it to its fileName
.filter(file -> !file.contains("someString")) //Filter
.forEach(fileName -> { //Rest of logic
try {
List<String> loadedFile = readContent(filePath);
storeFiles(fileName, filePath, loadedFile);
} catch (Exception e) {
LOGGER.info("ERROR WHILE READING THE CONTENT OF FILE");
LOGGER.error(e.getMessage());
}
});
Which will map to a String representation of the before you do the forEach
I would like to make you think about a tiny problem using the method printStackTrace(PrintWriter s). I need to use it in append mode.
The following example is explaining what I mean:
try {
} catch (Exception e) {
try {
e.printStackTrace(new PrintWriter(new FileWriter("mylog.txt", true)));
} catch (IOException ioe) {
System.out.println("I can't open the file mylog.txt");
}
}
Note that
new FileWriter("mylog.txt", true);
is the way I open the file (and create it the first time because it doesn't exist) in append mode.
The result is that in the file there is only the last exception and not a series of exceptions. One time it occurred that the method opened the file in which it didn't write anything.
How can I solve this problem?
Thank you.
Adding to what krzyk mentioned
Per OutputStreamWriter.close() : Closes the stream, flushing it first. Once the stream has been closed, further write() or flush() invocations will cause an IOException to be thrown. Closing a previously closed stream has no effect.
As mentioned, if you do not call close and this try{}catch is getting fired frequently, you are not flushing content to file.
It should written like
try {
} catch (Exception e) {
try {
FileWriter fw = new FileWriter("mylog.txt", true)
e.printStackTrace(new PrintWriter(fw));
fw.close();
} catch (IOException ioe) {
System.out.println("I can't open the file mylog.txt");
}
}
A better approach will be
FileWriter fw = new FileWriter("mylog.txt", true);
PrintWriter pw = new PrintWriter(fw);
try {
} catch (Exception e) {
try {
e.printStackTrace(pw);
} catch (IOException ioe) {
System.out.println("I can't open the file mylog.txt");
}
}finally {
pw.close();
fw.close();
}
You should close the created Writers, not closing it might cause the problems you describe.
try (PrintWriter writer = new PrintWriter(new FileWriter("mylog.txt", true))) {
e.printStackTrace(writer);
}
I'm a moderately-experienced C++ guy slowly learning Java. I'm writing a program which needs to do the following:
Create a simple text file, default directory is fine
As the program runs, periodically write one line of data to the file. Depending on a number of factors, the program may write to the file once or a million times. There is no way of knowing which write will be the last.
I've been researching different ways to do this, and this is the working code I've come up with. There are two files, "PeteProgram.java" and "PeteFileMgr.java" :
/*
"PeteProgram.java"
*/
import java.io.*;
import java.lang.String;
public class PeteProgram {
public static void main(String[] args) throws IOException {
String PeteFilename="MyRecordsFile.txt";
Writer writer = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(PeteFilename), "utf-8"));
PeteFileMgr MyPeteFileMgr = new PeteFileMgr(writer);
MyPeteFileMgr.AddThisString(writer, "Add this line #1\n");
MyPeteFileMgr.AddThisString(writer, "Add this line #2\n");
MyPeteFileMgr.AddThisString(writer, "Add this line #3\n");
}
}
//=====================================================================================================
//=====================================================================================================
/*
"PeteFileMgr.java"
*/
import java.io.*;
public class PeteFileMgr {
public PeteFileMgr(Writer writer) {
try {
writer.write("File created!");
} catch (IOException ex) {
// report
} finally {
try {writer.close();} catch (Exception ex) {}
}
}
void AddThisString(Writer writer, String AddThis) {
try {
writer.append(AddThis);
} catch (IOException ex) {
// report
} finally {
try {writer.close();} catch (Exception ex) {}
}
}
}
The initial creation of the file works just fine. However, the to-be-added lines are not written into the file. Because the program compiles and runs with no errors, I assume the program tries to write the added lines, fails, and throws an exception. (Unfortunately, I am working with a primitive compiler/debugger and can't see if this is the case.)
Does anyone spot my mistake?
Many thanks!
-P
That's because you're not flushing the Writer. You should call flush from time to time. Also, you should close your Writer at the end of your app, not after writing content into it. close method automatically flushes the contents of the writer.
So, this is how your code should look like:
public class PeteProgram {
public static void main(String[] args) {
String peteFilename = "MyRecordsFile.txt";
//here's when the physical file is created
Writer writer = null;
try {
writer = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(peteFilename), "utf-8"));
PeteFileMgr peteFileMgr = new PeteFileMgr(writer);
peteFileMgr.addThisString(writer, "Add this line #1\n");
peteFileMgr.addThisString(writer, "Add this line #2\n");
peteFileMgr.addThisString(writer, "Add this line #3\n");
} catch (IOException e) {
//handle the exception
//basic handling
e.printStacktrace();
} finally {
//this is a must!
try { writer.close(); } catch(IOException silent) { }
}
}
}
public class PeteFileMgr {
public PeteFileMgr(Writer writer) {
try {
//this method is not creating the physical file
writer.write("File created!");
} catch (IOException ex) {
// report
} finally {
//remove this call to close
//try {writer.close();} catch (Exception ex) {}
}
}
public void addThisString(Writer writer, String addThis) {
try {
writer.append(addThis);
} catch (IOException ex) {
// report
} finally {
//remove this call to close
//try {writer.close();} catch (Exception ex) {}
}
}
}
Or if using Java 7 or superior using the try-with-resources:
public class PeteProgram {
public static void main(String[] args) {
String peteFilename = "MyRecordsFile.txt";
//here's when the physical file is created
try (Writer writer = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(peteFilename), "utf-8"))) {
PeteFileMgr peteFileMgr = new PeteFileMgr(writer);
peteFileMgr.addThisString(writer, "Add this line #1\n");
peteFileMgr.addThisString(writer, "Add this line #2\n");
peteFileMgr.addThisString(writer, "Add this line #3\n");
} catch (IOException e) {
//handle the exception
//basic handling
e.printStacktrace();
}
}
}
I'm a beginner still, and currently learning about handling exceptions. The exercise in my book I'm trying to figure out tells me to add a Finally block to close out the file I opened, and I don't understand what I'm doing wrong. Keep in mind the file name and path are fake but here is what I have:
public static String readLineWithFinally()
{
System.out.println("Starting readLineWithFinally method.");
RandomAccessFile in = new RandomAccessFile("products.ran", "r");
try
{
String s = in.readLine();
return s;
}
catch (IOException e)
{
System.out.println(e.toString());
return null;
}
finally
{
try
{
in.close();
}
catch (Exception e)
{
System.out.println("Generic Error Message");
}
}
}
To add on to Taylor Hx's answer, you can take advantage of Java 7's try-with-resources construct to avoid having to use finally altogether in your case.
public static String readLineWithFinally() {
System.out.println("Starting readLineWithFinally method.");
try (RandomAccessFile in = new RandomAccessFile("products.ran", "r")) {
return in.readLine();
} catch (IOException e) {
System.out.println(e.toString());
return null;
}
}
You'll also want to be certain that your usage is consistent with what the API mandates for RandomAccessFile.
The code that you posted shouldn't compile, as RandomFile(String, String) can possibly throw FileNotFoundException. As such, we must include it in the try block.
System.out.println("Starting readLineWithFinally method.");
RandomAccessFile in = null;
try {
in = new RandomAccessFile("products.ran", "r");
String s = in.readLine();
return s;
} catch (IOException e) {
System.out.println(e.toString());
return null;
} finally {
try {
if(in != null) {
in.close();
}
} catch (Exception e) {
System.out.println("Generic Error Message");
}
}
Keep in mind the file name and path are fake but here is what I have:
That is why you will have a FileNotFoundException while creating RandomAccessFile("products.ran", "r") with read access mode "r".
From the documentation: RandomAccessFile(String name, String mode)
This constructor throws a FileNotFoundException if the mode is
"r" but the given string does not denote an existing regular file,
or if the mode begins with "rw" but the given string does not denote
an existing, writable regular file and a new regular file of that name
cannot be created, or if some other error occurs while opening or
creating the file