java split string[] array to multiple files - java

I'm having a problem figuring out how to split a string to multiple files. At the moment I should get two files both with JSON data. The code below writes to the first file but leaves the second empty. Any ideas why?
public void splitFile(List<String> results) throws IOException {
int name = 0;
for (int i=0; i<results.size(); i ++) {
write = new FileWriter("/home/tom/files/"+ name +".json");
out = new BufferedWriter(write);
out.write(results.get(i));
if (results.get(i).startsWith("}")) {
name++;
}
}
}
Edit: it splits at line starting with { because that denotes the end of a JSON document.

Enhance the cut-control
Get togher this:
write = new FileWriter("/home/tom/files/"+ name +".json");
out = new BufferedWriter(write);
and this:
name++;
Check for starting, not for end
Check for line starting with {, and execute those three lines to open the file.
Remember to close and flush
If it's not the first line (i > 0) then close the last writer (write.close();).
Close the last opened writer
if (!results.isEmpty())
out.close();
Result
It should look something like this:
public void splitFile(List<String> results) throws IOException {
int name = 0;
BufferedWriter out = null;
for (int i=0; i<results.size(); i ++) {
String line = results.get(i);
if (line.startsWith("{")) {
if (out != null) // it's not the first
out.close(); // tell buffered it's going to close, it makes it flush
FileWriter writer = new FileWriter("/home/tom/files/"+ name +".json");
out = new BufferedWriter(writer);
name++;
}
if (out == null)
throw new IllegalArgumentException("first line doesn't start with {");
out.write(line);
}
if (out != null) // there was at least one file
out.close();
}

I would close your buffered writer after each completed write sequence. i.e. after each iteration through the loop before you assign write to a new FileWriter().
Closing the BufferedWriter will close the underlying FileWriter, and consequently force a flush on the data written to the disk.
Note: If you're using a distinct FileWriter per loop then I'd scope that variable to that inner loop e.g.
FileWriter write = new FileWriter("/home/tom/files/"+ name +".json");
The same goes for the BufferedWriter. In fact you can write:
BufferedWriter outer = new BufferedWriter(new FileWriter(...
and just deal with outer.

Try the following code..
public void splitFile(List<String> results) throws IOException {
int name = 0;
for (int i = 0; i < results.size(); i++) {
write = new FileWriter("/home/tom/files/" + name + ".json");
out = new BufferedWriter(write);
out.write(results.get(i));
out.flush();
out.close(); // you have to close your stream every time in your case.
if (results.get(i).startsWith("}")) {
name++;
}
}
}

Related

Write and read txt file

I'm trying to do project which contains patient information and print them if user want. But it is currently adding only one person. I want to add infinite information and I don't how can I fix it.
public static void saveChanges(ArrayList<Human> humans) throws IOException {
File veritabani = new File("patients.txt");
System.gc();
RandomAccessFile raf = new RandomAccessFile(veritabani, "rw");
raf.close();
veritabani.delete();
int ctrWhile = 0;
for (int yazdir = 0; yazdir < humans.size(); yazdir++) {
File f = new File("patients.txt");
PrintWriter pw = new PrintWriter(new FileOutputStream(f, true));
String tName=humans.get(yazdir).getNameAndSurname();
int tID=humans.get(yazdir).getTC();
int tAge=humans.get(yazdir).getAge();
boolean tInsuance=humans.get(yazdir).isInsurance();
String tComplain=humans.get(yazdir).getComplain();
if (ctrWhile== 0) {
pw.append(tName+"-"+tID+"-"+tAge+"-"+"-"+tInsuance+"-"+tComplain+"-");
ctrWhile++;
} else {
pw.append("\n"+tName+"-"+tID+"-"+tAge+"-"+"-"+tInsuance+"-"+tComplain+"-");
}
pw.close();
}
}
Each time through your loop, you appear to create a new file, write to it, and close it. Therefore each time through your loop you will overwrite the file created before.
Create the file before entering the loop, and close it after you've completed the loop; only write to it within the loop.

FileWriter output to csv file is blank

FileWriter outfile = new FileWriter("ouput.csv", true); //true = append
for(int len = 0; len < tempList.size(); len++) {
LineItem tempItem = tempList.get(len);
if ( len == 0 ) {
lastTime = tempItem.getTimeEnd();
tempItem.setStatus("OK");
//out
output( tempItem.toCSV(), outfile);
} else {
if ( tempItem.getTimeStart().compareTo(lastTime) <= 0 ) {
//WARN
if (!tempItem.getStatus().equals("OVERLAP")) {
tempItem.setStatus("WARN");
}
} else {
//OK
//System.out.println( "OK ;" + tempItem.toCSV());
if (!tempItem.getStatus().equals("OVERLAP")) {
tempItem.setStatus("OK");
}
}
// file out write
output( tempItem.toCSV(), outfile);
lastTime = tempItem.getTimeEnd();
}
}
}
private static void output(String line, FileWriter outfile) throws IOException {
System.out.println(line);
// Write each line to a new csv file
outfile.write(line + "\n");
}
Why is my output.csv file 0 kb and empty? But when I print to line I see each string in my console...
You aren't closing the FileWriter.
NB The suggestion to flush as well as close is redundant.
After output( tempItem.toCSV(), outfile); please add the below statement. You forgot to flush. Close automatically flush for you.
outfile.close();
When you flush(outfile) it will be written to the file.
When you close(outfile) it will be flushed too, automatically. Sometimes you want to flush() at other times, but often it's not necessary. You should always close files when you've finished with them.
Since Java 7, it's often a good idea to use try-with-resources:
try(FileWriter outfile = new FileWriter("output.csv", true)) {
// code that writes to outfile
}
Because FileWriter implements Closeable, it will call outfile.close() automatically when execution leaves this block.

How can I read lines from a inputted file and then store the most recently read lines in an array?

I am trying to create a program that takes an inputted text file and reads the lines one by one. It then needs to store the most recently read lines (the number of lines depends on the parameter lines) in an array and then I need to print the lines using PrintWriter.
I started the first part but I'm not sure if I have the right idea. If anyone can help me on the second part as well that would be very appreciated!
public void RecentLines(Reader in, Writer out, int lines) throws IOException {
BufferedReader r3ader = new BufferedReader(in);
String str;
while((str = r3ader.readLine()) != null){
String[] arr = str.split(" ");
for( int i =0; i < lines; i++){
arr[i] = r3ader.readLine();
}
}
EDIT
the full question is this:
Create a program which reads lines from IN, one line at the time until the end. Your method must maintain an internal buffer that stores the most recently read lines (this might be best done using an array). Once the method reaches the end of the file, it should print the lines stored in the internal buffer into out, probably best done by creating a PrintWriter to decorate this Writer. (Except for your debugging purposes during the development stage, this method should not print anything to System.out.)
Try this one:
public void RecentLines(Reader in, Writer out, int lines) throws IOException {
BufferedReader r3ader = new BufferedReader(in);
String str;
int i=0;
String[] lineArray = new String[lines];
while((str = r3ader.readLine()) != null){
lines[i%lines] = str;
i++;
if(!r3ader.hasNextLine()){
break;
}
}
sounds like a task for data structures. Queue seems to be the best fit for a given task.
public void RecentLines(Reader in, Writer out, int lines) throws IOException {
BufferedReader r3ader = new BufferedReader(in);
BufferedWriter wout = new BufferedWriter(out);
String str;
Queue<String> content = new LinkedList<String>();
int i = 0;
while ((str = r3ader.readLine()) != null) {
if (i >= lines) {
content.remove();
}
content.add(str);
i++;
}
wout.write(String.valueOf(content));
}

Processing and splitting large files with Java 8

I'm new to Java 8 and I have just started using the NIO package for file-handling. I need help in how to process large files--varying from 100,000 lines to 1,000,000 lines per file--by transforming each line into a specific format and writing the formatted lines to new files. The new file(s) generated must only contain a maximum of 100,000 lines per file. So:
if I have a 500,000-line file for processing, I must transform those
lines and distribute and print them on 5 new files.
if I have a 745,000-line file for processing, I must transform those
lines and print them on 8 new files.
I'm having a hard time figuring out an approach that will efficiently utilize the new features of Java 8. I've started out with determining the number of new files to be generated based on the line count of the large file, and then creating those new empty files:
Path largFile = Path.get("path\to\file");
long recordCount = Files.lines(file).count();
int maxRecordOfNewFiles = 100000;
int numberOfNewFiles = 1;
if (recordCount > maxRecordOfNewFiles) {
numberOfNewFiles = Math.toIntExact(recordCount / maxRecordOfNewFiles);
if (Math.toIntExact(recordCount % maxRecordOfNewFiles) > 0) {
numberOfNewFiles ++;
}
}
IntStream.rangeClosed(1, numberOfNewFiles).forEach((i)
-> {
try {
Path newFile = Paths.get("path\to\newFiles\newFile1.txt");
Files.createFile(cdpFile);
} catch (IOException iOex) {
}
});
But as I go through the the lines of the largeFile through the Files.lines(largeFile).forEach(()) capability, I got lost on how to proceed with formatting the first 100,000 lines and then determining the first of the new files and printing them on that file, and then the second batch of 100,000 to the second new file, and so on.
Any help will be appreciated. :)
When you start conceiving batch processes, I think you should consider using a framework specialized in that. You may want to handle restarts, scheduling... Spring Batch is very good for that and already provides what you want: MultiResourceItemWriter that writes to multiple files with max lines per file and FlatFileItemReader to read data from a file.
In this case, what you want is to loop over each line of an input file and write a transformation of each line in multiple output files.
One way to do that would be to create a Stream over the lines of the input file, map each line and send it to a custom writer. This custom writer would implement the logic of switching writer when it has reached the maximum number of lines per file.
In the following code MyWriter opens a BufferedWriter to a file. When the maxLines is reached (a multiple of it), this writer is closed and another one is opened, incrementing currentFile. This way, it is transparent for the reader that we're writing to multiple files.
public static void main(String[] args) throws IOException {
try (
MyWriter writer = new MyWriter(10);
Stream<String> lines = Files.lines(Paths.get("path/to/file"));
) {
lines.map(l -> /* do transformation here */ l).forEach(writer::write);
}
}
private static class MyWriter implements AutoCloseable {
private long count = 0, currentFile = 1, maxLines = 0;
private BufferedWriter bw = null;
public MyWriter(long maxLines) {
this.maxLines = maxLines;
}
public void write(String line) {
try {
if (count % maxLines == 0) {
close();
bw = Files.newBufferedWriter(Paths.get("path/to/newFiles/newFile" + currentFile++ + ".txt"));
}
bw.write(line);
bw.newLine();
count++;
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}
#Override
public void close() throws IOException {
if (bw != null) bw.close();
}
}
From what I understand in question. A simple way can be:
BufferedReader buff = new BufferedReader(new FileReader(new File("H:\\Docs\\log.txt")));
Pair<Integer, BufferedWriter> ans = buff.lines().reduce(new Pair<Integer, BufferedWriter>(0, null), (count, line) -> {
try {
BufferedWriter w;
if (count.getKey() % 1000 == 0) {
if (count.getValue() != null) count.getValue().close();
w = new BufferedWriter(new FileWriter(new File("f" + count.getKey() + ".txt")));
} else w = count.getValue();
w.write(line + "\n"); //do something
return new Pair<>(count.getKey() + 1, w);
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}, (x, y) -> {
throw new RuntimeException("Not supproted");
});
ans.getValue().close();

Reading a specific set of lines in a file [duplicate]

In Java, is there any method to read a particular line from a file? For example, read line 32 or any other line number.
For small files:
String line32 = Files.readAllLines(Paths.get("file.txt")).get(32)
For large files:
try (Stream<String> lines = Files.lines(Paths.get("file.txt"))) {
line32 = lines.skip(31).findFirst().get();
}
Unless you have previous knowledge about the lines in the file, there's no way to directly access the 32nd line without reading the 31 previous lines.
That's true for all languages and all modern file systems.
So effectively you'll simply read lines until you've found the 32nd one.
Not that I know of, but what you could do is loop through the first 31 lines doing nothing using the readline() function of BufferedReader
FileInputStream fs= new FileInputStream("someFile.txt");
BufferedReader br = new BufferedReader(new InputStreamReader(fs));
for(int i = 0; i < 31; ++i)
br.readLine();
String lineIWant = br.readLine();
Joachim is right on, of course, and an alternate implementation to Chris' (for small files only because it loads the entire file) might be to use commons-io from Apache (though arguably you might not want to introduce a new dependency just for this, if you find it useful for other stuff too though, it could make sense).
For example:
String line32 = (String) FileUtils.readLines(file).get(31);
http://commons.apache.org/io/api-release/org/apache/commons/io/FileUtils.html#readLines(java.io.File, java.lang.String)
You may try indexed-file-reader (Apache License 2.0). The class IndexedFileReader has a method called readLines(int from, int to) which returns a SortedMap whose key is the line number and the value is the line that was read.
Example:
File file = new File("src/test/resources/file.txt");
reader = new IndexedFileReader(file);
lines = reader.readLines(6, 10);
assertNotNull("Null result.", lines);
assertEquals("Incorrect length.", 5, lines.size());
assertTrue("Incorrect value.", lines.get(6).startsWith("[6]"));
assertTrue("Incorrect value.", lines.get(7).startsWith("[7]"));
assertTrue("Incorrect value.", lines.get(8).startsWith("[8]"));
assertTrue("Incorrect value.", lines.get(9).startsWith("[9]"));
assertTrue("Incorrect value.", lines.get(10).startsWith("[10]"));
The above example reads a text file composed of 50 lines in the following format:
[1] The quick brown fox jumped over the lazy dog ODD
[2] The quick brown fox jumped over the lazy dog EVEN
Disclamer: I wrote this library
Although as said in other answers, it is not possible to get to the exact line without knowing the offset (pointer) before. So, I've achieved this by creating an temporary index file which would store the offset values of every line. If the file is small enough, you could just store the indexes (offset) in memory without needing a separate file for it.
The offsets can be calculated by using the RandomAccessFile
RandomAccessFile raf = new RandomAccessFile("myFile.txt","r");
//above 'r' means open in read only mode
ArrayList<Integer> arrayList = new ArrayList<Integer>();
String cur_line = "";
while((cur_line=raf.readLine())!=null)
{
arrayList.add(raf.getFilePointer());
}
//Print the 32 line
//Seeks the file to the particular location from where our '32' line starts
raf.seek(raf.seek(arrayList.get(31));
System.out.println(raf.readLine());
raf.close();
Also visit the Java docs on RandomAccessFile for more information:
Complexity: This is O(n) as it reads the entire file once. Please be aware for the memory requirements. If it's too big to be in memory, then make a temporary file that stores the offsets instead of ArrayList as shown above.
Note: If all you want in '32' line, you just have to call the readLine() also available through other classes '32' times. The above approach is useful if you want to get the a specific line (based on line number of course) multiple times.
Another way.
try (BufferedReader reader = Files.newBufferedReader(
Paths.get("file.txt"), StandardCharsets.UTF_8)) {
List<String> line = reader.lines()
.skip(31)
.limit(1)
.collect(Collectors.toList());
line.stream().forEach(System.out::println);
}
No, unless in that file format the line lengths are pre-determined (e.g. all lines with a fixed length), you'll have to iterate line by line to count them.
In Java 8,
For small files:
String line = Files.readAllLines(Paths.get("file.txt")).get(n);
For large files:
String line;
try (Stream<String> lines = Files.lines(Paths.get("file.txt"))) {
line = lines.skip(n).findFirst().get();
}
In Java 7
String line;
try (BufferedReader br = new BufferedReader(new FileReader("file.txt"))) {
for (int i = 0; i < n; i++)
br.readLine();
line = br.readLine();
}
Source: Reading nth line from file
If you are talking about a text file, then there is really no way to do this without reading all the lines that precede it - After all, lines are determined by the presence of a newline, so it has to be read.
Use a stream that supports readline, and just read the first X-1 lines and dump the results, then process the next one.
It works for me:
I have combined the answer of
Reading a simple text file
But instead of return a String I am returning a LinkedList of Strings. Then I can select the line that I want.
public static LinkedList<String> readFromAssets(Context context, String filename) throws IOException {
BufferedReader reader = new BufferedReader(new InputStreamReader(context.getAssets().open(filename)));
LinkedList<String>linkedList = new LinkedList<>();
// do reading, usually loop until end of file reading
StringBuilder sb = new StringBuilder();
String mLine = reader.readLine();
while (mLine != null) {
linkedList.add(mLine);
sb.append(mLine); // process line
mLine = reader.readLine();
}
reader.close();
return linkedList;
}
Use this code:
import java.nio.file.Files;
import java.nio.file.Paths;
public class FileWork
{
public static void main(String[] args) throws IOException {
String line = Files.readAllLines(Paths.get("D:/abc.txt")).get(1);
System.out.println(line);
}
}
You can use LineNumberReader instead of BufferedReader. Go through the api. You can find setLineNumber and getLineNumber methods.
You can also take a look at LineNumberReader, subclass of BufferedReader. Along with the readline method, it also has setter/getter methods to access line number. Very useful to keep track of the number of lines read, while reading data from file.
public String readLine(int line){
FileReader tempFileReader = null;
BufferedReader tempBufferedReader = null;
try { tempFileReader = new FileReader(textFile);
tempBufferedReader = new BufferedReader(tempFileReader);
} catch (Exception e) { }
String returnStr = "ERROR";
for(int i = 0; i < line - 1; i++){
try { tempBufferedReader.readLine(); } catch (Exception e) { }
}
try { returnStr = tempBufferedReader.readLine(); } catch (Exception e) { }
return returnStr;
}
you can use the skip() function to skip the lines from begining.
public static void readFile(String filePath, long lineNum) {
List<String> list = new ArrayList<>();
long totalLines, startLine = 0;
try (Stream<String> lines = Files.lines(Paths.get(filePath))) {
totalLines = Files.lines(Paths.get(filePath)).count();
startLine = totalLines - lineNum;
// Stream<String> line32 = lines.skip(((startLine)+1));
list = lines.skip(startLine).collect(Collectors.toList());
// lines.forEach(list::add);
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
list.forEach(System.out::println);
}
EASY WAY - Reading a line using line number.
Let's say Line number starts from 1 till null .
public class TextFileAssignmentOct {
private void readData(int rowNum, BufferedReader br) throws IOException {
int n=1; //Line number starts from 1
String row;
while((row=br.readLine()) != null) { // Reads every line
if (n == rowNum) { // When Line number matches with which you want to read
System.out.println(row);
}
n++; //This increments Line number
}
}
public static void main(String[] args) throws IOException {
File f = new File("../JavaPractice/FileRead.txt");
FileReader fr = new FileReader(f);
BufferedReader br = new BufferedReader(fr);
TextFileAssignmentOct txf = new TextFileAssignmentOct();
txf.readData(4, br); //Read a Specific Line using Line number and Passing buffered reader
}
}
for a text file you can use an integer with a loop to help you get the number of the line, don't forget to import the classes we are using in this example
File myObj = new File("C:\\Users\\LENOVO\\Desktop\\test.txt");//path of the file
FileReader fr = new FileReader(myObj);
fr.read();
BufferedReader bf = new BufferedReader(fr); //BufferedReader of the FileReader fr
String line = bf.readLine();
int lineNumber = 0;
while (line != null) {
lineNumber = lineNumber + 1;
if(lineNumber == 7)
{
//show line
System.out.println("line: " + lineNumber + " has :" + line);
break;
}
//lecture de la prochaine ligne, reading next
line = bf.readLine();
}
They are all wrong I just wrote this in about 10 seconds.
With this I managed to just call the object.getQuestion("linenumber") in the main method to return whatever line I want.
public class Questions {
File file = new File("Question2Files/triviagame1.txt");
public Questions() {
}
public String getQuestion(int numLine) throws IOException {
BufferedReader br = new BufferedReader(new FileReader(file));
String line = "";
for(int i = 0; i < numLine; i++) {
line = br.readLine();
}
return line; }}

Categories