We Keep Coding

JAVA write on specific line using BufferedWriter [duplicate]

How do I replace a line of text found within a text file?
I have a string such as:
Do the dishes0
And I want to update it with:
Do the dishes1
(and vise versa)
How do I accomplish this?
ActionListener al = new ActionListener() {
#Override
public void actionPerformed(ActionEvent e) {
JCheckBox checkbox = (JCheckBox) e.getSource();
if (checkbox.isSelected()) {
System.out.println("Selected");
String s = checkbox.getText();
replaceSelected(s, "1");
} else {
System.out.println("Deselected");
String s = checkbox.getText();
replaceSelected(s, "0");
}
}
};
public static void replaceSelected(String replaceWith, String type) {
}
By the way, I want to replace ONLY the line that was read. NOT the entire file.

At the bottom, I have a general solution to replace lines in a file. But first, here is the answer to the specific question at hand. Helper function:
public static void replaceSelected(String replaceWith, String type) {
try {
// input the file content to the StringBuffer "input"
BufferedReader file = new BufferedReader(new FileReader("notes.txt"));
StringBuffer inputBuffer = new StringBuffer();
String line;
while ((line = file.readLine()) != null) {
inputBuffer.append(line);
inputBuffer.append('\n');
}
file.close();
String inputStr = inputBuffer.toString();
System.out.println(inputStr); // display the original file for debugging
// logic to replace lines in the string (could use regex here to be generic)
if (type.equals("0")) {
inputStr = inputStr.replace(replaceWith + "1", replaceWith + "0");
} else if (type.equals("1")) {
inputStr = inputStr.replace(replaceWith + "0", replaceWith + "1");
}
// display the new file for debugging
System.out.println("----------------------------------\n" + inputStr);
// write the new string with the replaced line OVER the same file
FileOutputStream fileOut = new FileOutputStream("notes.txt");
fileOut.write(inputStr.getBytes());
fileOut.close();
} catch (Exception e) {
System.out.println("Problem reading file.");
}
}
Then call it:
public static void main(String[] args) {
replaceSelected("Do the dishes", "1");
}
Original Text File Content:
Do the dishes0
Feed the dog0
Cleaned my room1
Output:
Do the dishes0
Feed the dog0
Cleaned my room1
----------------------------------
Do the dishes1
Feed the dog0
Cleaned my room1
New text file content:
Do the dishes1
Feed the dog0
Cleaned my room1
And as a note, if the text file was:
Do the dishes1
Feed the dog0
Cleaned my room1
and you used the method replaceSelected("Do the dishes", "1");,
it would just not change the file.
Since this question is pretty specific, I'll add a more general solution here for future readers (based on the title).
// read file one line at a time
// replace line as you read the file and store updated lines in StringBuffer
// overwrite the file with the new lines
public static void replaceLines() {
try {
// input the (modified) file content to the StringBuffer "input"
BufferedReader file = new BufferedReader(new FileReader("notes.txt"));
StringBuffer inputBuffer = new StringBuffer();
String line;
while ((line = file.readLine()) != null) {
line = ... // replace the line here
inputBuffer.append(line);
inputBuffer.append('\n');
}
file.close();
// write the new string with the replaced line OVER the same file
FileOutputStream fileOut = new FileOutputStream("notes.txt");
fileOut.write(inputBuffer.toString().getBytes());
fileOut.close();
} catch (Exception e) {
System.out.println("Problem reading file.");
}
}

Since Java 7 this is very easy and intuitive to do.
List<String> fileContent = new ArrayList<>(Files.readAllLines(FILE_PATH, StandardCharsets.UTF_8));
for (int i = 0; i < fileContent.size(); i++) {
if (fileContent.get(i).equals("old line")) {
fileContent.set(i, "new line");
break;
}
}
Files.write(FILE_PATH, fileContent, StandardCharsets.UTF_8);
Basically you read the whole file to a List, edit the list and finally write the list back to file.
FILE_PATH represents the Path of the file.

If replacement is of different length:
Read file until you find the string you want to replace.
Read into memory the part after text you want to replace, all of it.
Truncate the file at start of the part you want to replace.
Write replacement.
Write rest of the file from step 2.
If replacement is of same length:
Read file until you find the string you want to replace.
Set file position to start of the part you want to replace.
Write replacement, overwriting part of file.
This is the best you can get, with constraints of your question. However, at least the example in question is replacing string of same length, So the second way should work.
Also be aware: Java strings are Unicode text, while text files are bytes with some encoding. If encoding is UTF8, and your text is not Latin1 (or plain 7-bit ASCII), you have to check length of encoded byte array, not length of Java string.

I was going to answer this question. Then I saw it get marked as a duplicate of this question, after I'd written the code, so I am going to post my solution here.
Keeping in mind that you have to re-write the text file. First I read the entire file, and store it in a string. Then I store each line as a index of a string array, ex line one = array index 0. I then edit the index corresponding to the line that you wish to edit. Once this is done I concatenate all the strings in the array into a single string. Then I write the new string into the file, which writes over the old content. Don't worry about losing your old content as it has been written again with the edit. below is the code I used.
public class App {
public static void main(String[] args) {
String file = "file.txt";
String newLineContent = "Hello my name is bob";
int lineToBeEdited = 3;
ChangeLineInFile changeFile = new ChangeLineInFile();
changeFile.changeALineInATextFile(file, newLineContent, lineToBeEdited);
}
}
And the class.
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.FileReader;
import java.io.IOException;
import java.io.OutputStreamWriter;
import java.io.UnsupportedEncodingException;
import java.io.Writer;
public class ChangeLineInFile {
public void changeALineInATextFile(String fileName, String newLine, int lineNumber) {
String content = new String();
String editedContent = new String();
content = readFile(fileName);
editedContent = editLineInContent(content, newLine, lineNumber);
writeToFile(fileName, editedContent);
}
private static int numberOfLinesInFile(String content) {
int numberOfLines = 0;
int index = 0;
int lastIndex = 0;
lastIndex = content.length() - 1;
while (true) {
if (content.charAt(index) == '\n') {
numberOfLines++;
}
if (index == lastIndex) {
numberOfLines = numberOfLines + 1;
break;
}
index++;
}
return numberOfLines;
}
private static String[] turnFileIntoArrayOfStrings(String content, int lines) {
String[] array = new String[lines];
int index = 0;
int tempInt = 0;
int startIndext = 0;
int lastIndex = content.length() - 1;
while (true) {
if (content.charAt(index) == '\n') {
tempInt++;
String temp2 = new String();
for (int i = 0; i < index - startIndext; i++) {
temp2 += content.charAt(startIndext + i);
}
startIndext = index;
array[tempInt - 1] = temp2;
}
if (index == lastIndex) {
tempInt++;
String temp2 = new String();
for (int i = 0; i < index - startIndext + 1; i++) {
temp2 += content.charAt(startIndext + i);
}
array[tempInt - 1] = temp2;
break;
}
index++;
}
return array;
}
private static String editLineInContent(String content, String newLine, int line) {
int lineNumber = 0;
lineNumber = numberOfLinesInFile(content);
String[] lines = new String[lineNumber];
lines = turnFileIntoArrayOfStrings(content, lineNumber);
if (line != 1) {
lines[line - 1] = "\n" + newLine;
} else {
lines[line - 1] = newLine;
}
content = new String();
for (int i = 0; i < lineNumber; i++) {
content += lines[i];
}
return content;
}
private static void writeToFile(String file, String content) {
try (Writer writer = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(file), "utf-8"))) {
writer.write(content);
} catch (UnsupportedEncodingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
private static String readFile(String filename) {
String content = null;
File file = new File(filename);
FileReader reader = null;
try {
reader = new FileReader(file);
char[] chars = new char[(int) file.length()];
reader.read(chars);
content = new String(chars);
reader.close();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (reader != null) {
try {
reader.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
return content;
}
}

Sharing the experience with Java Util Stream
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;
public static void replaceLine(String filePath, String originalLineText, String newLineText) {
Path path = Paths.get(filePath);
// Get all the lines
try (Stream<String> stream = Files.lines(path, StandardCharsets.UTF_8)) {
// Do the line replace
List<String> list = stream.map(line -> line.equals(originalLineText) ? newLineText : line)
.collect(Collectors.toList());
// Write the content back
Files.write(path, list, StandardCharsets.UTF_8);
} catch (IOException e) {
LOG.error("IOException for : " + path, e);
e.printStackTrace();
}
}
Usage
replaceLine("test.txt", "Do the dishes0", "Do the dishes1");

//Read the file data
BufferedReader file = new BufferedReader(new FileReader(filepath));
StringBuffer inputBuffer = new StringBuffer();
String line;
while ((line = file.readLine()) != null) {
inputBuffer.append(line);
inputBuffer.append('\n');
}
file.close();
String inputStr = inputBuffer.toString();
// logic to replace lines in the string (could use regex here to be generic)
inputStr = inputStr.replace(str, " ");
//'str' is the string need to update in this case it is updating with nothing
// write the new string with the replaced line OVER the same file
FileOutputStream fileOut = new FileOutputStream(filer);
fileOut.write(inputStr.getBytes());
fileOut.close();

Well you would need to get a file with JFileChooser and then read through the lines of the file using a scanner and the hasNext() function
http://docs.oracle.com/javase/7/docs/api/javax/swing/JFileChooser.html
once you do that you can save the line into a variable and manipulate the contents.

just how to replace strings :) as i do
first arg will be filename second target string third one the string to be replaced instead of targe
public class ReplaceString{
public static void main(String[] args)throws Exception {
if(args.length<3)System.exit(0);
String targetStr = args[1];
String altStr = args[2];
java.io.File file = new java.io.File(args[0]);
java.util.Scanner scanner = new java.util.Scanner(file);
StringBuilder buffer = new StringBuilder();
while(scanner.hasNext()){
buffer.append(scanner.nextLine().replaceAll(targetStr, altStr));
if(scanner.hasNext())buffer.append("\n");
}
scanner.close();
java.io.PrintWriter printer = new java.io.PrintWriter(file);
printer.print(buffer);
printer.close();
}
}

Storing data into a file for one time and avoid any changes after running the code again

My code
Here I take a name from a user and save it to a file, my purpose is to give that name an index (Out) and save it into a file, so that each time I run the code, I will still have the same name and index(not new values). So how can I do that?
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.util.Scanner;
public class h_main {
public static void main(String[] args) {
contact[] table = new contact[3]; //Create an object of contact class
int tablesize = 3;
// Input from user
//**Inserting from user**//
Scanner reader = new Scanner(System.in); // Reading from System.in
System.out.println("Enter a name: ");
String names = reader.nextLine();
// Save the inserted name inside a table array with an index Out(from the hash function)
int Out = calc_hash(names, tablesize);
table[Out] = new contact();
table[Out].Name = names;
System.out.println(Out);
// Writing
for (int i = 0; i < table.length; i++) {
FileWriter fWriter = null;
BufferedWriter writer = null;
try {
fWriter = new FileWriter("text.txt");
writer = new BufferedWriter(fWriter);
writer.write(table[i].Name);
writer.write(table[i].phone);
writer.newLine();
writer.close();
}
// System.err.println("Your input of " + table[i].Name.length + " characters was saved.");
catch (Exception e) {
System.out.println("Error!");
}
// Reading
// The name of the file to open.
String fileName = "text.txt";
// This will reference one line at a time
String line = null;
try {
// FileReader reads text files in the default encoding.
FileReader fileReader = new FileReader("text.txt");
// Always wrap FileReader in BufferedReader.
BufferedReader bufferedReader = new BufferedReader(fileReader);
while ((line = bufferedReader.readLine()) != null) {
System.out.println("Name: " + line);
}
// Always close files.
bufferedReader.close();
} catch (FileNotFoundException ex) {
System.out.println(
"Unable to open file '" +
"text.txt" + "'");
} catch (IOException ex) {
System.out.println(
"Error reading file '" +
"text.txt" + "'");
// Or we could just do this:
// ex.printStackTrace();
}
}
}
//**Generate hash function**//
public static int calc_hash(String names, int table_size) {
int i, l = names.length();
int hash = 0;
for (i = 0; i < l; i++) {
hash += Character.getNumericValue(names.charAt(i));
hash += (hash << 10);
hash ^= (hash >> 6);
}
hash += (hash << 3);
hash ^= (hash >> 11);
hash += (hash << 15);
if (hash > 0) return hash % table_size;
else return -hash % table_size;
}
}
Class contact
public class contact {
String Name ;
int phone ;
}

Have the file not exist when you run the program for the first time. After you do, use the existence of the file to determine whether the program has been run before or not. Your code could look like this:
import java.nio.file.*;
import java.util.stream.*;
public class Main {
public static void main(String[] args) {
// It's recommended to follow Java naming and style conventions:
// Class names always begin with an uppercase letter
Contact[] table = new Contact[3];
String fileName = "text.txt";
Path file = Paths.get(fileName);
// Check for persistence file:
if(Files.exists(file)) {
// If all you need to do is print each line, try this:
try {
Files.lines(file).forEach(l -> System.out.println("Name: " + l));
} catch(IOException e) {
System.err.println("Error reading data file!");
}
} else {
// Take data input
Scanner scanner = new Scanner(System.in);
// ...
// When program is terminated, save everything:
try(BufferedWriter writer = Files.newBufferedWriter(file)) {
// use writer to write data...
} catch(IOException e) {
System.err.println("Error writing data file!");
}
}
}
}

Make faster a read from file

I'm going to pass my data from MongoDB to Neo4j.
So, I exported my MongoDB documents in .csv. As you can read here I have a problem with the array uniform.
So I wrote a java program to fix this problem.
Here is the .csv exported from MongoDB (note the different about uniform array):
_id,official_name,common_name,country,started_by.day,started_by.month,started_by.year,championship,stadium.name,stadium.capacity,palmares.first_prize,palmares.second_prize,palmares.third_prize,palmares.fourth_prize,average_age,squad_value,foreigners,uniform
0,yaDIXxLAOV,WWYWLqPcYM,QsVwiNmeGl,7,9,1479,oYKGgstIMv,qskcxizCkd,8560,10,25,9,29,16,58,6,"[""first_colour"",""second_colour"",""third_colour""]"
Here is how it must be to import in Neo4j:
_id,official_name,common_name,country,started_by.day,started_by.month,started_by.year,championship,stadium.name,stadium.capacity,palmares.first_prize,palmares.second_prize,palmares.third_prize,palmares.fourth_prize,average_age,squad_value,foreigners,uniform.0,uniform.1,uniform.2
0,yaDIXxLAOV,WWYWLqPcYM,QsVwiNmeGl,7,9,1479,oYKGgstIMv,qskcxizCkd,8560,10,25,9,29,16,58,6,first_colour,second_colour,third_colour
My code works, but I have to convert 500k line of the .csv file and the program it is too much slow(it's still working after 20 minutes :/):
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import java.io.PrintWriter;
public class ConvertireCSV {
public static void main(String[] args) throws IOException {
FileReader f;
f=new FileReader("output.csv");
BufferedReader b;
b=new BufferedReader(f);
String firstLine= b.readLine();
int uniform = firstLine.indexOf("uniform");
firstLine=firstLine.substring(0, uniform);
firstLine = firstLine + "uniform.0,uniform.1,uniform.2\n";
String line="";
String csv="";
while(true) {
line=b.readLine();
if(line==null)
break;
int u = line.indexOf("\"[");
line=line.substring(0, u);
line=line + "first_colour,second_colour,third_colour \n";
csv=csv+line;
}
File file = new File("outputForNeo4j.csv");
if(file.createNewFile()) {
PrintWriter pw = new PrintWriter(file);
pw.println(firstLine + csv);
System.out.println("New file \"outputForNeo4j.csv\" created.");
pw.flush();
pw.close();
}
}
}
How can I make it faster?

Okay some basic ways to improve your code:
Make sure that your variables got the minimal scope required. If you don't need line outside your loop, don't declare it outside your loop.
Concatenation of simple strings is in general slow. Use a StringBuilder to speed things to there.
Why are you buffering the string anyway? Seems like a waste of memory. Just open the output stream to your target file and write the lines to the new file as you process them.
Examples:
I don't think you need a example on the first point.
For the second things could look like this:
...
StringBuilder csv = new StringBuilder();
while(true) {
...
csv.append(line);
}
...
if(file.createNewFile()) {
...
pw.println(firstLine + csv.toString());
...
}
For the third point the rewriting would be a little more extensive:
public static void main(String[] args) throws IOException {
FileReader f;
f=new FileReader("output.csv");
BufferedReader b;
b=new BufferedReader(f);
String firstLine= b.readLine();
int uniform = firstLine.indexOf("uniform");
firstLine=firstLine.substring(0, uniform);
firstLine = firstLine + "uniform.0,uniform.1,uniform.2\n";
File file = new File("outputForNeo4j.csv");
if(!file.createNewFile()) {
// all work would be for nothing! Bailing out.
return;
}
PrintWriter pw = new PrintWriter(file);
pw.print(firstLine);
while(true) {
String line=b.readLine();
if(line==null)
break;
int u = line.indexOf("\"[");
line=line.substring(0, u);
line=line + "first_colour,second_colour,third_colour \n";
pw.print(line);
}
System.out.println("New file \"outputForNeo4j.csv\" created.");
pw.flush();
pw.close();
b.close()
}

csv=csv+line;
string concatenation is expensive operation. I would suggest using bufferedWriter.
something like this:
FileReader f;
f=new FileReader("output.csv");
BufferedReader b;
BufferedWriter out;
b=new BufferedReader(f);
try{
out = new BufferedWriter(new FileWriter("outputForNeo4j.csv"));
} catch(Exception e){
//cannot create file
}
System.out.println("New file \"outputForNeo4j.csv\" created.");
String firstLine= b.readLine();
int uniform = firstLine.indexOf("uniform");
firstLine=firstLine.substring(0, uniform);
firstLine = firstLine + "uniform.0,uniform.1,uniform.2\n";
String line="";
String csv="";
out.write(firstLine);
while(true) {
line=b.readLine();
if(line==null)
break;
int u = line.indexOf("\"[");
line=line.substring(0, u);
line=line + "first_colour,second_colour,third_colour \n";
out.write(line);
}
out.flush();
}

Results :
test0 : Runs: 241 iterations ,avarage milis = 246
test1 : Runs: 249 iterations ,avarage milis = 118
test2 : Runs: 269 iterations ,avarage milis = 5
test3 : Runs: 241 iterations ,avarage milis = 2
import java.io.*;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.Random;
public class Tester {
private static final String filePath = "c:\\bigFile.txt";
//private static final String filePath = "c:\\bigfileNewLine.txt";
private static final int numOfMethods = 4;
private static final int numOfIter = 1000;
public Tester() throws NoSuchMethodException {
System.out.println("Tester.Tester");
int[] milisArr = new int [numOfMethods];
int[] actualRun = new int [numOfMethods];
Random rnd = new Random(System.currentTimeMillis());
Long startMs = 0l, endMs = 0l;
Method[] method = new Method[numOfMethods];
for (int i = 0; i < numOfMethods; i++)
method[i] = this.getClass().getMethod("test" + i);
int testCount = 0;
while (testCount++ < numOfIter) {
int testMethod = rnd.nextInt(numOfMethods);
Method m = method[testMethod];
try {
System.gc();
startMs = System.currentTimeMillis();
String retval = (String) m.invoke(null);
endMs = System.currentTimeMillis();
} catch (IllegalAccessException e) {
e.printStackTrace(); //To change body of catch statement use File | Settings | File Templates.
} catch (InvocationTargetException e) {
e.printStackTrace(); //To change body of catch statement use File | Settings | File Templates.
}
milisArr[testMethod] += (endMs - startMs);
actualRun[testMethod]++;
System.out.println("Test name: " + m.getName() + " testCount=" + testCount + " Of " + numOfIter + " iteration, Total time :" + (endMs - startMs) / 1000.0 + " seconds");
}
System.out.println("Test Summery :");
for (int i = 0; i < numOfMethods; i++)
System.out.println("test" + i + " : Runs: " + actualRun[i] + " iterations ,avarage milis = " + milisArr[i]/numOfIter);
}
public static String test0() throws IOException {
InputStream file = getInputStream();
StringBuffer textBuffer = new StringBuffer();
int c;
while ((c = file.read()) != -1)
textBuffer.append((char) c);
file.close();
return textBuffer.toString();
}
public static String test1() throws IOException {
Reader reader = new FileReader(new File(filePath));
BufferedReader br = new BufferedReader(reader);
String line = br.readLine();
String result = line;
while (line != null) {
line = br.readLine();
if (line == null) {
} else {
result = result + "\n" + line;
}
}
br.close();
reader.close();
return result;
}
public static String test2() throws IOException {
byte[] buf = new byte[1024];
int l;
InputStream is = getInputStream();
StringBuffer tmpBuf = new StringBuffer();
while ((l = is.read(buf)) != -1) {
tmpBuf.append(new String(buf, 0, l));
}
is.close();
return tmpBuf.toString();
}
public static String test3() throws IOException {
File source = new File(filePath);
final DataInputStream dis = new DataInputStream(new BufferedInputStream(new FileInputStream(source)));
final byte[] buffer = new byte[(int) source.length()];
dis.readFully(buffer);
dis.close();
return new String(buffer, "UTF-8");
}
private static InputStream getInputStream() {
try {
return new FileInputStream(filePath);
} catch (FileNotFoundException e) {
e.printStackTrace();
return null;
}
}
public static void main(String[] args) {
try {
new Tester();
} catch (NoSuchMethodException e) {
e.printStackTrace(); //To change body of catch statement use File | Settings | File Templates.
}
}
}

JAVA : file I/O

I have got two text files with data in the following format
data.txt file as following format
A 10
B 20
C 15
data1.txt file is in format (start node,end node, distance):
A B 5
A C 10
B C 20
I am trying to implement a search strategy, for that I need to load the data from data.txt and ONLY the start node and end node from data1.txt (i.e. I dont need the distance). I need to store this information in a stack as I think it would be a best data structure for implementing greedy search.
Actually I am not sure how to get started with file I/O to read these files and store them in array to implement greedy search. So I would highly appreciate any starting idea on how to proceed.
I am new to this, so please bear with me. Any help is much appreciated. Thanks.
EDIT:
Here is what I have got till now
String heuristic_file = "data.txt";
try
{
FileReader inputHeuristic = new FileReader(heuristic_file);
BufferedReader bufferReader = new BufferedReader(inputHeuristic);
String line;
while ((line = bufferReader.readLine()) != null)
{
System.out.println(line);
}
bufferReader.close();
} catch(Exception e) {
System.out.println("Error reading file " + e.getMessage());
}

My approach, doesn't differ fundamentally from the others. Please regard the try/catch/finally blocks. Always put the closing statements into the finally block, so the opened file is guaranteed to be closed, even if an exception was thrown while reading the file.
The part between the two //[...] could surely be done more efficient. Maybe reading the whole file in one take and then parsing the text backwards and searching for a line-break? Maybe a Stream-API supports to set the reading position. I honestly don't know. I didn't need that, up to now.
I chose to use the verbose initialization of the BufferedReader, because then you can specify the expected encoding of the file. In your case it doesn't matter, since your files do not contain symbols out of the standard ASCII range, but I believe it's a semi-best-practice.
Before you ask: r.close() takes care of closing the underlying InputStreamReader and FileInputStream in the right order, till all readers and streams are closed.
public static void readDataFile(String dir, String file1, String file2)
throws IOException
{
File datafile1 = new File(dir, file1);
File datafile2 = new File(dir, file2);
if (datafile1.exists())
{
BufferedReader r = null;
try
{
r = new BufferedReader(
new InputStreamReader(
new FileInputStream(datafile1),
"UTF-8"
)
);
String row;
Stack<Object[]> s = new Stack<Object[]>();
String[] pair;
Integer datapoint;
while((row = r.readLine()) != null)
{
if (row != null && row.trim().length() > 0)
{
// You could use " " instead of "\\s"
// but the latter regular expression
// shorthand-character-class will
// split the row on tab-symbols, too
pair = row.split("\\s");
if (pair != null && pair.length == 2)
{
datapoint = null;
try
{
datapoint = Integer.parseInt(pair[1], 10);
}
catch(NumberFormatException f) { }
// Later you can validate datapairs
// by using
// if (s.pop()[1] != null)
s.add(new Object[] { pair[0], datapoint});
}
}
}
}
catch (UnsupportedEncodingException e1) { }
catch (FileNotFoundException e2) { }
catch (IOException e3) { }
finally
{
if (r != null) r.close();
}
}
// Do something similar with datafile2
if (datafile2.exists())
{
// [...do the same as in the first try/catch block...]
String firstrow = null, lastrow = null;
String row = null;
int i = 0;
do
{
lastrow = row;
row = r.readLine();
if (i == 0)
firstrow = row;
i++;
} while(row != null);
// [...parse firstrow and lastrow into a datastructure...]
}
}

use split
while ((line = bufferReader.readLine()) != null)
{
String[] tokens = line.split(" ");
System.out.println(line + " -> [" + tokens[0] + "]" + "[" + tokens[1] + "][" + tokens[2] + "]");
}
if you must have this in an array you can use the following:
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import java.text.ParseException;
import java.util.ArrayList;
import java.util.List;
public class NodeTest {
public static void main(String[] args) throws ParseException {
try {
File first = new File("data.txt");
File second = new File("data1.txt");
Node[] nodes1 = getNodes(first);
Node[] nodes2 = getNodes(second);
print(nodes1);
print(nodes2);
}
catch(Exception e) {
System.out.println("Error reading file " + e.getMessage());
}
}
public static final void print(Node[] nodes) {
System.out.println("======================");
for(Node node : nodes) {
System.out.println(node);
}
System.out.println("======================");
}
public static final Node[] getNodes(File file) throws IOException {
FileReader inputHeuristic = new FileReader(file);
BufferedReader bufferReader = new BufferedReader(inputHeuristic);
String line;
List<Node> list = new ArrayList<Node>();
while ((line = bufferReader.readLine()) != null) {
String[] tokens = line.split(" ");
list.add(new Node(tokens[0], tokens[1]));
}
bufferReader.close();
return list.toArray(new Node[list.size()]);
}
}
class Node {
String start;
String end;
public Node(String start, String end){
this.start = start;
this.end = end;
}
public String toString() {
return "[" + start + "][" + end + "]";
}
}

Something like this?
HashSet<String> nodes = new HashSet<String>();
try(BufferedReader br = new BufferedReader(new FileReader("data.txt"))) {
String line = br.readLine();
while (line != null) {
String[] l = line.split(" ");
nodes.add(l[0]);
line = br.readLine();
}
}
try(BufferedReader br = new BufferedReader(new FileReader("data1.txt"))) {
String line = br.readLine();
while (line != null) {
String[] l = line.split(" ");
if (nodes.contains(l[0]) || nodes.contains(l[1]))
// Do whatever you want ...
line = br.readLine();
}
}

Java - Read file and split into multiple files

I have a file which I would like to read in Java and split this file into n (user input) output files. Here is how I read the file:
int n = 4;
BufferedReader br = new BufferedReader(new FileReader("file.csv"));
try {
String line = br.readLine();
while (line != null) {
line = br.readLine();
}
} finally {
br.close();
}
How do I split the file - file.csv into n files?
Note - Since the number of entries in the file are of the order of 100k, I can't store the file content into an array and then split it and save into multiple files.

Since one file can be very large, each split file could be large as well.
Example:
Source File Size: 5GB
Num Splits: 5: Destination
File Size: 1GB each (5 files)
There is no way to read this large split chunk in one go, even if we have such a memory. Basically for each split we can read a fix size byte-array which we know should be feasible in terms of performance as well memory.
NumSplits: 10 MaxReadBytes: 8KB
public static void main(String[] args) throws Exception
{
RandomAccessFile raf = new RandomAccessFile("test.csv", "r");
long numSplits = 10; //from user input, extract it from args
long sourceSize = raf.length();
long bytesPerSplit = sourceSize/numSplits ;
long remainingBytes = sourceSize % numSplits;
int maxReadBufferSize = 8 * 1024; //8KB
for(int destIx=1; destIx <= numSplits; destIx++) {
BufferedOutputStream bw = new BufferedOutputStream(new FileOutputStream("split."+destIx));
if(bytesPerSplit > maxReadBufferSize) {
long numReads = bytesPerSplit/maxReadBufferSize;
long numRemainingRead = bytesPerSplit % maxReadBufferSize;
for(int i=0; i<numReads; i++) {
readWrite(raf, bw, maxReadBufferSize);
}
if(numRemainingRead > 0) {
readWrite(raf, bw, numRemainingRead);
}
}else {
readWrite(raf, bw, bytesPerSplit);
}
bw.close();
}
if(remainingBytes > 0) {
BufferedOutputStream bw = new BufferedOutputStream(new FileOutputStream("split."+(numSplits+1)));
readWrite(raf, bw, remainingBytes);
bw.close();
}
raf.close();
}
static void readWrite(RandomAccessFile raf, BufferedOutputStream bw, long numBytes) throws IOException {
byte[] buf = new byte[(int) numBytes];
int val = raf.read(buf);
if(val != -1) {
bw.write(buf);
}
}

import java.io.*;
import java.util.Scanner;
public class split {
public static void main(String args[])
{
try{
// Reading file and getting no. of files to be generated
String inputfile = "C:/test.txt"; // Source File Name.
double nol = 2000.0; // No. of lines to be split and saved in each output file.
File file = new File(inputfile);
Scanner scanner = new Scanner(file);
int count = 0;
while (scanner.hasNextLine())
{
scanner.nextLine();
count++;
}
System.out.println("Lines in the file: " + count); // Displays no. of lines in the input file.
double temp = (count/nol);
int temp1=(int)temp;
int nof=0;
if(temp1==temp)
{
nof=temp1;
}
else
{
nof=temp1+1;
}
System.out.println("No. of files to be generated :"+nof); // Displays no. of files to be generated.
//---------------------------------------------------------------------------------------------------------
// Actual splitting of file into smaller files
FileInputStream fstream = new FileInputStream(inputfile); DataInputStream in = new DataInputStream(fstream);
BufferedReader br = new BufferedReader(new InputStreamReader(in)); String strLine;
for (int j=1;j<=nof;j++)
{
FileWriter fstream1 = new FileWriter("C:/New Folder/File"+j+".txt"); // Destination File Location
BufferedWriter out = new BufferedWriter(fstream1);
for (int i=1;i<=nol;i++)
{
strLine = br.readLine();
if (strLine!= null)
{
out.write(strLine);
if(i!=nol)
{
out.newLine();
}
}
}
out.close();
}
in.close();
}catch (Exception e)
{
System.err.println("Error: " + e.getMessage());
}
}
}

Though its a old question but for reference I am listing out the code which I used to split large files to any sizes and it works with any Java versions above 1.4 .
Sample Split and Join blocks were like below:
public void join(String FilePath) {
long leninfile = 0, leng = 0;
int count = 1, data = 0;
try {
File filename = new File(FilePath);
//RandomAccessFile outfile = new RandomAccessFile(filename,"rw");
OutputStream outfile = new BufferedOutputStream(new FileOutputStream(filename));
while (true) {
filename = new File(FilePath + count + ".sp");
if (filename.exists()) {
//RandomAccessFile infile = new RandomAccessFile(filename,"r");
InputStream infile = new BufferedInputStream(new FileInputStream(filename));
data = infile.read();
while (data != -1) {
outfile.write(data);
data = infile.read();
}
leng++;
infile.close();
count++;
} else {
break;
}
}
outfile.close();
} catch (Exception e) {
e.printStackTrace();
}
}
public void split(String FilePath, long splitlen) {
long leninfile = 0, leng = 0;
int count = 1, data;
try {
File filename = new File(FilePath);
//RandomAccessFile infile = new RandomAccessFile(filename, "r");
InputStream infile = new BufferedInputStream(new FileInputStream(filename));
data = infile.read();
while (data != -1) {
filename = new File(FilePath + count + ".sp");
//RandomAccessFile outfile = new RandomAccessFile(filename, "rw");
OutputStream outfile = new BufferedOutputStream(new FileOutputStream(filename));
while (data != -1 && leng < splitlen) {
outfile.write(data);
leng++;
data = infile.read();
}
leninfile += leng;
leng = 0;
outfile.close();
count++;
}
} catch (Exception e) {
e.printStackTrace();
}
}
Complete java code available here in File Split in Java Program link.

a clean solution to edit.
this solution involves loading the entire file into memory.
set all line of a file in List<String> rowsOfFile;
edit maxSizeFile to choice max size of a single file splitted
public void splitFile(File fileToSplit) throws IOException {
long maxSizeFile = 10000000 // 10mb
StringBuilder buffer = new StringBuilder((int) maxSizeFile);
int sizeOfRows = 0;
int recurrence = 0;
String fileName;
List<String> rowsOfFile;
rowsOfFile = Files.readAllLines(fileToSplit.toPath(), Charset.defaultCharset());
for (String row : rowsOfFile) {
buffer.append(row);
numOfRow++;
sizeOfRows += row.getBytes(StandardCharsets.UTF_8).length;
if (sizeOfRows >= maxSizeFile) {
fileName = generateFileName(recurrence);
File newFile = new File(fileName);
try (PrintWriter writer = new PrintWriter(newFile)) {
writer.println(buffer.toString());
}
recurrence++;
sizeOfRows = 0;
buffer = new StringBuilder();
}
}
// last rows
if (sizeOfRows > 0) {
fileName = generateFileName(recurrence);
File newFile = createFile(fileName);
try (PrintWriter writer = new PrintWriter(newFile)) {
writer.println(buffer.toString());
}
}
Files.delete(fileToSplit.toPath());
}
method to generate Name of file:
public String generateFileName(int numFile) {
String extension = ".txt";
return "myFile" + numFile + extension;
}

Have a counter to count no of entries. Let's say one entry per line.
step1: Initially create new subfile, set counter=0;
step2: increment counter as you read each entry from source file to buffer
step3: when counter reaches limit to number of entries that you want to write in each sub file, flush contents of buffer to subfile. close the subfile
step4 : jump to step1 till you have data in source file to read from

There's no need to loop twice through the file. You could estimate the size of each chunk as the source file size divided by number of chunks needed. Then you just stop filling each cunk with data as it's size exceeds estimated.

Here is one that worked for me and I used it to split 10GB file. it also enables you to add a header and a footer. very useful when splitting document based format such as XML and JSON because you need to add document wrapper in the new split files.
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardOpenOption;
public class FileSpliter
{
public static void main(String[] args) throws IOException
{
splitTextFiles("D:\\xref.csx", 750000, "", "", null);
}
public static void splitTextFiles(String fileName, int maxRows, String header, String footer, String targetDir) throws IOException
{
File bigFile = new File(fileName);
int i = 1;
String ext = fileName.substring(fileName.lastIndexOf("."));
String fileNoExt = bigFile.getName().replace(ext, "");
File newDir = null;
if(targetDir != null)
{
newDir = new File(targetDir);
}
else
{
newDir = new File(bigFile.getParent() + "\\" + fileNoExt + "_split");
}
newDir.mkdirs();
try (BufferedReader reader = Files.newBufferedReader(Paths.get(fileName)))
{
String line = null;
int lineNum = 1;
Path splitFile = Paths.get(newDir.getPath() + "\\" + fileNoExt + "_" + String.format("%02d", i) + ext);
BufferedWriter writer = Files.newBufferedWriter(splitFile, StandardOpenOption.CREATE);
while ((line = reader.readLine()) != null)
{
if(lineNum == 1)
{
System.out.print("new file created '" + splitFile.toString());
if(header != null && header.length() > 0)
{
writer.append(header);
writer.newLine();
}
}
writer.append(line);
if (lineNum >= maxRows)
{
if(footer != null && footer.length() > 0)
{
writer.newLine();
writer.append(footer);
}
writer.close();
System.out.println(", " + lineNum + " lines written to file");
lineNum = 1;
i++;
splitFile = Paths.get(newDir.getPath() + "\\" + fileNoExt + "_" + String.format("%02d", i) + ext);
writer = Files.newBufferedWriter(splitFile, StandardOpenOption.CREATE);
}
else
{
writer.newLine();
lineNum++;
}
}
if(lineNum <= maxRows) // early exit
{
if(footer != null && footer.length() > 0)
{
writer.newLine();
lineNum++;
writer.append(footer);
}
}
writer.close();
System.out.println(", " + lineNum + " lines written to file");
}
System.out.println("file '" + bigFile.getName() + "' split into " + i + " files");
}
}

Below code used to split a big file into small files with lesser lines.
long linesWritten = 0;
int count = 1;
try {
File inputFile = new File(inputFilePath);
InputStream inputFileStream = new BufferedInputStream(new FileInputStream(inputFile));
BufferedReader reader = new BufferedReader(new InputStreamReader(inputFileStream));
String line = reader.readLine();
String fileName = inputFile.getName();
String outfileName = outputFolderPath + "\\" + fileName;
while (line != null) {
File outFile = new File(outfileName + "_" + count + ".split");
Writer writer = new OutputStreamWriter(new FileOutputStream(outFile));
while (line != null && linesWritten < linesPerSplit) {
writer.write(line);
line = reader.readLine();
linesWritten++;
}
writer.close();
linesWritten = 0;//next file
count++;//nect file count
}
reader.close();
} catch (Exception e) {
e.printStackTrace();
}

Split a file to multiple chunks (in memory operation), here I'm splitting any file to a size of 500kb(500000 bytes) :
public static List<ByteArrayOutputStream> splitFile(File f) {
List<ByteArrayOutputStream> datalist = new ArrayList<>();
try {
int sizeOfFiles = 500000;
byte[] buffer = new byte[sizeOfFiles];
try (FileInputStream fis = new FileInputStream(f); BufferedInputStream bis = new BufferedInputStream(fis)) {
int bytesAmount = 0;
while ((bytesAmount = bis.read(buffer)) > 0) {
try (OutputStream out = new ByteArrayOutputStream()) {
out.write(buffer, 0, bytesAmount);
out.flush();
datalist.add((ByteArrayOutputStream) out);
}
}
}
} catch (Exception e) {
//get the error
}
return datalist; }

I am a bit late to answer, But here's how I did it:
Approach:
First I determine how many bytes each of the individual files should contain then I split the large file by bytes. Only one file chunk worth of data is loaded into memory at a time.
Example:- if a 5 GB file is split into 10 files then only 500MB worth of bytes are loaded into memory at a time which are held in the buffer variable in the splitBySize method below.
Code Explaination:
The method splitFile first gets the number of bytes each of the individual file chunks should contain by calling the getSizeInBytes method, then it calls the splitBySize method which splits the large file by size (i..e maxChunkSize represents the number of bytes each of file chunks will contain).
public static List<File> splitFile(File largeFile, int noOfFiles) throws IOException {
return splitBySize(largeFile, getSizeInBytes(largeFile.length(), noOfFiles));
}
public static List<File> splitBySize(File largeFile, int maxChunkSize) throws IOException {
List<File> list = new ArrayList<>();
int numberOfFiles = 0;
try (InputStream in = Files.newInputStream(largeFile.toPath())) {
final byte[] buffer = new byte[maxChunkSize];
int dataRead = in.read(buffer);
while (dataRead > -1) {
list.add(stageLocally(buffer, dataRead));
numberOfFiles++;
dataRead = in.read(buffer);
}
}
System.out.println("Number of files generated: " + numberOfFiles);
return list;
}
private static int getSizeInBytes(long totalBytes, int numberOfFiles) {
if (totalBytes % numberOfFiles != 0) {
totalBytes = ((totalBytes / numberOfFiles) + 1)*numberOfFiles;
}
long x = totalBytes / numberOfFiles;
if (x > Integer.MAX_VALUE){
throw new NumberFormatException("Byte chunk too large");
}
return (int) x;
}
Full Code:
public class StackOverflow {
private static final String INPUT_FILE_PATH = "/Users/malkesingh/Downloads/5MB.zip";
private static final String TEMP_DIRECTORY = "/Users/malkesingh/temp";
public static void main(String[] args) throws IOException {
File input = new File(INPUT_FILE_PATH);
File outPut = fileJoin2(splitFile(input, 5));
try (InputStream in = Files.newInputStream(input.toPath()); InputStream out = Files.newInputStream(outPut.toPath())) {
System.out.println(IOUtils.contentEquals(in, out));
}
}
public static List<File> splitFile(File largeFile, int noOfFiles) throws IOException {
return splitBySize(largeFile, getSizeInBytes(largeFile.length(), noOfFiles));
}
public static List<File> splitBySize(File largeFile, int maxChunkSize) throws IOException {
List<File> list = new ArrayList<>();
int numberOfFiles = 0;
try (InputStream in = Files.newInputStream(largeFile.toPath())) {
final byte[] buffer = new byte[maxChunkSize];
int dataRead = in.read(buffer);
while (dataRead > -1) {
list.add(stageLocally(buffer, dataRead));
numberOfFiles++;
dataRead = in.read(buffer);
}
}
System.out.println("Number of files generated: " + numberOfFiles);
return list;
}
private static int getSizeInBytes(long totalBytes, int numberOfFiles) {
if (totalBytes % numberOfFiles != 0) {
totalBytes = ((totalBytes / numberOfFiles) + 1)*numberOfFiles;
}
long x = totalBytes / numberOfFiles;
if (x > Integer.MAX_VALUE){
throw new NumberFormatException("Byte chunk too large");
}
return (int) x;
}
private static File stageLocally(byte[] buffer, int length) throws IOException {
File outPutFile = File.createTempFile("temp-", "split", new File(TEMP_DIRECTORY));
try(FileOutputStream fos = new FileOutputStream(outPutFile)) {
fos.write(buffer, 0, length);
}
return outPutFile;
}
public static File fileJoin2(List<File> list) throws IOException {
File outPutFile = File.createTempFile("temp-", "unsplit", new File(TEMP_DIRECTORY));
FileOutputStream fos = new FileOutputStream(outPutFile);
for (File file : list) {
Files.copy(file.toPath(), fos);
}
fos.close();
return outPutFile;
}}

import java.util.*;
import java.io.*;
public class task13 {
public static void main(String[] args)throws IOException{
Scanner s =new Scanner(System.in);
System.out.print("Enter path:");
String a=s.next();
File f=new File(a+".txt");
Scanner st=new Scanner(f);
System.out.println(f.canRead()+"\n"+f.canWrite());
long l=f.length();
System.out.println("Length is:"+l);
System.out.print("Enter no.of partitions:");
int p=s.nextInt();
long x=l/p;
st.useDelimiter("\\Z");
String t=st.next();
int j=0;
System.out.println("Each File Length is:"+x);
for(int i=1;i<=p;i++){
File ft=new File(a+"-"+i+".txt");
ft.createNewFile();
int g=(j*(int)x);
int h=(j+1)*(int)x;
if(g<=l&&h<=l){
FileWriter fw=new FileWriter(a+"-"+i+".txt");
String v=t.substring(g,h);
fw.write(v);
j++;
fw.close();
}}
}}