I was wondering how do you manipulate big Textfiles in Java, if we assume that the Filesize is larger than the memory. I googled that topic and it shows that most people recommend java.niofor such a task.
Unfortunately I haven't found any documentation on how to manipulate the File. For example read every Line, modify it, write it. I tried something like this, but this doesn't work:
FileChannel fileChannel = null;
try {
fileChannel = new RandomAccessFile(file, "rw").getChannel();
ByteBuffer buffer = ByteBuffer.allocate(256);
while (fileChannel.read(buffer) != -1) {
buffer.rewind();
buffer.flip();
String nextLine = buffer.asCharBuffer().toString();
if (replaceBackSlashes) {
nextLine = nextLine.replace("\\\\", "/");
}
if (!(removeEmptyLines && StringUtils.isEmpty(nextLine))) {
buffer.flip();
buffer.asCharBuffer().put(nextLine);
}
buffer.clear();
}
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
if (fileChannel != null) {
try {
fileChannel.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
So what are your recommendations? Also the String nextline, doesn't match anything in my File. Maybe I need to set the encoding?
Line by line. Something like this ...
public static void main(String[] args) throws Exception {
File someFile = new File("someFile.txt");
File temp = File.createTempFile(someFile.getName(), null);
BufferedReader reader = null;
PrintStream writer = null;
try {
reader = new BufferedReader(new FileReader(someFile));
writer = new PrintStream(temp);
String line;
while ((line = reader.readLine())!=null) {
// manipulate line
writer.println(line);
}
}
finally {
if (writer!=null) writer.close();
if (reader!=null) reader.close();
}
if (!someFile.delete()) throw new Exception("Failed to remove " + someFile.getName());
if (!temp.renameTo(someFile)) throw new Exception("Failed to replace " + someFile.getName());
}
Kudos to xagyg for a nice, clean answer! The following just didn't fit into a comment:
If you're running Java 7 already, you can save a lot of boilerplate code by using try-with-resources for the processing loop:
File source = ...
File target = ...
try (BufferedReader in = new BufferedReader(new FileReader(source));
PrintStream out = new PrintStream(target)) {
String line;
while ((line = in.readLine()) != null) {
// manipulate line
out.println(line);
}
}
// no catch or finally clause!
No more of that initalize-to-null-try-catch-finally-close-if-not-null mess, Java will take care of that for you now. Less code, less potential to forget or screw up that crucial call to close().
Related
I'm currently trying to save the output of my code into a text file, when I run it on a different project it generates the output file and stores the output respectively, however when I run the same code in a different project it gives me blank output file and I do not really know what's the matter. I'm confused as to where to put the .close() function and the flush function as well. Thank you in advance!
FileWriter output = new FileWriter("set.txt");
BufferedWriter writer = new BufferedWriter(output);
InputStream fis_n = new FileInputStream("/Users/User/NetBeansProjects/Test/src/test/sample.txt");
InputStreamReader isr_n = new InputStreamReader(fis_n, Charset.forName("UTF-8"));
BufferedReader br_n = new BufferedReader(isr_n);
while ((input = br_n.readLine()) != null) {
String[] s = input.split(":");
if (s[1].equals(text)) {
writer.write(s[0] + "'s result is " + sample_text);
writer.newLine();
break;
}
}
writer.close();
output.close();
This is what the edited code looks like, yet still the output file "set.txt" is empty upon running the program.
FileWriter output = new FileWriter("set.txt");
BufferedWriter writer = new BufferedWriter(output);
InputStream fis_n = new FileInputStream("/Users/User/NetBeansProjects/Test/src/test/sample.txt");
InputStreamReader isr_n = new InputStreamReader(fis_n, Charset.forName("UTF-8"));
BufferedReader br_n = new BufferedReader(isr_n);
try {
while ((input = br_n.readLine()) != null) {
String[] s = input.split(":");
if (s[1].equals(text)) {
writer.write(s[0] + "'s result is " + sample_text);
writer.newLine();
break;
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
writer.close();
fis_n.close();
isr_n.close();
br_n.close();
} catch (IOException e) {
e.printStackTrace();
}
}
// fis_n.close();
//isr_n.close();
//br_n.close();
}
This is what the final code looks like:
public static void dictionary(String sample_text, String text) throws FileNotFoundException, IOException {
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(
new FileInputStream("/Users/User/NetBeansProjects/Test/src/test/sample.txt"),
Charset.forName("UTF-8")
));
try {
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(
new FileOutputStream("/Users/User/NetBeansProjects/Test/src/test/set.txt"),
Charset.forName("UTF-8")
));
try {
String input;
while ((input = reader.readLine()) != null) {
String[] s = input.split(":");
if (s[1].equals(text)) {
writer.write(s[0] + "'s result is " + sample_text);
writer.newLine();
break;
}
}
} finally {
writer.close();
}
} finally {
reader.close();
}
} catch (IOException e) {
// Error handling
}
}
This is the main method where the dictionary method is being called.
public static void main(String[] args) throws FileNotFoundException, IOException {
case 2: {
BufferedReader d_br = new BufferedReader(new InputStreamReader(
new FileInputStream("/Users/User/NetBeansProjects/Test/src/test/input_file.txt"),
Charset.forName("UTF-8")
));
try {
String d_line;
while ((d_line = d_br.readLine()) != null) {
String h_input = test(d_line);
dictionary(d_line, h_input);
}
} catch(IOException e){
}finally {
d_br.close();
}
break;
}
}
You should put writer.close() after the while loop, and preferable, into the finally section.
If there is no requirement to store partially-processed files (as in most cases), you may remove flush at all. In the other case, it is better to leave it where it is.
The generic case of resource usage on Java 7+ looks like follows (this syntax is called try-with-resources:
try (
Resource resource1 = // Resource 1 initialization
Resource resource2 = // Resource 2 initialization
...
) {
// Resource utilization
} catch (XXXException e) {
// Something went wrong
}
Resource are freed (closed) automatically by try-with-resources.
If you need to use Java 6 or earlier, the above code could be roughly translated to the following (actually there are some subtle differences, that is not important at this level of details).
try {
Resource1 resource1 = // Resource initialization
try {
Resource2 resource2 = // Resource initialization
try {
// Resource utilization
} finally {
// Free resource2
resource2.close();
}
} finally {
// Free resource1
resource1.close();
}
} catch (XXXException e) {
// Something went wrong
}
Notice, how nested try-finally blocks used for resource management.
In your particular case we need to manage two resources: Reader and Writer, so the code will look as follows:
try (
// Notice, we build BufferedReader for the file in a single expression
BufferedReader reader = new BufferedReader(new InputStreamReader(
new FileInputStream("sample.txt"),
StandardCharsets.UTF_8 // Better replacement for Charset.forName("UTF-8")
));
// Alternative way to do the same
// BufferedReader reader = Files.newBufferedReader(Paths.get("sample.txt"), StandardCharsets.UTF_8);
// Output charset for writer provided explicitly
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(
new FileOutputStream("set.txt"),
StandardCharsets.UTF_8
))
// Alternative way to do the same
// BufferedWriter writer = Files.newBufferedWriter(Paths.get("set.txt"), StandardCharsets.UTF_8)
) {
String input;
while ((input = reader.readLine()) != null) {
String[] s = input.split(":");
if (s[1].equals(text)) {
writer.write(s[0] + "'s result is " + text);
writer.newLine();
break;
}
}
} catch (IOException e) {
// Error handling
}
Or, using pre-Java7 syntax:
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(
new FileInputStream("sample.txt"),
Charset.forName("UTF-8")
));
try {
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(
new FileOutputStream("set.txt"),
Charset.forName("UTF-8")
));
try {
String input;
while ((input = reader.readLine()) != null) {
String[] s = input.split(":");
if (s[1].equals(text)) {
writer.write(s[0] + "'s result is " + text);
writer.newLine();
break;
}
}
} finally {
writer.close();
}
} finally {
reader.close();
}
} catch (IOException e) {
// Error handling
}
First of all, you call the flush method of a writer, whenever you want the current buffer to be written immediately. If you just write a file completely without any intermediate operation on your output, you do not need to call it explicitly, since the close call will do that for you.
Secondly, you only call the close method of the top-level reader or writer, in your case BufferedWriter. The close call is forwarded to the other assigned readers or writers. Multiple consecutive close calls do not have any effect on a previously closed instance, see here.
As a general note to using readers and writers, consider this pattern:
// This writer must be declared before 'try' to
// be visible in the finally block
AnyWriter writer = null;
try {
// Instantiate writer here, because it can already
// throw an IOException
writer = new AnyWriter();
// The the writing in a loop or as you wish
// If you need to write out the buffer in
// between, call flush
} catch (IOException e) {
// Something went wrong while writing
} finally {
try {
if (writer != null)
writer.close();
} catch (IOException e) {
// Exception while trying to close
}
}
The finally block is ALWAYS executed. If you need a more compact syntax and you use at least Java 7, you can have a look at the try-with notation here.
embedded system project in which i will get some response from my microcontroller to android device using bluetooth module and i cant get the bytes from this line bytes = "mmInStream.read(buffer)"..
when i convert the byte[] buffer into String using this
String data=new String(bytes) I dont get data that i had sent from my microcontroller properly. Sometimes charactors are missing..
public void run() {
Log.i(TAG, "BEGIN mConnectedThread");
byte[] buffer = new byte[1024];
int bytes;
// Keep listening to the InputStream while connected
while (true) {
try {
// Read from the InputStream
bytes = mmInStream.read(buffer);
String data=new String(bytes);
System.out.println(data);
// Send the obtained bytes to the UI Activity
} catch (IOException e) {
Log.e(TAG, "disconnected", e);
connectionLost();
break;
}
}
}
Please Help me
Try to use BufferedReader instead.
It reads text from a character-input stream, buffering characters so
as to provide for the efficient reading of characters, arrays, and
lines.
If you use Java 7 or older the following code will help:
try (BufferedReader reader = new BufferedReader(new InputStreamReader(mmInStream))){
String line = null;
while((line = reader.readLine()) != null) {
System.out.println(line);
}
connectionLost();
} catch(IOException e) {
e.printStackTrace();
}
If you use Java 6 or younger than use this code:
BufferedReader reader = null;
try {
reader = new BufferedReader(new InputStreamReader(mmInStream));
String line = null;
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
if (reader != null) {
try {
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
connectionLost();
}
But this approach has disadvantages. You can read about them, e.g. here
I'm performing certain commands through command prompt and storing the values in a text file.
wmic logicaldisk where drivetype=3 get deviceid > drive.txt
Now I want to read the string stored in the text file from my java file. When I try to do this:
try {
File file = new File("drive.txt");
FileReader reader = new FileReader(file);
BufferedReader in = new BufferedReader(reader);
int i=0;
while ((string[i] = in.readLine()) != null) {
System.out.println(string[i]);
++i;
}
in.close();
} catch (IOException e) {
e.printStackTrace();
}
I get the output as follows:
ÿþD[]E[]V[]I[]C[]E[]
how to avoid this?
while ((string[i] = in.readLine()) != null) {
System.out.println(string[2]);
}
over there you are missing the i++;
However I would advise you to use this structure: Use a ArrayList instead of an array, since this allows you to have a self-resizing structure, also instead in the while use the method ready(); from the BufferedRead in order to check the end from the document, at the end the for it's just to display the elements in String ArrayList.
ArrayList<String> string = new ArrayList<String>();
try {
File file = new File("drive.txt");
BufferedReader entrada;
entrada = new BufferedReader(new FileReader(file));
entrada.readLine();
while (entrada.ready()) {
string.add(entrada.readLine());
}
entrada.close();
} catch (IOException e) {
e.printStackTrace();
}
for (String elements : string) {
System.out.println(elements);
}
Why do you need a string array here? The size of the array may be wrong? Simply use a string instead of array. I tried this and works fine for me:
try {
String string;
File file = new File("drive.txt");
FileReader reader = new FileReader(file);
BufferedReader in = new BufferedReader(reader);
int i = 0;
while ((string = in.readLine()) != null) {
System.out.println(string);
++i;
}
in.close();
} catch (IOException e) {
e.printStackTrace();
}
If you are using eclipse IDE, change the encoding type. Go to Edit->Set Encoding-> Others->UTF-8.
I have few text files. Each text file contains some path and/or the reference of some other file.
File1
#file#>D:/FilePath/File2.txt
Mod1>/home/admin1/mod1
Mod2>/home/admin1/mod2
File2
Mod3>/home/admin1/mod3
Mod4>/home/admin1/mod4
All I want is, copy all the paths Mod1, Mod2, Mod3, Mod4 in another text file by supplying only File1.txt as input to my java program.
What I have done till now?
public void readTextFile(String fileName){
try {
br = new BufferedReader(new FileReader(new File(fileName)));
String line = br.readLine();
while(line!=null){
if(line.startsWith("#file#>")){
String string[] = line.split(">");
readTextFile(string[1]);
}
else if(line.contains(">")){
String string[] = line.split(">");
svnLinks.put(string[0], string[1]);
}
line=br.readLine();
}
} catch (Exception e) {
e.printStackTrace();
}
}
Currently my code reads the contents of File2.txt only, control does not come back to File1.txt.
Please ask if more inputs are required.
First of all you are jumping to another file without closing the current reader and when you come back you lose the cursor. Read one file first and then write all its contents that match to another file. Close the current reader (Don't close the writer) and then open the next file to read and so on.
Seems pretty simple. You need to write your file once your svnLinks Map is populated, assuming your present code works (haven't seen anything too weird in it).
So, once the Map is populated, you could use something along the lines of:
File newFile = new File("myPath/myNewFile.txt");
// TODO check file can be written
// TODO check file exists or create
FileOutputStream fos = null;
OutputStreamWriter osw = null;
BufferedWriter bw = null;
try {
fos = new FileOutputStream(newFile);
osw = new OutputStreamWriter(fos);
bw = new BufferedWriter(osw);
for (String key: svnLinks.keySet()) {
bw.write(key.concat(" my separator ").concat(svnLinks.get(key)).concat("myNewLine"));
}
}
catch (Throwable t) {
// TODO handle more gracefully
t.printStackTrace();
if (bw != null) {
try {
bw.close();
}
catch (Throwable t) {
t.printStackTrace();
}
}
Here is an non-recursive implementation of your method :
public static void readTextFile(String fileName) throws IOException {
LinkedList<String> list = new LinkedList<String>();
list.add(fileName);
while (!list.isEmpty()) {
BufferedReader br = null;
try {
br = new BufferedReader(new FileReader(new File(list.pop())));
String line;
while ((line = br.readLine()) != null) {
if (line.startsWith("#file#>")) {
String string[] = line.split(">");
list.add(string[1]);
} else if (line.contains(">")) {
String string[] = line.split(">");
svnLinks.put(string[0], string[1]);
}
}
} catch (Exception e) {
e.printStackTrace();
} finally {
br.close();
}
}
}
Just used a LinkedList to maintain the order. I suggest you to add some counter if you to limit the reading of files to a certain number(depth). eg:
while (!list.isEmpty() && readCount < 10 )
This will eliminate the chance of running the code to infinity(in case of circular reference).
I need to read some data until file is opened at different times, but I'm not sure if pointer to data that have not been read yet is automatic increased?
My method:
//method for copy binary data from file to binaryDataBuffer
void readcpy(String fileName, int pos, int len) {
try {
File lxDirectory = new File(Environment.getExternalStorageDirectory().getPath() + "/DATA/EXAMPLE/");
File lxFile = new File(lxDirectory, (fileName);
FileInputStream mFileInputStream = new FileInputStream(lxFile);
mFileInputStream.read(binaryDataBuffer, pos, len);
}
catch (Exception e) {
Log.d("Exception", e.getMessage());
}
}
So, if I call this method first time and read and save 5 bytes for example, will be on next call of the method read out bytes from 5th byte? I don't close file after reading.
When you create an InputStream (because a FileInputStream is an InputStream), the stream is created anew each time, and starts at the beginning of the stream (therefore the file).
If you want to read from where you left off the last time, you need to retain the offset and seek -- or retain the initial input stream you have opened.
While you can seek into a stream (using .skip()), it is in any event NOT recommended to reopen each time, it is costly; also, when you are done with a stream, you should close it:
// with Java 7: in automatically closed
try (InputStream in = ...;) {
// do stuff
} catch (WhateverException e) {
// handle exception
}
// with Java 6
InputStream in = ...;
try {
// do stuff
} catch (WhateverException e) {
// handle exception
} finally {
in.close();
}
Try this code:
public String getStringFromFile (String filePath) throws Exception {
File fl = new File(filePath);
FileInputStream fin = new FileInputStream(fl);
BufferedReader reader = new BufferedReader(new InputStreamReader(fin));
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line).append("\n");
}
String ret = sb.toString();
//Make sure you close all streams.
fin.close();
reader.close();
return ret;
}
I find RandomAccessFile, it has offset which I need in my case.