I have a method that reads text from a file; decompression may be required, depending on an input parameter:
public static String readText(File inFile, boolean compressed) {
InputStream in = null;
InputStreamReader isr = null;
StringBuilder sb = new StringBuilder();//constant resizing is costly, so set the STRING_SIZE
try {
in = new FileInputStream(inFile);
if (compressed) {
in = new GZIPInputStream(in);
}
isr = new InputStreamReader(in);
int length = 0;
char[] cbuf = new char[8 * 1024];
while ((length = isr.read(cbuf)) != -1) {
sb.append(cbuf, 0, length);
}
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
in.close();
} catch (Exception e1) {
e1.printStackTrace();
}
}
return sb.toString();
}
It was suggested that I use InputStream like this so it is easier to write, and so that in the end I only have to close one thing. I am still a bit worried this might cause a memory leak. So my question is: does anyone knows if the code above is OK? Or do I have to get back to a dozen of streams and close them one by one in a finally block?
Thanks a lot.
Yes, closing the outermost stream/reader is sufficient.
However, your code has another potential bug: new InputStreamReader(in) will use the platform default encoding, which depends on the OS region/language settings. You should specify the encoding of the text file and use it explicitly in the constructor.
Here's one point to add: see if 'in' is null before calling 'in.close()' as the exception could happen without the first assignment succeeding.
Also, it's good form to only catch possible exceptions (e.g. IOException). That way if you add more code and the IDE tells you that a new exception type isn't handled you can add the proper specific code rather than never hearing about it because the catch (Exception ) which was originally for IOException is also (mishandling?) every other type.
Here's the clean Java 7 way which works for anything that implements AutoCloseable/Closeable:
try (InputStream in = compressed ?
new GZIPInputStream(new FileInputStream(inFile))
: new FileInputStream(inFile);
InputStreamReader isr = new InputStreamReader(in))
{
int length = 0;
char[] cbuf = new char[8 * 1024];
while ((length = isr.read(cbuf)) != -1) {
sb.append(cbuf, 0, length);
}
}
catch (Exception e) {
e.printStackTrace();
}
If you're wondering what happens if there's an exception while closing the resource, read about getSuppressedExceptions() which was also added.
Related
When I start with an unbounded Stream and use the limit() method to put a bound on it, how can I clean up the resources that were used by the Stream once the limit is reached? For example, if I'm trying to do what the Files.lines method does, but with floats instead of strings, I would write a function that looks something like this:
public static Stream<Float> floats(File f) throws FileNotFoundException {
FileInputStream fis = new FileInputStream(f);
return Stream.generate(() -> {
byte[] buff = new byte[4];
try { fis.read(buff); }
catch (Exception e) { e.printStackTrace(); }
return ByteBuffer.wrap(buff).getFloat(0);
}).limit(f.length()/4);
}
This should stream a large binary file full of floats until I reach the end of the file. I divide the length of the file in bytes by the four bytes that make up a float. However, I'd like to find some way to execute fis.close() once this limit is reached. Does the Streams API have any way of letting you do this?
I came up with another possibility. It's kind of a hack, but it seems to work. I just run the Stream through a filter that always returns true, but in the process of doing so, it checks for EOF and closes the input stream if it's reached. I was hoping there was a cleaner way to do this built into the API.
Stream<Float> answer;
FileInputStream fis = new FileInputStream(f);
answer = Stream
.generate(() -> {
byte[] buff = new byte[4];
try { fis.read(buff); }
catch (Exception e) { e.printStackTrace(); }
return ByteBuffer.wrap(buff).getFloat(0);})
.filter(x -> {
try { if (fis.available() == 0) fis.close(); }
catch (Exception e) { e.printStackTrace(); }
return true;})
.limit(f.length()/4);
return answer;
I’d use an entirely different approach:
public static Stream<Float> floats(File f) throws FileNotFoundException {
try(FileChannel fch = FileChannel.open(f.toPath(), StandardOpenOption.READ)) {
ByteBuffer bb = fch.map(FileChannel.MapMode.READ_ONLY, 0, fch.size());
FloatBuffer fb = bb.asFloatBuffer();
return IntStream.range(0, fb.remaining()).mapToObj(fb::get);
}
catch(IOException ex) { // or consider to declare "throws IOException" instead
FileNotFoundException fnfe = new FileNotFoundException(ex.getMessage());
fnfe.initCause(ex);
throw fnfe;
}
}
with this, the FileChannel has been closed already when the method returns. It’s also potentially more efficient. Note that for a lot of operations, it might be more efficient to process the data as DoubleStream, even when the source values are floats and only narrow the type of the final result back to float, if necessary:
public static DoubleStream floatsAsDouble(File f) throws FileNotFoundException {
try(FileChannel fch = FileChannel.open(f.toPath(), StandardOpenOption.READ)) {
ByteBuffer bb = fch.map(FileChannel.MapMode.READ_ONLY, 0, fch.size());
FloatBuffer fb = bb.asFloatBuffer();
return IntStream.range(0, fb.remaining()).mapToDouble(fb::get);
}
catch(IOException ex) { // or consider to declare throwsIOException instead
FileNotFoundException fnfe = new FileNotFoundException(ex.getMessage());
fnfe.initCause(ex);
throw fnfe;
}
}
…
float sum=(float)floatsAsDouble(new File("path")).sum();
Also, you may consider to directly work with Paths instead of File.
Recently, I used the AppScan Source to scan the coding, and it found out one of the finding which I don't know how to fix and pass to the scanner or is it a false alarm?
Here's my code.
public static void copyFileUsingFileStreams(File source, File dest)
throws IOException
{
InputStream input = null;
OutputStream output = null;
try
{
input = new FileInputStream(source);
output = new FileOutputStream(dest);
byte[] buf = new byte[1024];
int bytesRead;
while ((bytesRead = input.read(buf)) > 0) {
output.write(buf, 0, bytesRead); //Scanner reported that's a vulnerability in API OutputStream.write()
}
}
finally
{
input.close();
output.close();
}
}
Looks fine to me. There's no need for an encoding if you're writing bytes. If you were writing characters, that would be a different matter.
That said, you really ought to be wrapping your streams in buffered versions:
output = new BufferedOutputStream(new FileOutputStream(dest));
and similarly for input. That doesn't affect anything to do with encodings, but it will make the file I/O more efficient.
You should also change your finally block:
finally
{
if (input!=null)
input.close();
if (output!=null)
output.close();
}
When you start to catch and deal with IOExceptions, as you ought to be doing, you'll be risking a NullPointerException if anything goes wrong while your streams are being created.
I need to read some data until file is opened at different times, but I'm not sure if pointer to data that have not been read yet is automatic increased?
My method:
//method for copy binary data from file to binaryDataBuffer
void readcpy(String fileName, int pos, int len) {
try {
File lxDirectory = new File(Environment.getExternalStorageDirectory().getPath() + "/DATA/EXAMPLE/");
File lxFile = new File(lxDirectory, (fileName);
FileInputStream mFileInputStream = new FileInputStream(lxFile);
mFileInputStream.read(binaryDataBuffer, pos, len);
}
catch (Exception e) {
Log.d("Exception", e.getMessage());
}
}
So, if I call this method first time and read and save 5 bytes for example, will be on next call of the method read out bytes from 5th byte? I don't close file after reading.
When you create an InputStream (because a FileInputStream is an InputStream), the stream is created anew each time, and starts at the beginning of the stream (therefore the file).
If you want to read from where you left off the last time, you need to retain the offset and seek -- or retain the initial input stream you have opened.
While you can seek into a stream (using .skip()), it is in any event NOT recommended to reopen each time, it is costly; also, when you are done with a stream, you should close it:
// with Java 7: in automatically closed
try (InputStream in = ...;) {
// do stuff
} catch (WhateverException e) {
// handle exception
}
// with Java 6
InputStream in = ...;
try {
// do stuff
} catch (WhateverException e) {
// handle exception
} finally {
in.close();
}
Try this code:
public String getStringFromFile (String filePath) throws Exception {
File fl = new File(filePath);
FileInputStream fin = new FileInputStream(fl);
BufferedReader reader = new BufferedReader(new InputStreamReader(fin));
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line).append("\n");
}
String ret = sb.toString();
//Make sure you close all streams.
fin.close();
reader.close();
return ret;
}
I find RandomAccessFile, it has offset which I need in my case.
Is there a way to check whether a file was correctly written, I mean if there is an EOF at the end?
I'm asking that because I have a program that takes some file, merge them in a very big file and then use it to get statistics from it.
The point is that the second part never ends because it doesn't recognize the end of file.
The relevant parts of the code are the following:
(please do not ask for the whole code as I cannot post for important reasons)
FileWriter file=null;
PrintWriter pw = null;
String pathToRead=null;
InputStreamReader isr = null;
BufferedReader br = null ;
FileInputStream fis = null ;
TestJFileChooser d=new TestJFileChooser();
int c=1;
String line=null;
....
//here i select the files
selectedFile=new File(pathToRead);
//here I get one buffer reader for each file got with listFiles()
for(File file_sel:app){
if (file_sel.getName().startsWith("gtou")){
System.out.println(file_sel.getName());
fis = null;
try {
fis = new FileInputStream(file_sel);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
isr=new InputStreamReader(fis);
br=new BufferedReader(isr);
map.put(i, br);
num_file++;
i++;
}
}
//then I select the output file and open a print writer for it
fileToWrite=new File(pathToRead);
try {
file = new FileWriter(fileToWrite);
pw= new PrintWriter(file);
} catch (IOException e1) {
e1.printStackTrace();
}
//merging part
....
line=br.readLine();
while(line!=null){
System.out.println("line is:"+line);
....
line=br.readLine();
}
//end of merging ....
pw.flush();
pw.close();
try {
if (file!=null) file.close();
fis.close();
isr.close();
br.close();
for(int fi=0;fi<num_file;fi++){
br2=map.get(fi);
br2.close();
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
so.kill();
Runtime r=Runtime.getRuntime();
r.gc();
//this is a popup that comes out
GlitchSquad gli=new GlitchSquad("Completed");
the problem is that as output I get:
line is: null ;
line is: null ;
line is: null ;
etc
And never get to "completed" popup =(
I cannot understand what is exactly that null because the control line!=null doesn't work.
I also tried to use that null as a string ..but nothing..
I thought that was a problem in how I close the streams but now the code seems correct to me ..but still no way to stop it..
Suggestion?
Thanks in advance!
p.s. it is a summarized version in order to focus on the streams.. variables are correctly declared and the same is for imports etc
edit: code updated
EOF is EOF. There is no more data. Unless you have an expected EOF mark within the file, or a self-describing protocol that tells you where the EOF mark should be, there is no way to determine whether the file was completely written.
I don't know if it will solve your problem, but I'd be using this code instead:
try {
fis = new FileInputStream(file_sel);
isr=new InputStreamReader(fis);
br=new BufferedReader(isr);
map.put(num_file++, br);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
Otherwise there may be uncaught "NullPointer"-exceptions or strange BufferedReaders in your "map". ( I don't right now know how new InputStreamReader(null) will behave.)
It looks like i and num_file have always equal values, so just drop i. Or use a LinkedList and drop both.
If there's not a special merging that you have to do, I'd just do it like this:
OutputStream os;
try {
os = new FileOuputStream(outfile);
} catch (FileNotFoundException e) {
os = null;
e.printStackTrace();
}
if (os != null) {
for(File file_sel:app) {
if (file_sel.getName().startsWith("gtou")) {
System.out.println(file_sel.getName());
InputStream is = null;
try {
is = new FileInputStream(file_sel);
byte[] buffer = new byte[1024];
int readBytes = 0;
while ((readBytes = is.read(buffer)) > 0) {
os.write(buffer, 0, readBytes);
}
fos.flush();
is.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
}
}
}
}
If you read files with different encodings, you will have to modify at least the reading of course.
If it doesn't work, I'd suggest you build a "summarized" and runable sample program.
The core of your question is this code:
BufferedReader br = ...
String line = br.readLine();
while (line != null) {
System.out.println("line is:" + line);
...
line = br.readLine();
}
You say that this repeatedly outputs this:
line is: null ;
line is: null ;
(Notice the " ;" on the end!!!)
The only way that can happen is if the file you are reading contains at least one line that look like this:
null ;
Indeed, unless the "..." code includes a continue statement, there must must be lots of those lines in the input file.
Is there a way to check whether a file was correctly written?
Yea. Look at it using a text editor and/or check its file size.
I mean if there is an EOF at the end?
In modern file systems, EOF is a position not a marker. Specifically it is the position after the last byte of the file. So it is logically impossible for a file to not have an EOF. (You'd have to have a file that is infinite in length for there to be no EOF.)
I'm relatively new to Java and I'm attempting to write a simple android app. I have a large text file with about 3500 lines in the assets folder of my applications and I need to read it into a string. I found a good example about how to do this but I have a question about why the byte array is initialized to 1024. Wouldn't I want to initialize it to the length of my text file? Also, wouldn't I want to use char, not byte? Here is the code:
private void populateArray(){
AssetManager assetManager = getAssets();
InputStream inputStream = null;
try {
inputStream = assetManager.open("3500LineTextFile.txt");
} catch (IOException e) {
Log.e("IOException populateArray", e.getMessage());
}
String s = readTextFile(inputStream);
// Add more code here to populate array from string
}
private String readTextFile(InputStream inputStream) {
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
inputStream.length
byte buf[] = new byte[1024];
int len;
try {
while ((len = inputStream.read(buf)) != -1) {
outputStream.write(buf, 0, len);
}
outputStream.close();
inputStream.close();
} catch (IOException e) {
Log.e("IOException readTextFile", e.getMessage());
}
return outputStream.toString();
}
EDIT: Based on your suggestions, I tried this approach. Is it any better? Thanks.
private void populateArray(){
AssetManager assetManager = getAssets();
InputStream inputStream = null;
Reader iStreamReader = null;
try {
inputStream = assetManager.open("List.txt");
iStreamReader = new InputStreamReader(inputStream, "UTF-8");
} catch (IOException e) {
Log.e("IOException populateArray", e.getMessage());
}
String String = readTextFile(iStreamReader);
// more code here
}
private String readTextFile(InputStreamReader inputStreamReader) {
StringBuilder sb = new StringBuilder();
char buf[] = new char[2048];
int read;
try {
do {
read = inputStreamReader.read(buf, 0, buf.length);
if (read>0) {
sb.append(buf, 0, read);
}
} while (read>=0);
} catch (IOException e) {
Log.e("IOException readTextFile", e.getMessage());
}
return sb.toString();
}
This example is not good at all. It's full of bad practices (hiding exceptions, not closing streams in finally blocks, not specify an explicit encoding, etc.). It uses a 1024 bytes long buffer because it doesn't have any way of knowing the length of the input stream.
Read the Java IO tutorial to learn how to read text from a file.
You are reading the file into a buffer of 1024 Bytes.
Then those 1024 bytes are written to outputStream.
This process repeats until the whole file is read into the outputStream.
As JB Nizet mentioned the example is full of bad practices.
Wouldn't I want to initialize it to the length of my text file? Also, wouldn't I want to use char, not byte?
Yes, and yes ... and as other answers have said, you've picked an example with a number of errors in it.
However, there is a theoretical problem doing both; i.e. setting the buffer length to the file length and using a character buffer rather than a byte buffer. The problem is that the file size is measured in bytes, but the size of the buffer needs to be measured in characters. This is normally fine, but it is theoretically possible that you will need more characters than the file size in bytes; e.g. if the input file used a 6 bit character set and packed 4 characters into 3 bytes.
To read from a file I usaully use a Scanner and a StringBuilder.
Scanner scan = new Scanner(new BufferedInputStream(new FileInputStream(filename)), "UTF-8");
StringBuilder sb = new StringBuilder();
while (scan.hasNextLine()) {
sb.append(scan.nextLine());
sb.append("\n");
}
scan.close
return sb.toString();
Try to throw your exceptions instead of swallowing them. The caller must know there was a problem reading your file.
Edit: Also note that using a BufferedInputStream is important. Otherwise it will try to read bytes by bytes which can be slow.