I read from an input stream and in my finally block try to close it by calling in.close();
However the execution of main gets blocked . What is the way out ?
On suggestion , the code I used is ,
if (!processed) {
System.out.println("in processed");
byte[] contents = new byte[(int) fileSplit.getLength()];
Path file = fileSplit.getPath();
FileSystem fs = file.getFileSystem(conf);
FSDataInputStream in = null;
try {
in = fs.open(file);
IOUtils.readFully(in, contents, 0, contents.length);
value.set(contents, 0, contents.length);
} finally {
System.out.println("before close stream");
IOUtils.closeStream(in);
}
processed = true;
return true;
}
System.out.println("out of processed");
return false;
}
java.io.InputStream.close is not blocking, at least API never says that. Compare
InputStream.read
Reads the next byte of data from the input stream. The value byte is
returned as an <code>int</code> in the range <code>0</code> to
<code>255</code>. If no byte is available because the end of the stream
has been reached, the value <code>-1</code> is returned. This method
blocks until input data is available, the end of the stream is detected,
or an exception is thrown.
and InputStream.close
Closes this file input stream and releases any system resources
associated with the stream.
As for solving your problem I would suggest to use Java 7 Files.readAllBytes and forget about your puzzle.
Related
I wrote a piece of Java code to send PDF-turned postscript scripts to a network printer via Socket.
The files were printed in perfect shape but every job comes with one or 2 extra pages with texts like ps: stack underflow or error undefined offending command.
At beginning I thought something is wrong with the PDF2PS process so I tried 2 PS files from this PS Files. But the problem is still there.
I also verified the ps files with GhostView. Now I think there may be something wrong with the code. The code does not throw any exception.
The printer, Toshiba e-studion 5005AC, supports PS3 and PCL6.
File file = new File("/path/to/my.ps");
Socket socket = null;
DataOutputStream out = null;
FileInputStream inputStream = null;
try {
socket = new Socket(printerIP, printerPort);
out = new DataOutputStream(socket.getOutputStream());
DataInputStream input = new DataInputStream(socket.getInputStream());
inputStream = new FileInputStream(file);
byte[] buffer = new byte[8000];
while (inputStream.read(buffer) != -1) {
out.write(buffer);
}
out.flush();
} catch (IOException e) {
e.printStackTrace();
}
You are writing the whole buffer to the output stream regardless of how much actual content there is.
That means that when you write the buffer the last time it will most probably have a bunch of content from the previous iteration at the end of the buffer.
Example
e.g. imagine you have the following file and you use a buffer of size 10:
1234567890ABCDEF
After first inputStream.read() call it will return 10 and in the buffer you will have:
1234567890
After second inputStream.read() call it will return 6 and in the buffer you will have:
ABCDEF7890
After third inputStream.read() call it will return -1 and you will stop reading.
A printer socket will receive these data in the end:
1234567890ABCDEF7890
Here the last 7890 is an extra bit that the printer does not understand, but it can successfully interpret the first 1234567890ABCDEF.
Fix
You should consider the length returned by inputStream.read():
byte[] buffer = new byte[8000];
for (int length; (length = inputStream.read(buffer)) != -1; ){
out.write(buffer, 0, length);
}
Also consider using try-with-resources to avoid problems with unclosed streams.
according to :
Note that while some implementations of InputStream will return the
total number of bytes in the stream, many will not. It is never
correct to use the return value of this method to allocate a buffer
intended to hold all data in this stream.
from:
http://docs.oracle.com/javase/7/docs/api/java/io/InputStream.html#available%28%29
and this note
In particular, code of the form
int n = in.available();
byte buf = new byte[n];
in.read(buf);
is not guaranteed to read all of the remaining bytes from the given input stream.
http://docs.oracle.com/javase/8/docs/technotes/guides/io/troubleshooting.html
dose it mean that using below function cause not to read file completely?
/**
* Reads a file from /raw/res/ and returns it as a byte array
* #param res Resources instance for Mosembro
* #param resourceId ID of resource (ex: R.raw.resource_name)
* #return byte[] if successful, null otherwise
*/
public static byte[] readRawByteArray(Resources res, int resourceId)
{
InputStream is = null;
byte[] raw = new byte[] {};
try {
is = res.openRawResource(resourceId);
raw = new byte[is.available()];
is.read(raw);
}
catch (IOException e) {
e.printStackTrace();
raw = null;
}
finally {
try {
is.close();
}
catch (IOException e) {
e.printStackTrace();
}
}
return raw;
}
available() returns the number of bytes that can be read without blocking. There is no necessary correlation between that number, which can be zero, and the total length of the file.
Yes it does not necessarily read all. Like RandomAccessFile.read(byte[]) as opposed to RandomAccessFile.readFully(byte[]). Furthermore the code actually physically reads 0 bytes.
It probably reads only the first block, if it were a slow device like a file system.
The principle:
The file is being read by the underlying system software, normally
buffered, so you have a couple of blocks already in memory, and
sometimes already reading further. The software reads asynchrone
blocks, and blocks if trying to read more than the system has
already read.
So in general one has in the software a read loop of a block, and regularly at a read the read operation blocks till the physical read sufficiently buffers.
To hope for a non-blocking you would need to do:
InputStream is = res.openRawResource(resourceId);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
for (;;) {
// Read bytes until no longer available:
for (;;) {
int n = is.available();
if (n == 0) {
break;
}
byte[] part = new byte[n];
int nread = is.read(part);
assert nread == n;
baos.write(part, 0, nread);
}
// Still a probably blocking read:
byte[] part = new byte[128];
int nread = is.read(part);
if (nread <= 0) {
break; // End of file
}
baos.write(part, 0, nread);
}
return baos.toByteArray();
Now, before you copy that code, simply do a blocking read loop immediately. I cannot see an advantage of using available() unless you can do something with partial data while reading the rest.
I'm trying to make a simple transfer of a text .txt file from client to server, and no matter how much I think I know, and understand what I'm doing, and what exactly happening, I always get it wrong. I can really use some help here please.
So, this is the code, two function that transfer a .txt file from one to another:
Client side:
private void sendFileToServer(String file_name) throws IOException {
File file=new File(file_name);
int file_size=(int)file.length();
byte[] bytes=new byte[file_size];
FileInputStream os=null;
try {
os = new FileInputStream(file);
} catch (FileNotFoundException e) {
System.out.println("The file "+file+" wasn't found");
return;
}
BufferedInputStream bos=new BufferedInputStream(os);
bos.read(bytes);
output.write(bytes,0,bytes.length);
/* 'output' is a PrintStream object, that holds the output stream
* for the client's socket, meaning:
* output=new PrintStream(client_socket.getOutputStream()); */
output.flush();
bos.close();
}
this will buffer everything into BufferedInputStream, will copy it to bytes and will then send it to the other side - the server.
Server side:
public static String receiveFileFromClient(Client client) throws IOException {
int buffer_size=client.getSocket().getReceiveBufferSize();
byte[] bytes=new byte[buffer_size];
FileOutputStream fos=new FileOutputStream("transfered_file.txt");
BufferedOutputStream bos=new BufferedOutputStream(fos);
DataInputStream in=client.getInputStream();
int count;
System.out.println("this will be printed out");
while ((count=in.read(bytes))>0) { // execution is blocked here!
bos.write(bytes, 0, count);
}
System.out.println("this will not be printed");
bos.flush();
bos.close();
return "transfered_file.txt";
}
My intention here is to keep reading bytes from the client (the while loop), until the other side (the client) have no more bytes to send, and this is where in.read(bytes) should return 0 and the loop should break, but this is never happens, it just get blocked, even though all the bytes from the client's input-stream were successfully transferred!
Why doesn't the loop breaks?
From Javadoc:
If no byte is available because the stream is at end of file, the
value -1 is returned
doesn't the last byte is considered "end of file"? I made sure that the function sendFileToServer properly writes the entire file to the output instance (PrintStream object) and returns.
Any help would be appreciated.
As i understand it, the read() method will block until either it read[bytes] OR the socket is closed. So there is nothing for the read() what would indicate that it should stop reading, because it does not "understand" the file, its just some data.
A solution...
You could determine the number of bytes the client will send (on the client side) and then send the NUMBER over to the server. Now the server can process this number and knows how many bytes to read before the file is complete. So you can break the loop (or even don't use a loop) when the transfer is completed.
You could also process the data the server receives, and let the client send some "flag" after the file is complete, so the server knows when it is done. But this is more difficult, because you have to find something, that is not contained in the file-byte data
read() method will block for further input if you dont close the stream. So eather close the stream, or remove the loop and only read the number of bytes, you receive from the client
I was trying to read a file into an array by using FileInputStream, and an ~800KB file took about 3 seconds to read into memory. I then tried the same code except with the FileInputStream wrapped into a BufferedInputStream and it took about 76 milliseconds. Why is reading a file byte by byte done so much faster with a BufferedInputStream even though I'm still reading it byte by byte? Here's the code (the rest of the code is entirely irrelevant). Note that this is the "fast" code. You can just remove the BufferedInputStream if you want the "slow" code:
InputStream is = null;
try {
is = new BufferedInputStream(new FileInputStream(file));
int[] fileArr = new int[(int) file.length()];
for (int i = 0, temp = 0; (temp = is.read()) != -1; i++) {
fileArr[i] = temp;
}
BufferedInputStream is over 30 times faster. Far more than that. So, why is this, and is it possible to make this code more efficient (without using any external libraries)?
In FileInputStream, the method read() reads a single byte. From the source code:
/**
* Reads a byte of data from this input stream. This method blocks
* if no input is yet available.
*
* #return the next byte of data, or <code>-1</code> if the end of the
* file is reached.
* #exception IOException if an I/O error occurs.
*/
public native int read() throws IOException;
This is a native call to the OS which uses the disk to read the single byte. This is a heavy operation.
With a BufferedInputStream, the method delegates to an overloaded read() method that reads 8192 amount of bytes and buffers them until they are needed. It still returns only the single byte (but keeps the others in reserve). This way the BufferedInputStream makes less native calls to the OS to read from the file.
For example, your file is 32768 bytes long. To get all the bytes in memory with a FileInputStream, you will require 32768 native calls to the OS. With a BufferedInputStream, you will only require 4, regardless of the number of read() calls you will do (still 32768).
As to how to make it faster, you might want to consider Java 7's NIO FileChannel class, but I have no evidence to support this.
Note: if you used FileInputStream's read(byte[], int, int) method directly instead, with a byte[>8192] you wouldn't need a BufferedInputStream wrapping it.
A BufferedInputStream wrapped around a FileInputStream, will request data from the FileInputStream in big chunks (512 bytes or so by default, I think.) Thus if you read 1000 characters one at a time, the FileInputStream will only have to go to the disk twice. This will be much faster!
It is because of the cost of disk access. Lets assume you will have a file which size is 8kb. 8*1024 times access disk will be needed to read this file without BufferedInputStream.
At this point, BufferedStream comes to the scene and acts as a middle man between FileInputStream and the file to be read.
In one shot, will get chunks of bytes default is 8kb to memory and then FileInputStream will read bytes from this middle man.
This will decrease the time of the operation.
private void exercise1WithBufferedStream() {
long start= System.currentTimeMillis();
try (FileInputStream myFile = new FileInputStream("anyFile.txt")) {
BufferedInputStream bufferedInputStream = new BufferedInputStream(myFile);
boolean eof = false;
while (!eof) {
int inByteValue = bufferedInputStream.read();
if (inByteValue == -1) eof = true;
}
} catch (IOException e) {
System.out.println("Could not read the stream...");
e.printStackTrace();
}
System.out.println("time passed with buffered:" + (System.currentTimeMillis()-start));
}
private void exercise1() {
long start= System.currentTimeMillis();
try (FileInputStream myFile = new FileInputStream("anyFile.txt")) {
boolean eof = false;
while (!eof) {
int inByteValue = myFile.read();
if (inByteValue == -1) eof = true;
}
} catch (IOException e) {
System.out.println("Could not read the stream...");
e.printStackTrace();
}
System.out.println("time passed without buffered:" + (System.currentTimeMillis()-start));
}
Please explain why we are using -1 inside while loop
<%
File file = new File(file1);
int ch;
strContent = new StringBuffer("");
FileInputStream fin = null;
try {
fin = new FileInputStream(file);
while ((ch = fin.read()) != -1)
strContent.append((char) ch);
fin.close();
} catch (Exception e) {
System.out.println(e);
}
System.out.println(strContent.toString());
%>
In the above code explain why we are using -1,i don't understand why we are using the -1
Documentation of read:
Returns: the total number of bytes read into the buffer, or -1 if there is no more data because the end of the file has been reached.
So, the loop ends when there is nothing more to read
You should read the doc :
the method read() returns -1 if the end of the file is reached.
So you want to stop your while loop when you reach the end of your file.
This condition checks for EOF (End Of File). That is stop reading once, End of file is reached.
This is detailed in the Java docs:
Returns:
the total number of bytes read into the buffer, or -1 if there is no more data because the end of the stream has been reached.
Read more here:
http://docs.oracle.com/javase/6/docs/api/java/io/InputStream.html
public int read()
throws IOException
Reads a byte of data from this input stream. This method blocks if no
input is yet available.
Specified by:
read in class InputStream
Returns:
the next byte of data, or -1 if the end of the file is reached.
Throws:
IOException - if an I/O error occurs.