Why i cant read POST-request with 150k chars?
I can only read ~15k chars all time
InputStream is = socket.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
while (is.available() > 0 && (length = is.read(buffer)) != -1) {
baos.write(buffer, 0, length);
}
System.out.println(baos.toString(StandardCharsets.UTF_8.name()));
UPD: if we ignored is.available(), code freezes in the while:
InputStream is = socket.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
while ((length = is.read(buffer)) != -1) {
baos.write(buffer, 0, length);
}
System.out.println(baos.toString(StandardCharsets.UTF_8.name()));
There are no exceptions.
Docs for avaiable() says:
available()
Returns an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking
So I'm going to guess that you internally have 15k buffers and you're only reading up to the end of your own buffer, not to the end of the stream. You should frankly be ignoring availabe() in this case and just call read( byte[] ) until it returns -1.
Your updated code example looks almost exactly like the code I use to read streams. I think the problem must be on the sender's side. Either the sender is not closing the stream properly, or there's some network issue that doesn't allow enough packets through.
For reference, here's the code I use to read an entire stream. (Lightly tested.)
public static ByteArrayOutputStream readFully( InputStream ins )
throws IOException
{
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] bytes = new byte[ 1024 ];
for( int length; ( length = ins.read( bytes ) ) != -1; )
bos.write( bytes, 0, length );
return bos;
}
Related
When try to convert InputStream into byte array to know size of the file being uploaded. I am able to get size, but InputStream.read() becomes -1. How to check file size before writing it to server?
My current code gives me size , but InputStream reaches the end.
private static byte[] readFully(InputStream input) throws IOException
{
byte[] buffer = new byte[8192];
int bytesRead;
byte []bytes=null;
ByteArrayOutputStream output = new ByteArrayOutputStream();
while ((bytesRead = input.read(buffer)) != -1)
{
System.out.println("Buffer is "+input.read(buffer));
output.write(buffer, 0, bytesRead);
}
bytes=output.toByteArray();
output.close();
return bytes;
}
If you are implementing a web-server in Java, please take a look at the following link:
http://www.prasannatech.net/2008/11/http-web-server-java-post-file-upload.html
You must try to read the incomming information until you found its boundary.
You can't use read() method now, because the InputStream may not be ready to be read yet.
I'm currently trying to read in a image file from the server but either getting a incomplete data or
Exception in thread "main"
java.lang.NegativeArraySizeException.
Has this something to do with the buffer size? I have tried to use static size instead of contentlength. Please kindly advise.
URL myURL = new URL(url);
HttpURLConnection connection = (HttpURLConnection)myURL.openConnection();
connection.setRequestMethod("GET");
status = connection.getResponseCode();
if (status == 200)
{
int size = connection.getContentLength() + 1024;
byte[] bytes = new byte[size];
InputStream input = new ByteArrayInputStream(bytes);
FileOutputStream out = new FileOutputStream(file);
input = connection.getInputStream();
int data = input.read(bytes);
while(data != -1){
out.write(bytes);
data = input.read(bytes);
}
out.close();
input.close();
Let's examine the code:
int size = connection.getContentLength() + 1024;
byte[] bytes = new byte[size];
why do you add 1024 bytes to the size? What's the point? The buffer size should be something large enough to avoid too many reads, but small enough to avoid consuming too much memory. Set it at 4096, for example.
InputStream input = new ByteArrayInputStream(bytes);
FileOutputStream out = new FileOutputStream(file);
input = connection.getInputStream();
Why do you create a ByteArrayInputStream, and then forget about it completely? You don't need a ByteArrayInputStream, since you don't read from a byte array, but from the connection's input stream.
int data = input.read(bytes);
This reads bytes from the input. The max number of bytes read is the length of the byte array. The actual number of bytes read is returned and stored in data.
while (data != -1) {
out.write(bytes);
data = input.read(bytes);
}
So you have read data bytes, but you don't write only the first data bytes of the array. You write the whole array of bytes. That is wrong. Suppose your array if of size 4096 and data is 400, instead of writing the 400 bytes that have been read, you write the 400 bytes + the remaining 3696 bytes of the array, which could be 0, or could have values coming from a previous read. It should be
out.write(bytes, 0, data);
Finally:
out.close();
input.close();
If any exception occurs before, those two streams will never be closed. Do that a few times, and your whold OS won't have file descriptos available anymore. Use the try-with-resources statement to be sure your streams are closed, no matter what happens.
This code can help you
input = connection.getInputStream();
byte[] buffer = new byte[4096];
int n = - 1;
OutputStream output = new FileOutputStream( file );
while ( (n = input.read(buffer)) != -1)
{
if (n > 0)
{
output.write(buffer, 0, n);
}
}
output.close();
Here is the code but got error:
bin = new ByteArrayInputStream(socket.getInputStream());
Is it possible to receive byte[] using ByteArrayInputStream from a socket?
No. You use ByteArrayInputStream when you have an array of bytes, and you want to read from the array as if it were a file. If you just want to read arrays of bytes from the socket, do this:
InputStream stream = socket.getInputStream();
byte[] data = new byte[100];
int count = stream.read(data);
The variable count will contain the number of bytes actually read, and the data will of course be in the array data.
You can't get an instance of ByteArrayInputStream by reading directly from socket.
You require to read first and find byte content.
Then use it to create an instance of ByteArrayInputStream.
InputStream inputStream = socket.getInputStream();
// read from the stream
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] content = new byte[ 2048 ];
int bytesRead = -1;
while( ( bytesRead = inputStream.read( content ) ) != -1 ) {
baos.write( content, 0, bytesRead );
} // while
Now, as you have baos in hand, I don't think you still need a bais instance.
But, to make it complete,
you can generate byte array input stream as below
ByteArrayInputStream bais = new ByteArrayInputStream( baos.toByteArray() );
When I am trying to download a large file which is of 260MB from server, I get this error: java.lang.OutOfMemoryError: Java heap space. I am sure my heap size is less than 252MB. Is there any way I can download large files without increasing heap size?
How I can download large files without getting this issue? My code is given below:
String path= "C:/temp.zip";
response.addHeader("Content-Disposition", "attachment; filename=\"test.zip\"");
byte[] buf = new byte[1024];
try {
File file = new File(path);
long length = file.length();
BufferedInputStream in = new BufferedInputStream(new FileInputStream(file));
ServletOutputStream out = response.getOutputStream();
while ((in != null) && ((length = in.read(buf)) != -1)) {
out.write(buf, 0, (int) length);
}
in.close();
out.close();
There are 2 places where I can see you could potentially be building up memory usage:
In the buffer reading your input file.
In the buffer writing to your output stream (HTTPOutputStream?)
For #1 I would suggest reading directly from the file via FileInputStream without the BufferedInputStream. Try this first and see if it resolves your issue. ie:
FileInputStream in = new FileInputStream(file);
instead of:
BufferedInputStream in = new BufferedInputStream(new FileInputStream(file));
If #1 does not resolve the issue, you could try periodically flushing the output stream after so much data is written (decrease chunk size if necessary):
ie:
try
{
FileInputStream fileInputStream = new FileInputStream(file);
byte[] buf=new byte[8192];
int bytesread = 0, bytesBuffered = 0;
while( (bytesread = fileInputStream.read( buf )) > -1 ) {
out.write( buf, 0, bytesread );
bytesBuffered += bytesread;
if (bytesBuffered > 1024 * 1024) { //flush after 1MB
bytesBuffered = 0;
out.flush();
}
}
}
finally {
if (out != null) {
out.flush();
}
}
Unfortunately you have not mentioned what type out is. If you have memory issues I guess it is ByteArrayOutpoutStream. So, replace it by FileOutputStream and write the byte you are downloading directly to file.
BTW, do not use read() method that reads byte-by-byte. Use read(byte[] arr) instead. This is much faster.
First you can remove the (in != null) from your while statement, it's unnecessary. Second, try removing the BufferedInputStream and just do:
FileInputStream in = new FileInputStream(file);
There's nothing wrong (in regard to memory usage) with the code you're show. Either the servlet container is configured to buffer the entire response (look at the web.xml configuration), or the memory is being leaked elsewhere.
I use following code to download file from URL..
while(status==Status.DOWNLOADING){
HttpURLConnection conn=(HttpURLConnection)url.openConnection();
conn.connect();
int size=conn.getContentLength();
BufferedInputStream bin=new BufferedInputStream(conn.getInputStream());
byte[] buffer=new byte[1024];
int read=bin.read(buffer);
if(read==-1)
break;
downloaded+=read;
}
for some URL's read() method return -1 before reading upto size(content length) of download..
can anybody suggest me, what's happening with this code..
pls provide your suggestion..
Its not guaranteed that a webserver provides a content length in the http header. Therfore you should not rely on it. Just read the stream like this:
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
int len;
while ((len = bin.read(buf)) > 0) {
bos.write(buf, 0, len);
}
byte[] data = bos.toByteArray();