I use following code to download file from URL..
while(status==Status.DOWNLOADING){
HttpURLConnection conn=(HttpURLConnection)url.openConnection();
conn.connect();
int size=conn.getContentLength();
BufferedInputStream bin=new BufferedInputStream(conn.getInputStream());
byte[] buffer=new byte[1024];
int read=bin.read(buffer);
if(read==-1)
break;
downloaded+=read;
}
for some URL's read() method return -1 before reading upto size(content length) of download..
can anybody suggest me, what's happening with this code..
pls provide your suggestion..
Its not guaranteed that a webserver provides a content length in the http header. Therfore you should not rely on it. Just read the stream like this:
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
int len;
while ((len = bin.read(buf)) > 0) {
bos.write(buf, 0, len);
}
byte[] data = bos.toByteArray();
Related
Why i cant read POST-request with 150k chars?
I can only read ~15k chars all time
InputStream is = socket.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
while (is.available() > 0 && (length = is.read(buffer)) != -1) {
baos.write(buffer, 0, length);
}
System.out.println(baos.toString(StandardCharsets.UTF_8.name()));
UPD: if we ignored is.available(), code freezes in the while:
InputStream is = socket.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
while ((length = is.read(buffer)) != -1) {
baos.write(buffer, 0, length);
}
System.out.println(baos.toString(StandardCharsets.UTF_8.name()));
There are no exceptions.
Docs for avaiable() says:
available()
Returns an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking
So I'm going to guess that you internally have 15k buffers and you're only reading up to the end of your own buffer, not to the end of the stream. You should frankly be ignoring availabe() in this case and just call read( byte[] ) until it returns -1.
Your updated code example looks almost exactly like the code I use to read streams. I think the problem must be on the sender's side. Either the sender is not closing the stream properly, or there's some network issue that doesn't allow enough packets through.
For reference, here's the code I use to read an entire stream. (Lightly tested.)
public static ByteArrayOutputStream readFully( InputStream ins )
throws IOException
{
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] bytes = new byte[ 1024 ];
for( int length; ( length = ins.read( bytes ) ) != -1; )
bos.write( bytes, 0, length );
return bos;
}
We have a JSP which is supposed to fetch a PDF from an internal URL and pass this PDF on to the client (like a proxy).
The resulting download is corrupted. After about 18'400 bytes we only get 00 bytes till the end. Interestingly the download is exactly the right size in bytes.
// Get the download
URL url = new URL(url);
HttpURLConnection req = (HttpURLConnection)url.openConnection();
req.setDoOutput(true);
req.setRequestMethod("GET");
// Get Binary Response
int contentLength = req.getContentLength();
byte ba[] = new byte[contentLength];
req.getInputStream().read(ba);
ByteArrayInputStream in = new ByteArrayInputStream(ba);
// Prepare Reponse Headers
response.setContentType(req.getContentType());
response.setContentLength(req.getContentLength());
response.setHeader("Content-Disposition", "attachment; filename=download.pdf");
// Stream to Response
OutputStream output = response.getOutputStream();
//OutputStream output = new FileOutputStream("c:\\temp\\op.pdf");
int count;
byte[] buffer = new byte[8192];
while ((count = in.read(buffer)) > 0) output.write(buffer, 0, count);
in.close();
output.close();
req.disconnect();
UPDATE 1: I'm not the only one seeing Java stop streaming at 4379 bytes (link).
UPDATE 2: If I do output.flush after every write I get more data 14599 bytes and then the nulls. Must have something to do with tomcat's output buffer limit.
int contentLength = req.getContentLength();
byte ba[] = new byte[contentLength];
req.getInputStream().read(ba);
ByteArrayInputStream in = new ByteArrayInputStream(ba);
// Prepare Reponse Headers
response.setContentType(req.getContentType());
response.setContentLength(req.getContentLength());
response.setHeader("Content-Disposition", "attachment; filename=download.pdf");
// Stream to Response
OutputStream output = response.getOutputStream();
//OutputStream output = new FileOutputStream("c:\\temp\\op.pdf");
int count;
byte[] buffer = new byte[8192];
while ((count = in.read(buffer)) > 0) output.write(buffer, 0, count);
This code is all nonsense. You are ignoring the result of the first read() and you are also wasting both time and space with the ByteArrayInputStream. All you need is this:
int contentLength = req.getContentLength();
// Prepare Reponse Headers
response.setContentType(req.getContentType());
response.setHeader("Content-Disposition", "attachment; filename=download.pdf");
// Stream to Response
InputStream in = req.getInputStream();
OutputStream output = response.getOutputStream();
int count;
byte[] buffer = new byte[8192];
while ((count = in.read(buffer)) > 0) output.write(buffer, 0, count);
Note that the Content-Length is already set for you.
I am currently trying to read in data from a server response. I am using a Socket to connect to a server, creating a http GET request, then am using a Buffered Reader to read in data. Here is what the code looks like compacted:
Socket conn = new Socket(server, 80);
//Request made here
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String response;
while((response = inFromServer.readLine()) != null){
System.out.println(response);
}
I would like to read in the data, instead of as a String, as a byte array, and write it to a file. How is this possible? Any help is greatly appreciated, thank you.
You need to use a ByteArrayOutputStream, do something like the below code:
Socket conn = new Socket(server, 80);
//Request made here
InputStream is = conn.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
int readBytes = -1;
while((readBytes = is.read(buffer)) > 1){
baos.write(buffer,0,readBytes);
}
byte[] responseArray = baos.toByteArray();
One way is to use Apache commons-io IOUtils
byte[] bytes = IOUtils.toByteArray(inputstream);
With plain java:
ByteArrayOutputStream output = new ByteArrayOutputStream();
try(InputStream stream = new FileInputStream("myFile")) {
byte[] buffer = new byte[2048];
int numRead;
while((numRead = stream.read(buffer)) != -1) {
output.write(buffer, 0, numRead);
}
} catch(IOException e) {
e.printStackTrace();
}
// and here your bytes
byte[] myDesiredBytes = output.toByteArray();
If you are not using Apache commons-io library in your project,I have pretty simple method to do the same without using it..
/*
* Read bytes from inputStream and writes to OutputStream,
* later converts OutputStream to byte array in Java.
*/
public static byte[] toByteArrayUsingJava(InputStream is)
throws IOException{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int reads = is.read();
while(reads != -1){
baos.write(reads);
reads = is.read();
}
return baos.toByteArray();
}
I've seen examples with text files but is saving an audio file directly to a server done the same way with URLConnection?
Yes, the same. Although make sure you use a binary output stream to write the content to disk.
Something like:
URLConnection conn = new URL("http://www.gravatar.com/avatar/fd9e8761fad999a1bf1e095fc8f53ffe?s=32&d=identicon&r=PG")
.openConnection();
InputStream is = conn.getInputStream();
FileOutputStream outstream = new FileOutputStream("/tmp/myfile");
byte[] buffer = new byte[4096];
int len;
while ((len = is.read(buffer)) > 0) {
outstream.write(buffer, 0, len);
}
outstream.close();
is.close();
The example uses your gravatar, but same difference.
I need a very fast way to copy text from a file to the body of a HttpServletResponse.
Actually I'm copying byte by byte in a loop, from a bufferedReader to the response.getWriter() but I believe there must be a faster and more straightforward way of doing it.
Thanks!
I like using the read() method that accepts a byte array since you can tweak the size and change the performance.
public static void copy(InputStream is, OutputStream os) throws IOException {
byte buffer[] = new byte[8192];
int bytesRead;
BufferedInputStream bis = new BufferedInputStream(is);
while ((bytesRead = bis.read(buffer)) != -1) {
os.write(buffer, 0, bytesRead);
}
is.close();
os.flush();
os.close();
}
There's no need to do this stuff yourself. It is such a common requirement that open source, battle-tested, optimised solutions exist.
Apache Commons IO has an IOUtils class with a range of static copy methods. Perhaps you could use
IOUtils.copy(reader, writer);
http://commons.apache.org/io/api-1.4/org/apache/commons/io/IOUtils.html#copy(java.io.Reader, java.io.Writer)
This is how I do it in my Servlet with a 4K buffer,
// Send the file.
OutputStream out = response.getOutputStream();
BufferedInputStream is = new BufferedInputStream(new FileInputStream(file));
byte[] buf = new byte[4 * 1024]; // 4K buffer
int bytesRead;
while ((bytesRead = is.read(buf)) != -1) {
out.write(buf, 0, bytesRead);
}
is.close();
out.flush();
out.close();