I want to download a file from Gdrive. So I m using drive API v3 (java).
here the code snippet.
Drive.Files.Get request = driveService.files().get(fileId);
// Get the file metadata
File downloadFile = request.execute();
Long fileSize = downloadFile.getSize(); // always returns null.
OutputStream out = new FileOutputStream(fileName);
request.getMediaHttpDownloader().setProgressListener(
new DriveDownloadProgressListener(messageQueue));
request.executeMediaAndDownloadTo(out);
here downloadFile.getSize() always returns null.
Is there any way to get the file size from the MediaHttpDownloader?
I have to add required fields myself in the request.
like this
Drive.Files.Get request = driveService.files().get(fileId).setFields("size");
File file = request.execute(); // contains only size field.Other fields will be empty
then execute and get the expected response(It only send fields setted by yourself)
For example, if you need name and size of the file.
Drive.Files.Get request = driveService.files().get(fileId).setFields("name,size"); // name and size
File file = request.execute(); // contains only name and size field.Other fields will be empty(respective get methods will return null only)
I am downloading a file from the web via HtmlUnit.
This is what my (working) code looks like:
Page dlPage = client.getPage(url);
FileOutputStream fos = new FileOutputStream(destinationFile);
try
{
IOUtils.copy(dlPage.getWebResponse().getContentAsStream(), fos);
}
catch (Exception ex)
{
ex.printStackTrace();
}
finally
{
fos.close();
}
I want to show the download progress. Therefore I need to know the file size. The Content-Length header is sent by the server but the problem is that I can read the headers after the file is downloaded. the getPage() method is blocking until the file is downloaded.
Is there any way to read just the response headers first in HtmlUnit and then the content? Or is there any other way to solve this problem?
Thanks!
You can use getContentLength() method of URLConnection to get the length when the connection is established.
Okay, I figured out a way to get this working: Before the download I send a HEAD request to the server so I can use the response to read the content length:
WebRequest wr = new WebRequest(new URL(url), HttpMethod.HEAD);
Page wrPage = client.getPage(wr);
long contentLength = Integer.valueOf(wrPage.getWebResponse().getResponseHeaderValue("Content-Length"));
System.out.println(contentLength);
OkHttp version: 2.0.0
I've received the following exception stack trace through Google Play:
java.lang.IllegalStateException: Cannot stream a request body without
chunked encoding or a known content length! at
com.squareup.okhttp.internal.http.HttpTransport.createRequestBody(HttpTransport.java:68)
at
com.squareup.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:254)
at com.squareup.okhttp.Call.getResponse(Call.java:198) at
com.squareup.okhttp.Call.access$200(Call.java:36) at
com.squareup.okhttp.Call$AsyncCall.execute(Call.java:143) at
com.squareup.okhttp.internal.NamedRunnable.run(NamedRunnable.java:33)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
at java.lang.Thread.run(Thread.java:848)
This is exactly how requests are made:
Request.Builder requestBuilder = new Request.Builder();
requestBuilder.addHeader("User-Agent", userAgent);
requestBuilder.addHeader("Accept", "application/hal+json");
requestBuilder.url(transformedURL);
RequestBody requestBody = body == null ? null : RequestBody.create(MediaType.parse("application/json"), gson.toJson(body));
requestBuilder.method(method, requestBody);
okHttpClient.newCall(requestBuilder.build()).enqueue(new com.squareup.okhttp.Callback() { ...
At first it seemed related to this bug, but it's already been fixed. Also, we've tested POST requests with an empty body several times during development.
This app only does GET and POST requests.
What could have possibly led to this error?
How are you creating the HTTP request? If you can create a reproducible test case, please file a bug with OkHttp and we'll fix.
This exception is being thrown from following piece of code:
package okhttp3.internal.http;
Http1xStream.java
#Override public Sink createRequestBody(Request request, long contentLength) throws IOException {
if ("chunked".equalsIgnoreCase(request.header("Transfer-Encoding"))) {
// Stream a request body of unknown length.
return newChunkedSink();
}
if (contentLength != -1) {
// Stream a request body of a known length.
return newFixedLengthSink(contentLength);
}
throw new IllegalStateException(
"Cannot stream a request body without chunked encoding or a known content length!");
}
And to solve this issue, I required to add one header to my request which can be added like this: .addHeader("Transfer-Encoding","chunked"):
OP's sample :
Request.Builder requestBuilder = new Request.Builder();
requestBuilder.addHeader("User-Agent", userAgent);
requestBuilder.addHeader("Accept", "application/hal+json");
requestBuilder..addHeader("Transfer-Encoding","chunked")
requestBuilder.url(transformedURL);
Here is my code:
url = paths[0];
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
int length = connection.getContentLength(); // i get negetive length
InputStream is = (InputStream) url.getContent();
byte[] imageData = new byte[length];
int buffersize = (int) Math.ceil(length / (double) 100);
int downloaded = 0;
int read;
while (downloaded < length) {
if (length < buffersize) {
read = is.read(imageData, downloaded, length);
} else if ((length - downloaded) <= buffersize) {
read = is.read(imageData, downloaded, length - downloaded);
} else {
read = is.read(imageData, downloaded, buffersize);
}
downloaded += read;
publishProgress((downloaded * 100) / length);
}
Bitmap bitmap = BitmapFactory.decodeByteArray(imageData, 0,
length);
if (bitmap != null) {
Log.i(TAG, "Bitmap created");
} else {
Log.i(TAG, "Bitmap not created");
}
is.close();
return bitmap;
I looked at this in Java documentation and the length is negative because of the following reason:
"the number of bytes of the content, or a negative number if unknown. If the content >length is known but exceeds Long.MAX_VALUE, a negative number is returned."
What might be the reason for this? I am trying to download an image. I would like to point out that this is the fourth method that I am trying to download images. The other three are mentioned here.
Edit:
As requested, here is the full method I am using.
protected Bitmap getImage(String imgurl) {
try {
URL url = new URL(imgurl);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
int length = connection.getContentLength();
InputStream is = (InputStream) url.getContent();
byte[] imageData = new byte[length];
int buffersize = (int) Math.ceil(length / (double) 100);
int downloaded = 0;
int read;
while (downloaded < length) {
if (length < buffersize) {
read = is.read(imageData, downloaded, length);
} else if ((length - downloaded) <= buffersize) {
read = is.read(imageData, downloaded, length
- downloaded);
} else {
read = is.read(imageData, downloaded, buffersize);
}
downloaded += read;
// publishProgress((downloaded * 100) / length);
}
Bitmap bitmap = BitmapFactory.decodeByteArray(imageData, 0,length);
if (bitmap != null) {
System.out.println("Bitmap created");
} else {
System.out.println("Bitmap not created");
}
is.close();
return bitmap;
} catch (MalformedURLException e) {
System.out.println(e);
} catch (IOException e) {
System.out.println(e);
} catch (Exception e) {
System.out.println(e);
}
return null;
}
By default, this implementation of HttpURLConnection requests that servers use gzip compression.
Since getContentLength() returns the number of bytes transmitted, you cannot use that method to predict how many bytes can be read from getInputStream().
Instead, read that stream until it is exhausted: when read() returns -1.
Gzip compression can be disabled by setting the acceptable encodings in the request header:
urlConnection.setRequestProperty("Accept-Encoding", "identity");
So try this:
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestProperty("Accept-Encoding", "identity"); // <--- Add this line
int length = connection.getContentLength(); // i get negetive length
Source (Performance paragraph): http://developer.android.com/reference/java/net/HttpURLConnection.html
There are two possible common explanations for this:
The content length is not known. Or more specifically, the server is not setting a "Content-Length" header in the response message.
The content length is greater than Integer.MAX_VALUE. If that happens, getContentLength() returns -1. (The javadocs recommend that getContentLengthLong() is used instead of getContentLength() to avoid that problem.)
Either way, it is better to NOT preallocate a fixed sized byte array to hold the image.
One alternative is to create a local ByteArrayOutputStream and copy bytes read from the socket to that. Then call toByteArray to grab the full byte array.
Another alternative is to save the data in a temporary file in the file system.
Apparently a common underlying cause of this is that some implementations will by default request "gzip" encoding for the response data. That forces the server to set the content length to -1. You can prevent that like this:
connection.setRequestProperty("Accept-Encoding", "identity");
... but that means that the response won't be compressed. So that is
(IMO) a substandard solution.
Your existing client-side code is broken in another respect as well. If you get an IOException or some other exception, the code block will "exit abnormally" without closing the URLConnection. This will result in the leakage of a file descriptor. Do this too many times and your application will fail due to exhaustion of file descriptors ... or local port numbers.
It is best practice to use a try / finally to ensure that URLConnections, Sockets, Streams and so on that tie down external resources are ALWAYS closed.
Preallocating a buffer based on the (purported) content length sets you up for a denial of service attack. Imagine what if the bad guys send you a lot of request with dangerously large "Content-Length" headers and then slow-send the data. OOMEs or worse.
It seems the server doesn't offer Content-Length in its response headers, did you get the Transfer-Encoding=chunked header from the response headers?
My situation is : I perform a HttpURLConnection and consider the server would response to me the "Content-Length" with positive value, but it didn't, then I turn to
the AndroidHttpClient which android HttpClient implementation, perform the same request again and got the right Content-Length.
I used Wireshark to analysis the two requests, found a little difference about request headers.
the header list that use AndroidHttpClient :
---------------------------------- request headers
Range : bytpe=0-
Host : download.game.yy.com
Connection : Keep-Alive
User-Agent : com.duowan.mobile.netroid
---------------------------------- response headers
Server : nginx
Content-Type : text/plain; charset=utf-8
ETag : "535e2578-84e350"
Cache-Control : max-age=86400
Accept-Ranges : bytes
Content-Range : bytes 0-8708943/8708944
Content-Length : 8708944
the request header list that use HttpURLConnection :
---------------------------------- request headers
Range : bytpe=0-
Host : download.game.yy.com
Connection : Keep-Alive
User-Agent : com.duowan.mobile.netroid
Accept-Encoding : gzip // difference here
---------------------------------- response headers
Server : nginx
Content-Type : text/plain; charset=utf-8
Cache-Control : max-age=86400
Transfer-Encoding : chunked
X-Android-Received-Millis : 1398861612782
X-Android-Sent-Millis : 1398861608538
The only difference with request header is Accept-Encoding which isn't added by myself, it was Android default setting for HttpURLConnection, after that, I set it to identity then perform request again, below is the full header stacks :
---------------------------------- request headers
Range : bytpe=0-
Host : download.game.yy.com
Connection : Keep-Alive
User-Agent : com.duowan.mobile.netroid
Accept-Encoding : identity
---------------------------------- response headers
Server : nginx
Content-Type : text/plain; charset=utf-8
ETag : "535e2578-84e350"
Cache-Control : max-age=86400
Accept-Ranges : bytes
Content-Range : bytes 0-8708943/8708944
Content-Length : 8708944
X-Android-Received-Millis : 1398862186902
X-Android-Sent-Millis : 1398862186619
as you can see, after I set the Accept-Encoding to "identity" replace system default value "gzip", the server provided "Content-Length" positive, that's why AndroidHttpClient could take the right value of Content-Length and HttpURLConnection not.
the gzip compression encoding may cause chunked response that consider by server-side, and if server think you can receive chunked encoding response, it may not offer the Content-Length header, try to disable gzip acceptable behavior then see what difference with that.
See here: http://download.oracle.com/javase/6/docs/api/java/net/URLConnection.html#getContentLength%28%29
It says,
Returns:
the content length of the resource that this connection's URL references, or -1 if the content length is not known.
It simply means header does not contains the content-length field. If you are having control over server code. You should set the content-lenth somehow. Something like ServletResponse#setContentLength
I also meet this problem in my Wallpaper app. The problem is because your server doesn't provide Content-Length in its http header. Here is a snapshot about a normal http header with Content-length.
I am using a share host so I can not change the server configuration. In my app, I am set progress dialog max value with an approximated value (which is bigger than my real file size) like this:
int filesize = connection.getContentLength();
if(filesize < 0) {
progressDialog.setMax(1000000);
} else {
progressDialog.setMax(filesize);
}
You can also check my full example source code here:
Android Progress Dialog Example.
I am late here but this might help someone. I was facing same issue i was always getting -1 value, when ever i was trying get the content length.
previously i was using below method to get content length.
long totalByte=connection.getContentLength();
I used below method which solved my problem .
long totalByte=Long.parseLong(connection.getHeaderField("Content-Length"));
I'm attempting to use Panda with my GWT application. I can upload videos directly to my panda server using
POST MY_PANDA_SERVER/videos/MY_VIDEO_ID/upload
However I would like hide my panda server behind my J2EE (glassfish) server. I would like to achieve this:
Start upload to some servlet on my J2EE server
Authenticate user
POST the file to my panda server while still uploading to servlet
Ideally I would like to never store the file on the J2EE server, but just use it as a proxy to get to the panda server.
Commons FileUpload is nice, but not sufficient in your case. It will parse the entire body in memory before providing the file items (and streams). You're not interested in the individual items. You basically just want to stream the request body from the one to other side transparently without altering it or storing it in memory in any way. FileUpload would only parse the request body into some "useable" Java objects and HttpClient would only create the very same request body again based on those Java objects. Those Java objects consumes memory as well.
You don't need a library for this (or it must be Commons IO to replace the for loop with an oneliner using IOUtils#copy()). Just the basic Java NET and IO API's suffices. Here's a kickoff example:
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
URLConnection connection = new URL("http://your.url.to.panda").openConnection();
connection.setDoOutput(true); // POST.
connection.setRequestProperty("Content-Type", request.getHeader("Content-Type")); // This one is important! You may want to check other request headers and copy it as well.
// Set streaming mode, else HttpURLConnection will buffer everything.
int contentLength = request.getContentLength();
if (contentLength > -1) {
// Content length is known beforehand, so no buffering will be taken place.
((HttpURLConnection) connection).setFixedLengthStreamingMode(contentLength);
} else {
// Content length is unknown, so send in 1KB chunks (which will also be the internal buffer size).
((HttpURLConnection) connection).setChunkedStreamingMode(1024);
}
InputStream input = request.getInputStream();
OutputStream output = connection.getOutputStream();
byte[] buffer = new byte[1024]; // Uses only 1KB of memory!
for (int length = 0; (length = input.read(buffer)) > 0;) {
output.write(buffer, 0, length);
output.flush();
}
output.close();
connection.getInputStream(); // Important! It's lazily executed.
}
You can use apache commons file upload to receive the file. Then you can use http client to upload the file to your panda server with POST. With apache commons file upload you can process the file in memory so you don't have to store it.
Building upon Enrique's answer, I also recommend to use FileUpload and HttpClient. FileUpload can give you a stream of the uploaded file:
// Create a new file upload handler
ServletFileUpload upload = new ServletFileUpload();
// Parse the request
FileItemIterator iter = upload.getItemIterator(request);
while (iter.hasNext()) {
FileItemStream item = iter.next();
String name = item.getFieldName();
InputStream stream = item.openStream();
if (item.isFormField()) {
System.out.println("Form field " + name + " with value "
+ Streams.asString(stream) + " detected.");
} else {
System.out.println("File field " + name + " with file name "
+ item.getName() + " detected.");
// Process the input stream
...
}
}
You could then use HttpClient or HttpComponents to do the POST. You can find an example here.
The best solution is to use apache-camel servlet component:
http://camel.apache.org/servlet.html