We Keep Coding

OkHttp3 How I can retrieve the Http Content Length from http header?

I try to download a file and check whether has been downloaded as a whole:
Request request = new Request.Builder().url("http://example.com/myfile.tar.gz").build();
Response response = client.newCall(request).execute();
// Status code checks goes here
if (downloadedTar == null) {
throw new SettingsFailedException();
}
ResponseBody downloadedTar = response.body();
double contentLength = Double.parseDouble(response.header("content-length"));
File file = File.createTempFile(System.currentTimeMillis()+"_file", ".tar.gz", getContext().getCacheDir());
FileOutputStream download = new FileOutputStream(file);
download.write(downloadedTar.body().bytes());
download.flush();
download.close();
if(file.exists() && (double)file.length() == contentLength){
// file has been downloaded
}
But the line:
double contentLength = Double.parseDouble(response.header("content-length"));
But response.header("content-length") is Null and has no integer value, I also tried the following variation response.header("Content-Length") and response.header("Content-Length") without any success.
So why I cannot retrieve Content-Length header and how I can ensure that file has been sucessfully downloaded?

Content-Length is removed in a number of cases such as Gzip responses
https://github.com/square/okhttp/blob/3ad1912f783e108b3d0ad2c4a5b1b89b827e4db9/okhttp/src/jvmMain/kotlin/okhttp3/internal/http/BridgeInterceptor.kt#L98
But generally isn't guaranteed to be present for streamed responses (chunked of h2).
You should try to avoid requiring content-length as it is guaranteed to be present and may change. Also you can probably optimise your IO with
Okio.buffer(Okio.sink(file)).writeAll(response.body().source())
or kotlin
file.sink().buffer().writeAll(response.body!!.source())

parse multipart/form-data response in spring

I need to receive multipart/form-data response
but i have no idea on how to parse this kind of response
For example
--mf8sckatxs4PpMnOLF6ltSv26ZJc5qxy9qq
Content-Disposition: form-data; name="arguments"
Content-Type: text/plain;charset=UTF-8
Content-Length: 311
[{"code":200,"message":"123"}]
Content-Disposition: form-data; name="_0"; filename="0_BODY_feature"
Content-Type: application/octet-stream
Content-Length: 407
binarydata

Since you're using Spring you already have the Apache fileupload and http classes available. MultipartStream is designed to be attached to an incoming byte stream and can provide progress updates as data arrives.
This simple example illustrates a non-streaming scenario where you've already buffered the whole incoming body.
byte[] yourResponse = ... // the whole response as a byte array
String yourContentType = ... // the Content-Type header string
ContentType contentType = ContentType.parse(yourContentType);
MultipartStream multipartStream = new MultipartStream(
new ByteArrayInputStream(yourResponse),
contentType.getParameter("boundary").getBytes(),
1024, // internal buffer size (you choose)
null); // progress indicator (none)
boolean nextPart = multipartStream.skipPreamble();
while (nextPart) {
ByteArrayOutputStream output = new ByteArrayOutputStream();
String partHeaders = multipartStream.readHeaders();
multipartStream.readBodyData(output);
// do something with the multi-line part headers
// do something with the part 'output' byte array
nextPart = multipartStream.readBoundary();
}
Add exception handling as required.

You can store and separate those values in an Array of Strings:
String[] array = "allyourinputtext".split(";");
This will separate the values after a semicolon. Then, you can access each value by doing this:
String content = array[0];
String name = array[1];
...
This doesn't solve the WHOLE problem(since not all values are separated by semicolons), but you can play with the arguments you pass to split() to separate your values.
Note: If you want to parse a String to int (the length value for example) you can use:
int length = Integer.parseInt(array[index];

since HttpExchange require that sendResponseHeaders(int,long) must be before getResponseBody(), how do I handle transmitting file?

The following code is no longer feasible
server.createContext("/", exchange -> {
URL url = new URL("");
try (final BufferedInputStream in = new BufferedInputStream(url.openStream()); final BufferedOutputStream out = new BufferedOutputStream(exchange.getResponseBody())) {
} catch (IOException e) {
}
});
server.start();
The reason is that sendResponseHeaders(int,long) must be before getResponseBody() and the long in sendResponseHeaders(int,long) is the length of content to be written into ResponseBody...but now I cannot figure out the value.
so what should I do?
The size of the file is unknown: it can be smaller than 1MB but also can be bigger than 200MB, so creating a big enough byte array cannot be accepted. To convert the file to String or bytes directly without buffer cannot be accepted too.
What can I do?
Thanks a lot.

Had the same issue - in my case, helped to read the documentation in more detail...
responseLength - if > 0, specifies a fixed response body length and
that exact number of bytes must be written to the stream acquired from
getResponseBody(), or else if equal to 0, then chunked encoding is
used, and an arbitrary number of bytes may be written. if <= -1, then
no response body length is specified and no response body may be
written.
So zero should work here as well

WebResourceResponse can't read full inputstream from HttpConnection (Android)

I'm working with an ionic application(like hybrid) which can play some videos.I want to add some headers to the request so that I override the "shouldInterceptRequest".
URL myUrl = new URL(real);
HttpURLConnection connection = (HttpURLConnection) myUrl.openConnection();
for (Map.Entry < String, String > entry: headers.entrySet()) {
connection.setRequestProperty(entry.getKey(), entry.getValue());
}
InputStream in = connection.getInputStream();
WebResourceResponse response = new WebResourceResponse("video/mp4", "UTF-8", in );
for (Map.Entry < String, List < String >> entry: connection.getHeaderFields().entrySet()) {
resHeaders.put(entry.getKey(), entry.getValue().get(0));
}
This code can't work.The video tag in html can't play the video.
So I add some code.
byte[] bytes = new byte[30 * 1024 * 1024];
ByteArrayOutputStream byteBuffer = new ByteArrayOutputStream();
int len = 0;
while ((len = in.read(bytes)) != -1) {
byteBuffer.write(bytes, 0, len);
}
bytes = byteBuffer.toByteArray();
WebResourceResponse response = new WebResourceResponse("video/mp4", "UTF-8", new ByteArrayInputStream(bytes));
When I read inputstream into bytes and send bytes to WebResourceResponse, the video can be play.However it means my application will use lots of memory if the video is large.
So that, I want to know is there any way to play the video without saving inputstream into bytes.

OK.Fianlly,I found my way.
Actually, my goal is to add custom headers to resource request and now I found there is an easier way to do it.
For example, if I want to load a image,I will use img tag like this,<img src="the_url_of_image">,and I can't add any header to the request unless I interecept the request.
However,we can use blob now.We can request the resource by using something like ajax and use createObjectURL to create a url links to the resource.

Java Proxy Servlet for submitting files

I'm attempting to use Panda with my GWT application. I can upload videos directly to my panda server using
POST MY_PANDA_SERVER/videos/MY_VIDEO_ID/upload
However I would like hide my panda server behind my J2EE (glassfish) server. I would like to achieve this:
Start upload to some servlet on my J2EE server
Authenticate user
POST the file to my panda server while still uploading to servlet
Ideally I would like to never store the file on the J2EE server, but just use it as a proxy to get to the panda server.

Commons FileUpload is nice, but not sufficient in your case. It will parse the entire body in memory before providing the file items (and streams). You're not interested in the individual items. You basically just want to stream the request body from the one to other side transparently without altering it or storing it in memory in any way. FileUpload would only parse the request body into some "useable" Java objects and HttpClient would only create the very same request body again based on those Java objects. Those Java objects consumes memory as well.
You don't need a library for this (or it must be Commons IO to replace the for loop with an oneliner using IOUtils#copy()). Just the basic Java NET and IO API's suffices. Here's a kickoff example:
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
URLConnection connection = new URL("http://your.url.to.panda").openConnection();
connection.setDoOutput(true); // POST.
connection.setRequestProperty("Content-Type", request.getHeader("Content-Type")); // This one is important! You may want to check other request headers and copy it as well.
// Set streaming mode, else HttpURLConnection will buffer everything.
int contentLength = request.getContentLength();
if (contentLength > -1) {
// Content length is known beforehand, so no buffering will be taken place.
((HttpURLConnection) connection).setFixedLengthStreamingMode(contentLength);
} else {
// Content length is unknown, so send in 1KB chunks (which will also be the internal buffer size).
((HttpURLConnection) connection).setChunkedStreamingMode(1024);
}
InputStream input = request.getInputStream();
OutputStream output = connection.getOutputStream();
byte[] buffer = new byte[1024]; // Uses only 1KB of memory!
for (int length = 0; (length = input.read(buffer)) > 0;) {
output.write(buffer, 0, length);
output.flush();
}
output.close();
connection.getInputStream(); // Important! It's lazily executed.
}

You can use apache commons file upload to receive the file. Then you can use http client to upload the file to your panda server with POST. With apache commons file upload you can process the file in memory so you don't have to store it.

Building upon Enrique's answer, I also recommend to use FileUpload and HttpClient. FileUpload can give you a stream of the uploaded file:
// Create a new file upload handler
ServletFileUpload upload = new ServletFileUpload();
// Parse the request
FileItemIterator iter = upload.getItemIterator(request);
while (iter.hasNext()) {
FileItemStream item = iter.next();
String name = item.getFieldName();
InputStream stream = item.openStream();
if (item.isFormField()) {
System.out.println("Form field " + name + " with value "
+ Streams.asString(stream) + " detected.");
} else {
System.out.println("File field " + name + " with file name "
+ item.getName() + " detected.");
// Process the input stream
...
}
}
You could then use HttpClient or HttpComponents to do the POST. You can find an example here.

The best solution is to use apache-camel servlet component:
http://camel.apache.org/servlet.html