We Keep Coding

How to completely abort the output stream download?

we're currently working on the service that would archive the data and return it to the user as a ZipOutputStream. What we're currently looking for is an option to completely terminate the operation if something goes wrong on the server side. With our current implementation (just closing the response output stream) errors result in a malformed zip at the user side, but it can't be told if the archive is malformed or not before attempting to unzip it. The desired behavior would be something like download termination (from a browser perspective, for instance, it would result in an unsuccessful download indication (red cross icon or something similar, depending on the browser) explicitly telling the user that something went wrong). We're using Spring Boot, so any java code examples would really be appreciated, but if you know the underlying HTTP mechanism that is responsible for this kind of behavior, and can point in the right direction, that would be much appreciated too.
Here's what we have as of now (output being a response output stream of a Spring REST controller (HttpServletResponse.getOutputStream()) :
try (ZipOutputStream zipOutputStream = new ZipOutputStream(outputStream)) {
try {
for (ZipRecordFile fileInfo : zipRecord.listZipFileOverride()) {
InputStream fileStream = getFileStream(fileInfo.s3region(), fileInfo.s3bucket(),
fileInfo.s3key());
ZipEntry zipEntry = new ZipEntry(fileInfo.fileName());
zipOutputStream.putNextEntry(zipEntry);
fileStream.transferTo(zipOutputStream);
}
}
catch (Exception e) {
outputStream.close();
}
}

There isn't a (clean) way to do what you want:
Once you have started writing the ZIP file to the output stream, it is too late to change the HTTP response code. The response code is sent at the start of response.
Therefore, there is no proper way for the HTTP server to tell the HTTP client: "Hey ... ignore that ZIP file I sent you 'cos it is corrupt".
So what are the alternatives?
On the server side, create the entire ZIP as an in-memory object or write it to a temporary file. If you succeed, send an 2xx response followed by the ZIP data. If you fail, send a 4xx or 5xx response.
The main problem is that you need enough memory or file system space to hold the ZIP file.
Redesign your HTTP API so that the client can sent a second request to check if the first request's response contained a complete ZIP file.
You might be able to exploit MIME multipart encoding; see RFC 1341. Each part of a well-formed MIME multipart has a start marker and an end-marker. What you could try is to have your web-app construct the multipart stream containing the ZIP "by hand". If it decides it must abort the ZIP, it could just close the output stream without adding the required end marker.
The main problem with this is that you are depending on the HTTP stack on the client side to tell the browser (or whatever) that the multipart is corrupted. Furthermore, the browser (or whatever) must not pass on the partial (i.e. corrupt) ZIP file on to the user. I'm not sure if you can rely on (particular) web browsers to do that.
If you are running the download via custom code on the client side, you could conceivably implement your own encapsulation protocol. The effect would be the same as for 3 ... but you wouldn't be abusing the MIME spec.

How can i read a file in a ftp server using a servlet and then send this as a downloadable file to the user?

I have developed a servlet that offers some services.
I am using apache-commons-net FTPClient to log into a ftp server and read a file.
I want to make this file downloadle (aka send it to the outputstream maybe?) , but the only ways of reading a file that i know of are:
FTPClient.retrieveFileStream(String remote) and FTPClient.retrieveFile(String remote, OutputStream local).
I tried the first one and then wrote the InputStream i got to the outputStream of the servlet:
InputStream myFileStream = FTPClient.retrieveFileStream(fileName);
byte[] buffer = new byte[4096];
int length;
resp.reset();
resp.setContentType("text/csv");
resp.setHeader("Content-disposition","attachment; filename=\""+fileName+"\"");
OutputStream out = resp.getOutputStream();
while((length=myFileStream.read(buffer)) > 0){
out.write(buffer, 0, length);
}
myFileStream.close();
out.flush();
The Second One:
myClient.retrieveFile(fileName, resp.getOutputStream());
In both cases i get the text content of the file as a response and not the file itself.
Is there any way i can do this.
P.s. this code belongs to a medhod that is being called by the doPost() with http req and http resp as parameters.

If you want to download the file instead of just showing it, you have to change the content type you're sending to the browser (because it's browser's business to either display the data or save them as a file). Thus, do e.g.
resp.setContentType("application/octet-stream");
(instead of text/csv) to "hide" the real nature of the data from the browser and force it to save the data.

The problem was that i was using a google extension (DHC) to test my web service. and it displayed the file content instead of initializing the download.
I was making the file download in a doPost() method.
Solution:
I made it in a doGet() method and when accessed directly via browser everything works ok.
So i think it was only the extensions problem, which wrote the content of the response back to me instead of downloading the file attachment.
Thanks for the feedback to #Jozef

Why does HTTPURLConnection.getInputStream() takes time

I have a task to download & upload a file using HTTP protocol in Android (Java platform).
I am using following code for uploading a file:
HttpURLConnection httpURLConnection = (HttpURLConnection) serverUrl.openConnection();
....
httpURLConnection.connect();
OutputStream os = httpURLConnection.getOutputStream();
And Using following code for downloading a file:
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
...
urlConnection.connect();
DataInputStream stream = new DataInputStream(urlConnection.getInputStream());
As per my observation connect() for both the case takes time because it is communicating with network at this point. And for file upload, getOutputStream() gets execute very fast so does it means it is not communicating to network?
Whereas getInputStream() (in file download) takes some time (around 200 to 2500 mili sec) to execute. Does it mean it is communicating with network at this point? If yes then why so?
Experts, Please provide your comments on this & correct me if I am wrong anywhere.

HTTP is a request/response protocol. You need a TCP connection. The connect() method creates that. Then you need to send a request. You call getOutputStream() for that, and you write it.
At this point nothing has been written to the network (in normal transfer mode), because the content-length header has to be set, and Java doesn't know when you've finished writing. So when you call getInputStream() (or getResponseCode()), Java sets the content-length header, writes the request, waits for the server to start generating a response, reads all the response headers, and then gives you an input stream positioned at the beginning of the body of the response. All those steps take time.

You must limit buffering by specifying the streaming mode either by giving the final length of the uploaded information via setFixedLengthStreamingMode method, or setting mode to streaming if final length is not known via setChunkedStreamingMode method:
// For best performance, you should call either setFixedLengthStreamingMode(int) when the body length is known in advance,
// or setChunkedStreamingMode(int) when it is not. Otherwise HttpURLConnection will be forced to buffer the complete request body in memory
// before it is transmitted, wasting (and possibly exhausting) heap and increasing latency.
//
// see: https://developer.android.com/reference/java/net/HttpURLConnection.html
_connection.setChunkedStreamingMode(1024);
If you don't, the real transfer will occur when you call getInputStream().
See https://developer.android.com/reference/java/net/HttpURLConnection.html

Java: Stream Contents of Zipfile via HTTP

I have quite some amount of streamable data (>100MB), which, for the sake of compression, i would like to host packed in a zipfile on an http-server. So this zipfile contains a single file.
Now is it possible for a java-client to stream the data via http, even though it is packed in a zipfile?
According to wikipedia, ZIPs are not sequentially...
http://en.wikipedia.org/wiki/ZIP_(file_format)#Structure
If this is still possible somehow, then how?
edit: about gzip: as i said, i use a custom java client (not a webbrowser) is gzip available in the java http implementation?

Here's a snippet of code (that works) that the client can use to read from the zipped stream:
static void processZippedInputStream(InputStream in, String entryNameRegex)
throws IOException
{
ZipInputStream zin = new ZipInputStream(in);
ZipEntry ze;
while ((ze = zin.getNextEntry()) != null)
{
if (ze.getName().matches(entryNameRegex))
{
// treat zin as a normal input stream - ie read() from it till "empty" etc
break;
}
zin.closeEntry();
}
zin.close();
}
The main difference with a normal InputStream is iterating through the entries. You may know, for example, that you want the first entry, so no need for the name matching parameter etc.

Java supports the gzip format with the GZipInputStream (decompressing) and GZipOutputStream (compressing). Both zip and gzip use the same compressing format internally, the main difference is in the metadata: zip has it at the end of the file, gzip at the beginning (and gzip only supports one enclosed file easily).
For your of streaming one big file, using gzip will be the better thing to do - even more as you don't need access to the metadata.
I'm not sure if the HTTPConnection sends Accept-Encoding: gzip and then handles inflating the content automatically if the server delivers it with Content-Encoding: gzip, but you surely can do it manually if the server simply sends a the .gz file as such (i.e. with Content-Encoding: identity).
(By the way, make sure to read from the stream with not too small buffers, as each deflate call will have a native call overhead, since Java's GZipInputStream uses the native zlib implementation.)

Would it make more sense to let the web server do the zipping? If you are simply trying to reduce the amount of bandwidth being used, rather than really wanting to store the file zipped up on the server, this would simply be a matter of configurations, for example see:
http://tomcat.apache.org/tomcat-5.5-doc/config/http.html
for HTTP/1.1 GZIP compression. The server can force the response to the client to be zipped.
See also http://en.wikipedia.org/wiki/HTTP_compression.
The client will receive zipped packets and handle the unzipping. It should be possible to stream the file too, so the client doesn't need all the file before it can do something useful, because the server can zip individual chunks.

Yes you can, Stream the zip and use the MIME type as application/zip
If you actually want to play stream music on the other end, then it can't be done trivially as you can only unpack once the entire zip is available on client.
If size is you concern, you can either turn down your mp3 bit-rate or use formats such as ogg/vorbis

Use GZIP and then you can stream. Gzip uses the default compression algorithm of zip anyway.

Possible to send an image to my Servlet as a ByteBuffer?

I am attempting to have my android phone connect to my servlet and send it a certain image. The way I figured I would do this, is to use the copyPixelsToBuffer() function and then attempt to send this to the servlet through some output stream(similar to how I would do it in a normal stand alone java application). Will this way work? If so, what kind of stream do I use exactly? Should I just use DataOutputStream and just do something like the following:
ByteBuffer imgbuff;
Bitmap bm = BitmapFactory.decodeResource(getResources(), R.drawable.icon);
bm.copyPixelsToBuffer(bm);
...code...
URLConnection sc = server.openConnection();
sc.setDoOutput(true);
DataOutputStream out = new DataOutputStream( sc.getOutputStream() );
out.write(imgbuff.array());
out.flush();
out.close();
Note: I understand that this may not be the proper way of connecting to a server using the Android OS but at the moment I'm working on just how to send the image, not the connection (unless this is relevant on how the image is sent).
If this is not a way you'd recommend sending the image to the servlet (I figured a byte buffer would be best but I could be wrong), how would you recommend this to be done?

Since a HttpServlet normally listens on HTTP requests, you'd like to use multipart/form-data encoding to send binary data over HTTP, instead of raw (unformatted) like that.
From the client side on, you can use URLConnection for this as outlined in this mini tutorial, but it's going to be pretty verbose. You can also use Apache HttpComponents Client for this. This adds however extra dependencies, I am not sure if you'd like to have that on Android.
Then, on the server side, you can use Apache Commons FileUpload to parse the items out of a multipart/form-data encoded request body. You can find a code example in this answer how the doPost() of the servlet should look like.
As to your code example: wrapping in the DataOutputStream is unnecessary. You aren't taking benefit of the DataOutputStream's facilities. You are just using write(byte[]) method which is already provided by the basic OutputStream as returned by URLConnection#getOutputStream(). Further, the Bitmap has a compress() method which you can use to compress it using a more standard and understandable format (PNG, JPG, etc) into an arbitrary OutputStream. E.g.
output = connection.getOutputStream();
// ...
bitmap.compress(CompressFormat.JPEG, 100, output);
Do this instead of output.write(bytes) as in your code.