We Keep Coding

How to completely abort the output stream download?

we're currently working on the service that would archive the data and return it to the user as a ZipOutputStream. What we're currently looking for is an option to completely terminate the operation if something goes wrong on the server side. With our current implementation (just closing the response output stream) errors result in a malformed zip at the user side, but it can't be told if the archive is malformed or not before attempting to unzip it. The desired behavior would be something like download termination (from a browser perspective, for instance, it would result in an unsuccessful download indication (red cross icon or something similar, depending on the browser) explicitly telling the user that something went wrong). We're using Spring Boot, so any java code examples would really be appreciated, but if you know the underlying HTTP mechanism that is responsible for this kind of behavior, and can point in the right direction, that would be much appreciated too.
Here's what we have as of now (output being a response output stream of a Spring REST controller (HttpServletResponse.getOutputStream()) :
try (ZipOutputStream zipOutputStream = new ZipOutputStream(outputStream)) {
try {
for (ZipRecordFile fileInfo : zipRecord.listZipFileOverride()) {
InputStream fileStream = getFileStream(fileInfo.s3region(), fileInfo.s3bucket(),
fileInfo.s3key());
ZipEntry zipEntry = new ZipEntry(fileInfo.fileName());
zipOutputStream.putNextEntry(zipEntry);
fileStream.transferTo(zipOutputStream);
}
}
catch (Exception e) {
outputStream.close();
}
}

There isn't a (clean) way to do what you want:
Once you have started writing the ZIP file to the output stream, it is too late to change the HTTP response code. The response code is sent at the start of response.
Therefore, there is no proper way for the HTTP server to tell the HTTP client: "Hey ... ignore that ZIP file I sent you 'cos it is corrupt".
So what are the alternatives?
On the server side, create the entire ZIP as an in-memory object or write it to a temporary file. If you succeed, send an 2xx response followed by the ZIP data. If you fail, send a 4xx or 5xx response.
The main problem is that you need enough memory or file system space to hold the ZIP file.
Redesign your HTTP API so that the client can sent a second request to check if the first request's response contained a complete ZIP file.
You might be able to exploit MIME multipart encoding; see RFC 1341. Each part of a well-formed MIME multipart has a start marker and an end-marker. What you could try is to have your web-app construct the multipart stream containing the ZIP "by hand". If it decides it must abort the ZIP, it could just close the output stream without adding the required end marker.
The main problem with this is that you are depending on the HTTP stack on the client side to tell the browser (or whatever) that the multipart is corrupted. Furthermore, the browser (or whatever) must not pass on the partial (i.e. corrupt) ZIP file on to the user. I'm not sure if you can rely on (particular) web browsers to do that.
If you are running the download via custom code on the client side, you could conceivably implement your own encapsulation protocol. The effect would be the same as for 3 ... but you wouldn't be abusing the MIME spec.

Java http post: Difference in reading bytes through curl and in java

I am stuck in a strange issue, I am posting some image data to a server, now I created the requested using curl and then traced back it.
Next was to create similar request in java.
The code posts raw binary image data to server, but when I compare the binary data in java to that posted by curl, there is a minute difference due to which I am getting 400 response code from server.
The difference I think is in few dots.
Below is the request generate by curl (linux).
Generate by curl
Now here is the request generate by Java, when I read bytes.
Click here
Java code looks something like this:
PrintWriter out = new PrintWriter(os);
out.println("POST /1izb0rx1 HTTP/1.1");
out.println("User-Agent: curl/7.35.0");
out.println("Host: requestb.in");
out.println("Accept: */*");
out.println("Content-Disposition:inline; filename=test.png");
out.println("Authorization: Basic YW5kcm9pZDpUZXN0dGVzdDExISE=");
out.println("Content-Length: "+"24143");
out.println("Content-Type: application/x-www-form-urlencoded");
out.println();
out.println("."+imgBytes);
Any idea what can be causing this issue ?
Thanks

So,
I got it working, the problem was that certain classes on Android are broken and not behaving as the way they behave on core Java.
The same code that was working on Java, wasn't working here, reason being, a change in header occurring here (On Android).
This issue is very well mentioned here also:
https://github.com/WP-API/WP-API/issues/1862
Thus I was facing a similar issue, and adding external updated jars were conflicting with the ones on Android.
Finally I used a small HTTP Request library: https://github.com/kevinsawicki/http-request
The code is below:
HttpRequest request = HttpRequest.post(url);
request.authorization("Basic "+ah);
request.part("file", fName+".png", "image/png", new File(file));
request.part("title", "test");
if(request.code()==201) {
StringWriter sw = new StringWriter();
request.receive(sw);
onMedia(Media.parse(new JsonParser().parse(sw.toString()).getAsJsonObject()));
}
Thanks

Do not use PrintWriter to send raw bytes.

uploading png to server with java using POST data

Hi ive been having some trouble trying to transfer a png image to my webserver using java and php Ive tried using FTP but the software that Im scripting for blocks port 21 rendering it useless
I was directed to use form urlencoded data then use a POST request to get it
im completely lost on this topic and could just use some direction apparently file and image hosting sites use the same method to transfer files and images from the users computer to their servers.
maybe just an explanation of whats going on might help so that I can grasp what exactly im trying to do with java and php
Any help would be much appreciated!

I've also been facing the same kind of problem a short time ago.
After some researches, I found out that the HttpComponents library from Apache (http://hc.apache.org/) contains pretty much everything you'll need to build HTTP-POST request in a quite simple way.
Here is a method that will send a POST request with a file to a certain URL:
public static void upload(URL url, File file) throws IOException, URISyntaxException {
HttpClient client = new DefaultHttpClient(); //The client object which will do the upload
HttpPost httpPost = new HttpPost(url.toURI()); //The POST request to send
FileBody fileB = new FileBody(file);
MultipartEntity request = new MultipartEntity(); //The HTTP entity which will holds the different body parts, here the file
request.addPart("file", fileB);
httpPost.setEntity(request);
HttpResponse response = client.execute(httpPost); //Once the upload is complete (successful or not), the client will return a response given by the server
if(response.getStatusLine().getStatusCode()==200) { //If the code contained in this response equals 200, then the upload is successful (and ready to be processed by the php code)
System.out.println("Upload successful !");
}
}
In order to complete the upload, you must have a php code that handle that POST request,
here it is:
<?php
$directory = 'Set here the directory you want the file to be uploaded to';
$filename = basename($_FILES['file']['name']);
if(strrchr($_FILES['file']['name'], '.')=='.png') {//Check if the actual file extension is PNG, otherwise this could lead to a big security breach
if(move_uploaded_file($_FILES['file']['tmp_name'], $directory. $filename)) { //The file is transfered from its temp directory to the directory we want, and the function returns TRUE if successfull
//Do what you want, SQL insert, logs, etc
}
}
?>
The URL object given to the Java method must point to the php code, like http://mysite.com/upload.php and can be build very simply from a String. The file can also be build from a String representing its path.
I didn't take the time to test it properly, but it was build upon proper working solution, so I hope this will help you.

Java - Append data to a text file kept on webserver

I am trying to append some information to a text file kept on webserver using java using:
public class Main {
public static void main(String[] args) {
try {
URL url = new URL("http://www.abcd.com/info.txt");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
if (connection != null) {
System.out.println("Established URL connection");
}
connection.setDoOutput(true);
connection.setRequestMethod("PUT");
connection.setDoInput(true);
connection.setUseCaches(false);
connection.setRequestProperty("Content-Type", "text/html");
System.out.println(connection.getOutputStream().toString());
OutputStreamWriter writer = new OutputStreamWriter(connection.getOutputStream());
writer.write("This is a sample text");
writer.close();
}
catch (Exception ex) {
ex.printStackTrace();
}
}
}
Neither the text file is not being updated nor getting any error.. The reason for doing this is - I have developed a small software and the updates for this will be kept on web site. If any user updates the data, this code will update the text file. This way I will be able to get the information of user who have updated.

As far as I know, you first need to get the data written in the file, to client, Using a GET call , then append the data, and the finally do a POST call to rewrite the file with appended data

You would have to make the changes at server side to do that. You cannot acheive the same using HttpURLConnection.
You can try using FTP if its feasible for you. In case of FTP you should download the file, append the text and upload the same again.

I'm a bit confused - you're attempting to open an HTTP connection to a file and modify it on the fly?
I feel like I might be missing something - plain HTTP doesn't support this. Can you imagine the nightmare it would be if everybody could go around overwriting everybody else's websites (without authentication, even, as your code seems to suggest)?
What you're doing here is calling PUT on the /info.txt resource with your text as the entity body. I'm fairly sure that never has and never will overwrite the corresponding file.
What you need to do is either go through a protocol that supports file writing (WebDav, FTP...) or write server-side code that accepts a content submission (through, for example, a POST or PUT call with an entity body on a specific resource), analyses that input and modify its local file system.
Again, I might be misunderstanding your question entirely, in which case I apologise if I come off as somewhat patronising.

Large file transfer from HTTP server running Java Jersey Rest API

I am writing a web service using Java JDK 1.7 and Jersey Web service Framework. One of the things I need to provide is a way to allow authenticated clients to download certain large data files ( 1-3 GB). Ideally I would like this to be a pause and resume type downloadable option. I tried the jersey multi-part API and was able to get it to work on my client machine upto 400 MB but beyond that it ran into out-of memory issues. I am also worried that the server might fail when faced with simultaneous download requests. Any thoughts on how this can be done? Is Netty an option? Any pointers on how Netty can be integrated into a existing Jersey based web service? Are there other frame works available to help accomplish this? I do have to use java for the web service. Any pointers will be helpful.

If you are getting stuck on out-of-memory issues, you should check how you are handling the data you are downloading. If you are using Jersey's ClientResponse, make sure you are using getEntityInputStream() and not getEntity(). This way, you can stream the data, write it to file, and toss it aside, rather than letting it build up in the Java heap space.
I can't really speak about your simultaneous download concerns, but if you are using the web services framework, then it should be handled properly.
For both issues, more info on your specific implementation, especially code, will help you get a better response.

The server and the client must both support HTTP chunked encoding which allows one to stream data using HTTP. The code below should work with Jersey 2.11.
For downloading large files, try this on the server:
#GET
#Path("/files/{fileName}")
#Produces(MediaType.APPLICATION_OCTET_STREAM)
public StreamingOutput getFile(#PathParam("fileName") final String fileName) throws Exception {
//create instance of StreamingOutput here
return streamingOutput;
}
Try this for a client GET request using steams to download a file.
public String getFileReq(File outFile) throws IOException {
client = ClientBuilder.newClient(new ClientConfig());
client.property(ClientProperties.REQUEST_ENTITY_PROCESSING, "CHUNKED");
WebTarget target = client.target(URI)
OutputStream fileOutputStream = new FileOutputStream(outFile);
InputStream fileInputStream = target.request().get(InputStream.class);
writeFile(fileInputStream, fileOutputStream);
}
public static void writeFile(InputStream fileInputStream, OutputStream outputStream) throws IOException {
try {
byte[] buffer = new byte[1024];
int bytesRead;
while((bytesRead = fileInputStream.read(buffer)) !=-1) {
outputStream.write(buffer, 0, bytesRead);
}
fileInputStream.close();
outputStream.flush();
} catch (IOException e) {
e.printStackTrace();
} finally {
outputStream.close();
}