We Keep Coding

How to completely abort the output stream download?

we're currently working on the service that would archive the data and return it to the user as a ZipOutputStream. What we're currently looking for is an option to completely terminate the operation if something goes wrong on the server side. With our current implementation (just closing the response output stream) errors result in a malformed zip at the user side, but it can't be told if the archive is malformed or not before attempting to unzip it. The desired behavior would be something like download termination (from a browser perspective, for instance, it would result in an unsuccessful download indication (red cross icon or something similar, depending on the browser) explicitly telling the user that something went wrong). We're using Spring Boot, so any java code examples would really be appreciated, but if you know the underlying HTTP mechanism that is responsible for this kind of behavior, and can point in the right direction, that would be much appreciated too.
Here's what we have as of now (output being a response output stream of a Spring REST controller (HttpServletResponse.getOutputStream()) :
try (ZipOutputStream zipOutputStream = new ZipOutputStream(outputStream)) {
try {
for (ZipRecordFile fileInfo : zipRecord.listZipFileOverride()) {
InputStream fileStream = getFileStream(fileInfo.s3region(), fileInfo.s3bucket(),
fileInfo.s3key());
ZipEntry zipEntry = new ZipEntry(fileInfo.fileName());
zipOutputStream.putNextEntry(zipEntry);
fileStream.transferTo(zipOutputStream);
}
}
catch (Exception e) {
outputStream.close();
}
}

There isn't a (clean) way to do what you want:
Once you have started writing the ZIP file to the output stream, it is too late to change the HTTP response code. The response code is sent at the start of response.
Therefore, there is no proper way for the HTTP server to tell the HTTP client: "Hey ... ignore that ZIP file I sent you 'cos it is corrupt".
So what are the alternatives?
On the server side, create the entire ZIP as an in-memory object or write it to a temporary file. If you succeed, send an 2xx response followed by the ZIP data. If you fail, send a 4xx or 5xx response.
The main problem is that you need enough memory or file system space to hold the ZIP file.
Redesign your HTTP API so that the client can sent a second request to check if the first request's response contained a complete ZIP file.
You might be able to exploit MIME multipart encoding; see RFC 1341. Each part of a well-formed MIME multipart has a start marker and an end-marker. What you could try is to have your web-app construct the multipart stream containing the ZIP "by hand". If it decides it must abort the ZIP, it could just close the output stream without adding the required end marker.
The main problem with this is that you are depending on the HTTP stack on the client side to tell the browser (or whatever) that the multipart is corrupted. Furthermore, the browser (or whatever) must not pass on the partial (i.e. corrupt) ZIP file on to the user. I'm not sure if you can rely on (particular) web browsers to do that.
If you are running the download via custom code on the client side, you could conceivably implement your own encapsulation protocol. The effect would be the same as for 3 ... but you wouldn't be abusing the MIME spec.

Upload chunked video to Twitter with spring-social-twitter / postman

I am currently trying to build an application to post videos to Twitter on behalf of a user.
So I currently have the application-key, application-secret, access-token and access-secret.
final TwitterTemplate twitterTemplate = new TwitterTemplate(
"application-key",
"application-secret",
"access-token",
"access-secret");
So using this I can actually post a tweet containing only text.
If I want to include an image I have to include the workaround "solution" posted on a bug of spring-social-twitter. This resolves the image but the video still cannot be uploaded.
So the rational thinking was to try out this upload with postman to "isolate" the call itself.
In the previous image we can see the Authorization process. This is the same for Every single call I make.
with this Auth a simple POST to https://api.twitter.com/1.1/statuses/update.json?status=hello works. Same with 2 calls for image.
POST to https://upload.twitter.com/1.1/media/upload.json?media_category=tweet_image for the image upload ( body -> media : image ).
POST to https://api.twitter.com/1.1/statuses/update.json merge the media_id with a new tweet.
But back to the video following again the official guide of twitter when I send this
the response is
when the request media_type does not contain / then a valid response will return from tweeter with a media_id, lets call it X.
So I append a video ( second command ) on X and then finalize the video ( third command ) on X. But as expected the response is
as the media_type was never provided. On the other hand if the video on the second step is pushed as base64 encoded ( and including the header for base64 encoding) then a response of
No matter what I have done so far I cannot make it post a video. I even used Postman as a proxy for twurl and captured the request of twurl that did upload the video. Changed the auth with mine ( because of nonce it requires to be recreated ) and the request failed to upload the video!
some notes :
the credentials are up to dates and work from twurl.
the video is valid and can be uploaded from both tweeter ui and twurl upload command.
the base64 convert was encoded / decoded with linux base64 tool piped to file and validated that the size is the same.
If any other clarification is required please let me know!
Thank you in advance

I know it's late but in case someone has same question,
to upload video on twitter you have to use chunked-media-upload method.
here's the reference
https://developer.twitter.com/en/docs/media/upload-media/uploading-media/chunked-media-upload

REST: Tell client to send only csv and text format

In restfull WS, how to tell client to send only csv and text format file.
In content-type header, client set the format in which it is sending request and in Accept header, client set the format in which it want to accept response.
But how to tell client to send only content-type csv or file ? Is this through some documentation ?

The 415 status code seems to be suitable for this situation:
6.5.13. 415 Unsupported Media Type
The 415 (Unsupported Media Type) status code indicates that the
origin server is refusing to service the request because the payload
is in a format not supported by this method on the target resource.
The format problem might be due to the request's indicated
Content-Type or Content-Encoding, or as a result of inspecting the
data directly.
The response payload could contain a list of the media types supported by the server.

Image you have an endpoint called /textfiles - the developer using your API is usually reading your documentation on how to implement this endpoint. Unless you're not doing some auto-discovery magic (which I guess is still not common).
If we take Facebook for example, they just state in their documentation which files you can send:
We accept the following files: 3g2, 3gp, 3gpp, [...]
Your question:
But how to tell client to send only content-type csv or file ?
is also a bit unclear. When the user has sent the request, he already attached the files he thought he could send. So here you would rather send an error with a message, which files are allowed. So are we talking about some "pre"-requests here?
From a backend developers point of view I can just tell you: It's in the documentation. Handle errors properly, document and your implementing developer will not hate you :)

if i develop a restful application using spring i would set the produces attribute to return csv or plain text ( https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/web/bind/annotation/RequestMapping.html) . if the client tries to request a resource other than csv or text it will recieve an error . probably 415

Finding the downloaded filetype from HTTP message

I am trying to build a basic downloaded file scanning extension for the popular open source security application ZAP. using the built in sniffer, I can access the HTTP response messages. I am unable to determine the filetype of the file being downloaded. Although the Mozilla blog regarding HTTP talks about using the MIME Type in the 'Content-Type' header to determine the file type, I find that none of the response messages that I get have anything other than application/json or text/html or application/octet-stream. How do I determine if the corresponding HTTP response body contains any particular file type? . I am thus stuck at a dead end!
I am a beginner in this field and there might be something that I am over looking. Any help or pointers would be greatly appreciated.

The Content-Type entity-header field indicates the media type of the entity-body sent to the recipient or, in the case of the HEAD method, the media type that would have been sent had the request been a GET.
Taken from https://www.rfc-editor.org/rfc/rfc2616 under "14.17 Content-Type"
They give this as an example:
Content-Type: text/html; charset=ISO-8859-4
This HTTP request or response contains text in the form of a body of HTML.
If you do not trust this header (which most of the time you can), the next step would be analyzing the file contents. For example, if the file contains opening and closing HTML tags, then there is a good chance that the file is an HTML file. If the file begins with a [ or { and ends with a ] or } then there is a good chance that it is a JSON file. An actual analysis would and should be much more detailed, of course.

Include attachment in .ics file for a response to a meeting request

I am using the following tag while creating my ics object;
Attach;FMTTYE= text/plain ;ENCODING =BASE64; VALUE =BINARY : (base 64 encoded text for a image)
But it seems server not able to parse the ATTACH tag - the attachment is not getting displayed in the exchange server.
Can any one tell me how to include attachment in .ics file for a response to a meeting request?