We Keep Coding

Issues while File download

I'm trying to download a file from Angular UI, even after I got exception in backed code still I'm getting 200 ok as the response.
Here is the code I have :
public ResponseEntity<Object> downloadDocument(#PathVariable("docId") Long docId,
HttpServletResponse response) {
OutPutStream outputStream = null;
try {
outputStream = response.getOutputStream();
docService.downloadDocument(docId,outputStream);
return ResponseEntity.ok().contentType(MediaType.parseMediaType(MediaType.APPLICATION_OCTET_STREAM_VALUE))
.body("Success");
} catch(Exception e) {
return ResponseEntity.badRequest().body(e.getMessage());
} finally {
if (Objects.nonNull(outputStream)) {
IOUtils.closeQuietly(outputStream);
}
}
Can you please help me out what's wring here.

When you open the outputstream, the headers are sent. period.
There's no backtracking from that point on. You can't first open the outputstream and then later go: Oh, wait! No! nevermind! bad request!
Here's how it works - you pick a side and stick to it. You can either:
Handle it yourself; use response and the methods available there to set up your response; you can set headers, the return code and message, and you can obtain an outputstream for the response body, and send data that way. If you do this, you can't ALSO return a ResponseEntity!
Do NOT even add an HttpServletResponse parameter, and instead return a ResponseEntity object.
You're doing both, which is not allowed.
I'm frankly surprised; spring is a bit broken and ought to be throwing exceptions here, as it cannot possibly serve up what you're asking it to do here.
NB: Note that the type of an exception is usually more informative than the message (many exceptions don't even have a message).
Putting it all together:
public ResponseEntity<?> downloadDocument(#PathVariable("docId") Long docId) {
try {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
docService.downloadDocument(docId, baos);
return ResponseEntity.ok()
.contentType(MediaType.parseMediaType(MediaType.APPLICATION_OCTET_STREAM_VALUE))
.body(baos.toByteArray());
} catch(Exception e) {
return ResponseEntity.badRequest().body(e.getMessage());
}
}
'toString' defaults to printing its own type if there is no message, and its own type plus the message if there is, so you get e.g.:
NullPointerException
or
NullPointerException: parameter foo
which is what you want (versus a literal blank string in the former case, and just 'parameter foo' in the latter, which isn't particularly insightful either).
messages are generally intended not to necessarily make sense without the context of the type of the exception.
NB: This will cache the entire content of the downloaded document into the memory of the server before sending it onwards. If the document is large, this is a bad idea, but, if you want to 'stream' it, you have a pretty serious problem inherent in the HTTP protocol: Once you start sending, you've already sent the 'error status' (i.e. you already sent 200 OK), so if the document download process throws an exception halfway through, you cannot go back and send an error message. You need some sort of wire protocol where you send the body in chunks and have a 'type' code you send, so that the recipient can scan for these and knows 'this type means there's more data flowing', and 'this type means an error occured and now I can read the error description'. That all gets quite complicated. By caching it, you avoid this mess. But if that document can be very large you're going to have to deal with the above, there are no quick fixes.

When does HttpURLConnection on Android really call the request

I have the following code:
HttpURLConnection conn = null;
BufferedReader in = null;
StringBuilder sb = null;
InputStream is = null;
conn = (HttpURLConnection) url.openConnection();
// Break-point A
conn.setDoInput(true);
conn.setDoOutput(true);
conn.setRequestMethod("POST");
// Break-point B
conn.setRequestProperty("X-TP-APP", Constants.X_TP_APP);
conn.setRequestProperty("X-TP-DEVICE", Constants.X_TP_DEVICE);
conn.setRequestProperty("X-TP-LOCALE", Constants.X_TP_LOCALE);
conn.setRequestProperty("Content-Type", contentType);
conn.setRequestProperty("Accept", accept);
conn.setRequestProperty("Authorization", SystemApi.TOKEN_STR);
conn.setUseCaches(false);
conn.setConnectTimeout(30000);
conn.getOutputStream().write(req.getBytes("UTF-8"));
conn.getOutputStream().flush();
conn.getOutputStream().close();
is = conn.getInputStream();
in = new BufferedReader(new InputStreamReader(is));
int statusCode = conn.getResponseCode();
// Break-point C
The code is running fine without problem (when breakpoint(A,B) is disabled)
I tried to find out when does HttpURLConnection really call the request and place breakpoint(A) after conn = getConnection(strURL);
and continue the code, but then at the end, at breakpoint(C), server would return me 401 - Unauthorized, which mean my Authorization header is not in the request.
It seem like that we are trying to open a connection first, and then set the header as fast as we can. If we are not fast enough, then the request is called anyway, which doesn't seem right.
My question and concern:
When does HttpURLConnection really call the request?
Is this what is actually happening? Is this the correct way to do so?
Is there a better way to make sure the header is set before calling the request?

Per the docs, the actual connection is made when the connect() method is invoked on the [Http]UrlConnection. That may be done manually, or it may be done implicitly by certain other methods. The Javadocs for UrlConnection.connect() say, in part:
URLConnection objects go through two phases: first they are created, then they are connected. After being created, and before being connected, various options can be specified (e.g., doInput and UseCaches). After connecting, it is an error to try to set them. Operations that depend on being connected, like getContentLength, will implicitly perform the connection, if necessary.
Note in particular the last sentence. I don't see anything in your code that would require the connection to be established until the first conn.getOutputStream(), and I read the docs as saying that the connection object will not enter the "connected" state until some method is invoked on it that requires that. Until such a time, it is ok to set connection properties.
Moreover, the docs definitely state that methods that set properties on the connection (and setRequestProperty() in particular) will throw an IllegalStateException if invoked when the connection object is already connected.
It is possible that your Java library is buggy in the manner you describe, but that would certainly be in conflict with the API specification. I think it's more likely that the explanation for the behavior you observe is different, and I recommend you capture and analyze the actual HTTP traffic to determine what's really going on.

Actually what really happened is, in the debug mode, I used conn.getResponseCode() in the expressions, which force the conn.getResponseCode() to run.
When it is not connected yet, getResponseCode() would calls connect() before the request is prepared.
Hence it would return me 401.

Since Android using the same HttpURLConnection, I did some capture the packet exchange to see what is happening under the hood.
I detailed my experiment in this post Can you explain the HttpURLConnection connection process?
To outline the network activity for your program.
At Breakpoint A No physical connection is made to the remote server. You get a logical handle to a local connection object.
At Breakpoint B You just configure the local connection object, nothing more.
conn.getOutputStream() Network connection starts here, but no payload is transferred to the server.
conn.getInputStream() Payload (http headers, content) are sent to the server, and you get the response (buffered into input stream, and also the response code etc.)
To Answer your question
When does HttpURLConnection really call the request?
getInputStream() triggers network layer to send out application payload and got responses.
Is this what is actually happening? Is this the correct way to do so?
No. openConnection() does not initiate network activity. You are getting back a local handle for future connection, not an active connection.
Is there a better way to make sure the header is set before calling the request?
You don't need to make sure header is set. The header payload isn't sent to the server until you ask for response (such as getting the response code, or opening a inputStream )

When does URL redirect happen?

I have a URL that redirects to another one, that redirects to another one, and I'm trying to get the end domain. I thought that the redirection would happen when connecting, but running this code tells me otherwise:
public static void main(String[] args)
{
System.out.println(resolveEndDomain("http://goo.gl/ELHEjl"));
}
private static String resolveEndDomain(String deepLink)
{
HttpURLConnection httpConnection = null;
try
{
httpConnection = (HttpURLConnection) new URL(deepLink).openConnection();
//httpConnection.getResponseCode();
}
catch (Exception e)
{
e.printStackTrace();
}
return httpConnection.getURL().getHost();
}
So, if I run it, it will just give me goo.gl. But if I uncomment the line to get the response code, it goes and fetches it from the end domain, and prints www.thomann.de. The same occurs if I use getHeaderFields(), getContent() or similar, but connect() doesn't seem to help. I'm not interested in any of those responses so, how can I make it go and resolve the redirections? When does the connection effectively occur?

As for your question, the redirect happens when the server being connected replies with a 301 or 302 HTTP Status Code. Note that Status Code are part of the answer from the server, so you must have actually requested the resource from it (usually with connect()).
As you have already found, HttpURLConnection usually handles the part of checking for the redirect Status Code and redirects you to the URL provided as part of the redirect response (up to a maximum of redirections, I am not sure where that limit is defined).
You could set followsRedirects to false and interpret the HTTP headers yourself to find if you are going to be redirected or not, but until you get a 2X Status code (or 4XX -not found, forbidden- or 5XX -server error-) you will not be sure if the URL you are being redirected to is the last one.
For example, you could connect to http://goo.gl/ELHEjl and find that it redirects you to http://www.bit.ly/s34314313. But then, unless you connect to that URL and get a not 3XX Status Code, you cannot be sure if connecting to http://www.bit.ly/s34314313 is the final resource location or it will just redirect you again.

Retrieve redirected URL with Java / HttpURLConnection

Given a URL (String ref), I am attempting to retrieve the redirected URL as follows:
HttpURLConnection con = (HttpURLConnection)new URL(ref).openConnection();
con.setInstanceFollowRedirects(false);
con.setRequestProperty("User-Agent","");
int responseType = con.getResponseCode()/100;
while (responseType == 1)
{
Thread.sleep(10);
responseType = con.getResponseCode()/100;
}
if (responseType == 3)
return con.getHeaderField("Location");
return con.getURL().toString();
I am having several (conceptual and technical) problems with it:
Conceptual problem:
It works in most cases, but I don't quite understand how.
All methods of the 'con' instance are called AFTER the connection is opened (when 'con' is instanciated).
So how do they affect the actual result?
How come calling 'setInstanceFollowRedirects' affects the returned value of 'getHeaderField'?
Is there any point calling 'getResponseCode' over and over until the returned value is not 1xx?
Bottom line, my general question here: is there another request/response sent through the connection every time one of these methods is invoked?
Technical problem:
Sometimes the response-code is 3xx, but 'getHeaderField' does not return the "final" URL.
I tried calling my code with the returned value of 'getHeaderField' until the response-code was 2xx.
But in most other cases where the response-code is 3xx, 'getHeaderField' DOES return the "final" URL, and if I call my code with this URL then I get an empty string.
Can you please advise how to approach the two problems above in order to have a "100% proof" code for retrieving the "final" URL?
Please ignore cases where the response-code is 4xx or 5xx (or anything else other than 1xx / 2xx / 3xx for that matter).
Thanks

Conceptual problems:
0.) Can one URLConnection or HttpURLConnection object be reused?
No, you can not reuse such an object. You can use it to fetch the content of one URL just once. You can not use it to retrieve another URL, nor to fetch the content twice (speaking on the network level).
If you want to fetch another URL or to fetch the URL a second time, you have to call the openConnection() method of the URL class again to instanciate a new connection object.
1.) When is the URLConnection actually connected?
The method name openConnection() is misleading. It only instanciates the connection object. It does not do anything on the network level.
The interaction on the network level starts in this line, which implicitly connects the connection (= the TCP socket under the hood is opened and data is sent and received):
int responseType = con.getResponseCode()/100;
.
Alternatively, you can use HttpURLConnection.connect() to explicitly connect the connection.
2.) How does setInstanceFollowRedirects work?
setInstanceFollowRedirects(true) causes the URLs to be fetched "under the hood" again and again until there is a non-redirect response. The response code of the non-redirect response is returned by your call to getResponseCode().
UPDATE:
Yes, this allows to write simple code if you do not want to bother about the redirects yourself. You can simply switch on to follow redirects and then you can read the final response of the location to which you get redirected as if there was no redirect taking place.

I would be more careful in evaluating the response code. Not every 3xx-code is automatically a kind of redirection. For example the code 304 just stands for "Not modified."
Look at the original definitions here.

Java - Append data to a text file kept on webserver

I am trying to append some information to a text file kept on webserver using java using:
public class Main {
public static void main(String[] args) {
try {
URL url = new URL("http://www.abcd.com/info.txt");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
if (connection != null) {
System.out.println("Established URL connection");
}
connection.setDoOutput(true);
connection.setRequestMethod("PUT");
connection.setDoInput(true);
connection.setUseCaches(false);
connection.setRequestProperty("Content-Type", "text/html");
System.out.println(connection.getOutputStream().toString());
OutputStreamWriter writer = new OutputStreamWriter(connection.getOutputStream());
writer.write("This is a sample text");
writer.close();
}
catch (Exception ex) {
ex.printStackTrace();
}
}
}
Neither the text file is not being updated nor getting any error.. The reason for doing this is - I have developed a small software and the updates for this will be kept on web site. If any user updates the data, this code will update the text file. This way I will be able to get the information of user who have updated.

As far as I know, you first need to get the data written in the file, to client, Using a GET call , then append the data, and the finally do a POST call to rewrite the file with appended data

You would have to make the changes at server side to do that. You cannot acheive the same using HttpURLConnection.
You can try using FTP if its feasible for you. In case of FTP you should download the file, append the text and upload the same again.

I'm a bit confused - you're attempting to open an HTTP connection to a file and modify it on the fly?
I feel like I might be missing something - plain HTTP doesn't support this. Can you imagine the nightmare it would be if everybody could go around overwriting everybody else's websites (without authentication, even, as your code seems to suggest)?
What you're doing here is calling PUT on the /info.txt resource with your text as the entity body. I'm fairly sure that never has and never will overwrite the corresponding file.
What you need to do is either go through a protocol that supports file writing (WebDav, FTP...) or write server-side code that accepts a content submission (through, for example, a POST or PUT call with an entity body on a specific resource), analyses that input and modify its local file system.
Again, I might be misunderstanding your question entirely, in which case I apologise if I come off as somewhat patronising.