can't get response header location using Java's URLConnection - java

can someone kindly suggest what I'm doing wrong here?
I'm trying to get the header location for a certain URL using Java
here is my code:
URLConnection conn = url.openConnection();
String location = conn.getHeaderField("Location");
it's strange since I know for sure the URL i'm refering to return a Location header and using methods like getContentType() or getContentLength() works perfectly


Perhaps Location header is returned as a part of redirect response. If so, URLConnection handles redirect automatically by issuing the second request to the pointed resource, so you need to disable it:
((HttpURLConnection) conn).setInstanceFollowRedirects(false);
EDIT:
If you actually need a URL of the redirect target and don't want to disable redirect handling, you may call getURL() instead (after connection is established).


Just a follow up to axtavt's answer... If the url has multiple redirects, you could do something like this in order to obtain the direct link:
String location = "http://www.example.com/download.php?getFile=1";
HttpURLConnection connection = null;
for (;;) {
URL url = new URL(location);
connection = (HttpURLConnection) url.openConnection();
connection.setInstanceFollowRedirects(false);
String redirectLocation = connection.getHeaderField("Location");
if (redirectLocation == null) break;
location = redirectLocation;
}

Related

Is using url.openConnection() function safe in java?

I have a shortened URL. Now I am using HttpUrlConnection to open the connection with the shortened link.
URL url = new URL(myshortened url);
Now I open the connection by calling:
HttpURLConnection httpurlconnection = url.openConnection();
Finally I am extracting the location header containing the actual destination URL by calling:
String expandedurl = httpurlconnection.getHeaderField("Location");
At the end I disconnect the httpurlconnection by calling:
httpurlconnection.disconnect();
I want to know if the URL I have used is of a malicious website, can it cause any harm to the calling host? If yes, then what are the possible ways it can attack the calling host?
Edit: I have even disabled redirect by calling:
httpurlconnection.setInstanceFollowRedirects(false);

It depends on what you do with the result. For example if you use it to query a database, it could be vulnerable for SQL injection.

Connection to URL

This is my code
try{
URL url = new URL("url to tes");
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.connect();
if(urlConnection.getResponseCode()==HttpURLConnection.HTTP_OK){
test=true
}catch()...
if(test){
tv.setText("yesssss");
}else{
tv.setText("noooo");
info: so I put this code on my mainActivity to test if I know how to connect on a url. My goal is to connect to web service that return me a json. But first I want to know how to connect.
Do you know a url that I can test with this code?
And my code here, is he good for connection?
Also tv is my TextView it work. It was to minimise the code.
I know after that I got to catch the InputStream and work with it to catch my json.

Use URLUtil to validate the URL as below.
URLUtil.isValidUrl(url)
It will return True if URL is valid and false if URL is invalid.

How to make REST API calls that have filter added

I am new to REST API and I want to make a REST API call which returns a JSON object
http://smlookup.dev/sec/products?search={"ABC.CP":"123A5"} - Runs fine in a browser and gives a JSON object
how do i get '?search={"ABC.CP":"12345"}' this expression to work as it filter the records
Code i am using is
String serverUrl="http://smlookup.dev/sec/products?search=";
String search=URLEncoder.encode("={\"ABC.CP\":\"12345\"}");
URL url = new URL(serverUrl+search);
HttpURLConnection httpCon = (HttpURLConnection) url.openConnection();
httpCon.setDoOutput(true);
httpCon.setRequestMethod("GET");
OutputStream out = httpCon.getOutputStream();
//FAILS GIVING 405 STATUS CODE
int responseCode = httpCon.getResponseCode();
All help or suggestions are helpful
Thanks!

Not sure if its normal but you dont send any data in your POST.
Furthermore you should urlencode your url, the inverted comma are not accepted like that.
URLEncoder.encode("={\"Xref.CP\":\"XAW3123A5\"}");

Retrieve the final location of a given URL in Java

I am trying to retrieve the final location of a given URL (String ref) as follows:
HttpURLConnection con = (HttpURLConnection)new URL(ref).openConnection();
con.setInstanceFollowRedirects(true);
con.setRequestProperty("User-Agent","");
int responseCode = con.getResponseCode();
return con.getURL().toString();
It works in most cases, but rarely returns a URL which yet contains another redirection.
What am I doing wrong here?
Why do I get responseCode = 3xx, even after calling setInstanceFollowRedirects(true)?
UPDATE:
OK, responseCode can sometimes be 3xx.
If it happens, then I will return con.getHeaderField("Location") instead.
The code now is:
HttpURLConnection con = (HttpURLConnection)new URL(ref).openConnection();
con.setInstanceFollowRedirects(true);
con.setRequestProperty("User-Agent","");
int responseType = con.getResponseCode()/100;
while (responseType == 1)
{
Thread.sleep(10);
responseType = con.getResponseCode()/100;
}
if (responseType == 3)
return con.getHeaderField("Location");
return con.getURL().toString();
Will appreciate comment should anyone see anything wrong with the code above.
UPDATE
Removed the handling of code 1xx, as according to most commenters it is not necessary.
Testing if the Location header exists before returning it, in order to handle code 304.
HttpURLConnection con = (HttpURLConnection)new URL(ref).openConnection();
con.setInstanceFollowRedirects(true);
con.setRequestProperty("User-Agent","");
if (con.getResponseCode()/100 == 3)
{
String target = con.getHeaderField("Location");
if (target != null)
return target;
}
return con.getURL().toString();

HttpURLConnection will not follow redirects if the protocol changes, such as http to https or https to http. In that case, it will return the 3xx code and you should be able to get the Location header. You may need to open a connection again in case that new url also redirects. So basically, use a loop and break it when you get a non-redirect response code. Also, watch out for infinite redirect loops, you could set a limit for the number of iterations or check if each new url has been visited already.

If you just want the redirect url, the response header should give you that:
if (con.getResponseCode() == 301) {
String redirectUrl = con.getHeaderField("Location");
}

There probably can easily be multiple levels of redirection - imagine a bit.ly pointing to a youtu.be address pointing to youtube.com. Perhaps you need to loop until you get your 200 OK or until you hit a redirection cycle.
I have trouble locating the source code to check but I believe what I said is true. See e.g. java urlconnection get the final redirected URL
You also might need to handle protocol redirects, e.g. HTTP -> HTTPS: URLConnection Doesn't Follow Redirect

I think I now understand what you want. I now think that you are trying to retrieve the final address, not the content of the final address. Please correct me if my assumption is wrong.
For doing this (not the content, but the address), you need a different approach. You need to switch off follow-redirects and you then need to handle the iterational redirect-following on your own until you find a non-redirecting response. Bear in mind that you can not reuse a URLConnection.
The approaches for finding the final address and the other approach for retrieving the content of the final address are so different, because URLConnection does not reveal the followed-to address if you switch on follow-redirects.
In your code, you seem to expect URLConnection.getURL() to return the followed-to address. This is not the behavior of this method. It returns the original URL which you used to create the URLConnection. It does this no matter if you switch on follow-redirects or not.
However, if you switch it on, you will not be able to get the followed-to URL address. This is because getHeaderField("Location"), with follow-redirects, makes no sense: it returns the redirection-target of the final redirect, which should not exist, since it's the final address.

Sometime it is loading in the field of requestURI. Use like this code:
val declaredField = con.javaClass.getDeclaredField("requestURI")
declaredField.isAccessible=true
val loc = declaredField.get(con).toString()

HttpURLConnection with Parameters

I have a URL which I pass in that looks like this
http://somecompany.com/restws/ebi/SVI/4048/?Name=Tra&Brand=Software: WebSphere - Open App Servers
It does not like the 2nd parameter (Brand). From a browser this querystring above works fine but as soon as I execute in Java, it fails. When I change the webservice to accept a single parameter, then this URL works fine
http://somecompany.com/restws/ebi/SVI/4048/?Name=Tra
It seems java is having issues with the 2nd parameter. I have tried escape characters and everything else I can think of but nothing seems to work. Please Help!
String uri = "somecompany.com/restws/ebi/SVI/4048/?Name="
+ name+ "&Brand=Software: WebSphere - Open App Servers";
URL url;
try {
url = new URL(uri);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("Accept", "application/xml");
}
...

Try url encoding your parameters.
Something like this:
String uri = "somecompany.com/restws/ebi/SVI/4048/?Name=" +name+ "&Brand=";
uri = URLEncoder.encode("Software: WebSphere - Open App Servers", "utf-8");

I'd perhaps use a library that can handle HTTP parameters properly and provide suitable encoding etc. See HttpComponents and the tutorial for more info.

Categories