URL.openstream() method can't read from link with & - java

I have a link to a .csv which looks like this:
http://website.com/data.csv?since=2019-03-21T00:00Z&sensor=xxx
My method get this link as URL
BufferedReader in = new BufferedReader(new InputStreamReader(URL.openStream()));
The problem is that it is not properly downloading .csv. I have realized that if I add 'amp;' after & it works properly. Is there any easy way to work this out?


I have found the solution: https://www.mkyong.com/java/java-httpurlconnection-follow-redirect-example/
public URL getFinalUrl(URL url) {
try {
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setReadTimeout(5000);
conn.addRequestProperty("Accept-Language", "en-US,en;q=0.8");
conn.addRequestProperty("User-Agent", "Mozilla");
conn.addRequestProperty("Referer", "google.com");
boolean redirect = false;
int status = conn.getResponseCode();
if (status != HttpURLConnection.HTTP_OK) {
if (status == HttpURLConnection.HTTP_MOVED_TEMP
|| status == HttpURLConnection.HTTP_MOVED_PERM
|| status == HttpURLConnection.HTTP_SEE_OTHER)
redirect = true;
}
String newUrl = null;
if (redirect) {
// get redirect url from "location" header field
newUrl = conn.getHeaderField("Location");
// get the cookie if need, for login
String cookies = conn.getHeaderField("Set-Cookie");
// open the new connnection again
conn = (HttpURLConnection) new URL(newUrl).openConnection();
conn.setRequestProperty("Cookie", cookies);
conn.addRequestProperty("Accept-Language", "en-US,en;q=0.8");
conn.addRequestProperty("User-Agent", "Mozilla");
conn.addRequestProperty("Referer", "google.com");
}
System.out.println(newUrl);
return new URL(newUrl);
} catch (IOException ioe) {
System.out.println("Wrong sensor URL");
}
return null;
}

Related

How to call subsequent request using j_security_check

How to call subsequent requests using j_security_check, I have followed below code but it's not working .
String url = "https://myhost/jsp/j_security_check?j_username=myuser&j_password=mypassword";
HttpURLConnection connection = (HttpURLConnection) new
URL(url).openConnection();
if (connection.getResponseCode() == 200) {
String cookie = connection.getHeaderField("Set-Cookie").split(";", 2)[0];
url = "https://myhost/getusers";
connection = (HttpURLConnection) new URL(url).openConnection();
connection.setRequestProperty("Cookie", cookie);
connection.setRequestProperty("Cache-Control", "no-cache");
InputStream input = connection.getInputStream();
try (BufferedReader buffer = new BufferedReader(new InputStreamReader(connection.getInputStream()))) {
String result = buffer.lines().collect(Collectors.joining("\n"));
System.out.println(result);
} catch (Exception e) {
e.printStackTrace();
}
}
Can anyone help me on it.

Not able to get Jsonstring from this link

String fileURL="https://tools.keycdn.com/geo.json?host=192.168.6.9";
URL url = new URL(fileURL);
HttpURLConnection httpConn = (HttpURLConnection) url.openConnection();
int responseCode = httpConn.getResponseCode();
System.out.println(responseCode);
Full code:
try {
String url = "http://tools.keycdn.com/geo.json?host=192.168.6.9";
URL obj = new URL(url);
HttpURLConnection conn = (HttpURLConnection) obj.openConnection();
conn.setInstanceFollowRedirects(true);
HttpURLConnection.setFollowRedirects(true);
conn.setReadTimeout(5000);
conn.addRequestProperty("Accept-Language", "en-US,en;q=0.8");
conn.addRequestProperty("User-Agent", "Mozilla");
conn.addRequestProperty("Referer", "google.com");
System.out.println("Request URL ... " + url);
boolean redirect = false;
// normally, 3xx is redirect
int status = conn.getResponseCode();
if (status != HttpURLConnection.HTTP_OK) {
if (status == HttpURLConnection.HTTP_MOVED_TEMP
|| status == HttpURLConnection.HTTP_MOVED_PERM
|| status == HttpURLConnection.HTTP_SEE_OTHER)
redirect = true;
}
System.out.println("Response Code ... " + status);
if (redirect) {
// get redirect url from "location" header field
String newUrl =conn.getHeaderField("Location");
newUrl=newUrl.replace("https://", "http://");
// get the cookie if need, for login
String cookies = conn.getHeaderField("Set-Cookie");
// open the new connnection again
conn = (HttpURLConnection) new URL(newUrl).openConnection();
conn.setRequestProperty("Cookie", cookies);
conn.addRequestProperty("Accept-Language", "en-US,en;q=0.8");
conn.addRequestProperty("User-Agent", "Mozilla");
conn.addRequestProperty("Referer", "google.com");
System.out.println("Redirect to URL : " + newUrl);
}
BufferedReader in = new BufferedReader(
new InputStreamReader(conn.getInputStream()));
String inputLine;
StringBuffer html = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
html.append(inputLine);
}
in.close();
System.out.println("URL Content... \n" + html.toString());
System.out.println("Done");
} catch (Exception e) {
e.printStackTrace();
}
}
OUTPUT:
Request URL ... http://tools.keycdn.com/geo.json?host=192.168.6.9
Response Code ... 301
Redirect to URL : http://tools.keycdn.com/geo.json?host=192.168.6.9
URL Content...
301 Moved Permanently301 Moved Permanentlynginx
Done
Iam not able to get json from this link, It gives 301 response code, I tried to get the redirected URL from HTTP header part, eventhough, It returns same URL and same 301 response code, please give java code solution to get JSON string from this URL.

Try this :
String fileURL="https://tools.keycdn.com/geo.json?host=192.168.6.9";
URL url;
try
{
final String USER_AGENT = "Mozilla/5.0";
url = new URL(fileURL);
HttpURLConnection httpConn = (HttpURLConnection) url.openConnection();
// optional default is GET
httpConn.setRequestMethod("GET");
//add request header
httpConn.setRequestProperty("User-Agent", USER_AGENT);
int responseCode = httpConn.getResponseCode();
System.out.println(responseCode);
BufferedReader in = new BufferedReader(
new InputStreamReader(httpConn.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
//print result
System.out.println(response.toString());
}
catch (Exception e)
{
e.printStackTrace();
}

Log in to facebook using httpurlconnection

I'm trying to login to facebook using httpurlconnect. I think i need to get the cookie from facebook.com/ajax/bz and then pass it along with my login attempt but maybe not. The response just takes me back to the login page and I never log in. Can someone help me get this to log in?
I do not want to use the official API.
private String getRequestCookie() throws IOException {
// URL myUrl = new URL("http://www.hccp.org/cookieTest.jsp");
/* URLConnection urlConn = null;
try {
urlConn = url.openConnection(currentProxy);
urlConn.connect();
} catch (IOException e) {
e.printStackTrace();
}*/
String cookies = "";
URL myUrl = new URL("https://www.facebook.com/ajax/bz");
URLConnection urlConn = myUrl.openConnection();
urlConn.connect();
String headerName=null;
for (int i=1; (headerName = urlConn.getHeaderFieldKey(i))!=null; i++) {
if (headerName.equals("Set-Cookie")) {
String cookie = urlConn.getHeaderField(i);
cookie = cookie.substring(0, cookie.indexOf(";"));
String cookieName = cookie.substring(0, cookie.indexOf("="));
String cookieValue = cookie.substring(cookie.indexOf("=") + 1, cookie.length());
cookies = cookies+cookieName+"="+cookieValue+"; ";
//cookieNameList.add(cookieValue);
System.out.println("da cookies:" + cookies);
return cookies;
}
}
// must have failed yo
cookies = "failed";
return cookies;
}
private String performJsonPost() {
try {
String cookies = getRequestCookie();
String query = "email=EMAILNAME%40EMAIL.com&pass=PASSWORD&lsd=AVrVVnf1&default_persistent=0&timezone=&lgnrnd=132329_hCSS&lgnjs=n&locale=tr_TR";
URL urls = new URL("https://www.facebook.com/login.php?login_attempt=1");
final HttpURLConnection conn = (HttpURLConnection)urls.openConnection();
final byte[] payloadAsBytes = query.getBytes(Charset.forName("UTF-8"));
conn.setConnectTimeout(15000);
conn.setReadTimeout(15000);
conn.setRequestMethod("POST");
conn.setRequestProperty("Cookie", "reg_fb_gate=https%3A%2F%2Fwww.facebook.com%2F; reg_fb_ref=https%3A%2F%2Fwww.facebook.com%2F; reg_ext_ref=deleted; Max-Age=0; datr=0aFzVK8Fu0Gl7M8cn_6TSqlZ");
conn.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:25.0) Gecko/20100101 Firefox/25.0");
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
conn.setRequestProperty("Content-Length", "" + payloadAsBytes.length);
conn.setRequestProperty("Content-Language", "en-US");
conn.setUseCaches(false);
conn.setDoInput(true);
conn.setDoOutput(true);
final DataOutputStream outStream = new DataOutputStream(conn.getOutputStream());
outStream.write(payloadAsBytes);
// outStream.write(payloadAsBytes);
outStream.flush();
outStream.close();
try {
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
/* catch (Exception e2) {
//inStream = conn.getErrorStream();
}*/
final StringBuilder response = new StringBuilder();
final byte[] buffer = new byte[1024];
int bytesRead;
//System.out.println(inStream.toString() + "WHAT ARE U SAYING BRO");
//while ((bytesRead = inStream.read(buffer)) > 0) {
//response.append(new String(buffer, "UTF-8").substring(0, bytesRead));
while ((line = rd.readLine()) != null) {
System.out.println(line);
response.append(line);
// inStream = conn.getInputStream();
}
return response.toString();
}catch (IOException e) {
e.printStackTrace();
return null;
}
// System.out.println("Response!:"+response.toString());
// return response.toString();
}
catch (IOException e) {
e.printStackTrace();
return null;
}
}

It doesn't matter that you want to do it; according to the Automated Data Collection Terms, it's not allowed. You app/server can be blocked. Don't do this. Instead, just use the Graph API and all documented features.

HTTP Request to Wikipedia gives no result

I try to fetch HTML per Code. When fetching from "http://www.google.com" for example it works perfect. When trying to fetch from "http://en.wikipedia.org/w/api.php" I do not get any results.
Does someone have any idea ?
Code:
String sURL="http://en.wikipedia.org/w/api.php?action=query&generator=categorymembers&gcmtitle=Category:Countries&prop=info&gcmlimit=500&format=json";
String sText=readfromURL(sURL);
public static String readfromURL(String sURL){
URL url = null;
try {
url = new URL(sURL);
} catch (MalformedURLException e1) {
e1.printStackTrace();
}
URLConnection urlconnect = null;
try {
urlconnect = url.openConnection();
urlconnect.setRequestProperty("User-Agent","Mozilla/5.0 (Windows NT 5.1; rv:19.0) Gecko/20100101 Firefox/19.0");
} catch (IOException e) {
e.printStackTrace();
}
BufferedReader in = null;
try {
in = new BufferedReader(new InputStreamReader(urlconnect.getInputStream()));
} catch (IOException e) {
e.printStackTrace();
}
String inputLine;
String sEntireContent="";
try {
while ((inputLine = in.readLine()) != null) {
System.out.println(inputLine);
sEntireContent=sEntireContent+inputLine;
}
} catch (IOException e) {
e.printStackTrace();
}
try {
in.close();
} catch (IOException e) {
e.printStackTrace();
}
return sEntireContent;
}

It looks like the request limit. Try to check the response code.
From the documentation (https://www.mediawiki.org/wiki/API:Etiquette):
If you make your requests in series rather than in parallel (i.e. wait
for the one request to finish before sending a new request, such that
you're never making more than one request at the same time), then you
should definitely be fine.
Be sure that you do not do few request at a time
Update
I did verification on my local your code - you are correct it does not work. Fix - you need to use https, so it would work:
https://en.wikipedia.org/w/api.php?action=query&generator=categorymembers&gcmtitle=Category:Countries&prop=info&gcmlimit=500&format=json
result:
{"batchcomplete":"","query":{"pages":{"5165":{"pageid":5165,"ns":0,"title":"Country","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-20T20:09:05Z","lastrevid":686706429,"length":12695},"5112305":{"pageid":5112305,"ns":14,"title":"Category:Countries by continent","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-18T17:31:54Z","lastrevid":681415612,"length":133},"14353213":{"pageid":14353213,"ns":14,"title":"Category:Countries by form of government","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-08-13T23:33:29Z","lastrevid":675984011,"length":261},"5112467":{"pageid":5112467,"ns":14,"title":"Category:Countries by international organization","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-18T05:11:12Z","lastrevid":686245148,"length":123},"4696391":{"pageid":4696391,"ns":14,"title":"Category:Countries by language","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-20T01:17:18Z","lastrevid":675966601,"length":333},"5112374":{"pageid":5112374,"ns":14,"title":"Category:Countries by status","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-08-13T21:05:47Z","lastrevid":675966630,"length":30},"708617":{"pageid":708617,"ns":14,"title":"Category:Lists of countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-18T05:08:45Z","lastrevid":681553760,"length":256},"46624537":{"pageid":46624537,"ns":14,"title":"Category:Caspian littoral states","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-09-23T08:40:34Z","lastrevid":663549987,"length":50},"18066512":{"pageid":18066512,"ns":14,"title":"Category:City-states","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-09-29T20:14:14Z","lastrevid":679367764,"length":145},"2019528":{"pageid":2019528,"ns":14,"title":"Category:Country classifications","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-09-25T09:09:13Z","lastrevid":675966465,"length":182},"935240":{"pageid":935240,"ns":14,"title":"Category:Country codes","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-17T06:05:53Z","lastrevid":546489724,"length":222},"36819536":{"pageid":36819536,"ns":14,"title":"Category:Countries in fiction","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-03T06:09:16Z","lastrevid":674147667,"length":169},"699787":{"pageid":699787,"ns":14,"title":"Category:Fictional countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-17T18:43:25Z","lastrevid":610289877,"length":356},"804303":{"pageid":804303,"ns":14,"title":"Category:Former countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-09-21T09:58:52Z","lastrevid":668632882,"length":403},"7213567":{"pageid":7213567,"ns":14,"title":"Category:Island countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-22T22:10:37Z","lastrevid":648502876,"length":157},"3046541":{"pageid":3046541,"ns":14,"title":"Category:Landlocked countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-04T00:45:24Z","lastrevid":648502892,"length":54},"743058":{"pageid":743058,"ns":14,"title":"Category:Middle Eastern countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-12T14:41:59Z","lastrevid":677900732,"length":495},"41711462":{"pageid":41711462,"ns":14,"title":"Category:Mongol states","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-23T07:36:21Z","lastrevid":687093637,"length":121},"30645082":{"pageid":30645082,"ns":14,"title":"Category:Country names","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-03-07T06:33:19Z","lastrevid":561256656,"length":94},"21218559":{"pageid":21218559,"ns":14,"title":"Category:Outlines of countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-07T18:04:29Z","lastrevid":645312408,"length":248},"37943702":{"pageid":37943702,"ns":14,"title":"Category:Proposed countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-20T02:30:25Z","lastrevid":668630396,"length":130},"15086044":{"pageid":15086044,"ns":14,"title":"Category:Turkic states","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-20T06:23:35Z","lastrevid":677424552,"length":114},"32809189":{"pageid":32809189,"ns":14,"title":"Category:Works about countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-17T08:45:32Z","lastrevid":620016516,"length":153},"27539189":{"pageid":27539189,"ns":14,"title":"Category:Wikipedia books on countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-04-11T05:12:25Z","lastrevid":546775798,"length":203},"35317198":{"pageid":35317198,"ns":14,"title":"Category:Wikipedia categories named after countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-17T18:35:14Z","lastrevid":641689352,"length":202}}}}
{"batchcomplete":"","query":{"pages":{"5165":{"pageid":5165,"ns":0,"title":"Country","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-20T20:09:05Z","lastrevid":686706429,"length":12695},"5112305":{"pageid":5112305,"ns":14,"title":"Category:Countries by continent","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-18T17:31:54Z","lastrevid":681415612,"length":133},"14353213":{"pageid":14353213,"ns":14,"title":"Category:Countries by form of government","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-08-13T23:33:29Z","lastrevid":675984011,"length":261},"5112467":{"pageid":5112467,"ns":14,"title":"Category:Countries by international organization","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-18T05:11:12Z","lastrevid":686245148,"length":123},"4696391":{"pageid":4696391,"ns":14,"title":"Category:Countries by language","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-20T01:17:18Z","lastrevid":675966601,"length":333},"5112374":{"pageid":5112374,"ns":14,"title":"Category:Countries by status","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-08-13T21:05:47Z","lastrevid":675966630,"length":30},"708617":{"pageid":708617,"ns":14,"title":"Category:Lists of countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-18T05:08:45Z","lastrevid":681553760,"length":256},"46624537":{"pageid":46624537,"ns":14,"title":"Category:Caspian littoral states","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-09-23T08:40:34Z","lastrevid":663549987,"length":50},"18066512":{"pageid":18066512,"ns":14,"title":"Category:City-states","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-09-29T20:14:14Z","lastrevid":679367764,"length":145},"2019528":{"pageid":2019528,"ns":14,"title":"Category:Country classifications","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-09-25T09:09:13Z","lastrevid":675966465,"length":182},"935240":{"pageid":935240,"ns":14,"title":"Category:Country codes","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-17T06:05:53Z","lastrevid":546489724,"length":222},"36819536":{"pageid":36819536,"ns":14,"title":"Category:Countries in fiction","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-03T06:09:16Z","lastrevid":674147667,"length":169},"699787":{"pageid":699787,"ns":14,"title":"Category:Fictional countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-17T18:43:25Z","lastrevid":610289877,"length":356},"804303":{"pageid":804303,"ns":14,"title":"Category:Former countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-09-21T09:58:52Z","lastrevid":668632882,"length":403},"7213567":{"pageid":7213567,"ns":14,"title":"Category:Island countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-22T22:10:37Z","lastrevid":648502876,"length":157},"3046541":{"pageid":3046541,"ns":14,"title":"Category:Landlocked countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-04T00:45:24Z","lastrevid":648502892,"length":54},"743058":{"pageid":743058,"ns":14,"title":"Category:Middle Eastern countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-12T14:41:59Z","lastrevid":677900732,"length":495},"41711462":{"pageid":41711462,"ns":14,"title":"Category:Mongol states","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-23T07:36:21Z","lastrevid":687093637,"length":121},"30645082":{"pageid":30645082,"ns":14,"title":"Category:Country names","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-03-07T06:33:19Z","lastrevid":561256656,"length":94},"21218559":{"pageid":21218559,"ns":14,"title":"Category:Outlines of countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-07T18:04:29Z","lastrevid":645312408,"length":248},"37943702":{"pageid":37943702,"ns":14,"title":"Category:Proposed countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-20T02:30:25Z","lastrevid":668630396,"length":130},"15086044":{"pageid":15086044,"ns":14,"title":"Category:Turkic states","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-20T06:23:35Z","lastrevid":677424552,"length":114},"32809189":{"pageid":32809189,"ns":14,"title":"Category:Works about countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-17T08:45:32Z","lastrevid":620016516,"length":153},"27539189":{"pageid":27539189,"ns":14,"title":"Category:Wikipedia books on countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-04-11T05:12:25Z","lastrevid":546775798,"length":203},"35317198":{"pageid":35317198,"ns":14,"title":"Category:Wikipedia categories named after countries","contentmodel":"wikitext","pagelanguage":"en","touched":"2015-10-17T18:35:14Z","lastrevid":641689352,"length":202}}}}

The reason you didn't receive the response back is due to a HTTP Redirect 3XX. Wikipedia redirects your HTTP Request. Please try the below source code to fetch Response from Redirected URL. Please refer How to send HTTP request GET/POST in Java
public static String readfromURLwithRedirect(String url) {
String response = "";
try {
URL obj = new URL(url);
HttpURLConnection conn = (HttpURLConnection) obj.openConnection();
conn.setReadTimeout(5000);
conn.addRequestProperty("Accept-Language", "en-US,en;q=0.8");
conn.addRequestProperty("User-Agent", "Mozilla");
conn.addRequestProperty("Referer", "google.com");
System.out.println("Request URL ... " + url);
boolean redirect = false;
// normally, 3xx is redirect
int status = conn.getResponseCode();
if (status != HttpURLConnection.HTTP_OK) {
if (status == HttpURLConnection.HTTP_MOVED_TEMP
|| status == HttpURLConnection.HTTP_MOVED_PERM
|| status == HttpURLConnection.HTTP_SEE_OTHER) {
redirect = true;
}
}
System.out.println("Response Code ... " + status);
if (redirect) {
// get redirect url from "location" header field
String newUrl = conn.getHeaderField("Location");
// get the cookie if need, for login
String cookies = conn.getHeaderField("Set-Cookie");
// open the new connnection again
conn = (HttpURLConnection) new URL(newUrl).openConnection();
conn.setRequestProperty("Cookie", cookies);
conn.addRequestProperty("Accept-Language", "en-US,en;q=0.8");
conn.addRequestProperty("User-Agent", "Mozilla");
conn.addRequestProperty("Referer", "google.com");
System.out.println("Redirect to URL : " + newUrl);
}
BufferedReader in = new BufferedReader(
new InputStreamReader(conn.getInputStream()));
String inputLine;
StringBuffer responseBuffer = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
responseBuffer.append(inputLine);
}
in.close();
System.out.println("URL Content... \n" + responseBuffer.toString());
response = responseBuffer.toString();
System.out.println("Done");
} catch (Exception e) {
e.printStackTrace();
}
return response;
}

Android HttpURLConnection: Handle HTTP redirects

I'm using HttpURLConnection to retrieve an URL just like that:
URL url = new URL(address);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setInstanceFollowRedirects(true);
// ...
I now want to find out if there was a redirect and if it was a permanent (301) or temporary (302) one in order to update the URL in the database in the first case but not in the second one.
Is this possible while still using the redirect handling of HttpURLConnection and if, how?

Simply call getUrl() on URLConnection instance after calling getInputStream():
URLConnection con = new URL(url).openConnection();
System.out.println("Orignal URL: " + con.getURL());
con.connect();
System.out.println("Connected URL: " + con.getURL());
InputStream is = con.getInputStream();
System.out.println("Redirected URL: " + con.getURL());
is.close();
If you need to know whether the redirection happened before actually getting it's contents, here is the sample code:
HttpURLConnection con = (HttpURLConnection) (new URL(url).openConnection());
con.setInstanceFollowRedirects(false);
con.connect();
int responseCode = con.getResponseCode();
System.out.println(responseCode);
String location = con.getHeaderField("Location");
System.out.println(location);

private HttpURLConnection openConnection(String url) throws IOException {
HttpURLConnection connection;
boolean redirected;
do {
connection = (HttpURLConnection) new URL(url).openConnection();
int code = connection.getResponseCode();
redirected = code == HTTP_MOVED_PERM || code == HTTP_MOVED_TEMP || code == HTTP_SEE_OTHER;
if (redirected) {
url = connection.getHeaderField("Location");
connection.disconnect();
}
} while (redirected);
return connection;
}

Categories