I am trying to validate the linkedIn profile of 100K person and wrote a dummy code but its giving "java.io.IOException: Server returned HTTP response code: 403 for URL: https://www.linkedin.com/in/test.user"
I have tried setting different setRequestProperty but not working.
public static void main(final String[] args) {
String output = "";
int TIMEOUT_VALUE = 99999999;
HttpURLConnection conn = null;
BufferedReader br = null;
String urlEndPoint = "";
String authUser = "";
String authPwd = "";
try {
long start = System.nanoTime();
urlEndPoint = "https://www.linkedin.com/in/test.user";
authUser = "linkedin-username";
authPwd = "linkedin-password";
URL url = new URL(urlEndPoint);
conn = (HttpURLConnection) url.openConnection();
conn.setRequestProperty("username", authUser);
conn.setRequestProperty("password", authPwd);
conn.setRequestProperty("Connection", "Keep-Alive");
conn.setRequestProperty("Keep-Alive", "header");
conn.setRequestProperty("accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8");
conn.setConnectTimeout(TIMEOUT_VALUE);
conn.setReadTimeout(TIMEOUT_VALUE);
conn.setRequestMethod("POST");
conn.setRequestProperty("Accept-Language", "en-US,en;q=0.9,mt;q=0.8");
conn.setRequestProperty("Accept-Encoding", "gzip,deflate,br");
conn.setRequestProperty("Host", "www.linkedin.com");
conn.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.80 Safari/537.36");
conn.setRequestProperty("http.agent", "Chrome/71.0.3578.80 (Windows NT 10.0; Win64; x64)");
conn.setDoOutput(true);
String userPassword = authUser + ":" + authPwd;
String encoding = Base64Encoder.encode(userPassword);
conn.setRequestProperty("Authorization", "Basic " + encoding);
OutputStream os = conn.getOutputStream();
os.flush();
conn.connect();
br = new BufferedReader(new InputStreamReader((conn.getInputStream())));
while ((output = br.readLine()) != null) {
System.out.println(output);
}
if (br != null) {
br.close();
}
if (os != null) {
os.close();
}
long elapsed = System.nanoTime() - start;
} catch (MalformedURLException e) {
//this.logger.error("Error occurred during processPartyTerrRelationship ", e);
e.printStackTrace();
} catch (IOException e) {
//this.logger.error("Error occurred during processPartyTerrRelationship ", e);
e.printStackTrace();
} catch (Exception e) {
//this.logger.error("Error occurred during processPartyTerrRelationship ", e);
e.printStackTrace();
} finally {
try {
if (conn != null) {
conn.disconnect();
}
} catch (Exception e) {
//this.logger.error("Error occurred during processPartyTerrRelationship ", e);
e.printStackTrace();
}
}
//logger.info("processPartyTerrRelationship called ends");
}
The outcode of above code is :
java.io.IOException: Server returned HTTP response code: 403 for URL: https://www.linkedin.com/in/test.user
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1894)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:263)
at ValidateLinkedInProfiles.main(ValidateLinkedInProfiles.java:57)
The HTTP error code 403 is an error related to the authorization to the requested resource:
HTTP 403 provides a distinct error case from HTTP 401; while HTTP 401 is returned when the client has not authenticated, and implies that a successful response may be returned following valid authentication, HTTP 403 is returned when the client is not permitted access to the resource for some reason besides authentication
It's hard to understand how you're working. The LinkedIn link requires login. But you indeed need to debug it somehow and need a raw real output to the server with the correct one otherwise you will not complete it. If you have Java example program, see if they or you have a typo, but again without screenshot or text from LinkedIn I cannot debug it. Maybe try to add the examples and I will try to help you (just make me login with my public profile to other places). Also make sure there is your real password and your user account in the correct fields of course (authUsr,authPwd shall not be copy paste unlike everything else).
HTTP 403 is a legitimate response from a server. So the behavior is valid. However, I would recommend to use some HTTP client utility rather then writing your own code to make Http request. This will reduce the chance of a problem caused by your own code. As some Http clients I would suggest Apache Http Client or OK Http client or MgntUtils Http Client (see MgntUtils HttpClient javadoc here, Full MgntUtils library on github is here and Maven repository is here). Disclaimer: MgntUtils library is written by me
HTTP 403 is a standard HTTP status code communicated to clients by an HTTP server to indicate that the server understood the request, but will not fulfill it. There are a number of sub-status error codes that provide a more specific reason for responding with the 403 status code.
You either do not have access to the site(try logging in from a browser and try to run the script from the same browser, if your access is shared across different tabs of the same browser that is also fine, but make sure you're authorized) or the request to the link contains sensitive information which the site doesn't want to share.
I am using java.net.HttpUrlConnection to make Http requests to my Server. I realized that if the Server return an error status (400 for example). HttpUrlConnection will throw an IOException corresponding to the error status.
My question is: Does HttpUrlConnection always throw an IOException if the server return an error status (4xx, 5xx)?
I take a look at HttpUrlConnection API description but I couldn't answer my question.
Updated: Please see my test:
public static void main(String[] args) {
try {
String url = "https://www.googleapis.com/oauth2/v3/userinfo?access_token=___fake_token";
HttpURLConnection conn = (HttpURLConnection)new URL(url).openConnection();
conn.getResponseCode();
conn.getInputStream();
} catch (Exception ex) {
ex.printStackTrace();
// Print IOException: java.io.IOException: Server returned HTTP response code: 401 for URL: https://www.googleapis.com/oauth2/v3/userinfo?access_token=abc
// If I commented conn.getResponseCode() -> The same IOException
}
}
Thank you!
Not if you check getResponseCode() before getInputStream() and the problem is an HTTP return code rather than a connect error.
I'm doing an Android app with an API with Python. The API is on a Google App Engine cloud and everything works fine when I tested it with Postman.
I'm trying to do a Login with a POST method. That method returns json with the user information I keep getting that error: FileNotFoundException
Here is some of my code:
try{
String account = params[0].get(0);
String password = params[0].get(1);
URL url = new URL("http", WEB_SERVICE_URL, PORT, REST_LOGIN);
HttpURLConnection httpURLConnection = (HttpURLConnection) url.openConnection();
httpURLConnection.setConnectTimeout(CONNECTION_TIMEOUT);
httpURLConnection.setRequestMethod("POST");
httpURLConnection.setDoOutput(true);
httpURLConnection.setRequestProperty("Content-Type", "application/json");
httpURLConnection.setRequestProperty("Accept", "application/json");
JSONObject json = jsonParser.serialJsonLogin(nomCompte, motPasse);
osw = new OutputStreamWriter(httpURLConnection.getOutputStream(),"UTF-8");
osw.write(json.toString());
osw.flush();
String body = readStream(httpURLConnection.getInputStream());
osw.close();
Log.i(TAG, "Return : " + body);
user = jsonParser.deserializeJsonUser(body);
}catch (Exception e) {
mException = e;
}finally {
if (mHttpURLConnection != null) {
mHttpURLConnection.disconnect();
}
}
return user;
At: String body = readStream(httpURLConnection.getInputStream()); I'm getting a java.io.FileNotFoundException: http://10.0.2.2:8080/login
My readStream method is fine, I tested it. If I look in my Google App Engine logs, I can see that there is no 404, or anything wrong. If I find the user I get a 201 if not a 403. So even if the error says FileNotFound, I see status code which means that actually the URL is right.
UPDATE: My API was giving me a 201 and getInputStream apparently doesn't work on 201 status. Changed my return status to 200 in my API and it works fine.
I hava a java application that calls a service. The service call will take about 5-10 minutes to complete its operation and return a status log as a response. (The reason behind the long duration is to copy files/images from one server to another). In the meantime any other call made to the service will be rejected with an error response. What's the best way to wait for this usually long response?
I tried the following request with no luck:
1.HTTPUrlConnection:
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setReadTimeout(10 * 60000);
conn.setConnectTimeout(10 * 60000);
conn.connect();
try {
conn.getInputStream();
} catch (Exception e) {
//error handling
}
results: after 5 minutes or so the request fails, error message indicating the service is already in progress. Seems like request is being call twice to the service after some kind of timeout.
2.HttpClient
HttpClient client = new HttpClient ();
GetMethod method = new GetMethod(requestUrl);
try {
int statusCode = client.executeMethod(method);
if(statusCode == HttpStatus.SC_OK) {
// success
} else {
String errorMessage = method.getResponseBodyAsString();
//error handling
}
}
results: after 5 minutes or so, the httpClient fails with an "Connection Reset" error.
Note that I'm able to run the url request on a browser, and is able to get a successful response back. Did I miss anything?
Thanks.
I'm trying to retrieve a json string from a comet URL link.
Here is the API link: http://www.plurk.com/API#realtime
Here is the description:
You'll get an URL from /APP/Realtime/getUserChannel and you do GET requests to this URL to get new data. Your request will sleep for about 50 seconds before returning a response if there is no new data added to your channel. You won't get notifications on responses that the logged in user adds, but you will get notifications for new plurks.
I was able to obtain the comet_server url and paste that to firefox and get the result manually. However, when I tried to get these json string in android, I only got timeout error.
01:48:51.698 com.net.xerothermic.plurk INFO PLURK http://comet58.plurk.com:80/comet?channel=...&offset=0
01:53:43.680 com.net.xerothermic.plurk ERROR PLURK HTTP con. get
response error:Connection timed out
Here is the code I used to retrieve the data.
URL url = new URL(urlString);
HttpURLConnection conn = null;
try
{
conn = (HttpURLConnection) url.openConnection();
}
catch (IOException ex)
{
Log.e("PLURK", "HTTP con. open error:" + ex.getMessage());
return "";
}
try
{
conn.setRequestMethod("GET");
}
catch (ProtocolException ex)
{
Log.e("PLURK", "HTTP con. set method error:" + ex.getMessage());
}
try
{
return conn.getResponseMessage();
}
catch (IOException ex)
{
Log.e("PLURK", "HTTP con. get response error:" + ex.getMessage());
return "";
}
Any suggestion is much appreciated!
EDIT: here is the output from a browser. Did I miss to set some properties?
Even though the timeout value was set to 0 by default (meaning wait infinitely), I found I still need to explicit set the timeout value in order to not raise IOException.
setConnectTimeout(70000);
setReadTimeout(70000);
This is only needed on android but not Windows...