I am trying to make a web proxy for HTTP communication between server and client. The GET method is working fine but I am POST method part is not working. I am sure I have missed out something. I want to know what have I missed or not implemented.
// request from client is handle from here
while ((inputLine = in.readLine()) != null) {
try {
StringTokenizer tok = new StringTokenizer(inputLine);
tok.nextToken();
} catch (Exception e) {
break;
}
if (cnt == 0) {
System.out.println("inputLine "+inputLine);
String[] tokens = inputLine.split(" ");
urlToCall = tokens[1];
//hum inputline sy URL nikaal rahay hai
if(tokens[0]=="POST")
{
f=1;
}
System.out.println("Request for : " + urlToCall);
}
cnt++;
}
BufferedReader rd = null;
try {
//yaha sy hum ab server ko request send karay gy
URL url = new URL(urlToCall);
URLConnection conn = url.openConnection();
HttpURLConnection huc = (HttpURLConnection) conn;
conn.setDoInput(true);
conn.setDoOutput(false);
// now we will get the response from the server
if (f == 1) {
huc.setDoOutput(true);
huc.setInstanceFollowRedirects(false);
huc.setRequestMethod("POST");
huc.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
huc.setRequestProperty("charset", "utf-8");
}
InputStream is = null;
if (conn.getContentLength() > 0)
{
try {
is = conn.getInputStream();
rd = new BufferedReader(new InputStreamReader(is));
} catch (IOException ioe) {
System.out.println(
"********* IO EXCEPTION **********: " + ioe);
}
}
What is the error you are getting on the post? And what is a sample GET request that is being passed into your code?
When I use a simple GET request, the code fails because the urlToCall does not have the host or protocol. The below code worked for me, but I would highly suggest you change your code to not hide the exceptions that are being thrown because they will have important information about what is going wrong with your code.
if (cnt == 1) {
System.out.println("host: " + inputLine);
String[] tokens = inputLine.split(" ");
urlToCall = "HTTP://" + tokens[1] + urlToCall;
}
Related
how to use the perspective api, in java, I have tried several codes and it does not work too much .
I have do this, but not work.
`
try {
String url = "https://commentanalyzer.googleapis.com/$discovery/rest?version=v1alpha1&", key = "key=my_key";
final URL serverUrl = new URL(url + key);
URLConnection urlConnection = serverUrl.openConnection();
HttpURLConnection httpConnection = (HttpURLConnection)urlConnection;
httpConnection.setRequestMethod("POST");
httpConnection.setRequestProperty("Content-Type", "application/json");
httpConnection.setDoOutput(true);
BufferedWriter httpRequestBodyWriter = new BufferedWriter(new
OutputStreamWriter(httpConnection.getOutputStream()));
httpRequestBodyWriter.write("{\"comment\": {\"text\": \"" + "Salut mec" + "\"},"
+ "\"requestedAttributes\": {\"TOXICITY\": {}}}");
httpRequestBodyWriter.flush();
httpRequestBodyWriter.close();
System.out.println("CODE : " + httpConnection.getResponseCode());
BufferedReader responseBuffer = new BufferedReader(new InputStreamReader((httpConnection.getInputStream())));
String output;
while ((output = responseBuffer.readLine()) != null) {
System.out.println(output);
}
httpConnection.disconnect();
} catch (IOException e1) {
e1.printStackTrace();
}`
I have Code 404, and Caused by: java.io.FileNotFoundException: https://commentanalyzer.googleapis.com/$discovery/rest?version=v1alpha1&key=my_key
Have you an idea ?
I want to scrape posts from Imgur.
Let's say we have this link(permalink): https://imgur.com/gallery/ZXjNfqu/comment/1717354251
I want to load this url via HttpURLConnection and read some data from content(HTML).
I have this code:
try{
//create connection
HttpURLConnection connection = (HttpURLConnection)(new URL(url)).openConnection();
connection.setRequestMethod("GET");
//get response
int responseCode = connection.getResponseCode();
if(responseCode == HttpURLConnection.HTTP_OK){
//read html
StringBuilder body = new StringBuilder();
try(var reader = new BufferedReader(new InputStreamReader(connection.getInputStream()))){
String line;
while((line = reader.readLine()) != null){
body.append(line);
}
}
//print html
System.out.println(body.toString());
}else{
//read error content
StringBuilder body = new StringBuilder();
try(var reader = new BufferedReader(new InputStreamReader(connection.getErrorStream()))){
String line;
while((line = reader.readLine()) != null){
body.append(line);
}
}
String bodyStr = body.toString().trim();
throw new Exception("Bad HTTP Request: " + responseCode +
" - " + connection.getResponseMessage() +
(bodyStr.isEmpty() ? "" : " - " + bodyStr));
}
}catch(Exception e){ e.printStackTrace(); }
Via HttpURLConnection I get only a part of the real page(what I see in browser). From HTML is missing at least image url and comments.
How I can get all content via HttpURLConnection?
P.S.: Is not necessary to use HttpURLConnection
Edit 1: I think it's because of javascript. But I'm not sure.
I'm trying to set the OAuth Authorization header of a HttpsURLConnection object and below is the java code for that
String url1 = "/data/ServiceAccount?schema=1.0&form=json&byBillingAccountId={EQUALS,xyz#pqr.edu}";
String url = "https://secure.api.abc.net/data/ServiceAccount?schema=1.0&byBillingAccountId={EQUALS,xyz#pqr.edu}";
String header = OAuthClient.prepareURLWithOAuthSignature(url1);
HttpsURLConnection con = null;
try {
URL obj = new URL(url);
con = (HttpsURLConnection) obj.openConnection();
con.setRequestMethod("GET");
con.setRequestProperty("Authorization", "OAuth " + header);
System.out.println("Request properties = " + con.getRequestProperty("Authorization"));
int responseCode = con.getResponseCode();
System.out.println("Response Code = " + responseCode);
BufferedReader in = new BufferedReader(
new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
con.disconnect();
//print result
System.out.println("Response = " + response.toString());
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if(con!=null) con.disconnect();
}
And below is the code for prepareURLWithOAuthSignature
public String prepareURLWithOAuthSignature(String url)
{
String signature = null;
setOAuthParameters();
setOAuthQParams();
try
{
httpURL = URLEncoder.encode(baseURL+url, "UTF-8");
signature = OAuthSignatureService.getSignature(httpURL, URLEncoder.encode(URLEncodedUtils.format(qparams, "UTF-8"), "UTF-8"), consumer_secret);
OAuthParameters.put("oauth_signature", signature);
} catch (Exception e) {
e.printStackTrace();
}
return getOAuthAuthorizationHeader();
}
public String getOAuthAuthorizationHeader()
{
String OAuthHeader = "oauth_consumer_key=\"" + OAuthParameters.get("oauth_consumer_key") + "\"" +
",oauth_signature_method=\"" + OAuthParameters.get("oauth_signature_method") + "\"" +
",oauth_timestamp=\"" + OAuthParameters.get("oauth_timestamp") + "\"" +
",oauth_nonce=\"" + OAuthParameters.get("oauth_nonce") + "\"" +
",oauth_version=\"" + OAuthParameters.get("oauth_version") + "\"" +
",oauth_signature=\"" + OAuthParameters.get("oauth_signature") + "\"";
byte[] authEncBytes = Base64.encodeBase64(OAuthHeader.getBytes());
String authStringEnc = new String(authEncBytes);
return authStringEnc;
}
The problem is that
1) while I'm printing the con.getRequestProperty("Authorization") I'm getting a null value which means the Authorization header is not set
2) The final response I'm getting from the server is 403
Any idea what's going wrong here?
I know this might not be an answer but looks like this issue was submitted as a bug to sun and here is the relevant part of the reply.
This behavior is intentional in order to prevent a security hole that
getRequestProperty() opened. setRequestProperty("Authorization")
should still work, you just won't be able to proof the results via
getRequestProperty().
For the original forum post, please see: http://www.coderanch.com/t/205485/sockets/java/setRequestProperty-authorization-JDK
I would not be able to advice why you're getting a 403 but try adding the "Content-Type" request header to your connection and see if it makes any difference. Until I added that header in my code, I was getting a 404 back from the Spring Security module.
So I want to send a GET request with parameters. But it only seems to have conventions for the url you send the request to. Unlike the POST request, I see no way to pass parameters in it.
How I send the GET request now, without parameters (might be wrong):
String url = "http://api.netatmo.net/api/getuser";
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
// optional default is GET
con.setRequestMethod("GET");
//add request header
con.setRequestProperty("User-Agent", USER_AGENT);
int responseCode = con.getResponseCode();
Log.v(TAG, ("\nSending 'GET' request to URL : " + url));
Log.v(TAG, ("Response Code : " + responseCode));
BufferedReader in = new BufferedReader(
new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
//print result
Log.v(TAG, (response.toString()));
How I send the POST request with parameters:
String url = "https://api.netatmo.net/oauth2/token";
URL obj = new URL(url);
HttpsURLConnection con = (HttpsURLConnection) obj.openConnection();
//add request header
con.setRequestMethod("POST");
con.setRequestProperty("User-Agent", USER_AGENT);
con.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
String urlParameters = "grant_type=password&client_id=myid&client_secret=mysecret&username=myusername&password=mypass";
// Send post request
con.setDoOutput(true);
DataOutputStream wr = new DataOutputStream(con.getOutputStream());
wr.writeBytes(urlParameters);
wr.flush();
wr.close();
int responseCode = con.getResponseCode();
Log.v(TAG, "\nSending 'POST' request to URL : " + url);
Log.v(TAG, "Post parameters : " + urlParameters);
Log.v(TAG, "Response Code : " + responseCode);
BufferedReader in = new BufferedReader(
new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
//print result
Log.v(TAG, response.toString());
access_token = response.substring(17, 74);
refresh_token = response.substring(93,150);
getRequest = "/api/getuser?access_token=" + access_token + " HTTP/1.1";
Log.v(TAG, access_token);
Log.v(TAG, refresh_token);
Log.v(TAG, getRequest);
As per the HTTP specification GET supports only path params or url params and hence you cannot put the params in HTTP request body as you do in POST request.
As Sotirios mentioned in the comments, technically you can still push params in the GET body, but if the APIs are respecting the specs, they will not provide you a way to do it.
Have you tried to add the query params to the request java.net.URL?
String url = "http://api.netatmo.net/api/getuser?access_token=" + access_token;
URL obj = new URL(url);
I was encountering the same problem, trying this:
String bla = "http://api.netatmo.net/api/devicelist?access_token=" + AUTH_TOKEN;
URL url = new URL(bla);
BufferedReader reader = new BufferedReader(new InputStreamReader(url.openStream()));
String line = "";
String message = "";
while ((line = reader.readLine()) != null)
{
message += line;
}
I got an exception that the syntax was not correct. When I changed the syntax (by for example encoding with UTF 8) the API would just return errors (like 404 not found...).
I finally got it working using this:
try
{
System.out.println("Access Token: " + AUTH_TOKEN);
String url = "http://api.netatmo.net/api/devicelist";
String query = "access_token=" + URLEncoder.encode(AUTH_TOKEN, CHARSET);
URLConnection connection = new URL(url + "?" + query).openConnection();
connection.setRequestProperty("Accept-Charset", CHARSET);
InputStream response = connection.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(response));
String line = "";
String message = "";
while ((line = reader.readLine()) != null)
{
message += line;
}
return message;
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Note: CHARSET = "UTF-8"
Turns out the url the API provided confused me greatly. I fixed the url and it works now.
I have a java servlet class that is performing a GET to a specific URL. I am also passing data as part of the GET.
What I need, is in my HTTP Server code that recieves this data, how do I insert user based response data into the Header back so my calling Java servlet class can read it.
I can read standard response things like .getResponseCode() etc, but I need to insert my own response into the header some how. How can this be done? and how can I read it?
This is my java servlet send class:
public void sendRequest(String data, String sendUrl) throws Throwable{
String messageEncoded = URLEncoder.encode(data, "UTF-8");
String message = URLDecoder.decode(messageEncoded);
System.out.println("messageEncoded : " + messageEncoded);
System.out.println("messageDecoded : " + message);
try {
URL url = new URL(sendUrl);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setRequestMethod("GET");
OutputStreamWriter writer = new OutputStreamWriter(connection.getOutputStream());
writer.write(message);
writer.close();
BufferedReader rd = null;
StringBuilder sb = null;
String line = null;
System.out.println(" *** headers ***");
for (Entry<String, List<String>> headernew : connection.getHeaderFields().entrySet()) {
System.out.println(headernew.getKey() + "=" + headernew.getValue());
}
System.out.println(" \n\n*** Body ***");
rd = new BufferedReader(new InputStreamReader(connection.getInputStream()));
sb = new StringBuilder();
while ((line = rd.readLine()) != null) {
sb.append(line + '\n');
}
System.out.println("body=" + sb.toString());
System.out.println("connection.getResponseCode() : " + connection.getResponseCode());
System.out.println("connection.getResponseMessage()" + connection.getResponseMessage());
if (connection.getResponseCode() == HttpURLConnection.HTTP_OK) {
// Ok
} else {
// Server returned HTTP error code.
}
} catch (MalformedURLException e) {
// ...
System.out.println(this.getClass() + " : MalformedURLException Error occured due to: " + e);
} catch (IOException e) {
System.out.println(this.getClass() + " : IOException Error occured due to: " + e);
}
}