I am trying to use some code that I got from a website that has sports data served publically via an API (http://developer.fantasydata.com).
The site provide some sample JAVA code to make the http request. For some reason the setEntity method for the declared request (request) is showing a "cannot find symbol error.
package epl.fixtures.test.app;
import java.net.URI;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.utils.URIBuilder;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;
public class EPLFixturesTestApp {
/**
* #param args the command line arguments
*/
public static void main(String[] args) {
// TODO code application logic here
HttpClient httpclient = HttpClients.createDefault();
try
{
URIBuilder builder = new URIBuilder("https://api.fantasydata.net/soccer/v2/json/CompetitionDetails/EPL");
URI uri = builder.build();
HttpGet request = new HttpGet(uri);
request.setHeader("Ocp-Apim-Subscription-Key", "****************");
// Request body
StringEntity reqEntity = new StringEntity("{body}");
request.setEntity(reqEntity);
HttpResponse response = httpclient.execute(request);
HttpEntity entity = response.getEntity();
if (entity != null)
{
System.out.println(EntityUtils.toString(entity));
}
}
catch (Exception e)
{
System.out.println(e.getMessage());
}
}
}
The line causing the issue is the request.setEntity(reqEntity); line
Can anyone explain this to me please? I have all the relevant jar files from apache added to the project libraries directory.
Thanks
HttpGet does not have a setEntity method.
This makes sense, since the request body has no meaning in GET requests.
Only classes implementing HttpEntityEnclosingRequest have this method.
I don't know why the documentation uses it, but it seems to work when omitting those two lines (which look meaningless anyway). Code:
URIBuilder builder = new URIBuilder("https://api.fantasydata.net/soccer/v2/json/CompetitionDetails/EPL");
URI uri = builder.build();
HttpGet request = new HttpGet(uri);
request.setHeader("Ocp-Apim-Subscription-Key", "****************");
HttpResponse response = httpclient.execute(request);
HttpEntity entity = response.getEntity();
if (entity != null)
{
System.out.println(EntityUtils.toString(entity));
}
I am receiving an error org.apache.http.client.HttpResponseException: Bad Request when I try to run this program.
Could you please help me understand where I should modify the code ?
I am using the following libraries
httpclient-4.4.1.jar
httpcore-4.4.1.jar
commons-logging-1.1.2.jar
org.apache.http.client.HttpResponseException: Bad Request
Here is the code :
import org.apache.http.client.HttpClient;
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.BasicResponseHandler;
import org.apache.http.impl.client.DefaultHttpClient;
public class Test {
public static void main(String args[]) throws Exception {
HttpClient httpclient = new DefaultHttpClient();
HttpPost postRequest = new HttpPost("http://localhost:8080/engine-rest/process-definition/key/demo-scaling/start");
try {
StringEntity input = new StringEntity("(\"variables\":{}, \"businessKey\" : \"AAA001\")");
postRequest.addHeader("Accept", "application/json");
postRequest.setEntity(input);
postRequest.addHeader("Content-Type", "application/json");
ResponseHandler<String> responseHandler = new BasicResponseHandler();
String responseBody = httpclient.execute(postRequest, responseHandler);
System.out.println(responseBody);
} catch (Exception e) {
e.printStackTrace();
} finally {
httpclient.getConnectionManager().shutdown();
}
}
}
Ok, try this:
HttpClient httpClient = HttpClientBuilder.create().build();
try {
HttpPost request = new HttpPost("http://localhost:8080/engine-rest/process-definition/key/demo-scaling/start");
StringEntity params =new StringEntity("variables={\"businessKey\":\"AAA001\"}");
request.addHeader("content-type", "application/x-www-form-urlencoded");
request.setEntity(params);
HttpResponse response = httpClient.execute(request);
System.out.println(response);
}catch (Exception ex) {
// handle Exceptions
}
Use httpclientbuilder to get client (DefaultHttpClient -> Deprecated) and i am not sure if you have a valid JSON data, this is only my suggest.
Please forgive me if I ask something stupid, I am a novice here. I need to implement OAuth in my Java application to authenticate against launchpad.net API. The documentation specifies an initiation of a token request with three parameters : oauth_consumer_key e.g. (name of my application), oauth_signature_method e.g. "PLAINTEXT" and oauth_signature e.g. The string "&". I realised that most OAuth libraries require that
I have already acquired a Consumer key and Consumer Id/Secret from
the OAuth provider (e.g as issued in Twitter), and most examples are organised in this manner. However, launchpad.net will issue these parameters only after issuance of request token (they use no third party provider). How can I proceed?I am currently stuck after trying some libraries that threw errors. Many thanks for any useful information. The official launchpad library is in python.
My initial code is below:
public class Quicky {
public static void main(String[] args) throws Exception {
CloseableHttpClient httpclient = HttpClients.createDefault();
try {
HttpGet httpGet = new HttpGet("https://launchpad.net/+request-token");
CloseableHttpResponse response1 = httpclient.execute(httpGet);
try {
System.out.println("Your current GET request status:" + response1.getStatusLine());
HttpEntity entity1 = response1.getEntity();
EntityUtils.consume(entity1);
} finally {
response1.close();
}
HttpRequest request;
HttpPost httpPost = new HttpPost("https://launchpad.net/+request-token");
PostMethod poster = new PostMethod();
List <NameValuePair> postParams = new ArrayList <NameValuePair>();
postParams.add(new BasicNameValuePair("oauth_customer_key", "XXXX"));
postParams.add(new BasicNameValuePair("oauth_signature_method", "PLAINTEXT"));
postParams.add(new BasicNameValuePair("oauth_signature", "&"));
httpPost.setEntity(new UrlEncodedFormEntity(postParams, "utf-8"));
// httpPost.setEntity(entity1);
httpclient.execute(httpPost);
HttpParameters requestParams = (HttpParameters) postParams;
CloseableHttpResponse response2 = httpclient.execute(httpPost);
try {
System.out.println("Your current POST request status:" + response2.getStatusLine());
HttpEntity entity2 = response2.getEntity();
// do something useful with the response body
// and ensure it is fully consumed
EntityUtils.consume(entity2);
} finally {
response2.close();
}
} finally {
httpclient.close();
}
}
}
I finally resolved the issue error messages after some research and code re-factoring. The correct code is below, maybe it could be useful to someone out there.
public class LaunchPadTokenRetriever {
public static void main(String[] args) throws ClientProtocolException, IOException{
CloseableHttpClient httpclient = HttpClients.createDefault();
HttpPost httpPost = new HttpPost("https://launchpad.net/+request-token");
httpPost.addHeader("Content-Type", "application/x-www-form-urlencoded");
List <NameValuePair> urlParams = new ArrayList <NameValuePair>();
urlParams.add(new BasicNameValuePair("oauth_signature", "&"));
urlParams.add(new BasicNameValuePair("oauth_consumer_key", "tester"));
urlParams.add(new BasicNameValuePair("oauth_signature_method", "PLAINTEXT"));
httpPost.setEntity(new UrlEncodedFormEntity(urlParams));
CloseableHttpResponse response = httpclient.execute(httpPost);
System.out.println(response);
try {
System.out.println(response.getStatusLine());
HttpEntity entity = response.getEntity();
ResponseHandler<String> responseHandler = new BasicResponseHandler();
String responseBody = httpclient.execute(httpPost, responseHandler);
System.out.println("Initial credentials ---> "+ responseBody);
System.out.println();
String getresponse = responseBody;
EntityUtils.consume(entity);
} finally {
response.close();
}
}
}
I am attempting to write a code to scrape websites and use httpclient. I am trying to import the proper classes to run my program, but it is saying the package does not exist. I have looked at their API to try to figure it out and still cannot. My code is:
import java.io.IOException;
import org.apache.commons.httpclient.*;
import org.apache.commons.httpclient.methods.*;
import java.util.Scanner
public class Scraper3 {
public static String scrapeWebsite() throws IOException {
HttpClient client = new DefaultHttpClient();
HttpGet get = new HttpGet("http://ichart.finance.yahoo.com/table.csv?s=MSFT");
HttpResponse response = client.execute(get);
HttpEntity entity = response.getEntity();
if (entity != null) {
Scanner scanner = new Scanner(entity.getContent());
while (scanner.hasNextLine()) {
System.out.println(scanner.nextLine());
}
}
}
}
try: org.apache.http.client.HttpClient
I know there used to be a way to get it with Apache Commons as documented here:
http://hc.apache.org/httpclient-legacy/apidocs/org/apache/commons/httpclient/HttpMethod.html
...and an example here:
http://www.kodejava.org/examples/416.html
...but I believe this is deprecated.
Is there any other way to make an http get request in Java and get the response body as a string and not a stream?
Here are two examples from my working project.
Using EntityUtils and HttpEntity
HttpResponse response = httpClient.execute(new HttpGet(URL));
HttpEntity entity = response.getEntity();
String responseString = EntityUtils.toString(entity, "UTF-8");
System.out.println(responseString);
Using BasicResponseHandler
HttpResponse response = httpClient.execute(new HttpGet(URL));
String responseString = new BasicResponseHandler().handleResponse(response);
System.out.println(responseString);
Every library I can think of returns a stream. You could use IOUtils.toString() from Apache Commons IO to read an InputStream into a String in one method call. E.g.:
URL url = new URL("http://www.example.com/");
URLConnection con = url.openConnection();
InputStream in = con.getInputStream();
String encoding = con.getContentEncoding();
encoding = encoding == null ? "UTF-8" : encoding;
String body = IOUtils.toString(in, encoding);
System.out.println(body);
Update: I changed the example above to use the content encoding from the response if available. Otherwise it'll default to UTF-8 as a best guess, instead of using the local system default.
Here's an example from another simple project I was working on using the httpclient library from Apache:
String response = new String();
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(1);
nameValuePairs.add(new BasicNameValuePair("j", request));
HttpEntity requestEntity = new UrlEncodedFormEntity(nameValuePairs);
HttpPost httpPost = new HttpPost(mURI);
httpPost.setEntity(requestEntity);
HttpResponse httpResponse = mHttpClient.execute(httpPost);
HttpEntity responseEntity = httpResponse.getEntity();
if(responseEntity!=null) {
response = EntityUtils.toString(responseEntity);
}
just use EntityUtils to grab the response body as a String. very simple.
This is relatively simple in the specific case, but quite tricky in the general case.
HttpClient httpclient = new DefaultHttpClient();
HttpGet httpget = new HttpGet("http://stackoverflow.com/");
HttpResponse response = httpclient.execute(httpget);
HttpEntity entity = response.getEntity();
System.out.println(EntityUtils.getContentMimeType(entity));
System.out.println(EntityUtils.getContentCharSet(entity));
The answer depends on the Content-Type HTTP response header.
This header contains information about the payload and might define the encoding of textual data. Even if you assume text types, you may need to inspect the content itself in order to determine the correct character encoding. E.g. see the HTML 4 spec for details on how to do that for that particular format.
Once the encoding is known, an InputStreamReader can be used to decode the data.
This answer depends on the server doing the right thing - if you want to handle cases where the response headers don't match the document, or the document declarations don't match the encoding used, that's another kettle of fish.
Below is a simple way of accessing the response as a String using Apache HTTP Client library.
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.BasicResponseHandler;
//...
HttpGet get;
HttpClient httpClient;
// initialize variables above
ResponseHandler<String> responseHandler = new BasicResponseHandler();
String responseBody = httpClient.execute(get, responseHandler);
The Answer by McDowell is correct one. However if you try other suggestion in few of the posts above.
HttpEntity responseEntity = httpResponse.getEntity();
if(responseEntity!=null) {
response = EntityUtils.toString(responseEntity);
S.O.P (response);
}
Then it will give you illegalStateException stating that content is already consumed.
How about just this?
org.apache.commons.io.IOUtils.toString(new URL("http://www.someurl.com/"));
We can use the below code also to get the HTML Response in java
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.HttpResponse;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import org.apache.log4j.Logger;
public static void main(String[] args) throws Exception {
HttpClient client = new DefaultHttpClient();
// args[0] :- http://hostname:8080/abc/xyz/CheckResponse
HttpGet request1 = new HttpGet(args[0]);
HttpResponse response1 = client.execute(request1);
int code = response1.getStatusLine().getStatusCode();
try (BufferedReader br = new BufferedReader(new InputStreamReader((response1.getEntity().getContent())));) {
// Read in all of the post results into a String.
String output = "";
Boolean keepGoing = true;
while (keepGoing) {
String currentLine = br.readLine();
if (currentLine == null) {
keepGoing = false;
} else {
output += currentLine;
}
}
System.out.println("Response-->" + output);
} catch (Exception e) {
System.out.println("Exception" + e);
}
}
Here's a lightweight way to do so:
String responseString = "";
for (int i = 0; i < response.getEntity().getContentLength(); i++) {
responseString +=
Character.toString((char)response.getEntity().getContent().read());
}
With of course responseString containing website's response and response being type of HttpResponse, returned by HttpClient.execute(request)
Following is the code snippet which shows better way to handle the response body as a String whether it's a valid response or error response for the HTTP POST request:
BufferedReader reader = null;
OutputStream os = null;
String payload = "";
try {
URL url1 = new URL("YOUR_URL");
HttpURLConnection postConnection = (HttpURLConnection) url1.openConnection();
postConnection.setRequestMethod("POST");
postConnection.setRequestProperty("Content-Type", "application/json");
postConnection.setDoOutput(true);
os = postConnection.getOutputStream();
os.write(eventContext.getMessage().getPayloadAsString().getBytes());
os.flush();
String line;
try{
reader = new BufferedReader(new InputStreamReader(postConnection.getInputStream()));
}
catch(IOException e){
if(reader == null)
reader = new BufferedReader(new InputStreamReader(postConnection.getErrorStream()));
}
while ((line = reader.readLine()) != null)
payload += line.toString();
}
catch (Exception ex) {
log.error("Post request Failed with message: " + ex.getMessage(), ex);
} finally {
try {
reader.close();
os.close();
} catch (IOException e) {
log.error(e.getMessage(), e);
return null;
}
}
Here is a vanilla Java answer:
import java.net.http.HttpClient;
import java.net.http.HttpResponse;
import java.net.http.HttpRequest;
import java.net.http.HttpRequest.BodyPublishers;
...
HttpClient client = HttpClient.newHttpClient();
HttpRequest request = HttpRequest.newBuilder()
.uri(targetUrl)
.header("Content-Type", "application/json")
.POST(BodyPublishers.ofString(requestBody))
.build();
HttpResponse response = client.send(request, HttpResponse.BodyHandlers.ofString());
String responseString = (String) response.body();
If you are using Jackson to deserialize the response body, one very simple solution is to use request.getResponseBodyAsStream() instead of request.getResponseBodyAsString()
Using Apache commons Fluent API, it can be done as mentioned below,
String response = Request.Post("http://www.example.com/")
.body(new StringEntity(strbody))
.addHeader("Accept","application/json")
.addHeader("Content-Type","application/json")
.execute().returnContent().asString();