We Keep Coding

Sending http request get syntax error 400 message from API on Android

I'm using Positionstack API to build my APP with a location function on Android. The API works well when I test it on java in the local environment. However, it keeps returning a syntax error message and an Error 400 code when I send the request on Android Studio through an activity.
The error message
I/System.out: 400
I/System.out: {"error":{"code":"bad_request","message":"Could not decode value from JSON format. Error was: \u0022Syntax error\u0022."}}
The class of sending requests. It works well in the local environment but fails on the emulator. It establishes a HttpUrlConnection and uses the GET method to retrieve the result from API. It returns a 400 status code on the Android Studio emulator.
import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
public class NetTest {
public static String sendRequest(String urlParam, String coordinate){
HttpURLConnection con = null;
BufferedReader buffer = null;
StringBuffer resultBuffer = null;
InputStream is;
try{
// prepare the params and send request
URL url = new URL(urlParam);
con = (HttpURLConnection) url.openConnection();
con.setConnectTimeout(5000);
con.setReadTimeout(5000);
con.setRequestMethod("POST");
con.setRequestProperty("Content-Type","application/json;charset=UTF-8");
con.setDoOutput(true);
con.set
// DataOutputStream wr = new DataOutputStream(con.getOutputStream());
// wr.writeBytes("access_key=xxx");
// wr.writeBytes("query=-33.7,127");
// wr.flush();
// wr.close();
System.out.println("message out");
// receive response
int responseCode = con.getResponseCode();
System.out.println(responseCode);
if (responseCode == 200) {
is = con.getInputStream();
}else {
is = con.getErrorStream();
}
resultBuffer = new StringBuffer();
String line;
buffer = new BufferedReader(new InputStreamReader(is,"UTF-8"));
while ((line = buffer.readLine()) != null){
resultBuffer.append(line);
}
return resultBuffer.toString();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return "";
}
public static void main(String[] args) {
String coordinate = "-33.7,127";
String url = "http://api.positionstack.com/v1/reverse";
System.out.println(sendRequest(url,coordinate));
}
}
The manifest
<application
android:usesCleartextTraffic="true">
...
</application>
<uses-permission android:name="android.permission.INTERNET" />
Thanks a lot!
The problem may be the encoding method of Android, but I don't know how to change it, or even see it.

Use this codeblock -> first create jsonObject and add your key values and then set in output Stream. you can use this link for reference -> https://www.baeldung.com/httpurlconnection-post
JSONObject jsonObject = new JSONObject();
jsonObject.put("access_key", "Enter your access key here");
jsonObject.put("query", "Enter your query string here");
DataOutputStream wr = new DataOutputStream(con.getOutputStream());
String jsonString = jsonObject.toString();
byte[] input = jsonString.getBytes(StandardCharsets.UTF_8);
wr.write(input, 0, input.length);
wr.flush();
wr.close();
Also check the logger for what is sent in request body, use okhttp as search string in logger

Thanks to #abhishekrajak. I think his answer can solve most questions in this case.
In my case, however, I received an Error 403 status code from the API that said that my subscription couldn't use this service. I think the problem is that using OutputStream of HttpUrlConnection will automatically transfer your request method from "GET" to "POST", and my subscription is only able to receive the "GET" service.
Anyway, no doubt that #abhishekrajak 's answer can successfully pass the parameter in the correct encoding.
I solved this problem by using an alternative API. The original API may not be a good choice in the first place.

GlassFish 3 - 400 Bad Request on GET/POST/PUT/DELETE

In a script I have, I've created a small and simple REST client. The script itself is a prototype, and therefore the code is not 'production worthy' - so ignore lazy catch expressions and alike.
There are two types of servers that contain the REST service that I fetch data from; either a WildFly 8.2.0 or a GlassFish 3.1.2.2. And the catch here is: My REST client works fine for fetching data from the Wildfly server, but the GlassFish server returns an HTTP 400 Bad Request, for any request.
I can access the REST service for both servers through a web browser, so I know that they are both working properly. I can even do a raw connection though a socket to both servers and they response with the correct data.
So, what could be the reason for GlassFish to not accept the requests?
Socket connection (for testing)
import java.net.Socket;
Socket s = new Socket("localhost", 8080);
String t = "GET /rest/appointment/appointments/search/?fromDate=2016-11-21&branchId=3 HTTP/1.1\nhost: localhost:8080\nAuthorization: Basic base64encodedUsername:PasswordHere\n\n"
OutputStream out = s.getOutputStream();
out.write(t.getBytes());
InputStream inn = s.getInputStream();
Scanner scan = new Scanner(inn);
String line;
while ((line = scan.nextLine()) != null) {
println line;
}
s.close();
REST client code:
import groovy.json.JsonSlurper;
import javax.xml.bind.DatatypeConverter;
/*
REST-client (a very simple one)
*/
public class RESTclient {
public static Object get(URL url, Map<String, String> headers) {
return http(url, "GET", null, headers);
}
public static Object post(URL url, String data, Map<String, String> headers) {
return http(url, "POST", data, headers);
}
public static Object put(URL url, String data, Map<String, String> headers) {
return http(url, "PUT", data, headers);
}
public static Object delete(URL url, String data, Map<String, String> headers) {
return http(url, "DELETE", data, headers);
}
private static Object http(URL url, String method, String data, Map<String, String> headers) {
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
Authenticator.setDefault(new Authenticator() {
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication("username", "password".toCharArray());
}
});
connection.setRequestMethod(method);
for (String header : headers.keySet()) {
connection.setRequestProperty(header, headers.get(header));
}
if (data != null) {
connection.setRequestProperty("Content-Type", "application/json");
connection.setDoOutput(true);
OutputStream outputStream =connection.getOutputStream();
outputStream.write(data.getBytes());
}
int responseCode = connection.getResponseCode();
switch (responseCode) {
case HttpURLConnection.HTTP_NO_CONTENT:
// This happens when the server doesn't give back content, but all was ok.
return (new HashMap());
case HttpURLConnection.HTTP_OK:
InputStream inputStream = connection.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
String response = reader.readLine();
JsonSlurper parser = new JsonSlurper();
Object jsonResponse = parser.parseText(response); // This can be either a List or a Map
// Close the connection
try { connection.close(); } catch (Exception e) { /* Already closed */ }
return jsonResponse;
default:
println "response code: " + responseCode;
println connection.getResponseMessage();
println connection.getHeaderFields();
// Close the connection
try { connection.close(); } catch (Exception e) { /* Already closed */ }
return null;
}
}
}
Usage:
URL appointmentSearchURL = new URL("http://localhost:8080/rest/appointment/appointments/search/?fromDate=2016-11-21&branchId=3");
Object response = RESTclient.get(appointmentSearchURL, new HashMap<String, String>());
println response;
All that is printed out:
response code: 400
Bad Request
[null:[HTTP/1.1 400 Bad Request], Server:[GlassFish Server Open Source Edition 3.1.2.2], Connection:[close], Set-Cookie:[rememberMe=deleteMe; Path=/; Max-Age=0; Expires=Tue, 22-Nov-2016 08:43:29 GMT, SSOcookie=2a86cf4b-a772-435a-b92e-f12845dc20a2; Path=/; HttpOnly], Content-Length:[1090], Date:[Wed, 23 Nov 2016 08:43:28 GMT], Content-Type:[text/html], X-Powered-By:[Servlet/3.0 JSP/2.2 (GlassFish Server Open Source Edition 3.1.2.2 Java/Oracle Corporation/1.7)]]
null

I found my answer! So, I will leave this here if any other stumble across the same issue in the future:
There was a missing Accept header, I guess the server-side only accept json content. I have not researched further on why the WildFly server does not response with a 400 bad request, but I suppose WildFly tries to guess/deduce the incoming data.
So the whole issue was resolved by adding the following:
connection.setRequestProperty("Accept", "application/json");

What is the difference between the different HttpClients available?

I am trying to write a simple HttpClient program.
This is the first time I am working with HttpClient, I am quite confused which jars to include.
I have included the apache-httpcomponents-httpclient.jar and org.apache.commons.httpclient.jar with these ones when I create a HttpClient object I see different methods in the client object
package com.comverse.rht;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.ArrayList;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpException;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.httpclient.NameValuePair;
import org.apache.commons.httpclient.URI;
import org.apache.commons.httpclient.URIException;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpGet;
public class HttpClientTest {
public static void main(String[] args) throws URIException {
URI url = new URI("http://www.google.com/search?q=httpClient");
HttpClient client = new HttpClient();
GetMethod get = new GetMethod();
PostMethod post = new PostMethod();
String responseString;
StringBuilder sb = new StringBuilder();
String line;
// add request header
get.setURI(url);
get.addRequestHeader("User-Agent", "shaiksha429");
try {
int respCode = client.executeMethod(get);
System.out.println("Response Code:" +respCode);
System.out.println(
"PCRF HTTP Status" + HttpStatus.getStatusText(respCode)
);
responseString = get.getResponseBodyAsString();
BufferedReader rd = null;
rd = new BufferedReader(
new InputStreamReader(get.getResponseBodyAsStream())
);
while ((line = rd.readLine()) != null) {
sb.append(line + '\n');
}
System.out.println(sb);
} catch (HttpException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
But when I google I see a different example as below. What is the difference between the two? Why one HttpClient has "execute" and the other has "executeMethod". Which one I need to use?
String url = "http://www.google.com/search?q=httpClient";
HttpClient client = HttpClientBuilder.create().build();
HttpGet request = new HttpGet(url);
// add request header
request.addHeader("User-Agent", USER_AGENT);
HttpResponse response = client.execute(request);
System.out.println("Response Code : " + response.getStatusLine().getStatusCode());
BufferedReader rd = new BufferedReader(
new InputStreamReader(response.getEntity().getContent())
);
StringBuffer result = new StringBuffer();
String line = "";
while ((line = rd.readLine()) != null) {
result.append(line);
}

There were a lot of changes from HttpClient version 3 to version 4. The second example is definitely from HttpClient 4, so the first example is probably from the previous version.
Here is code that will do your google search, and read the result into a string
PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager();
connectionManager.setMaxTotal(60);
connectionManager.setDefaultMaxPerRoute(6);
try (CloseableHttpClient client = HttpClients.custom().setConnectionManager(connectionManager).build()) {
HttpGet request = new HttpGet("http://www.google.com/search?q=httpClient");
request.setHeader("User-Agent", "HttpClient");
try (CloseableHttpResponse response = client.execute(request)) {
MediaType mediaType = MediaType.parseMediaType(response.getFirstHeader("Content-Type").getValue());
Charset charSet = mediaType.getCharSet();
HttpEntity entity = response.getEntity();
InputStream is = entity.getContent();
String body = CharStreams.toString(new InputStreamReader(is, charSet));
System.out.println("body = " + body);
EntityUtils.consume(entity);
}
}
First, you probably want to create a connection pool, so you can reuse the connection if you send multiple requests to the same server. The pool is typically created during application initialisation , for instance as a Spring singleton bean.
Here I used the ClosableHttpClient because it works with resource-try syntax, and you need to close both the httpClient, the response and the inputStream when you are done reading. The HttpClient is actually a lightweight object, the state like socket connection and cookies are stored elsewhere.
I use Spring's MediaType.parseMediaType() to get the char encoding, and Guavas CharStreams to convert the inputStream to a String. In my case google encoded the content using latin-1, because "search" is "søgning" in Danish.
The last step is to use EntityUtils.consume(entity), to ensure that all entity data has been read. If you use connection pooling this is important, because unread data will cause the connection to be thrown away, instead of being reused by the connection manager (this is extremely important if you are using https).

You're using a library whose interface has changed across its major versions. You can't casually copy jars and copy/paste examples without understanding which release you're using and which release an example or snippet was from.
Look at the examples that accompany the latest release and take anything old with a grain of salt.
Apache seems to move especially fast.

java restful web service explanation

I am completely new to Java web services. I have written following code:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
public class Main {
public static void main(String[] args) {
try {
URL url = new URL("www.somehost.com/somedata");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/json");
if (conn.getResponseCode() != 200) {
throw new RuntimeException("Failed: HTTP error code: " + conn.getResponseCode());
}
BufferedReader br = new BufferedReader(new InputStreamReader(
conn.getInputStream()
));
String output;
while ((output = br.readLine()) != null) {
System.out.println(output);
}
conn.disconnect();
} catch (IOException e) {
e.printStackTrace();
}
}
}
My task is to create a web service that returns data in JSON format from "some URL". I want to create a RESTful web service but I do not realize how to modify the code to serve it as a web service. Can anybody explain/show what else I should do?

Here is a Jersey resource example:
#Path("rest/heartbeat")
public class HeartbeatResource {
#GET
#Produces(MediaType.APPLICATION_XML)
public Response heartbeatGet() {
return Response.status(Status.OK).type(MediaType.APPLICATION_XML)
.entity(new Messages("I am alive!")).build();
}
}
Do some research and choose a solid REST framework, if it happens to be Jersey then you can find needed learning documents at: https://jersey.java.net/

I prefer Apace Wink to develop RESTful services.. It gives you capability to tune the API as per you need .
http://wink.apache.org/

How can I get an HTTP response body as a string?

I know there used to be a way to get it with Apache Commons as documented here:
http://hc.apache.org/httpclient-legacy/apidocs/org/apache/commons/httpclient/HttpMethod.html
...and an example here:
http://www.kodejava.org/examples/416.html
...but I believe this is deprecated.
Is there any other way to make an http get request in Java and get the response body as a string and not a stream?

Here are two examples from my working project.
Using EntityUtils and HttpEntity
HttpResponse response = httpClient.execute(new HttpGet(URL));
HttpEntity entity = response.getEntity();
String responseString = EntityUtils.toString(entity, "UTF-8");
System.out.println(responseString);
Using BasicResponseHandler
HttpResponse response = httpClient.execute(new HttpGet(URL));
String responseString = new BasicResponseHandler().handleResponse(response);
System.out.println(responseString);

Every library I can think of returns a stream. You could use IOUtils.toString() from Apache Commons IO to read an InputStream into a String in one method call. E.g.:
URL url = new URL("http://www.example.com/");
URLConnection con = url.openConnection();
InputStream in = con.getInputStream();
String encoding = con.getContentEncoding();
encoding = encoding == null ? "UTF-8" : encoding;
String body = IOUtils.toString(in, encoding);
System.out.println(body);
Update: I changed the example above to use the content encoding from the response if available. Otherwise it'll default to UTF-8 as a best guess, instead of using the local system default.

Here's an example from another simple project I was working on using the httpclient library from Apache:
String response = new String();
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(1);
nameValuePairs.add(new BasicNameValuePair("j", request));
HttpEntity requestEntity = new UrlEncodedFormEntity(nameValuePairs);
HttpPost httpPost = new HttpPost(mURI);
httpPost.setEntity(requestEntity);
HttpResponse httpResponse = mHttpClient.execute(httpPost);
HttpEntity responseEntity = httpResponse.getEntity();
if(responseEntity!=null) {
response = EntityUtils.toString(responseEntity);
}
just use EntityUtils to grab the response body as a String. very simple.

This is relatively simple in the specific case, but quite tricky in the general case.
HttpClient httpclient = new DefaultHttpClient();
HttpGet httpget = new HttpGet("http://stackoverflow.com/");
HttpResponse response = httpclient.execute(httpget);
HttpEntity entity = response.getEntity();
System.out.println(EntityUtils.getContentMimeType(entity));
System.out.println(EntityUtils.getContentCharSet(entity));
The answer depends on the Content-Type HTTP response header.
This header contains information about the payload and might define the encoding of textual data. Even if you assume text types, you may need to inspect the content itself in order to determine the correct character encoding. E.g. see the HTML 4 spec for details on how to do that for that particular format.
Once the encoding is known, an InputStreamReader can be used to decode the data.
This answer depends on the server doing the right thing - if you want to handle cases where the response headers don't match the document, or the document declarations don't match the encoding used, that's another kettle of fish.

Below is a simple way of accessing the response as a String using Apache HTTP Client library.
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.BasicResponseHandler;
//...
HttpGet get;
HttpClient httpClient;
// initialize variables above
ResponseHandler<String> responseHandler = new BasicResponseHandler();
String responseBody = httpClient.execute(get, responseHandler);

The Answer by McDowell is correct one. However if you try other suggestion in few of the posts above.
HttpEntity responseEntity = httpResponse.getEntity();
if(responseEntity!=null) {
response = EntityUtils.toString(responseEntity);
S.O.P (response);
}
Then it will give you illegalStateException stating that content is already consumed.

How about just this?
org.apache.commons.io.IOUtils.toString(new URL("http://www.someurl.com/"));

We can use the below code also to get the HTML Response in java
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.HttpResponse;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import org.apache.log4j.Logger;
public static void main(String[] args) throws Exception {
HttpClient client = new DefaultHttpClient();
// args[0] :- http://hostname:8080/abc/xyz/CheckResponse
HttpGet request1 = new HttpGet(args[0]);
HttpResponse response1 = client.execute(request1);
int code = response1.getStatusLine().getStatusCode();
try (BufferedReader br = new BufferedReader(new InputStreamReader((response1.getEntity().getContent())));) {
// Read in all of the post results into a String.
String output = "";
Boolean keepGoing = true;
while (keepGoing) {
String currentLine = br.readLine();
if (currentLine == null) {
keepGoing = false;
} else {
output += currentLine;
}
}
System.out.println("Response-->" + output);
} catch (Exception e) {
System.out.println("Exception" + e);
}
}

Here's a lightweight way to do so:
String responseString = "";
for (int i = 0; i < response.getEntity().getContentLength(); i++) {
responseString +=
Character.toString((char)response.getEntity().getContent().read());
}
With of course responseString containing website's response and response being type of HttpResponse, returned by HttpClient.execute(request)

Following is the code snippet which shows better way to handle the response body as a String whether it's a valid response or error response for the HTTP POST request:
BufferedReader reader = null;
OutputStream os = null;
String payload = "";
try {
URL url1 = new URL("YOUR_URL");
HttpURLConnection postConnection = (HttpURLConnection) url1.openConnection();
postConnection.setRequestMethod("POST");
postConnection.setRequestProperty("Content-Type", "application/json");
postConnection.setDoOutput(true);
os = postConnection.getOutputStream();
os.write(eventContext.getMessage().getPayloadAsString().getBytes());
os.flush();
String line;
try{
reader = new BufferedReader(new InputStreamReader(postConnection.getInputStream()));
}
catch(IOException e){
if(reader == null)
reader = new BufferedReader(new InputStreamReader(postConnection.getErrorStream()));
}
while ((line = reader.readLine()) != null)
payload += line.toString();
}
catch (Exception ex) {
log.error("Post request Failed with message: " + ex.getMessage(), ex);
} finally {
try {
reader.close();
os.close();
} catch (IOException e) {
log.error(e.getMessage(), e);
return null;
}
}

Here is a vanilla Java answer:
import java.net.http.HttpClient;
import java.net.http.HttpResponse;
import java.net.http.HttpRequest;
import java.net.http.HttpRequest.BodyPublishers;
...
HttpClient client = HttpClient.newHttpClient();
HttpRequest request = HttpRequest.newBuilder()
.uri(targetUrl)
.header("Content-Type", "application/json")
.POST(BodyPublishers.ofString(requestBody))
.build();
HttpResponse response = client.send(request, HttpResponse.BodyHandlers.ofString());
String responseString = (String) response.body();

If you are using Jackson to deserialize the response body, one very simple solution is to use request.getResponseBodyAsStream() instead of request.getResponseBodyAsString()

Using Apache commons Fluent API, it can be done as mentioned below,
String response = Request.Post("http://www.example.com/")
.body(new StringEntity(strbody))
.addHeader("Accept","application/json")
.addHeader("Content-Type","application/json")
.execute().returnContent().asString();