We Keep Coding

OkHttp: A simple GET request: response.body().string() returns unreadable escaped unicode symbols inside json can't convert to gson

When sending a request in Postman, I get this output:
{
"valid": false,
"reason": "taken",
"msg": "Username has already been taken",
"desc": "That username has been taken. Please choose another."
}
However when doing it using okhttp, I get encoding problems and can't convert the resulting json string to a Java object using gson.
I have this code:
public static void main(String[] args) throws Exception {
TwitterChecker checker = new TwitterChecker();
TwitterJson twitterJson = checker.checkUsername("dogster");
System.out.println(twitterJson.getValid()); //NPE
System.out.println(twitterJson.getReason());
System.out.println("Done");
}
public TwitterJson checkUsername(String username) throws Exception {
HttpUrl.Builder urlBuilder = HttpUrl.parse("https://twitter.com/users/username_available").newBuilder();
urlBuilder.addQueryParameter("username", username);
String url = urlBuilder.build().toString();
Request request = new Request.Builder()
.url(url)
.addHeader("Content-Type", "application/json; charset=utf-8")
.build();
OkHttpClient client = new OkHttpClient();
Call call = client.newCall(request);
Response response = call.execute();
System.out.println(response.body().string());
Gson gson = new Gson();
return gson.fromJson(
response.body().string(), new TypeToken<TwitterJson>() {
}.getType());
}
Which prints this:
{"valid":false,"reason":"taken","msg":"\u0414\u0430\u043d\u043d\u043e\u0435 \u0438\u043c\u044f \u0443\u0436\u0435 \u0437\u0430\u043d\u044f\u0442\u043e","desc":"\u0414\u0430\u043d\u043d\u043e\u0435 \u0438\u043c\u044f \u0443\u0436\u0435 \u0437\u0430\u043d\u044f\u0442\u043e. \u041f\u043e\u0436\u0430\u043b\u0443\u0439\u0441\u0442\u0430, \u0432\u044b\u0431\u0435\u0440\u0438\u0442\u0435 \u0434\u0440\u0443\u0433\u043e\u0435."}
and then throws a NullPointerException when trying to access a twitterJson. Debugger shows that object as being null.
TwitterJson:
#Generated("net.hexar.json2pojo")
#SuppressWarnings("unused")
public class TwitterJson {
#Expose
private String desc;
#Expose
private String msg;
#Expose
private String reason;
#Expose
private Boolean valid;
public String getDesc() {
return desc;
}
public String getMsg() {
return msg;
}
public String getReason() {
return reason;
}
public Boolean getValid() {
return valid;
}
...
How can I fix the encoding issues with okhttp?

It is because the response object can be consumed only once. OKHTTP says that in their documentation. After the execute is invoked, you are calling the response object twice. Store the result of response.body().string() to a variable and then do the convert into GSON.
If I were to use a hello world example...
private void testOkHttpClient() {
OkHttpClient httpClient = new OkHttpClient();
try {
Request request = new Request.Builder()
.url("https://www.google.com")
.build();
Call call = httpClient.newCall(request);
Response response = call.execute();
System.out.println("First time " + response.body().string()); // I get the response
System.out.println("Second time " + response.body().string()); // This will be empty
} catch (IOException e) {
e.printStackTrace();
}
}
The reason it is empty the second time is because the response object can be consumed only once. So you either
Return the response as it is. Do not do a sysOut
System.out.println(response.body().string()); // Instead of doing a sysOut return the value.
Or
Store the value of the response to a JSON then convert it to GSON and then return the value.
EDIT: Concerning Unicode characters. It turned out since my location is not an English-speaking country, the json i was accepting was not in English as well. I added this header:
.addHeader("Accept-Language", Locale.US.getLanguage())
to the request to fix that.

How to ask gson to avoid escaping json in a json response?

I have a response object like this:
public class TestResponse {
private final String response;
private final ErrorCodeEnum error;
private final StatusCodeEnum status;
// .. constructors and getters here
}
I am serializing above class using Gson library as shown below:
Gson gson = new GsonBuilder().setPrettyPrinting().serializeNulls().create();
System.out.println(gson.toJson(testResponseOutput));
And the response I am getting back is shown below:
{
"response": "{\"hello\":0,\"world\":\"0\"}",
"error": "OK",
"status": "SUCCESS"
}
As you can see, my json string in "response" field is getting escaped. Is there any way I can ask gson not to do that and instead return a full response like this:
{
"response": {"hello":0,"world":"0"},
"error": "OK",
"status": "SUCCESS"
}
And also - Is there any problem if I do it above way?
NOTE: My "response" string will always be JSON string or it will be null so only these two values will be there in my "response" string. In "response" field, I can have any json string since this library is calling a rest service which can return back any json string so I am storing that in a string "response" field.

If your response field can be arbitrary JSON, then you need to:
Define it as an arbitrary JSON field (leveraging the JSON type system already built into GSON by defining it as the root of the JSON hierarchy - JsonElement)
public class TestResponse {
private final JsonElement response;
}
Convert the String field to an appropriate JSON object representation. For this, you can use GSON's JsonParser class:
final JsonParser parser = new JsonParser();
String responseJson = "{\"hello\":0,\"world\":\"0\"}";
JsonElement json = parser.parse(responseJson); // Omits error checking, what if responseJson is invalid JSON?
System.out.println(gson.toJson(new TestResponse(json)));
This should print:
{
"response": {
"hello": 0,
"world": "0"
}
}
It should also work for any valid JSON:
String responseJson = "{\"arbitrary\":\"fields\",\"can-be\":{\"in\":[\"here\",\"!\"]}}";
JsonElement json = parser.parse(responseJson);
System.out.println(gson.toJson(new TestResponse(json)));
Output:
{
"response": {
"arbitrary": "fields",
"can-be": {
"in": [
"here",
"!"
]
}
}
}

I know this is old but just adding an potential answer in case it is needed.
Sounds like you just want to return the response without escaping. Escaping is a good thing, it will help to prevent security issues and prevent your JS application from crashing with errors.
However, if you still want to ignore escaping, try:
Gson gson = new GsonBuilder().setPrettyPrinting().disableHtmlEscaping().serializeNulls().create();

add simple TypeAdapter and use jsonValue(value)
gson 2.8.0
version 1:
#Test
public void correctlyShow() {
TestResponse2 src = new TestResponse2("{\"arbitrary\":\"fields\",\"can-be\":{\"in\":[\"here\",\"!\"]}}");
Gson create = new GsonBuilder().registerTypeAdapter(String.class, ADAPTER).create();
Stopwatch createStarted = Stopwatch.createStarted();
String json2 = create.toJson(src);
System.out.println(json2 + " correctlyShow4 " + createStarted.stop());
}
public class TestResponse2 {
private final String response;
public TestResponse2(String response) {
this.response = response;
}
public String getResponse() {
return response;
}
}
private static final TypeAdapter<String> ADAPTER = new TypeAdapter<String>() {
#Override
public String read(JsonReader in) throws IOException {
throw new UnsupportedOperationException("Unsupported Operation !!!");
}
#Override
public void write(JsonWriter out, String value) throws IOException {
out.jsonValue(value);
}
};
...
vesrion 2
#Test
public void correctlyShow() {
TestResponse2 src = new TestResponse2("{\"arbitrary\":\"fields\",\"can-be\":{\"in\":[\"here\",\"!\"]}}");
String json2 = new Gson().toJson(src);
System.out.println(json2 + " correctlyShow4 ");
}
public class TestResponse2 {
#JsonAdapter(value = AdapterStringJson.class)
private final String response;
public TestResponse2(String response) {
this.response = response;
}
public String getResponse() {
return response;
}
}
private class AdapterStringJson extends TypeAdapter<String> {
#Override
public String read(JsonReader in) throws IOException {
throw new UnsupportedOperationException("Unsupported Operation !!!");
}
#Override
public void write(JsonWriter out, String value) throws IOException {
out.jsonValue(value);
}
}

You should have a nested object.
public class Response {
private final Integer hello;
private final String world;
}
public class TestResponse {
private final Response response;
private final ErrorCodeEnum error;
private final StatusCodeEnum status;
// .. constructors and getters here
}

Instead of a String, depending on your needs, you could use a Map (or similar) or a nested Object. There should not be a problem representing it this way but in your example, if it were a String, there would be a problem if you didn't escape characters such as the double-quote.

Could not read JSON: Can not deserialize instance of modelName out of START_OBJECT token

I trying to read this json value and convert it to java object
{"message":"1"}
which is returned from this string.
http://mywebservice:8080/SpringService/service/updatepool/selectdynamicurl/~!~''WEB1500001''~!~
but i always get this error
Could not read JSON: Can not deserialize instance of
org.springframework.samples.petclinic.model.CheckOpjID[] out of
START_OBJECT token
Here is my CHeckOPJID.java class
public class CheckOpjID {
private String message;
public String getMessage() {
return message;
}
public void setMessage(String message) {
this.message = message;
}
}
and here is my controller class
String url = "http://mywebservice:8080/SpringService/service/updatepool/selectdynamicurl/~!~''WEB1500001''~!~";
RestTemplate restTemplate = new RestTemplate();
CheckOpjID[] messageArray = restTemplate.getForObject(url, CheckOpjID[].class);
which is i have same code, and it worked! then i trying to create new class in another controller, then it starting give me Could not read JSON error, until this line (below)
CheckOpjID[] messageArray = restTemplate.getForObject(url, CheckOpjID[].class);
what i missed here??

This is what i missed, i have to delete the array Object, as Nikolay Rusev suggested
CheckOpjID messageArray = restTemplate.getForObject(url,
CheckOpjID.class);

Changed & Improved passing data from WebServlet to a WebService

I'm pretty new to writing Servlet and REST Services, but now i'm at a problem that I'm not sure if i'm doing it correctly. My Service look like this:
#POST
#Produces ("application/json")
#Path ("/register")
public String register(
#FormParam("name") String name,
#FormParam("username") String username,
#FormParam("password") String password,
#Context HttpServletResponse servletResponse) throws IOException {
if( this.user_taken(username) ) return "USERNAME_TAKEN";
User user = new User(name,username,password);
.....
return mapper.writeValueAsString(user);
}
So as you can see the Service takes care of doing the back end (database and creating user) the Servlet on the other hand is in charge of taking request from the form, properly validating and passing it to the Service. Servlet Code:
... validate user input form ...
ClientConfig config = new DefaultClientConfig();
Client client = Client.create(config);
WebResource service = client.resource("http://localhost/Jaba");
String map = mapper.writeValueAsString(request.getParameterMap());
MultivaluedMap<String, String> obj = mapper.readValue(map, MultivaluedMap.class);
String result =
service.path("api").path("register")
.accept("application/json")
.post(String.class, obj);
As you can see the Client (Servlet) has to do a lot of nasty work, to pass data to the Service. How can this be changed/improved/optimized or better yet refactored ? I'm trying to follow best practices and how it would be in a real life example.

Here is what I might do:
Instead of doing
String result =
service.path("api").path("register")
.accept("application/json")
.post(String.class, obj);
I would do something more like creating a DTO object, filling it out and then passing it to your service. This is then were you would apply an aspect along with JSR validation and annotations (you can do this on what you have but it won't be nearly so nice) on the client call.
example:
#Aspect
public class DtoValidator {
private Validator validator;
public DtoValidator() {
}
public DtoValidator(Validator validator) {
this.validator = validator;
}
public void doValidation(JoinPoint jp){
for( Object arg : jp.getArgs() ){
if (arg != null) {
Set<ConstraintViolation<Object>> violations = validator.validate(arg);
if( violations.size() > 0 ){
throw buildError(violations);
}
}
}
}
private static BadRequestException buildError( Set<ConstraintViolation<Object>> violations ){
Map<String, String> errorMap = new HashMap<String, String>();
for( ConstraintViolation error : violations ){
errorMap.put(error.getPropertyPath().toString(), error.getMessage());
}
return new BadRequestException(errorMap);
}
}
You can annotatively declare you aspect or you can do it in config (makes it reusable). As such:
<aop:config proxy-target-class="true">
<aop:aspect id="dtoValidator" ref="dtoValidator" order="10">
<aop:before method="doValidation" pointcut="execution(public * com.mycompany.client.*.*(..))"/>
</aop:aspect>
</aop:config>
Now you can have a DTO like this:
#XmlAccessorType(XmlAccessType.FIELD)
#XmlRootElement
public class LoginRequest extends AbstractDto{
#NotNull
private String userName;
#NotNull
private String password;
private LoginRequest() {
}
public LoginRequest(String userName, String password) {
this.userName = userName;
this.password = password;
}
public String getUserName() {
return userName;
}
public String getPassword() {
return password;
}
}
When it fails those #NotNull checks you will get something like this:
{
"message":"{username=must not be null",
"httpStatusCode":400,
"httpMessage":"Bad Request",
"details":{
"username":"must not be null"
}
}
Then use a RestOperation client as such
org.springframework.web.client.RestOperations restClient ...
restClient.postForObject(URL,new Dto(...),args);
Place the aspect around this restClient and you're golden (and, actually for good measure, on your service calls too).

Getting rid of default values in spring rest

I'm using Spring rest with below code base:
When I invoke /info by passing string value in request body, I'm expecting the below response if this value is not present in my backend database.
{"output":-10}
but instead it returns me below response:
{"id": 0, "output":-10}
Can any one tell me how to get rid of this id default value? If there is a boolean variable in JSON mapper, then that would also get returned as
{"id": 0, "booleanVar": false, "output":-10}
Can any one tell me how to get rid of this default value?
Controller.java
#RequestMapping(value = "heartbeat", method = RequestMethod.GET, consumes="application/json")
public ResponseEntity<String> getHeartBeat() throws Exception {
String curr_time = myService.getCurrentTime();
return MyServiceUtil.getResponse(curr_time, HttpStatus.OK);
}
#RequestMapping(value = "info", method = RequestMethod.POST, consumes="application/json")
public ResponseEntity<String> getData(#RequestBody String body) throws Exception {
....
myInfo = myService.getMyInfo(myServiceJson);
return MyServiceUtil.getResponse(myInfo, responseHeader, HttpStatus.OK);
}
MyService.java
#Override
public String getCurrentTime() throws Exception {
String currentDateTime = null;
MyServiceJson json = new MyServiceJson();
ObjectMapper mapper = new ObjectMapper().configure(SerializationConfig.Feature.DEFAULT_VIEW_INCLUSION, false);
try {
Date currDate = new Date(System.currentTimeMillis());
currentDateTime = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss").format(currDate);
json.setCurrentDateTime(currentDateTime);
ObjectWriter writer = mapper.writerWithView(Views.HeartBeatAPI.class);
return writer.writeValueAsString(json);
} catch (Exception e) {
throw new Exception("Excpetion in getCurrentTime: ", HttpStatus.BAD_REQUEST);
}
}
#Override
public String getMyInfo(MyServiceJson myServiceJson) throws Exception {
MyServiceJson json = new MyServiceJson();
json.setFirstName("hhh");
json.setLastName("abc");
ObjectMapper mapper = new ObjectMapper();
return mapper.writeValueAsString(json);
}
Views.java
public class Views {
public static class HeartBeatAPI { }
}
MyServiceJson.java
#JsonSerialize(include = Inclusion.NON_NULL)
public class MyServiceJson {
private int id;
private String firstName;
private String lastName;
#JsonView(Views.HeartBeatAPI.class)
private String currentDateTime;
// Getter/Setter for the above variables here
.....
}

Use Integer class instead of int primitive type. Primitive types always hold default values, where class type defaults to null.