We Keep Coding

Json file error out of START_ARRAY token caused by these [ ] brackets

I need method to get json from website and return offers witch minSalary>1500&&marker_icon.equals("kotlin").I have tried this so far:
public void getRightOffers() {
ResponseEntity<JobObject> exchange = restTemplate.exchange(url, HttpMethod.GET, null, new ParameterizedTypeReference<>() {
});
ResponseEntity<Salary> exchangeForSalary = restTemplate.exchange(url, HttpMethod.GET, null, new ParameterizedTypeReference<>() {
});
Salary body1 = exchangeForSalary.getBody();
int minSalary = body1.from;
JobObject body = exchange.getBody();
String marker_icon = body.marker_icon.toLowerCase();
if (minSalary > 10000 &&marker_icon.equals("java")) {
//it should return these offers
}
}
But I got error: RestClientException: Error while extracting response for type [class packageOne.JobObject] and content type [application/json;charset=utf-8]; nested exception is org.springframework.http.converter.HttpMessageNotReadableException: JSON parse error: Cannot deserialize instance of packageOne.JobObject out of START_ARRAY token;
My json starts with [{"title... and ends with ...false}]. These [ ] couses the problem but I can't remove them from json.
Also I have JobObject,Salary and Skill classes

[...] means the JSON response is an array.
You have to define your Java ResponseEntity to a compatible type like List:
ResponseEntity<List<JobObject>> exchanges = ...

Spring Boot Custom HTTP EXCEPTION

I have this method to handle all request header missing exceptions, but in one controller is expected to receive a json as body. If it is a invalid json or is null, it drops a exception with custom messages:
#ExceptionHandler(value = {ServletRequestBindingException.class, HttpMessageNotReadableException.class})
public final ResponseEntity<ErrorResponse> handleHeaderException(Exception ex) {
List<String> details = new ArrayList<>();
details.add(ex.getMessage());
return new ResponseEntity<>(new ErrorResponse("Bad Request", details), HttpStatus.BAD_REQUEST);
}
{
"message": "Bad Request",
"details": [
"Required request body is missing: public org.springframework.http.ResponseEntity
packages.fazerLogin(packages.BodyLogin) throws java.io.IOException"
] }
{
"message": "Bad Request",
"details": [
"JSON parse error: Unexpected character ('\"' (code 34)): was expecting comma to separate Object entries; nested exception is
com.fasterxml.jackson.core.JsonParseException: Unexpected character
('\"' (code 34)): was expecting comma to separate Object entries\n at
[Source: (PushbackInputStream); line: 3, column: 3]"
] }
But I don't want a long message such as above. Just "Required request body" or "JSON parse error" is just fine. I want to know what can I do.
My controller:
#PostMapping(value = "v1/token", consumes = "application/json;charset=UTF-8")
public ResponseEntity<TokenOutputDto> doLogin(#RequestBody #Valid BodyLogin body) throws IOException {
return authenticationModel.auth(body.getEmail(), body.getPassword());
}
Also, should I create a #ExceptionHandler method to each one of the possibles exceptions (HttpClientErrorException, HttpServerErrorException etc)? It will be a bad pratice, because the code will repeat almost identically...

You can handle those exception handlers in your method by following way:
#ExceptionHandler(value = {ServletRequestBindingException.class, HttpMessageNotReadableException.class})
public final ResponseEntity<ErrorResponse> handleHeaderException(Exception ex) {
List<String> details = new ArrayList<>();
if (ex instanceof IOException ) {
if (ex.getCause() instanceof JsonParseException) {
details.add("JSON parse error");
} else {
details.add("Required request body");
}
} else {
details.add(ex.getMessage());
}
return new ResponseEntity<>(new ErrorResponse("Bad Request", details), HttpStatus.BAD_REQUEST);
}

How to handle jackson deserialization error for all kinds of data mismatch in spring boot

I know there is some similar questions here about how to parse ENUM, how to parse customize JSON structure. But here my question is how to just give a better message when the user submit some JSON with is not as expected.
This is the code:
#PutMapping
public ResponseEntity updateLimitations(#PathVariable("userId") String userId,
#RequestBody LimitationParams params) {
Limitations limitations = user.getLimitations();
params.getDatasets().forEach(limitations::updateDatasetLimitation);
params.getResources().forEach(limitations::updateResourceLimitation);
userRepository.save(user);
return ResponseEntity.noContent().build();
}
And the request body I expected is like this:
{
"datasets": {"public": 10},
"resources": {"cpu": 2}
}
But when they submit something like this:
{
"datasets": {"public": "str"}, // <--- a string is given
"resources": {"cpu": 2}
}
The response will show something like this in logs:
400 JSON parse error: Cannot deserialize value of type `java.lang.Integer` from String "invalid": not a valid Integer value; nested exception is com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `java.lang.Integer` from String "invalid": not a valid Integer value
at [Source: (PushbackInputStream); line: 1, column: 23] (through reference chain: com.openbayes.api.users.LimitationParams["datasets"]->java.util.LinkedHashMap["public"])
But what I want is a more human readable message.
I tried to use ExceptionHandler for com.fasterxml.jackson.databind.exc.InvalidFormatException but it doesn't work.

You can write a controller advice to catch exceptions and return corresponding error response.
Here is an example of controller advice in spring boot :
#RestControllerAdvice
public class ControllerAdvice {
#ExceptionHandler(InvalidFormatException.class)
public ResponseEntity<ErrorResponse> invalidFormatException(final InvalidFormatException e) {
return error(e, HttpStatus.BAD_REQUEST);
}
private ResponseEntity <ErrorResponse> error(final Exception exception, final HttpStatus httpStatus) {
final String message = Optional.ofNullable(exception.getMessage()).orElse(exception.getClass().getSimpleName());
return new ResponseEntity(new ErrorResponse(message), httpStatus);
}
}
#AllArgsConstructor
#NoArgsConstructor
#Data
public class ErrorResponse {
private String errorMessage;
}

The real exception is org.springframework.http.converter.HttpMessageNotReadableException.
Intercept this and it will work.
public ResponseEntity<String> handle(HttpMessageNotReadableException e) {
return ResponseEntity.badRequest().body("your own message" + e.getMessage());
}

Following error handling method is worked for me.
#ExceptionHandler(HttpMessageNotReadableException.class)
public ResponseEntity handleAllOtherErrors(HttpMessageNotReadableException formatException) {
String error = formatException.getMessage().toString();
return new ResponseEntity(error, HttpStatus.BAD_REQUEST);

Efficient way to detect strings?

I am working on a project in which I am making a call to one of my servers using RestTemplate which is running a restful service and getting the response back from them.
The response that I will be getting from my server can be either of these error responses (that's all I have for error response) if something has gone wrong -
{"warning": "user_id not found", "user_id": some_user_id}
{"error": "user_id for wrong partition", "user_id": some_user_id, "partition": some_partition}
{"error": "missing client id", "client_id":2000}
or below successful response (it can be any random json string key can also be different) -
{"#data": {"oo":"1205000384","p":"2047935"}
If I am getting any error response as mentioned above, then I am deserializing it (my bad :( ) so that I can log them as an error with a specific error or warning I got front the server which can be for example - user_id not found or missing client id.
If it is a successful response then also I am deserializing it which I don't need for my use case as we don't have any POJO and I just need to return the response as it is which I have got from the server.
In my use case, I don't need to deserialize my response string if it is a successful response as we don't have any POJO for that and we are returning the response string as it is which we have got from the server. But just for logging specific error messages (if I am getting error response from the server) I am deserializing it which I am thinking is unnecessary. There might be better solution for my use case.
Below is my Java client which is calling Callable task using future.get -
public class TestingClient implements IClient {
private ExecutorService service = Executors.newFixedThreadPool(10);
private RestTemplate restTemplate = new RestTemplate();
#Override
public String executeSync(ClientKey keys) {
String response = null;
try {
ClientTask ClientTask = new ClientTask(keys, restTemplate);
Future<String> future = service.submit(ClientTask);
response = handle.get(keys.getTimeout(), TimeUnit.MILLISECONDS);
} catch (TimeoutException e) {
} catch (Exception e) {
}
return response;
}
}
And now below is my ClientTask class which implements Callable interface. In the call method, I am generating an URL and then hit the server using RestTemplate and get the response back -
class ClientTask implements Callable<String> {
private ClientKey cKeys;
private RestTemplate restTemplate;
public ClientTask(ClientKey cKeys, RestTemplate restTemplate) {
this.restTemplate = restTemplate;
this.cKeys = cKeys;
}
#Override
public String call() throws Exception {
// .. some code here
String url = "some_url";
String response = restTemplate.getForObject(url, String.class);
String test = checkJSONResponse(response);
return test;
}
private String checkJSONResponse(final String response) throws Exception {
// may be there are some better way of doing it for my scenario instead of using GSON
Gson gson = new Gson();
String str = null;
JsonObject jsonObject = gson.fromJson(response, JsonObject.class); // parse it, may be performance issues here/
if (jsonObject.has("error") || jsonObject.has("warning")) {
final String error = jsonObject.get("error") != null ? jsonObject.get("error").getAsString() : jsonObject
.get("warning").getAsString();
// log specific `error` here using log4j
str = response;
} else {
str = response;
}
return str;
}
}
As you can see in my above code we are deserializing the JSON string only to log specific error messages if we are getting any error response back. But for successful response we don't need any deserialization but still we are doing it.
Is there any better way of solving this problem? Because currently I am seeing some performance issues with the GSON deserialization.
The only way I can identify successful response along with error response is with error or warning in the response so I am thinking of using regular expressions which can identify error or warning as the key in the response string. If they contain error or warning in the response string then extract the specific error or warning message and log it. But not sure whether this will have any performance benefit or not.
Is there any other better way of solving this problem without using GSON deserialization.

It is a good practice to use HTTP status codes for your responses (e.g. BAD_REQUEST, NOT_FOUND). Return one of them from the server and then check on the client. It will allow to parse response only if some error code is returned:
String result = restTemplate.execute("url", HttpMethod.GET, null, new HttpMessageConverterExtractor<String> {
#Override
public MyEntity extractData(ClientHttpResponse response)
throws IOException {
String result = super.extractData(response);
if (response.getStatusCode() != HttpStatus.OK) {
// parse message and log only for some error code
JsonObject errorJson = parse(result);
log.warn("Got {} status error, with message [{}]", response.getStatusCode(), errorJson.get("warning"));
}
return result;
}
});

You do not need to deserialize to a POJO.
A simple JSON parser such as the one found on json.org will provide minimal JSON parsing an return a JSONObject that you can query.
I very much doubt that
you can come up with a faster parsing of your json responses using regular expressions or otherwise, without taking the risk of failing in corner cases
given the size of your response strings, that the JSON parsing is the performance bottleneck in your code
Unless you have done some serious profiling, I would play safe and follow the first rule of program optimization

How to use regular expressions to identify the error message in the JSON String?

I am working on a project in which I am making a call to one of my servers using RestTemplate which is running a restful service and getting the response back from them.
The response that I will be getting from my server can be either of these error responses (that's all I have for error response) if something has gone wrong -
{"warning": "user_id not found", "user_id": some_user_id}
{"error": "user_id for wrong partition", "user_id": some_user_id, "partition": some_partition}
{"error": "missing client id", "client_id":2000}
or below successful response (it can be any random json string key can also be different) -
{"#data": {"oo":"1205000384","p":"2047935"}
If I am getting any error response as mentioned above, then I am deserializing it (my bad :( ) so that I can log them as an error with a specific error or warning I got front the server which can be for example - user_id not found or missing client id.
If it is a successful response then also I am deserializing it which I don't need for my use case as we don't have any POJO and I just need to return the response as it is which I have got from the server.
In my use case, I don't need to deserialize my response string if it is a successful response as we don't have any POJO for that and we are returning the response string as it is which we have got from the server. But just for logging specific error messages (if I am getting error response from the server) I am deserializing it which I am thinking is unnecessary. There might be better solution for my use case.
Below is my Java client which is calling Callable task using future.get -
public class TestingClient implements IClient {
private ExecutorService service = Executors.newFixedThreadPool(10);
private RestTemplate restTemplate = new RestTemplate();
#Override
public String executeSync(ClientKey keys) {
String response = null;
try {
ClientTask ClientTask = new ClientTask(keys, restTemplate);
Future<String> future = service.submit(ClientTask);
response = handle.get(keys.getTimeout(), TimeUnit.MILLISECONDS);
} catch (TimeoutException e) {
} catch (Exception e) {
}
return response;
}
}
And now below is my ClientTask class which implements Callable interface. In the call method, I am generating an URL and then hit the server using RestTemplate and get the response back -
class ClientTask implements Callable<String> {
private ClientKey cKeys;
private RestTemplate restTemplate;
public ClientTask(ClientKey cKeys, RestTemplate restTemplate) {
this.restTemplate = restTemplate;
this.cKeys = cKeys;
}
#Override
public String call() throws Exception {
// .. some code here
String url = "some_url";
String response = restTemplate.getForObject(url, String.class);
String test = checkJSONResponse(response);
return test;
}
private String checkJSONResponse(final String response) throws Exception {
// may be there are some better way of doing it for my scenario instead of using GSON
Gson gson = new Gson();
String str = null;
JsonObject jsonObject = gson.fromJson(response, JsonObject.class); // parse it, may be performance issues here/
if (jsonObject.has("error") || jsonObject.has("warning")) {
final String error = jsonObject.get("error") != null ? jsonObject.get("error").getAsString() : jsonObject
.get("warning").getAsString();
// log specific `error` here using log4j
str = response;
} else {
str = response;
}
return str;
}
}
As you can see in my above code we are deserializing the JSON string only to log specific error messages if we are getting any error response back. But for successful response we don't need any deserialization but still we are doing it.
Is there any better way of solving this problem? Because currently I am seeing some performance issues with the GSON deserialization.
The only way I can identify successful response along with error response is with error or warning in the response so I am thinking of using regular expressions which can identify error or warning as the key in the response string. If they contain error or warning in the response string then extract the specific error or warning message and log it.
I guess there might be some better way of solving this problem without paying the cost for deserialization.

Just relying on a regex is I think to dangerous. What if the server slightly changes the output format?
I would try to make a quick test, possibly with a simple regexp looking for the string "error" and if there is a chance that it is an error response do a full deserialization to determine if it really was an error or not.
You would pay the extra cost only for false positives when a regular response by chance triggers the quick check.

I would use http codes to control success/fail data parsing.