We Keep Coding

Nested json type won't serialize in jersey

This bounty has ended. Answers to this question are eligible for a +100 reputation bounty. Bounty grace period ends in 22 hours.
James Wierzba wants to draw more attention to this question.
I'm running a dropwizard/jersey java restful web app.
I have an endpoint that is defined like this in api.yaml:
swagger: '2.0'
info:
version: 0.0.1
basePath: /
schemes:
- https
- http
consumes:
- application/json
- application/x-protobuf
produces:
- application/json
- application/x-protobuf
paths:
/v1/event:
post:
summary: receive a event
operationId: receiveEvent ## this value names the generated Java method
parameters:
- name: event
in: body
schema:
$ref: "#/definitions/Event"
responses:
200:
description: success
schema:
type: object
$ref: '#/definitions/EventResponse'
definitions:
Stream:
properties:
vendor:
type: "string"
Event:
properties:
eventCity:
type: "string"
streams:
type: "array"
items:
$ref: "#/definitions/Stream"
EventResponse:
required:
- statusCode
properties:
statusCode:
type: "integer"
Endpoint is defined like so
#POST
#Consumes({ "application/json", "application/x-protobuf" })
#Produces({ "application/json", "application/x-protobuf" })
#Path("/event")
void receiveEvent(
#Suspended AsyncResponse response,
#Valid Event.EventModel event
);
When issuing json POST request, I cannot get the streams field to get serialized/deserialized property.
This is the payload
{
"eventCity": "San Diego",
"streams": [
{
"vendor": "CBS"
}
]
}
I test like this with curl
curl -X POST -H "Content-Type: application/json" -d '{"eventCity": "San Diego", "streams": [{"vendor": "CBS"}]}' https://localhost:8990/v1/event
In the server request handler:
#Override
public void receiveEvent(AsyncResponse response, Event.EventModel event) {
System.out.println(event.getEventCity());
System.out.println(event.getStreamCount()); // <-- this returns 0? why is the inner 'streams' list not getting serialized? it should have one element
}
And the output:
San Diego
0
Another observation, is that when I issue the same post, but with a protobuf payload, it works. The streams list is populated.
The protobuf was generated like so
// create proto
Stream.StreamModel stream = Stream.StreamModel.newBuilder()
.setVendor("CBS")
.build();
Event.EventModel event = Event.EventModel.newBuilder()
.setEventCity("San Diego")
.addStream(stream)
.build();
// write to file
FileOutputStream fos = new FileOutputStream("/Users/jameswierzba/temp/proto.out");
stream.writeTo(fos);
The output in the endpoint is as expected:
San Diego
1
This is the full generated code for the Event class: https://gist.github.com/wierzba3/84face6c21c4fb6ce554f90707ba6ef9
This is the full generated doe for the Stream class: https://gist.github.com/wierzba3/32664312df87c64049b281daab928f94

You can't use the same class as the payload for protobuf and json.
If you inspect the Event generated for protobuf processing you will notice that the the stream is stored as stream_ and has a setStream that returns a Builder as follows. A JSON deserialiser can't work with it:
/**
* <code>repeated .com.apple.amp.social.linearmasterplaylist.refresh.model.StreamModel stream = 2;</code>
*/
public Builder setStream(
int index, com.apple.amp.social.linearmasterplaylist.refresh.model.Stream.StreamModel value) {
if (streamBuilder_ == null) {
if (value == null) {
throw new NullPointerException();
}
ensureStreamIsMutable();
stream_.set(index, value);
onChanged();
} else {
streamBuilder_.setMessage(index, value);
}
return this;
}
A JSON deserialiser (probably jackson-databind) needs a definition of Event like this generated with openapi-generator-cli. In this case the stream and its setter look like this: (note that the #JsonProperty("streams") is redundant as the property is named streams)
#JsonProperty("streams")
#Valid
private List<Stream> streams = null;
and
public void setStreams(List<Stream> streams) {
this.streams = streams;
}
I have included the other model definitions here to allow you to try them with you controller to show that JSON is correctly consumed.
It is possible to write code to inspect the incoming media type and fork the processing. Eg the following generated by openapi-generator. I would recommend a separate controller method for each media type and some sort of mapping so that a single service layer call will suffice.
#RequestMapping(
method = RequestMethod.POST,
value = "/v1/event",
produces = { "application/json", "application/x-protobuf" },
consumes = { "application/json", "application/x-protobuf" }
)
default ResponseEntity<EventResponse> receiveEvent(
#Parameter(name = "event", description = "") #Valid #RequestBody(required = false) Event event
) {
getRequest().ifPresent(request -> {
for (MediaType mediaType: MediaType.parseMediaTypes(request.getHeader("Accept"))) {
if (mediaType.isCompatibleWith(MediaType.valueOf("application/json"))) {
String exampleString = "{ \"statusCode\" : 0 }";
ApiUtil.setExampleResponse(request, "application/json", exampleString);
break;
}
if (mediaType.isCompatibleWith(MediaType.valueOf("application/x-protobuf"))) {
String exampleString = "Custom MIME type example not yet supported: application/x-protobuf";
ApiUtil.setExampleResponse(request, "application/x-protobuf", exampleString);
break;
}
}
});
return new ResponseEntity<>(HttpStatus.NOT_IMPLEMENTED);
}
On a separate note, but I doubt it will help: it looks like the protobuf is using stream (singular) (eg getStreamList()) and the swagger get uses streams (plural).

So, the issue turned out to be a bug in our companies internal implementation of the code generation module.
Workflow looks like this: api.yaml specified -> .proto file generated -> .java code generated.
To be explicit. Using the schema in question. Note I changed the list to have name myStreamArrayInput to illustrate the issue at hand.
Event:
properties:
eventCity:
type: "string"
myStreamArrayInput:
type: "array"
items:
$ref: "#/definitions/Stream"
This would generate a proto like this:
message Event {
optional string eventCity = 1;
/*
this name should be `myStreamArrayInput` or something
close to it. but it is using the name of the TYPE
instead of the name specified by the dev in api.yaml
*/
repeated StreamModel stream;
}
And then this would generate a java class like this
public final class Event {
public static final class EventModel extends
com.google.protobuf.GeneratedMessageV3 implements EventModelOrBuilder {
private EventModel() {
eventCity_ = "";
stream_ = "";
}
}
}
YMMV, since this is an bug in our own internal code generation engine

import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.databind.ObjectMapper;
import javax.ws.rs.*;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import java.util.ArrayList;
import java.util.List;
#Path("/myresource")
public class MyResource {
#GET
#Produces(MediaType.APPLICATION_JSON)
public Response getJson() {
ObjectMapper mapper = new ObjectMapper();
List<MyObject> myObjects = new ArrayList<>();
MyObject myObject = new MyObject("foo", new NestedObject("bar"));
myObjects.add(myObject);
String json;
try {
json = mapper.writeValueAsString(myObjects);
} catch (Exception e) {
return Response.status(Response.Status.INTERNAL_SERVER_ERROR).build();
}
return Response.ok(json, MediaType.APPLICATION_JSON).build();
}
private static class MyObject {
private String name;
private NestedObject nestedObject;
public MyObject(String name, NestedObject nestedObject) {
this.name = name;
this.nestedObject = nestedObject;
}
#JsonProperty("name")
public String getName() {
return name;
}
#JsonProperty("nested_object")
public NestedObject getNestedObject() {
return nestedObject;
}
}
private static class NestedObject {
private String value;
public NestedObject(String value) {
this.value = value;
}
#JsonProperty("value")
public String getValue() {
return value;
}
}
}

How to return Array of two JSON Object in Jersey RESTFul API(JAX-RS)

I have designed login module in RESTFul API using jersey.
whenever any error occurred while login it will return error code and message like,
{
"errorFlag": 1,
"errorMessage": "Login Failed"
}
but whenever I get successful results it returns
{
"apiKey": "3942328b-fa65-496c-bf32-910aafbc1b0e",
"email": "caXXXX#gmail.inl",
"name": "Chandrakant"
}
I'm looking for results like below
{
"errorFlag": 0,
"errorMessage":{
"apiKey": "3942328b-fa65-496c-bf32-910aafbc1b0e",
"email": "caXXXX#gmail.inl",
"name": "Chandrakant"}
}

Use structure like below,
{
status/statusCode : 200/400, //eg. 200 for success, any other for failure.
statusMessage : "Success/<failureMessage>",
errorDetails : "Failed due to <reason>" //optional
data :{ //data will exists only in case of success call.
}
}
you can achieve this like below,
#GET
#Path("/images/{image}")
#Produces("image/*")
public Response getImage(#PathParam("image") String image) {
  File f = new File(image);
 
  if (!f.exists()) {
    throw new WebApplicationException(404);
  }
 
  String mt = new MimetypesFileTypeMap().getContentType(f);
  return Response.ok(f, mt).build();
}

You can return all the attributes in HashMap as key value .
Below piece of code worked for me
#POST
#Path("/test")
#Produces(MediaType.APPLICATION_JSON)
#Consumes(MediaType.APPLICATION_JSON)
public HashMap check(InputStream inputJsonObj) {
HashMap map = new HashMap();
map.put("key1", "value1");
return map;
}

return any exception in json in rest api

Is there any simple methods to return exception in JSON using Rest api?
I've already googled this question, but all solutions i see, was about throwing exceptions during some calculations. But what if income parameters are wrong? I mean what if there is sone string instead of int input parameter?
I created some DTO class for input data:
#XmlRootElement
public class RequestDTO implements Serializable{
private static final long serialVersionUID = 1L;
#XmlElement(name = "request_id")
private String requestId;
#XmlElement(name = "site")
private List<String> sitesIds;
#XmlElement(name = "date_begin")
#JsonSerialize(using = DateSerializer.class)
#JsonDeserialize(using = DateDeserializer.class)
private Date dateBegin;
#XmlElement(name = "date_end")
#JsonSerialize(using = JsonDateSerializer.class)
#JsonDeserialize(using = JsonDateDeserializer.class)
private Date dateEnd;
#XmlElement(name = "volume")
private double volume;
// there is getters and setters
}
If i sent something like 'qwerty' instead of 'volume' field in my json request i'l see error message like Runtime. Is it possible to handle it in someway? I mean to return error in json with such structure?
public class ExceptionDTO {
private String shortExceptionMessage;
private String stackTrace;
public ExceptionDTO(String shotExceptionMessage, String stackTrace){
this.shortExceptionMessage = shotExceptionMessage;
this.stackTrace = stackTrace;
}
public String getShortExceptionMessage() {
return shortExceptionMessage;
}
public String getStackTrace() {
return stackTrace;
}
}
UPD1:
#Provider
#Singleton
public class ExceptionMapperProvider implements ExceptionMapper<Exception>{
#Override
public Response toResponse(final Exception e) {
StringBuilder trace = new StringBuilder();
IntStream.range(0, e.getStackTrace().length)
.forEach(i -> trace.append(e.getStackTrace()[i]).append('\n'));
ExceptionDTO exceptionMessage = new ExceptionDTO(
e.toString(),
trace.toString()
);
return Response.status(500).entity(exceptionMessage).build();
}
}

As it's not really clear if you are interested on checking if field or value of the payload is correct, here are a few ways to work with both.
If you want to check if the value for a field is correct (ie volume field value should be greater than zero etc), check out bean validation. This makes use of annotations on the fields you want to verify.
// for example
#Min(value = 0, message = "invalid message")
private double range;
To use your ExceptionDTO as error response whenever one of those validation fails, you can do so by creating an ExceptionMapper<ConstraintViolationException>. check it here for more details.
If you are checking for the invalid field (ie client sends ragne fields instead of range), have a look at the stack trace on what exception is being thrown. Then register an exception mapper with your ExceptionDTO as body.
For example, if UnrecognizedPropertyException is thrown then you can add:
#Provider
public class UnrecognizedPropertyExceptionMapper implements ExceptionMapper<UnrecognizedPropertyException> {
#Override
public Response toResponse(UnrecognizedPropertyException e) {
ExceptionDTO myDTO = // build response
return Response.status(BAD_REQUEST).entity(myDTO).build();
}
}

If you want to validate input parameters in the request, you should return status code 400 (Bad Request) along with the error details. You can simply send json
{ "error": { "message": "string received for parameter x, where as int expected" } with the response status code 400.
`

I did a bit of research and determined that the best way to encode a Java exception in JSON is to use a convention developed by Oasis that looks like this:
{
"error": {
"code": "400",
"message": "main error message here",
"target": "approx what the error came from",
"details": [
{
"code": "23-098a",
"message": "Disk drive has frozen up again. It needs to be replaced",
"target": "not sure what the target is"
}
],
"innererror": {
"trace": [ ... ],
"context": [ ... ]
}
}
}
details is a list that should have an entry for each nested cause exception in the chain.
innererror.trace should include the stack trace if you wish, as a list of string values.
The response status code should be 400 unless you have a good reason for making it something else, and the code in the structure should match whatever you sent.
Write one method to convert a Java exception to this format, and you are done. Use it consistently and your JS code will be able to handle and display the exception values.
More of the details of the other approaches evaluated and dismissed are covered in this blog post on JSON REST API – Exception Handling
https://agiletribe.purplehillsbooks.com/2015/09/16/json-rest-api-exception-handling/
Here is the java method to convert an exception to this format:
public static JSONObject convertToJSON(Exception e, String context) throws Exception {
JSONObject responseBody = new JSONObject();
JSONObject errorTag = new JSONObject();
responseBody.put("error", errorTag);
errorTag.put("code", 400);
errorTag.put("target", context);
JSONArray detailList = new JSONArray();
errorTag.put("details", detailList);
String lastMessage = "";
Throwable runner = e;
while (runner!=null) {
String className = runner.getClass().getName();
String msg = runner.toString();
runner = runner.getCause();
JSONObject detailObj = new JSONObject();
detailObj.put("message",msg);
int dotPos = className.lastIndexOf(".");
if (dotPos>0) {
className = className.substring(dotPos+1);
}
detailObj.put("code",className);
System.out.println(" ERR: "+msg);
detailList.put(detailObj);
}
JSONObject innerError = new JSONObject();
errorTag.put("innerError", innerError);
JSONArray stackList = new JSONArray();
runner = e;
while (runner != null) {
for (StackTraceElement ste : runner.getStackTrace()) {
String line = ste.getFileName() + ":" + ste.getMethodName() + ":" + ste.getLineNumber();
stackList.put(line);
}
stackList.put("----------------");
runner = runner.getCause();
}
errorTag.put("stack", stackList);
return responseBody;
}

Elasticsearch initial configuration in Java code or in external script?

I am learning Elasticsearch and I have started a new project. Now I wonder where I should add the initial code for creating the mappings etc. Would you create an external script which holds the different cURL commands and then run that, or have for example a own package in the Java project where you have the configuration code and then run it when you need to? Which approach is the most appropriate and why?
Mapping that I want to try with XContentBuilder
{
"tweet" : {
"properties" : {
"message" : {
"type" : "string",
"store" : "yes",
"index" : "analyzed",
"null_value" : "na"
}
}
}
}

I like to have it in java:
public void putMappingFromString(String index, String type, String mapping) {
IndicesAdminClient iac = getClient().admin().indices();
PutMappingRequestBuilder pmrb = new PutMappingRequestBuilder(iac);
pmrb.setIndices(index);
pmrb.setType(type);
pmrb.setSource(mapping);
ListenableActionFuture<PutMappingResponse> laf = pmrb.execute();
PutMappingResponse pmr = laf.actionGet();
pmr.getAcknowledged();
}
You can also get the mapping for an index from the cluster state (indirectly):
public String getMapping(String index, String type) throws EsuException {
ClusterState cs = getClient().admin().cluster().prepareState().setFilterIndices(index).execute().actionGet().getState();
IndexMetaData imd = cs.getMetaData().index(index);
if (imd == null) {
throw new EsuIndexDoesNotExistException(index);
}
MappingMetaData mmd = imd.mapping(type);
if (mmd == null) {
throw new EsuTypeDoesNotExistException(index, type);
}
String mapping = "";
try {
mapping = mmd.source().string();
} catch (IOException e) {
mapping = "{ \"" + e.toString() + "\"}";
}
return mapping;
}
This allows for versioning your mappings along with your source code if you store your mappings as a resource on your class path

Consuming Json in Play Framework

I try to run this code but I get a Null Exception.
Java Code :
public static void updateData(List<Users> users){
for(Users u : users){ //Error
System.out.println(u.name); // Error
}
}
Extjs Code :
proxy: {
type: 'ajax',
api: {
update: '/Application/updateData'
},
reader: {
type: 'json',
root: 'users',
successProperty: 'success'
}
}
Json Array :
[{"name":"Ed","email":"a...#aa.com"},{"name":"Ez","email":"b...#bb.com"}]
So please tell how to bind json Array to Entity List on Play Framework
1.2.2.
Thanks ...

You've specified root: 'users' in your reader's config. This means that JSON Array should look like this:
{users: [{"name":"Ed","email":"a...#aa.com"},{"name":"Ez","email":"b...#bb.com"}]}

You have to use Gson :
List<User> userList = new Gson().fromJson(yourString, Users.class);
And have a Users class suitable for your JSON :
public class Users {
private String name;
private String email;
...
//[Add your getter and setter]
...
}
For more information you can read the GSON documentation