I am currently working on a REST based java application using the new Camel REST DSL as the foundation.
It mostly works except that I noticed when calling the URLs through a REST client (instead of say a browser) the JSON response is "garbled" and comes through with what I assume is the wrong encoding
MyRouteBuilder.java
#Component
public class MyRouteBuilder extends RouteBuilder{
#Autowired
LocalEnvironmentBean environmentBean;
#Override
public void configure() throws Exception {
restConfiguration().component("jetty").host("0.0.0.0").port(80)
.bindingMode(RestBindingMode.auto);
rest("/testApp")
.get("/data").route()
.to("bean:daoService?method=getData")
.setProperty("viewClass", constant(CustomeJsonViews.class))
.marshal("customDataFormat").endRest()
.get("/allData").route()
.to("bean:daoService?method=getDatas")
.setProperty("viewClass", constant(CustomeJsonViews.class))
.marshal("customDataFormat").endRest();
}
}
CustomeDataFormat.java
public class CustomDataFormat implements DataFormat{
private ObjectMapper jacksonMapper;
public CustomDataFormat(){
jacksonMapper = new ObjectMapper();
}
#Override
public void marshal(Exchange exchange, Object obj, OutputStream stream) throws Exception {
Class view = (Class) exchange.getProperty("viewClass");
if (view != null)
{
ObjectWriter w = jacksonMapper.writerWithView(view);
w.writeValue(stream, obj);
}
else
stream.write(jacksonMapper.writeValueAsBytes(obj));
}
#Override
public Object unmarshal(Exchange exchange, InputStream stream) throws Exception {
return null;
}
}
A full working version can be found here:
https://github.com/zwhitten/camel-rest-test
When going to the URL, {host}/testApp/data, in Chrome for example the response comes through as:
{
data: "Sherlock",
value: "Holmes",
count: 10
}
However using the Postman browser plugin as the client returns:
"W3siZGF0YSI6ImRhdGE6OjAiLCJ2YWx1ZSI6InZhbHVlOjowIiwiY291bnQiOjB9LHsiZGF0YSI6ImRhdGE6OjEiLCJ2YWx1ZSI6InZhbHVlOjoxIiwiY291bnQiOjF9LHsiZGF0YSI6ImRhdGE6OjIiLCJ2YWx1ZSI6InZhbHVlOjoyIiwiY291bnQiOjJ9LHsiZGF0YSI6ImRhdGE6OjMiLCJ2YWx1ZSI6InZhbHVlOjozIiwiY291bnQiOjN9LHsiZGF0YSI6ImRhdGE6OjQiLCJ2YWx1ZSI6InZhbHVlOjo0IiwiY291bnQiOjR9LHsiZGF0YSI6ImRhdGE6OjUiLCJ2YWx1ZSI6InZhbHVlOjo1IiwiY291bnQiOjV9LHsiZGF0YSI6ImRhdGE6OjYiLCJ2YWx1ZSI6InZhbHVlOjo2IiwiY291bnQiOjZ9LHsiZGF0YSI6ImRhdGE6OjciLCJ2YWx1ZSI6InZhbHVlOjo3IiwiY291bnQiOjd9LHsiZGF0YSI6ImRhdGE6OjgiLCJ2YWx1ZSI6InZhbHVlOjo4IiwiY291bnQiOjh9LHsiZGF0YSI6ImRhdGE6OjkiLCJ2YWx1ZSI6InZhbHVlOjo5IiwiY291bnQiOjl9XQ=="
The problem seems to be with the REST bind mode being "auto" and using a custom marshaller.
If I set the binding mode to "json" then both the browser and client responses get garbled.
If I set the binding mode to "json" and bypass the custom marshallers everything works correctly.
Is there a way to configure the route to use a custom marshaller and encode the responses correctly regardless of the client?
I think the solution is to use the default binding option(off) since you are using custom marshallers.
You have two ways to achieve it:
Turn off the RestBindingMode, because otherwise the RestBindingMarshalOnCompletion in RestBindingProcessor will be registered and manually (un)marshal.
Register your own DataFormat and use it within the RestBinding automatically. You configure the REST configuration via jsonDataFormat to set the custom data format.
Map<String, DataFormatDefinition> dataFormats = getContext().getDataFormats();
if (dataFormats == null) {
dataFormats = new HashMap<>();
}
dataFormats.put("yourFormat", new DataFormatDefinition(new CustomDataFormat()));
restConfiguration()....jsonDataFormat("yourFormat")
You can also create your own dataformat like so:
in your restconfiguration it will look sthg like this (see json-custom)
builder.restConfiguration().component("jetty")
.host(host(propertiesResolver))
.port(port(propertiesResolver))
.bindingMode(RestBindingMode.json)
.jsonDataFormat("json-custom")
;
You must create a file "json-custom"
that's the name of the file and that file should contain the class name that implements your own way to marshal and unmarshal...
it must be located in your jar : META-INF\services\org\apache\camel\dataformat
so the content of the file should be:
class=packageofmyclass.MyOwnDataformatter
The response you were receiving is JSON, but it had been encoded to base64. Taking the String from your post, I was able to decode it as:
[{"data":"data::0","value":"value::0","count":0},{"data":"data::1","value":"value::1","count":1},{"data":"data::2","value":"value::2","count":2},{"data":"data::3","value":"value::3","count":3},{"data":"data::4","value":"value::4","count":4},{"data":"data::5","value":"value::5","count":5},{"data":"data::6","value":"value::6","count":6},{"data":"data::7","value":"value::7","count":7},{"data":"data::8","value":"value::8","count":8},{"data":"data::9","value":"value::9","count":9}]
The answers above stop the response body being encoded to base64. The documentation from Apache Camel on bindingMode is illusive as to why it behaves that way when combined with explicit marshalling. Removing the explicit marshalling will return a JSON body, but you may also notice that it contains the any class names in the body. The documentation suggests that bindingMode is more for the transportation of classes and that you specifiy a type(Pojo.class) and optionally outType(Pojo.class) of your requests/responses. See http://camel.apache.org/rest-dsl.html (section Binding to POJOs Using) for more details.
Base64 is the safest way of transferring JSON across networks to ensure it is received exactly as the server sent it, according to some posts I've read. It is then the responsibility of the client to then decode the response.
The answers above do solve the problem. However, I'm not entirely convinced that mixing the data format in the service routes is such as good thing and should ideally be at a higher level of abstraction. This would then allow the data format to be changed in one place, rather than having to change it on every route that produces JSON. Though, I must admit, I've never seen a service that has change data format in its lifetime, so this really is a mute point.
We were also facing the same issue.
Our DataFormat was Json .Once we implented our own custom marshaller. Camel was encoding the data to base64.I tried the approach provided by Cshculz but our CustomDataFormatter was not getting called for some reason which i couldn't figure out.
So We added .marshal(YourDataFormatter) after every Bean call.This provided us with the formatted json but in the encoded form so in the end of the route we added .unmarshal().json(JsonLibrary.Jackson) to return a raw json to the client.
sample snippet :-
.to("xxxxxxx").marshal(YourDataFormatterBeanRef)
.to("xxxxxxx").marshal(YourDataFormatterBeanRef)
.to("xxxxxxx").marshal(YourDataFormatterBeanRef)
.to("xxxxxxx").marshal(YourDataFormatterBeanRef)
.end().unmarshal().json(JsonLibrary.Jackson)
Related
We are trying to create a Java client for an API created with Spring Data.
Some endpoints return hal+json responses containing _embedded and _links attributes.
Our main problem at the moment is trying to wrap our heads around the following structure:
{
"_embedded": {
"plans": [
{
...
}
]
},
...
}
When you hit the plans endpoint you get a paginated response the content of which is within the _embedded object. So the logic is that you call plans and you get back a response containing an _embedded object that contains a plans attribute that holds an array of plan objects.
The content of the _embedded object can vary as well, and trying a solution using generics, like the example following, ended up returning us a List of LinkedHashMap Objects instead of the expected type.
class PaginatedResponse<T> {
#JsonProperty("_embedded")
Embedded<T> embedded;
....
}
class Embedded<T> {
#JsonAlias({"plans", "projects"})
List<T> content; // This instead of type T ends up deserialising as a List of LinkedHashMap objects
....
}
I am not sure if the above issue is relevant to this Jackson bug report dating from 2015.
The only solution we have so far is to either create a paginated response for each type of content, with explicitly defined types, or to include a List<type_here> for each type of object we expect to receive and make sure that we only read from the populated list and not the null ones.
So our main question to this quite spread out issue is, how one is supposed to navigate such an API without the use of Spring?
We do not consider using Spring in any form as an acceptable solution. At the same time, and I may be quite wrong here, but it looks like in the java world Spring is the only framework actively supporting/promoting HAL/HATEOAS?
I'm sorry if there are wrongly expressed concepts, assumptions and terminology in this question but we are trying to wrap our heads around the philosophy of such an implementation and how to deal with it from a Java point of view.
You can try consuming HATEOS API using super type tokens. A kind of generic way to handle all kind of hateos response.
For example
Below generic class to handle response
public class Resource<T> {
protected Resource() {
this.content = null;
}
public Resource(T content, Link... links) {
this(content, Arrays.asList(links));
}
}
Below code to read the response for various objects
ObjectMapper objectMapper = new ObjectMapper();
Resource<ObjectA> objectA = objectMapper.readValue(response, new TypeReference<Resource<ObjectA>>() {});
Resource<ObjectB> objectB = objectMapper.readValue(response, new TypeReference<Resource<ObjectB>>() {});
You can refer below
http://www.java-allandsundry.com/2012/12/json-deserialization-with-jackson-and.html
http://www.java-allandsundry.com/2014/01/consuming-spring-hateoas-rest-service.html
I'm not too familiar with Anypoint and we will probably only use this program once, I've looked at tutorials on the website but I feel I can't find one that demonstrates the task we have. Basically we are trying to read a JSON file that will be sent to the software, from there we want to use JAVA to read/alter the contents into a desired XML formatting. Finally we then send back or redirect the XML results.
1: I am trying to accept a JSON file from a HTTP POST, I believe I have accomplished this part by using the tutorials and an HTTP element with metadata attached representing the JSON format that may come. Using postman to send the json data as well.
2: From here is where I start to get completely confused. I am wondering what I would need to do in order to pass the data into a Java class object to read the JSON file and begin using Java code (getters/setters) instead of the Anypoint interface in order to start designing the xml layout.
Thanks,
I think the next step is for you to define a model class(with getter/setter), which would represent your json file contents. Since you have already the POST part, the next thing you would do is to use JSON to Object Transformer which the model is injected in.
<json:json-to-object-transformer returnClass="com.alexfrndz.Person" doc:name="JSON to Object"/>
After adding the transformer, you could use a mule custom transformer,
Here is the custom transformer.
package com.alexfrndz;
import org.mule.api.MuleMessage;
import org.mule.api.transformer.TransformerException;
import org.mule.transformer.AbstractMessageTransformer;
public class PersonTransformer extends AbstractMessageTransformer {
#Override
public Object transformMessage(MuleMessage message, String outputEncoding) throws TransformerException {
Person person = (Person) message.getPayload();
//Do your transformation hear
return null;
}
}
Here is how you implement it,
<custom-transformer class="com.alexfrndz.PersonTransformer" doc:name="PersonTransformer"/>
Hope this will help you.
I followed my previous question Spring Cloud Stream message from/to JSON conversion configuration and configured stream as described, yet, I can't make it work correctly.
My setup is as follows. I have two apps A and B. App A uses input channel one, output two. App B uses input two. Channel two is configured with content type application/json.
App A. Properties.
spring.cloud.stream.bindings.input.destination=one
spring.cloud.stream.bindings.input.group=default
spring.cloud.stream.bindings.output.destination=two
spring.cloud.stream.bindings.output.content-type=application/json
Listener method.
#ServiceActivator(inputChannel = Processor.INPUT, outputChannel = Processor.OUTPUT)
public Dto handle(byte[] payload) throws IOException {
final Dto dto = new ObjectMapper().readValue(payload, Dto.class);
logger.info("{}", dto);
dto.setId(dto.getId() + 1000);
return dto;
}
App B. Properties.
spring.cloud.stream.bindings.input.destination=two
spring.cloud.stream.bindings.input.group=default
spring.cloud.stream.bindings.input.content-type=application/json
Listener method.
#ServiceActivator(inputChannel = Sink.INPUT)
public void handle(Dto dto) throws IOException {
logger.info("DTO {}", dto);
}
When I manually send a message with proper JSON string to channel one, it is processed correctly and send to channel two as a JSON message (headers exactly the same as described in above mentioned question). After that, it's received on channel two by App B and exception is thrown: Method handle(java.lang.String) cannot be found
Of course when I create both methods, handling Dto and String as an input, it works, but always String method is invoked and have to deserialize the payload by myself.
Am I mistaken somewhere? How do I setup method with such signature: public Dto handle(Dto incoming)?
You should change the content-type declaration of input of AppB to
application/x-java-object;type=your.package.Dto.
As it is specified in your question, of course you accept JSON strings only.
if use #StreamListener you don't have to use the answer way, but you have to remove (don't specify anything, otherwise it will be a json string):
spring.cloud.stream.bindings.input.content-type=application/json
from AppB properties
source (old docs but still valid):
https://docs.spring.io/spring-cloud-stream/docs/Brooklyn.RELEASE/reference/html/contenttypemanagement.html#__literal_streamlistener_literal_and_message_conversion
I am looking to get the key and value to each Json formatted call and use them as java objects such as String or Integer ,in a rest client i would enter
{
"Name":"HelloWorld"
}
And i would get back the HelloWorld mapped to its Key so far ive seen examples but im just having trouble finding out what each tag does and how to parse the body to give the above results
#POST
#Path("/SetFeeds")
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_JSON)
#JsonCreator
public String setFeed(String jsonBody,#Context UriInfo uriInfo){
// Code to manipulate the body of the request
return response;
}
First thing you need to understand is how request body parsing is done. In JAX-RS parsing (or unmarshalling/deserializing/whatever) is done with MessageBodyReaders. There are different readers that can handle different Content-Type. For instance if you have Content-Type application/octet-stream, there is a reader that will unmarshal to byte[] or File or InputStream. So the following would work out the box
#Consumes("application/octet-stream")
public Response post(File file) {} // or `byte[]` or `InputStream`
That being said, JAX-RS implementations come with very basic readers for "easily convertible" format. For example, most requests can be converted to String, so you get that free for most Content-Types, as you are with your current code.
If we want some more complex data types, like your HelloWorld for Content-Type application/json, there is no standard reader for this. For this to work, we either need to create our own reader or use a library that comes with a reader. Luckily, the most popular JSON library in Java, Jackson, has implemented a JAX-RS provider that has a reader and a writer (for serialization).
Now depending on what server/JAX-RS implementation you are using, different implementations create light wrappers around the core Jackson JAX-RS module. If I knew the JAX-RS implementation you were using, I could recommend which wrapper to use, or you can forget the wrapper and just go with the basic Jackson module, which is
<dependency>
<groupId>com.fasterxml.jackson.jaxrs</groupId>
<artifactId>jackson-jaxrs-json-provider</artifactId>
<version>2.2.3</version>
</dependency>
The above is a Maven dependency. If you are not using Maven, then basically you need to download all these jars.
You can find all of them here. Just search for them individually.
Then you need to register the provider. Again it depends on your JAX-RS implementation and how you are handling the configuration of your resource classes. I would need to see your application configuration (either web.xml or Java code) and maybe the server you are using to help with that. For the most part, the JacksonJsonProvider (which is the reader and writer) needs to be registered.
Once you have it registered then you need to understand the basics of how Jackson handles the serialization. At most basic level, Jackson looks for JavaBean properties (basic getter/setter) to match with JSON properties. For instance, if you have this bean property
public class HelloWorld {
private String name;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
The JSON should look like {"name": "whatever"}. The "name" key is the same as the bean property. In Bean property terms, the name of the property is all letters after the get/set with the first letter lowercased.
That's pretty much all there is to it. Now you can do
#Consumes("application/json")
public Response post(HelloWorld helloWorld) {
String name = helloWorld.getName(); // should == "whatever"
return Response.ok(helloWorld).build(); // we can also return objects
}
For more complex JSON formats, you should refer to the Jackson documentation or ask a question here on SO.
As far as the registering of the JacksonJsonProvider, if you are having trouble, please provide the information I requested, i.e. application configuration (web.xml or Java config) and the server you are using.
See Also:
JAX-RS Entity Provider to learn more about readers and writers
Level 3 RESTful API's feature custom media-types like application/vnd.service.entity.v1+json, for example. In my case I am using HAL to provide links between related resources in my JSON.
I'm not clear on the correct format for a custom media-type that uses HAL+JSON. What I have currently, looks like application/vnd.service.entity.v1.hal+json. I initially went with application/vnd.service.entity.v1+hal+json, but the +hal suffix is not registered and therefore violates section 4.2.8 of RFC6838.
Now Spring HATEOAS supports links in JSON out of the box but for HAL-JSON specifically, you need to use #EnableHypermediaSupport(type=EnableHypermediaSupport.HypermediaType.HAL). In my case, since I am using Spring Boot, I attach this to my initializer class (i.e., the one that extends SpringBootServletInitializer). But Spring Boot will not recognize my custom media-types out of the box. So for that, I had to figure out how to let it know that it needs to use the HAL object-mapper for media-types of the form application/vnd.service.entity.v1.hal+json.
For my first attempt, I added the following to my Spring Boot initializer:
#Bean
public HttpMessageConverters customConverters() {
MappingJackson2HttpMessageConverter converter = new MappingJackson2HttpMessageConverter();
converter.setSupportedMediaTypes(Arrays.asList(
new MediaType("application", "json", Charset.defaultCharset()),
new MediaType("application", "*+json", Charset.defaultCharset()),
new MediaType("application", "hal+json"),
new MediaType("application", "*hal+json")
));
CurieProvider curieProvider = getCurieProvider(beanFactory);
RelProvider relProvider = beanFactory.getBean(DELEGATING_REL_PROVIDER_BEAN_NAME, RelProvider.class);
ObjectMapper halObjectMapper = beanFactory.getBean(HAL_OBJECT_MAPPER_BEAN_NAME, ObjectMapper.class);
halObjectMapper.registerModule(new Jackson2HalModule());
halObjectMapper.setHandlerInstantiator(new Jackson2HalModule.HalHandlerInstantiator(relProvider, curieProvider));
converter.setObjectMapper(halObjectMapper);
return new HttpMessageConverters(converter);
}
This worked and I was getting the links back in proper HAL format. However, this was coincidental. This is because the actual media-type that ends up being reported as "compatible" with application/vnd.service.entity.v1.hal+json is *+json; it doesn't recognize it against application/*hal+json (see later for explanation). I didn't like this solution since it was polluting the existing JSON converter with HAL concerns. So, I made a different solution like so:
#Configuration
public class ApplicationConfiguration {
private static final String HAL_OBJECT_MAPPER_BEAN_NAME = "_halObjectMapper";
#Autowired
private BeanFactory beanFactory;
#Bean
public HttpMessageConverters customConverters() {
return new HttpMessageConverters(new HalMappingJackson2HttpMessageConverter());
}
private class HalMappingJackson2HttpMessageConverter extends MappingJackson2HttpMessageConverter {
public HalMappingJackson2HttpMessageConverter() {
setSupportedMediaTypes(Arrays.asList(
new MediaType("application", "hal+json"),
new MediaType("application", "*hal+json")
));
ObjectMapper halObjectMapper = beanFactory.getBean(HAL_OBJECT_MAPPER_BEAN_NAME, ObjectMapper.class);
setObjectMapper(halObjectMapper);
}
}
}
This solution does not work; I end up getting links in my JSON that don't conform to HAL. This is because application/vnd.service.entity.v1.hal+json is not recognized by application/*hal+json. The reason this happens is that MimeType, which checks for media-type compatibility, only recognizes media-types that start with *+ as valid wild-card media-types for subtypes (e.g., application/*+json). This is why the first solution worked (coincidentally).
So there are two problems here:
MimeType will never recognize vendor-specific HAL media-types of the form application/vnd.service.entity.v1.hal+json against application/*hal+json.
MimeType will recognize vendor-specific HAL media-types of the form application/vnd.service.entity.v1+hal+json against application/*+hal+json, however these are invalid mimetypes as per section 4.2.8 of RFC6838.
It seems like the only right way would be if +hal is recognized as a valid suffix, in which case the second option above would be fine. Otherwise there is no way any other kind of wild-card media-type could specifically recognize vendor-specific HAL media-types. The only option would be to override the existing JSON message converter with HAL concerns (see first solution).
Another workaround for now would be to specify every custom media-type you are using, when creating the list of supported media-types for the message converter. That is:
#Configuration
public class ApplicationConfiguration {
private static final String HAL_OBJECT_MAPPER_BEAN_NAME = "_halObjectMapper";
#Autowired
private BeanFactory beanFactory;
#Bean
public HttpMessageConverters customConverters() {
return new HttpMessageConverters(new HalMappingJackson2HttpMessageConverter());
}
private class HalMappingJackson2HttpMessageConverter extends MappingJackson2HttpMessageConverter {
public HalMappingJackson2HttpMessageConverter() {
setSupportedMediaTypes(Arrays.asList(
new MediaType("application", "hal+json"),
new MediaType("application", "vnd.service.entity.v1.hal+json"),
new MediaType("application", "vnd.service.another-entity.v1.hal+json"),
new MediaType("application", "vnd.service.one-more-entity.v1.hal+json")
));
ObjectMapper halObjectMapper = beanFactory.getBean(HAL_OBJECT_MAPPER_BEAN_NAME, ObjectMapper.class);
setObjectMapper(halObjectMapper);
}
}
}
This has the benefit of not polluting the existing JSON converter, but seems less than elegant. Does anyone know the right solution for this? Am I going about this completely wrong?
Although this question is a litte bit old, I recently stumbled upon the same problem so I wanted to give my 2 cents to this topic.
I think the problem here is the understanding of HAL regarding JSON. As you already pointed out here, all HAL is JSON but not all JSON is HAL. The difference between both is, from my understanding, that HAL defines some conventions to the semantics/structure, like telling you that behind an attribute like _links you'll find some links, whereas JSON just defines the format like key: [value] (as #zeroflagL already mentioned)
This is the reason, why the media type is called application/hal+json. It basically says it's the HAL style/semantics in the JSON format. This is also the reason that there exists a media type application/hal+xml (source ).
Now with a vendor specific media type, you define your own semantics and so your replacing the hal in application/hal+json and don't extend it.
If I understand you right, you basically want to say that you have a custom media type that uses the HAL style for it's JSON formatting. (This way, a client could use some HAL library to easily parse your JSON.)
So, at the end I think you basically have to decide wether you want to differentiate between JSON and HAL-based JSON and wether your API should provide one of these or both.
If you want to provide both, you'll have to define two different media types vnd.service.entity.v1.hal+json AND vnd.service.entity.v1+json. For the vnd.service.entity.v1.hal+json media type you then have to add your customized MappingJackson2HttpMessageConverter that uses the _halObjectMapper to return HAL-based JSON whereas the +json media type is supported by default returning your resource in good old JSON.
If you always want to provide HAL-based JSON, you have to enable HAL as the default JSON-Media type (for instance, by adding a customized MappingJackson2HttpMessageConverter that supports the +json media type and uses the _halObjectMapper mentioned before), so every request to application/vnd.service.entity.v1+json is handled by this converter returning HAL-based JSON.
From my opinion I think the right way is to only differentiate between JSON and other formats like XML and in your media type documentation you'd say, that your JSON is HAL-inspired in a way that clients can use HAL libs to parse the responses.
EDIT:
To bypass the problem that you'll have to add each vendor specific media type separately, you could override the isCompatibleWith method of the media type you're adding to your custom MappingJackson2HttpMessageConverter
converter.setSupportedMediaTypes(Arrays.asList(
new MediaType("application", "doesntmatter") {
#Override
public boolean isCompatibleWith(final MediaType other) {
if (other == null) {
return false;
}
else if (other.getSubtype().startsWith("vnd.") && other.getSubtype().endsWith("+json")) {
return true;
}
return super.isCompatibleWith(other);
}
}
));