We Keep Coding

WebClient does not return a "valid" list of Strings

I have a spring boot app that among others, has an endpoint that when hit, returns a list of strings. I also have another spring boot app that hits the first app's endpoint to get the data. The fetch code:
return webClient.get().uri("/sensors/get-cities").headers(httpHeaders -> {
httpHeaders.set("Authorization", auth);
}).retrieve()
.bodyToFlux(String.class).collectList().block();
The above yields a list but with this format when I inspect it in the debbuger, "["city"]". The outer double quotes, I get them because it's a string but the brackets and the inside double quotes, I do not. I tried replacing these characters but I had no luck with the brackets (tried regex). It is like they are not there, but at the same time they are. I am confused at this point. But I think that the behavior of the fetch code is not normal, it should yield a valid array of strings.

What you are probably getting (im guessing here) is a response body that looks something like this:
[
"New York",
"Madrid",
"London"
]
You then tell webflux that you want to convert the body to a Flux of String by calling bodyToFlux(String.class).
So the framework takes the entire response and makes a string out of it
// A string of the entire array (im escaping the quotation marks)
"[\"New York\",\"Madrid\",\"London\"]"
And then the framework will throw the entire thing into a Flux which means it takes the first position in the Flux. You then emit all the values into a List by calling collectList The equivalent code is sort of:
List<String> oneString = Flux.just("[\"New York\",\"Madrid\",\"London\"]")
.collectList()
.block();
So you get a list, with one string in it, which is the entire body.
What you probably want to do is to get a list out if it. And this is one way to do it:
List<String> strings = webClient.get()
.uri("/sensors/get-cities")
.headers(httpHeaders -> {
httpHeaders.set("Authorization", auth);
})
.retrieve()
.bodyToMono(new ParameterizedTypeReference<List<String>>() {})
.block();
Spring explains ParameterizedTypeReference:
The purpose of this class is to enable capturing and passing a generic Type. In order to capture the generic type and retain it at runtime
So its sort of a class that makes sure we can use generic types like List<T> and helps us with type information.
So what we do is that we now take the response and tell the framework that the body is a list of strings directly. We dont need to do collectList anymore as the framework will stick it in a list for us. We then call block to wait in the response.

Your Springboot API returns result as parsed to JSON (this is default behavior). So it first builds a list of Strings (in your case just a single String "city" and than serializes it to Json. In this case since it is a list it serializes it to JSON array as opposed to JSON Object. Read about JSON here. So in your second Springboot app that hits the API from the first one should assume that you are getting JSON which you need to parse to get your list. To parse it you can use readValue() method of ObjectMapper class of Json Jackson library which is a default JSON library in Springboot. your code would be
List<String> myList;
ObjectMapper = new ObjectMapper();
//Add setters for ObjectMapper configuration here if you want a specific config
try {
myList = objectMapper.readValue(myJsonString, List.class);
} catch(IOException ioe) {
...
}
In addition I wrote my own Open-source library called MgntUtils, that includes JsonUtils class which is a thin wrapper over Json Jackson library. It provides just Json parser and serializer, but in many cases that is all you need. With my library you would only need one dependency as oppose to Jackson, and JsonUtils class just have 4 methods, so by far easier to understand. But in your case if you use my library the code would be very similar to the above code. It would be something like this:
List<String> myList;
try {
myList = JsonUtils.readObjectFromJsonString(myJsonString, List.class);
} catch(IOException ioe) {
...
}
Note that in this case you won't have to instantiate and configure ObjectMapper instance as readObjectFromJsonString is a static method. Anyway if you are interested in using my library you can find maven artifacts here and The library itself with source code and javadoc is on Github here. Javadoc for JsonUtils class is here

Custom handler for response body in Jackson / Spring

I am trying to intercept the object that is being returned in my controller so that I can create a flat JSON structure of the response, before Spring invokes Jackson's serialization process.
I am going to support a query parameter that allows the client to flatten the response body. Something like:
/v1/rest/employees/{employeId}/id?flat=true
The controller method looks something like:
public Employee getEmployee(...) {}
I would like to avoid implementing this flattening logic in every one of my service calls and continue to return the Employee object.
Is there some kind of facility in Spring that would allow me to A) read the query string and B) intercept the object that is being returned as the response body?

Here's one idea. There may be a better way, but this will work:
Define an extra request mapping to do the flat mapping:
#RequestMapping(path = "/endpoint", params = {"flat"})
public String getFlatThing() {
return flatMapper.writeValueAsString(getThing());
}
// The Jackson converter will do its ordinary serialization here.
#RequestMapping(path = "/endpoint")
public Thing getFlatThing() {
return new Thing();
}
the "flatMapper" implementation can be whatever you like so long as it works.
One option is to use Jackson's ObjectMapper to write the value as json first and then use https://github.com/wnameless/json-flattener to flatten that to your desired output. There may also be a way to define a custom ObjectMapper that does flat mapping, though that would take some more work on your part.

Programmatically ignore (omit) specific fields in JSON response of REST service WITHOUT altering the DTO object class

I have a DTO class and some REST services that sometimes return (among other things) a List of those DTOs.
I cannot alter that DTO, as it's used in several places of the project.
However, only for one specific REST service, I need to exclude some of the fields of that DTO object.
Basically I need to be able to apply this solution only at a certain point.
I tried applying #JsonFilter("restrictionFilter") to my DTO class, but then I get an error if I don't use that filter with a mapper every time I marshall the object into a JSON, like this:
final String writeValueAsString = mapper.writer(
new SimpleFilterProvider()
.addFilter("restrictionFilter",
SimpleBeanPropertyFilter.filterOutAllExcept("name", "sizeInByte"))
).writeValueAsString(objectsList);
The error is Cannot resolve PropertyFilter with id 'restrictionFilter'; no FilterProvider configured...

This issue sounds like a perfect Decorator design pattern use.
Create a new DTO with a constructor that gets the original DTO and create which get methods you want or ignore whatever get methods you like.
For example:
public class NewDto {
OldDto oldDto;
public NewDto(OldDto oldDto){
this.oldDto = oldDto;
}
public String getName(){
return oldDto.getName();
}
}
Now you will only need to return the NewDto object, like so:
return new NewDto(oldDto)

Cannot do HAL+JSON Level 3 RESTful API with Spring HATEOAS due to lack of clarity surrounding HAL+JSON media-type

Level 3 RESTful API's feature custom media-types like application/vnd.service.entity.v1+json, for example. In my case I am using HAL to provide links between related resources in my JSON.
I'm not clear on the correct format for a custom media-type that uses HAL+JSON. What I have currently, looks like application/vnd.service.entity.v1.hal+json. I initially went with application/vnd.service.entity.v1+hal+json, but the +hal suffix is not registered and therefore violates section 4.2.8 of RFC6838.
Now Spring HATEOAS supports links in JSON out of the box but for HAL-JSON specifically, you need to use #EnableHypermediaSupport(type=EnableHypermediaSupport.HypermediaType.HAL). In my case, since I am using Spring Boot, I attach this to my initializer class (i.e., the one that extends SpringBootServletInitializer). But Spring Boot will not recognize my custom media-types out of the box. So for that, I had to figure out how to let it know that it needs to use the HAL object-mapper for media-types of the form application/vnd.service.entity.v1.hal+json.
For my first attempt, I added the following to my Spring Boot initializer:
#Bean
public HttpMessageConverters customConverters() {
MappingJackson2HttpMessageConverter converter = new MappingJackson2HttpMessageConverter();
converter.setSupportedMediaTypes(Arrays.asList(
new MediaType("application", "json", Charset.defaultCharset()),
new MediaType("application", "*+json", Charset.defaultCharset()),
new MediaType("application", "hal+json"),
new MediaType("application", "*hal+json")
));
CurieProvider curieProvider = getCurieProvider(beanFactory);
RelProvider relProvider = beanFactory.getBean(DELEGATING_REL_PROVIDER_BEAN_NAME, RelProvider.class);
ObjectMapper halObjectMapper = beanFactory.getBean(HAL_OBJECT_MAPPER_BEAN_NAME, ObjectMapper.class);
halObjectMapper.registerModule(new Jackson2HalModule());
halObjectMapper.setHandlerInstantiator(new Jackson2HalModule.HalHandlerInstantiator(relProvider, curieProvider));
converter.setObjectMapper(halObjectMapper);
return new HttpMessageConverters(converter);
}
This worked and I was getting the links back in proper HAL format. However, this was coincidental. This is because the actual media-type that ends up being reported as "compatible" with application/vnd.service.entity.v1.hal+json is *+json; it doesn't recognize it against application/*hal+json (see later for explanation). I didn't like this solution since it was polluting the existing JSON converter with HAL concerns. So, I made a different solution like so:
#Configuration
public class ApplicationConfiguration {
private static final String HAL_OBJECT_MAPPER_BEAN_NAME = "_halObjectMapper";
#Autowired
private BeanFactory beanFactory;
#Bean
public HttpMessageConverters customConverters() {
return new HttpMessageConverters(new HalMappingJackson2HttpMessageConverter());
}
private class HalMappingJackson2HttpMessageConverter extends MappingJackson2HttpMessageConverter {
public HalMappingJackson2HttpMessageConverter() {
setSupportedMediaTypes(Arrays.asList(
new MediaType("application", "hal+json"),
new MediaType("application", "*hal+json")
));
ObjectMapper halObjectMapper = beanFactory.getBean(HAL_OBJECT_MAPPER_BEAN_NAME, ObjectMapper.class);
setObjectMapper(halObjectMapper);
}
}
}
This solution does not work; I end up getting links in my JSON that don't conform to HAL. This is because application/vnd.service.entity.v1.hal+json is not recognized by application/*hal+json. The reason this happens is that MimeType, which checks for media-type compatibility, only recognizes media-types that start with *+ as valid wild-card media-types for subtypes (e.g., application/*+json). This is why the first solution worked (coincidentally).
So there are two problems here:
MimeType will never recognize vendor-specific HAL media-types of the form application/vnd.service.entity.v1.hal+json against application/*hal+json.
MimeType will recognize vendor-specific HAL media-types of the form application/vnd.service.entity.v1+hal+json against application/*+hal+json, however these are invalid mimetypes as per section 4.2.8 of RFC6838.
It seems like the only right way would be if +hal is recognized as a valid suffix, in which case the second option above would be fine. Otherwise there is no way any other kind of wild-card media-type could specifically recognize vendor-specific HAL media-types. The only option would be to override the existing JSON message converter with HAL concerns (see first solution).
Another workaround for now would be to specify every custom media-type you are using, when creating the list of supported media-types for the message converter. That is:
#Configuration
public class ApplicationConfiguration {
private static final String HAL_OBJECT_MAPPER_BEAN_NAME = "_halObjectMapper";
#Autowired
private BeanFactory beanFactory;
#Bean
public HttpMessageConverters customConverters() {
return new HttpMessageConverters(new HalMappingJackson2HttpMessageConverter());
}
private class HalMappingJackson2HttpMessageConverter extends MappingJackson2HttpMessageConverter {
public HalMappingJackson2HttpMessageConverter() {
setSupportedMediaTypes(Arrays.asList(
new MediaType("application", "hal+json"),
new MediaType("application", "vnd.service.entity.v1.hal+json"),
new MediaType("application", "vnd.service.another-entity.v1.hal+json"),
new MediaType("application", "vnd.service.one-more-entity.v1.hal+json")
));
ObjectMapper halObjectMapper = beanFactory.getBean(HAL_OBJECT_MAPPER_BEAN_NAME, ObjectMapper.class);
setObjectMapper(halObjectMapper);
}
}
}
This has the benefit of not polluting the existing JSON converter, but seems less than elegant. Does anyone know the right solution for this? Am I going about this completely wrong?

Although this question is a litte bit old, I recently stumbled upon the same problem so I wanted to give my 2 cents to this topic.
I think the problem here is the understanding of HAL regarding JSON. As you already pointed out here, all HAL is JSON but not all JSON is HAL. The difference between both is, from my understanding, that HAL defines some conventions to the semantics/structure, like telling you that behind an attribute like _links you'll find some links, whereas JSON just defines the format like key: [value] (as #zeroflagL already mentioned)
This is the reason, why the media type is called application/hal+json. It basically says it's the HAL style/semantics in the JSON format. This is also the reason that there exists a media type application/hal+xml (source ).
Now with a vendor specific media type, you define your own semantics and so your replacing the hal in application/hal+json and don't extend it.
If I understand you right, you basically want to say that you have a custom media type that uses the HAL style for it's JSON formatting. (This way, a client could use some HAL library to easily parse your JSON.)
So, at the end I think you basically have to decide wether you want to differentiate between JSON and HAL-based JSON and wether your API should provide one of these or both.
If you want to provide both, you'll have to define two different media types vnd.service.entity.v1.hal+json AND vnd.service.entity.v1+json. For the vnd.service.entity.v1.hal+json media type you then have to add your customized MappingJackson2HttpMessageConverter that uses the _halObjectMapper to return HAL-based JSON whereas the +json media type is supported by default returning your resource in good old JSON.
If you always want to provide HAL-based JSON, you have to enable HAL as the default JSON-Media type (for instance, by adding a customized MappingJackson2HttpMessageConverter that supports the +json media type and uses the _halObjectMapper mentioned before), so every request to application/vnd.service.entity.v1+json is handled by this converter returning HAL-based JSON.
From my opinion I think the right way is to only differentiate between JSON and other formats like XML and in your media type documentation you'd say, that your JSON is HAL-inspired in a way that clients can use HAL libs to parse the responses.
EDIT:
To bypass the problem that you'll have to add each vendor specific media type separately, you could override the isCompatibleWith method of the media type you're adding to your custom MappingJackson2HttpMessageConverter
converter.setSupportedMediaTypes(Arrays.asList(
new MediaType("application", "doesntmatter") {
#Override
public boolean isCompatibleWith(final MediaType other) {
if (other == null) {
return false;
}
else if (other.getSubtype().startsWith("vnd.") && other.getSubtype().endsWith("+json")) {
return true;
}
return super.isCompatibleWith(other);
}
}
));

How to serialize POJO fields differently based on Accepts header

We have a POJO that contains a collection and have it annotated thus:
#XmlElement(name = "<MyId")
#XmlElementWrapper(name = "MyIds")
private final Set<Long> myIds;
We are using JacksonJaxbJsonProvider in CXF to do the marshalling in our REST service.
The problem we are seeing is that if someone requests application/xml the response is correct, in that the user gets:
<MyIds>
<MyId>123</MyId>
<MyId>456</MyId>
...
</MyIds>
But when application/json is requested, the user gets (note the singular field name):
{
"MyId" : [123, 456, ...]
}
What I want to know is if there's a way to make that plural in the JSON response, and if so, how.
This feels like it may be bug in Jackson but there may be a perfectly good reason this is happening. Also, I realize that if everyone used the same POJO, we wouldn't have to care about what the marshalled text looked like, but in this case, one of the consumers cannot use our POJO.

If you're using version 2.1 or above of Jackson there's a feature which can be enabled to get the behaviour you want
ObjectMapper m = new ObjectMapper()
m.configure(MapperFeature.USE_WRAPPER_NAME_AS_PROPERTY_NAME, true);