We Keep Coding

Schemas disappear from components when programmatically adding security scheme

I've recently converted from Springfox to Springdoc-openapi for generating my OpenAPI for my Spring Boot Rest API service.
Everything was working perfectly until I added a security scheme. Once I did that, my schemes no longer appear and an error appears on the SwaggerUI page:
Could not resolve reference: Could not resolve pointer: /components/schemas/Ping does not exist in document
I am setting up my configuration programmatically, and have 2 groups.
I'm using Spring Boot v2.4.0 with springdoc-openapi-ui v1.5.1
Snippet of my pom.xml:
<dependency>
<groupId>org.springdoc</groupId>
<artifactId>springdoc-openapi-ui</artifactId>
<version>1.5.1</version>
</dependency>
<dependency>
<groupId>org.springdoc</groupId>
<artifactId>springdoc-openapi-hateoas</artifactId>
<version>1.5.1</version>
</dependency>
<dependency>
<groupId>org.springdoc</groupId>
<artifactId>springdoc-openapi-security</artifactId>
<version>1.5.1</version>
</dependency>
Snippet from configuration:
#Bean
public GroupedOpenApi apiV1() {
String[] paths = {"/v1/**"};
String[] packagesToScan = {"com.test.controller"};
return GroupedOpenApi.builder()
.group("v1")
.packagesToScan(packagesToScan)
.pathsToMatch(paths)
.addOpenApiCustomiser(buildV1OpenAPI())
.build();
}
#Bean
public GroupedOpenApi apiV2() {
String[] paths = {"/v2/**"};
String[] packagesToScan = {"com.test.controller"};
return GroupedOpenApi.builder()
.group("v2")
.packagesToScan(packagesToScan)
.pathsToMatch(paths)
.addOpenApiCustomiser(buildV2OpenAPI())
.build();
}
public OpenApiCustomiser buildV1OpenAPI() {
return openApi -> openApi.info(apiInfo().version("v1"));
}
public OpenApiCustomiser buildV2OpenAPI() {
final String securitySchemeName = "Access Token";
return openApi -> openApi.info(apiInfo().version("v2"))
.addSecurityItem(new SecurityRequirement().addList(securitySchemeName))
.components(new Components().addSecuritySchemes(securitySchemeName, new SecurityScheme()
.type(SecurityScheme.Type.APIKEY)
.in(SecurityScheme.In.HEADER)
.name(HttpHeaders.AUTHORIZATION)));
}
// Describe the apis
private Info apiInfo() {
return new Info()
.title("Title")
.description("API Description");
}
For my v1 group, everything works fine. My Schemas appear on the Swagger UI page and I see them in the components section of the generated api-doc.
"components": {
"schemas": {
"ApplicationErrorResponse": {
...
}
},
"Ping": {
...
}
}
}
For my v2 group, the Schemas are not generated.
"components": {
"securitySchemes": {
"Access Token": {
"type": "apiKey",
"name": "Authorization",
"in": "header"
}
}
}
Any idea why my Schemas are not automatically scanned and added when adding the security scheme to the OpenAPI components programmatically? Am I missing something in my config?
Here's my request mapping in my controller.
#Operation(summary = "Verify API and backend connectivity",
description = "Confirm connectivity to the backend, as well and verify API service is running.")
#OkResponse
#GetMapping(value = API_VERSION_2 + "/ping", produces = MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<Ping> getPingV2(HttpServletRequest request) {
...
}
And here's my #OkResponse annotation:
#Retention(RetentionPolicy.RUNTIME)
#Target({ElementType.TYPE, ElementType.METHOD})
#Documented
#ApiResponse(responseCode = HTTP_200,
description = HTTP_200_OK,
headers = {
#Header(name = CONTENT_VERSION_HEADER, description = CONTENT_VERSION_HEADER_DESCRIPTION, schema = #Schema(type = "string")),
#Header(name = DEPRECATION_MESSAGE_HEADER, description = DEPRECATION_MESSAGE_HEADER_DESCRIPTION, schema = #Schema(type = "string")),
#Header(name = DESCRIPTION_HEADER, description = DESCRIPTION_HEADER_DESCRIPTION, schema = #Schema(type = "string"))
})
public #interface OkResponse {
}
My v1 mappings are defined similarly.

So, it would seem that when solely relying on OpenApiCustomiser for creating the OpenAPI, the scanned components are ignored, or at least overwritten with just the components specified in the customizer (I could have also programmatically added all of my schemas, but this would have been very cumbersome to maintain).
Changing my config to the following resolved my issue:
#Bean
public OpenAPI customOpenAPI() {
final String securitySchemeName = "Access Token";
return new OpenAPI()
.addSecurityItem(new SecurityRequirement().addList(securitySchemeName))
.components(new Components().addSecuritySchemes(securitySchemeName, new SecurityScheme()
.type(SecurityScheme.Type.APIKEY)
.in(SecurityScheme.In.HEADER)
.name(HttpHeaders.AUTHORIZATION)))
.info(apiInfo());
}
#Bean
public GroupedOpenApi apiV1() {
String[] paths = {"/v1/**"};
String[] packagesToScan = {"com.test.controller"};
return GroupedOpenApi.builder()
.group("v1")
.packagesToScan(packagesToScan)
.pathsToMatch(paths)
.addOpenApiCustomiser(buildV1OpenAPI())
.build();
}
#Bean
public GroupedOpenApi apiV2() {
String[] paths = {"/v2/**"};
String[] packagesToScan = {"com.test.controller"};
return GroupedOpenApi.builder()
.group("v2")
.packagesToScan(packagesToScan)
.pathsToMatch(paths)
.addOpenApiCustomiser(buildV2OpenAPI())
.build();
}
public OpenApiCustomiser buildV1OpenAPI() {
return openApi -> openApi.info(openApi.getInfo().version("v1"));
}
public OpenApiCustomiser buildV2OpenAPI() {
return openApi -> openApi.info(openApi.getInfo().version("v2"));
}
// Describe the apis
private Info apiInfo() {
return new Info()
.title("Title")
.description("API Description.");
}
While this technically does also add the Authorize button and security scheme to the v1 group, it can be ignored because those API endpoints are not secured anyway (internal API and they should be going away soon anyway).
Probably a better solution anyway as the Info is basically identical between the groups.

Instead of creating new Components you should just modify them:
public OpenApiCustomiser buildV2OpenAPI() {
final String securitySchemeName = "Access Token";
return openApi -> {
openApi.info(apiInfo().version("v2"))
.addSecurityItem(new SecurityRequirement().addList(securitySchemeName));
openApi.getComponents().addSecuritySchemes(securitySchemeName, new SecurityScheme()
.type(SecurityScheme.Type.APIKEY)
.in(SecurityScheme.In.HEADER)
.name(HttpHeaders.AUTHORIZATION));
return openApi;
};
}

Generate json schema from POJO

I need to generate json schema from my POJOs. The requirement is that every POJO must be exported as a separate file and the references inside the json schema must be handled appropriately. It means that the library should keep track of which POJO is exported to which file. I found this library: https://github.com/mbknor/mbknor-jackson-jsonSchema and it works fine but it seems (or at least i cannot find such option) that i can't accomplish the requirements without custom coding. Do you know any other library that supports this?

You can use Jackson to generate the JSON schema using the following maven dependencies
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.9.8</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-jsonSchema</artifactId>
<version>2.9.8</version>
</dependency>
<dependency>
<groupId>org.reflections</groupId>
<artifactId>reflections</artifactId>
<version>0.9.11</version>
</dependency>
You can then generate the schema by writing something like this
public static void main(String[] args) throws IOException {
ObjectMapper mapper = new ObjectMapper();
JsonSchemaGenerator schemaGen = new JsonSchemaGenerator(mapper);
Reflections reflections = new Reflections("my.pojo.model",new SubTypesScanner(false));
Set<Class<?>> pojos = reflections.getSubTypesOf(Object.class);
Map<String, String> schemaByClassNameMap = pojos.stream()
.collect(Collectors.toMap(Class::getSimpleName, pojo -> getSchema(mapper, schemaGen, pojo)));
schemaByClassNameMap.entrySet().forEach(schemaByClassNameEntry->writeToFile(schemaByClassNameEntry.getKey(),schemaByClassNameEntry.getValue()));
}
private static void writeToFile(String pojoClassName, String pojoJsonSchema) {
try {
Path path = Paths.get(pojoClassName + ".json");
Files.deleteIfExists(path);
byte[] strToBytes = pojoJsonSchema.getBytes();
Files.write(path, strToBytes);
}catch (Exception e){
throw new IllegalStateException(e);
}
}
private static String getSchema(ObjectMapper mapper,JsonSchemaGenerator schemaGenerator,Class clazz){
try {
JsonSchema schema = schemaGenerator.generateSchema(clazz);
return mapper.writerWithDefaultPrettyPrinter().writeValueAsString(schema);
}catch (Exception e){
throw new IllegalStateException(e);
}
}

Cloud Translate API Java client - problems using source, target lang

I have an issue here that I'm hoping to resolve. First, when I call the cloud Translate service with source and target languages, I encounter the following error:
java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
at com.google.cloud.translate.TranslateImpl.optionMap(TranslateImpl.java:131)
at com.google.cloud.translate.TranslateImpl.access$000(TranslateImpl.java:40)
at com.google.cloud.translate.TranslateImpl$4.call(TranslateImpl.java:113)
at com.google.cloud.translate.TranslateImpl$4.call(TranslateImpl.java:110)
This is what I'm doing:
protected Translate getTranslationServiceClient() throws IOException {
if (translationServiceClient == null) {
synchronized (this) {
if (translationServiceClient == null) {
try (InputStream is = new FileInputStream(new File(getCredentialFilePath()))) {
final GoogleCredentials myCredentials = GoogleCredentials.fromStream(is);
translationServiceClient = TranslateOptions.newBuilder().setCredentials(myCredentials).build().getService();
} catch (IOException ioe) {
throw new NuxeoException(ioe);
}
}
}
}
return translationServiceClient;
}
public TranslationResponse translateText(String text, String sourceLanguage, String targetLanguage) throws IOException {
Translation response = translationService.translate(text, TranslateOption.sourceLanguage("en"), TranslateOption.sourceLanguage("es"));
//System.out.println(response.getTranslatedText());
GoogleTranslationResponse gtr = new GoogleTranslationResponse(response);
return gtr;
}
The error points to the Cloud's TranslateImpl class optionMap method and spills the NoSuchMethodError on the checkArgument. Am I Passing the TranslateOption's incorrectly??:
private Map<TranslateRpc.Option, ?> optionMap(Option... options) {
Map<TranslateRpc.Option, Object> optionMap = Maps.newEnumMap(TranslateRpc.Option.class);
for (Option option : options) {
Object prev = optionMap.put(option.getRpcOption(), option.getValue());
checkArgument(prev == null, "Duplicate option %s", option);
}
return optionMap;
}
In an effort to get any kind of response from the API, I've tried calling the service without passing any options or just the targetLanguage. Without any options, I don't have any errors and my texted is translated into english, as expected. If I just add TranslateOption.targetLanguage("es"), I still get the NoSuchMethodError.

I had this exact same error. The problem was an ancient version of Google Guava being brought in by some other dependency. I found this by running mvn dependency:tree. I had to exclude the ancient version of Guava like this
<exclusions>
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</exclusion>
</exclusions>

How to test if JSON path does not include a specific element, or if the element is present it is null?

I have been writing some simple unit testing routines for a simple spring web application. When I add #JsonIgnore annotation on a getter method of a resource, the resulting json object does not include the corresponding json element. So when my unit test routine tries to test if this is null (which is the expected behavior for my case, I don't want the password to be available in json object), test routine runs into an exception:
java.lang.AssertionError: No value for JSON path: $.password, exception: No results for path: $['password']
This is the unit test method I wrote, testing the 'password' field with is(nullValue()) method:
#Test
public void getUserThatExists() throws Exception {
User user = new User();
user.setId(1L);
user.setUsername("zobayer");
user.setPassword("123456");
when(userService.getUserById(1L)).thenReturn(user);
mockMvc.perform(get("/users/1"))
.andExpect(jsonPath("$.username", is(user.getUsername())))
.andExpect(jsonPath("$.password", is(nullValue())))
.andExpect(jsonPath("$.links[*].href", hasItem(endsWith("/users/1"))))
.andExpect(status().isOk())
.andDo(print());
}
I have also tried it with jsonPath().exists() which gets a similar exception stating that the path doesn't exist. I am sharing some more code snippets so that the whole situation becomes more readable.
The controller method I am testing looks something like this:
#RequestMapping(value="/users/{userId}", method= RequestMethod.GET)
public ResponseEntity<UserResource> getUser(#PathVariable Long userId) {
logger.info("Request arrived for getUser() with params {}", userId);
User user = userService.getUserById(userId);
if(user != null) {
UserResource userResource = new UserResourceAsm().toResource(user);
return new ResponseEntity<>(userResource, HttpStatus.OK);
} else {
return new ResponseEntity<>(HttpStatus.NOT_FOUND);
}
}
I am using spring hateos resource assembler for converting entity to resource objects and this is my resource class:
public class UserResource extends ResourceSupport {
private Long userId;
private String username;
private String password;
public Long getUserId() {
return userId;
}
public void setUserId(Long userId) {
this.userId = userId;
}
public String getUsername() {
return username;
}
public void setUsername(String username) {
this.username = username;
}
#JsonIgnore
public String getPassword() {
return password;
}
public void setPassword(String password) {
this.password = password;
}
}
I understand why this is getting an exception, also in a way, the test is successful that it could not find the password field. But what I want to do is, run this test to ensure that the field is not present, or if present, it contains null value. How can I achieve this?
There is a similar post in stack overflow:
Hamcrest with MockMvc: check that key exists but value may be null
In my case, the field may be non existent as well.
For the record, these are the versions of test packages I am using:
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
<version>2.6.1</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
<version>2.6.1</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.6.1</version>
</dependency>
<dependency>
<groupId>com.jayway.jsonpath</groupId>
<artifactId>json-path</artifactId>
<version>2.0.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.jayway.jsonpath</groupId>
<artifactId>json-path-assert</artifactId>
<version>2.0.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>1.10.19</version>
<scope>test</scope>
</dependency>
[EDIT]
To be more precise, say, you have to write a test for an entity where you know some of the fields need to be null or empty or should not even exists, and you don't actually go through the code to see if there is a JsonIgnore added on top of the property. And you want your tests to pass, how can I do this.
Please feel free to tell me that this is not practical at all, but still would be nice to know.
[EDIT]
The above test succeeds with the following older json-path dependencies:
<dependency>
<groupId>com.jayway.jsonpath</groupId>
<artifactId>json-path</artifactId>
<version>0.9.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.jayway.jsonpath</groupId>
<artifactId>json-path-assert</artifactId>
<version>0.9.1</version>
<scope>test</scope>
</dependency>
[EDIT] Found a quickfix that works with latest version of jayway.jasonpath after reading the documentation of spring's json path matcher.
.andExpect(jsonPath("$.password").doesNotExist())

I had the same problem with the newer version. It looks to me that the doesNotExist() function will verify that the key is not in the result:
.andExpect(jsonPath("$.password").doesNotExist())

There is a difference between the property that is present, but having null value, and the property not being present at all.
If the test should fail only when there is a non-null value, use:
.andExpect(jsonPath("password").doesNotExist())
If the test should fail as soon as the property is present, even with a null value, use:
.andExpect(jsonPath("password").doesNotHaveJsonPath())

#JsonIgnore is behaving as expected, not producing the password in the json output, so how could you expect to test something that you are explicitly excluding from the output?
The line:
.andExpect(jsonPath("$.property", is("some value")));
or even a test that the property is null:
.andExpect(jsonPath("$.property").value(IsNull.nullValue()));
correspond to a json like:
{
...
"property": "some value",
...
}
where the important part is the left side, that is the existence of "property":
Instead, #JsonIgnore is not producing the porperty in the output at all, so you can't expect it not in the test nor in the production output.
If you don't want the property in the output, it's fine, but you can't expect it in test.
If you want it empty in output (both in prod and test) you want to create a static Mapper method in the middle that is not passing the value of the property to the json object:
Mapper.mapPersonToRest(User user) {//exclude the password}
and then your method would be:
#RequestMapping(value="/users/{userId}", method= RequestMethod.GET)
public ResponseEntity<UserResource> getUser(#PathVariable Long userId) {
logger.info("Request arrived for getUser() with params {}", userId);
User user = Mapper.mapPersonToRest(userService.getUserById(userId));
if(user != null) {
UserResource userResource = new UserResourceAsm().toResource(user);
return new ResponseEntity<>(userResource, HttpStatus.OK);
} else {
return new ResponseEntity<>(HttpStatus.NOT_FOUND);
}
}
At this point, if your expectations are for Mapper.mapPersonToRest to return a user with a null password, you can write a normal Unit test on this method.
P.S. Of course the password is crypted on the DB, right? ;)

doesNotHaveJsonPath for checking that it is not in json body

I wanted to reuse the same code I use for testing for the parameter being supplied, and for it missing, and this is what I came up with
#Test
void testEditionFoundInRequest() throws JsonProcessingException {
testEditionWithValue("myEdition");
}
#Test
void testEditionNotFoundInRequest() {
try {
testEditionWithValue(null);
throw new RuntimeException("Shouldn't pass");
} catch (AssertionError | JsonProcessingException e) {
var msg = e.getMessage();
assertTrue(msg.contains("No value at JSON path"));
}
}
void testEditionWithValue(String edition) {
var HOST ="fakeHost";
var restTemplate = new RestTemplate();
var myRestClientUsingRestTemplate = new MyRestClientUsingRestTemplate(HOST, restTemplate);
MockRestServiceServer mockServer;
ObjectMapper objectMapper = new ObjectMapper();
String id = "userId";
var mockResponse = "{}";
var request = new MyRequest.Builder(id).edition(null).build();
mockServer = MockRestServiceServer.bindTo(restTemplate).bufferContent().build();
mockServer
.expect(method(POST))
// THIS IS THE LINE I'd like to say "NOT" found
.andExpect(jsonPath("$.edition").value(edition))
.andRespond(withSuccess(mockResponse, APPLICATION_JSON));
var response = myRestClientUsingRestTemplate.makeRestCall(request);
} catch (AssertionError | JsonProcessingException e) {
var msg = e.getMessage();
assertTrue(msg.contains("No value at JSON path"));
}

Serialization and deserialization of Map with Jersey and Jackson

I've got one Pojo object which is serialized by Jersey using jackson:
public class BookOfFriendsAnswer {
private Map<String, List<BookSummary>> books;
public BookOfFriendsAnswer() {
}
public BookOfFriendsAnswer(Map<String, List<BookSummary>> books) {
this.books = books;
}
public Map<String, List<BookSummary>> getBooks() {
return books;
}
public void setBooks(Map<String, List<BookSummary>> books) {
this.books = books;
}
}
The serialization produces a JSon like this one:
{
"books": {
"entry": [
{
"key": "54567bbce4b0e0ef9379993e",
"value": "BookSummary{id='54567bbde4b0e0ef9379993f', title='title 1', authors=[Steve,James] } BookSummary{id='54567bd9e4b0e0ef93799940', title='Title 2', authors=[Simon, Austin]}"
}
]
}
}
However, when I'm trying to deserialize the message from my client like this:
mapper.readValue(json, clazz)
I get the following error:
Unrecognized field "key" (class com.example.server.api.BookSummary), not marked as ignorable
I don't know if the problem comes from the JSOn produced by the server or the deserialization on client's side.
Do you know what is the problem and how to correct it?
Thanks a lot

So after a little testing with:
Jersey 1.18.1 (with jersey-json-1.18.1 for JSON support)
Jersey 2.13 (with jersey-media-json-jackson-2.13 for JSON support)
Jersey 2.13 (with jersey-media-moxy-2.13 for JSON support)
The last test (jersey-media-moxy-2.13) was the only one to produce this exact output
{
"books": {
"entry": [
{
"key": "54567bbce4b0e0ef9379993e",
"value": "BookSummary{id='54567bbde4b0e0ef9379993f', title='title 1', authors=[Steve,James] } BookSummary{id='54567bd9e4b0e0ef93799940', title='Title 2', authors=[Simon, Austin]}"
}
]
}
}
That being said, I'll make the assumption you're using a Jersey 2.x version. I'm not sure if there is any configuration in MOXy to better support this use case, but to make things easy, just add the following dependency, and get rid of the MOXy
<dependency>
<groupId>org.glassfish.jersey.media</groupId>
<artifactId>jersey-media-json-jackson</artifactId>
<version>${jersey-version}</version>
</dependency>
With this, you will get the correct JSON output
{ // BookOfFriendsAnswer object
"books": { // Map<String, List<BookSummary>> books
"randomKey": [ // String (key) , List<BookSummary> (value)
{ // BookSummary object
"id": "54567bbde4b0e0ef9379993f",
"title": "Title 1",
"authors": ["Steve", "James"]
}
]
}
}
Simple Test
Resource method
#GET
#Produces(MediaType.APPLICATION_JSON)
public Response getResponse() {
BookOfFriendsAnswer books = new BookOfFriendsAnswer();
String id = "randomKey"; <===== Not sure if you want the key to be
the BookSummary id
BookSummary summary = new BookSummary();
summary.setId(id);
summary.setTitle("Title 1");
summary.getAuthors().add("Steve");
summary.getAuthors().add("James");
List<BookSummary> summaries = new ArrayList<>();
summaries.add(summary);
books.getBooks().put("randomKey", summaries);
return Response.ok(books).build();
}
Test With ObjectMapper
#Test
public void testGetIt() throws Exception {
String responseMsg = target.path("book").request().get(String.class);
ObjectMapper mapper = new ObjectMapper();
BookOfFriendsAnswer books = mapper.readValue(
responseMsg, BookOfFriendsAnswer.class);
System.out.println(books);
}
Test Without ObjectMapper - Using the automatically configured Jackson provider
#Test
public void testGetIt() throws Exception {
BookOfFriendsAnswer responseMsg
= target.path("book").request().get(BookOfFriendsAnswer.class);
System.out.println(responseMsg);
}

I think that you should create specific Map type and provide it into deserialization process:
TypeFactory typeFactory = mapper.getTypeFactory();
MapType mapType = typeFactory.constructMapType(HashMap.class, String.class, ArrayList.class);
HashMap<String, List<BookSummary>> map = mapper.readValue(json, mapType);