We Keep Coding

Batch operations in JAX-RS

Context
I am currently working on a JavaEE project with a lot of existing resource based JAX-RS services. For this project we would like to have batch processing to prevent a lot of separate calls and, most importantly, to execute these different methods in a transactional context for rollback purposes with the native MongoDB driver. We want to avoid manually creating new methods for all possible combinations. I could not find any solution to this issue on Stack Overflow so I started analyzing the implementation of RESTEasy and I came up with the following solution.
Below a simplified/pseudo version of my code:
JAX-RS method
#POST
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_JSON)
#Path("execute")
public Response executeBatch(BatchRequestWrapper batchRequestWrapper) throws UnsupportedEncodingException
{
// Retrieve information from context
HttpServletRequest httpServletRequest = ResteasyProviderFactory.getContextData(HttpServletRequest.class);
HttpServletResponse httpServletResponse = ResteasyProviderFactory.getContextData(HttpServletResponse.class);
ServletContext servletContext = ResteasyProviderFactory.getContextData(ServletContext.class);
HttpResponse httpResponse = ResteasyProviderFactory.getContextData(HttpResponse.class);
SynchronousDispatcher dispatcher = (SynchronousDispatcher) ResteasyProviderFactory.getContextData(Dispatcher.class);
ResteasyHttpHeaders httpHeaders = (ResteasyHttpHeaders) ResteasyProviderFactory.getContextData(HttpHeaders.class);
ResteasyUriInfo uriInfo = (ResteasyUriInfo) ResteasyProviderFactory.getContextData(UriInfo.class);
// Create Mongo Client Session object and save it in a Singleton which contains a ThreadLocal object so that DAO layer can reuse the client session object for all methods.
// Iterate over all the methods and invoke dispatcher
for (BatchRequest batchRequest : batchRequestWrapper.getBatchRequests())
{
// Update URI based on specific endpoint
uriInfo.setRequestUri(URI.create(batchRequest.getUri()));
// Temporary use mock response for the response
MockHttpResponse response = new MockHttpResponse();
// Create httpservletinput message from RESTEasy lib to pass to the dispatcher. It will automatically resolve all parameters/methods etc.
HttpServletInputMessage request = new HttpServletInputMessage(httpServletRequest, httpServletResponse, servletContext, httpResponse, httpHeaders, uriInfo, batchRequest.getHttpMethod(), dispatcher);
// Set body in input stream if body is specified. This will inject the correct 'body' parameters in the methods. Query and Path parameters are already resolved in the method above.
if(!Strings.isNullOrEmpty(batchRequest.getBody()))
{
InputStream targetStream = new ByteArrayInputStream(batchRequest.getBody().getBytes(StandardCharsets.UTF_8));
request.setInputStream(targetStream);
}
// Actual invoke
dispatcher.invoke(request, response);
// Do something with response object
}
// Clean or abort session based on invoke result
return Response.ok().entity(null).build();
}
Request Object
public class BatchRequestWrapper
{
private List<BatchRequest> batchRequests;
public List<BatchRequest> getBatchRequests()
{
return batchRequests;
}
public void setBatchRequests(List<BatchRequest> batchRequests)
{
this.batchRequests = batchRequests;
}
}
public class BatchRequest
{
private String uri;
private String httpMethod;
private String body;
public String getUri()
{
return uri;
}
public void setUri(String uri)
{
this.uri = uri;
}
public String getHttpMethod()
{
return httpMethod;
}
public void setHttpMethod(String httpMethod)
{
this.httpMethod = httpMethod;
}
public String getBody()
{
return body;
}
public void setBody(String body)
{
this.body = body;
}
}
My solution works with one new REST method and let's me reuse all the existing JAX-RS annotated methods in the project. Before I actually fully implement this and bring it to production, I would like to know if this is the way to actually do this or are there better alternatives? I am not a big fan of the hard dependency on RESTEasy though.

multipart/form-data MutlipartFile is not set through all args constructor

Background
The problem came up on my day job while implementing a POST multipart/form-data endpoint for a file upload with some meta information. I am not an expert in the Spring Boot ecosystem; it is likely that the problem is solved by a simple fix and that I am just missing the right term to search for.
Problem statement
To implement an endpoint for a file-upload with additional meta information, I wrote the following #RestController:
#RestController
#RequestMapping(Resource.ROOT)
#AllArgsConstructor(onConstructor = #__({#Inject}))
public class Resource {
public static final String ROOT = "/test";
private final Logger logger;
#PostMapping(consumes = MediaType.MULTIPART_FORM_DATA, produces = MediaType.APPLICATION_JSON)
public ResponseEntity<Void> test(#Valid final Request request) {
logger.info("request = {}", request);
return ResponseEntity.ok().build();
}
}
With Request being specified as:
#Value
#AllArgsConstructor
public class Request {
#NotNull
String name;
#NotNull
MultipartFile file;
}
And a small happy path test:
#SpringBootTest
#AutoConfigureMockMvc
class TestCase {
#Autowired
private MockMvc mockMvc;
#Test
void shouldReturnOk() throws Exception {
// GIVEN
final byte[] content = Files.readAllBytes(Path.of(".", "src/test/resources/PenPen.png"));
final String name = "name";
// WHEN
// #formatter:off
mockMvc.perform(MockMvcRequestBuilders
.multipart(Resource.ROOT)
.file("file", content)
.param("name", name))
// THEN
.andExpect(status().isOk());
// #formatter:on
}
}
A complete MRE can be found on Bitbucket, branch problem-with-immutable-request.
When running the test (./mvnw test), it fails with the endpoint returning a 400 BAD REQUEST instead of 200 OK. Reading the logs reveals that request parameter file is null:
...
Content type = text/plain;charset=UTF-8
Body = file: must not be null.
...
I partially understand why it is null. With this partial knowledge, I was able to circumvent the problem by making the field file in Request mutable:
#ToString
#Getter
#AllArgsConstructor
public class Request {
#NotNull
private final String name;
#Setter
#NotNull
private MultipartFile file;
}
The code "fixing" the problem can be found on Bitbucket, branch problem-solved-by-making-field-mutable.
This, however, makes the Request mutable, which I would like to prevent. To further investigate, I unrolled the lombok annotations on Request and added some logging:
public class Request {
private static final Logger LOGGER = LoggerFactory.getLogger(Request.class);
#NotNull
private final String name;
#NotNull
private MultipartFile file;
public Request(final String name, final MultipartFile file) {
this.name = name;
this.setFile(file);
}
public #NotNull String getName() {
return this.name;
}
public #NotNull MultipartFile getFile() {
return this.file;
}
public String toString() {
return "Request(name=" + this.getName() + ", file=" + this.getFile() + ")";
}
public void setFile(final MultipartFile file) {
LOGGER.info("file = {}", file);
this.file = file;
}
}
Code of unrolled version can be found on Bitbucket, branch lombok-unrolled-for-debugging.
When looking at the log statements of the now successful test, we can see that Request::setFile is called twice:
2020-09-05 09:42:31.049 INFO 11012 --- [ main] d.turing85.springboot.multipart.Request : file = null
2020-09-05 09:42:31.056 INFO 11012 --- [ main] d.turing85.springboot.multipart.Request : file = org.springframework.mock.web
The first call comes from the constructor invocation. The second call, I imagine, comes from somewhere within Spring's mapping mechanism for the form parameters.
I know that there is the possibility to define the form parameters individually on the endpoint and constructing the Request instance within the method:
public class Resource {
...
#PostMapping(consumes = MediaType.MULTIPART_FORM_DATA, produces = MediaType.APPLICATION_JSON)
public ResponseEntity<Void> test(
#RequestPart(name = "name") final String name,
#RequestPart(name = "file") final MultipartFile file) {
final Request request = new Request(name, file);
logger.info("request = {}", request);
return ResponseEntity.ok().build();
}
}
This will, however, result in other problems. For example, we would have to add an additional exception mapper for MissingServletRequestPartException and align the returned HTTP response with the existing response for BindException. I would like to avoid this if possible.
A search on the topic turned up Spring Boot controller - Upload Multipart and JSON to DTO. The solution, however, did not work for me since I do not use MVC (I think).
Question
Is there a possibility to keep Request immutable such that Spring is able to pass the MultipartFile to the all args constructor instead of setting it through the setter afterwards? Writing a custom mapper/converter is acceptable, but I did not find a possibility to write a mapper for either a specific endpoint or a specific type.

#PostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, produces = MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<Void> test(#Valid #ModelAttribute final RequestDto request) {
return ResponseEntity.ok().build();
}
It is still working with rest api call. But i really do not get immutability concern of yours.
If you define setter the multipart data you can use ModelAttribute.
#SpringBootTest
#AutoConfigureMockMvc
class FileUploadControllerIT {
#Autowired
private MockMvc mockMvc;
#Test
void shouldReturnOk() throws Exception {
// GIVEN
final byte[] content = Files.readAllBytes(Paths.get(Thread.currentThread().getContextClassLoader().getResource("text.txt").toURI()));
final String name = "name";
// WHEN
// #formatter:off
mockMvc.perform(MockMvcRequestBuilders
.multipart("/context/api/v1")
.file("multipartFile", content)
.param("name", name))
// THEN
.andExpect(status().isOk());
// #formatter:on
}
}
The above code works with ModelAttribute.
Also you are giving absolute path, i guess it is wrong. You can get file with classloader.

Spring Boot #Async annotation and MockRestServiceServer

I'm using Spring Boot 2.0.6 and Java 10. I did the following service that only hits an external rest api using RestTemplate.
#Service
#Slf4j
public class DbApiClientImpl implements DbApiClient {
private final String URL_DELIMITER = "/";
private RestTemplate restTemplate;
private String url;
public DbApiClientImpl(
RestTemplateBuilder restTemplate,
#Value("${dbapi.namespace}") String namespace,
#Value("${dbapi.url}") String uri,
#Value("${dbapi.username}") String username,
#Value("${dbapi.password}") String password) {
this.restTemplate = restTemplate.basicAuthorization(username,
password).build();
this.url = namespace.concat(uri);
}
#Override
#Async("asyncExecutor")
public Merchant fetchMerchant(String id) {
ResponseEntity<Merchant> response =
restTemplate.getForEntity(url.concat(URL_DELIMITER).concat(id),
Merchant.class);
return response.getBody();
}
}
And the following test using MockeRestServiceServer:
#RunWith(SpringRunner.class)
#RestClientTest(value = {DbApiClient.class})
public class DbApiClientTest {
private static final String TEST_NAME = "test";
private static final String TEST_NAME_BAD_REQUEST = "test-
1";
private static final String TEST_NAME_SERVER_ERROR =
"test-2";
#Autowired DbApiClient dbApiClient;
#Value("${dbapi.namespace}")
private String namespace;
#Value("${dbapi.url}")
private String dbApiUrl;
#Autowired private MockRestServiceServer mockServer;
#Autowired private ObjectMapper objectMapper;
#Test
public void test() throws
JsonProcessingException, IOException {
Merchant mockMerchantSpec = populateFakeMerchant();
String jsonResponse =
objectMapper.writeValueAsString(mockMerchantSpec);
mockServer
.expect(manyTimes(),
requestTo(dbApiUrl.concat("/").concat(TEST_NAME)))
.andExpect(method(HttpMethod.GET))
.andRespond(withSuccess(jsonResponse,
MediaType.APPLICATION_JSON));
assertNotNull(dbApiClient.fetchMerchant(TEST_NAME));
}
The thing is that I'm getting the following exception when I run the test "No further request expected HTTP GET http://localthost... excecuted"
So seems that the #Async is borking MockerServerService response...
Also, If I commented the #Async annotation everything works just fine and I get all test green.
Thanks in advance for your comments.
Update:
As per #M.Deinum's comment. I removed the CompletableFuture from the service but I'm still getting the same exception.

The problem is your code and not your test.
If you read the documentation (the JavaDoc) of AsyncExecutionInterceptor you will see the mention that only void or Future is supported as a return type. You are returning a plain object and that is internally treated as void.
A call to that method will always respond with null. As your test is running very quickly everything has been teared down already (or is in the process of being teared down) no more calls are expected to be made.
To fix, fix your method signature and return a Future<Merchant> so that you can block and wait for the result.
#Override
#Async("asyncExecutor")
public Future<Merchant> fetchMerchant(String id) {
ResponseEntity<Merchant> response =
restTemplate.getForEntity(url.concat(URL_DELIMITER).concat(id),
Merchant.class);
return CompletableFuture.completedFuture(response.getBody());
}
Now your calling code knows about the returned Future as well as the Spring Async code. Now in your test you can now call get on the returned value (maybe with a timeout to receive an error if something fails). TO inspect the result.

How to build get-query string for MockMvcRequestBuilders?

I'm using spring-boot-test with MockMvcRequestBuilders to test some GET rest webservice.
Question: is it possible to automatically translate a bean to a get-query?
Example:
#AutoConfigureMockMvc
public class WebTest {
#Autowired
protected MockMvc mvc;
#Test
public void test() {
MyRequest req = new MyRequest();
req.setFirstname("john");
req.setLastname("doe");
req.setAge(30);
mvc.perform(MockMvcRequestBuilders
.get(path)
.contentType(MediaType.APPLICATION_JSON)
.param(...) //TODO how to automatically add all params?
.andExpect(status().isOk());
}
}
public class MyRequest {
private String firstname;
private String lastname;
private int age;
}
I would need an auto translation to: ?firstname=john&lastname=doe&age=30, but in a more generic way not having to type the parameters statically.

I don't think there's anything available out-of-the-box for this specific requirement, but you can piece it together using a BeanWrapperImpl to access the properties from MyRequest and turn each into a call to param on the request builder:
MyRequest req = new MyRequest();
req.setFirstname("john");
req.setLastname("doe");
req.setAge(30);
MockHttpServletRequestBuilder requestBuilder = MockMvcRequestBuilders
.get(path).contentType(MediaType.APPLICATION_JSON);
for (PropertyDescriptor property : new BeanWrapperImpl(req).getPropertyDescriptors()) {
if (property.getWriteMethod() != null) {
requestBuilder.param(property.getName(),
property.getReadMethod().invoke(req).toString());
}
}
mvc.perform(requestBuilder).andExpect(status().isOk());

Spring REST caching RSS AbstractRssFeedView

I am developing a REST based webservice using springs to serve a RSS feed. Updates to the RSS are very rare (a couple of times a week) and hence I want to cache the RSS feed rather than building it every time someone requests for that. Here is my code. My first request after starting my webserver hits getRssFeed() method in SubscriptionEventHandler class and then goes into SubscriptionRssFeedView and calls buildFeedMetadata, buildFeedItems methods and so on which is correct. But when I make the second request, it skips getRssFeed() method in SubscriptionEventHandler BUT the buildFeedMetadata, buildFeedItems methods in SubscriptionRssFeedView gets called which in turn calls the getIncidents() and builds the RSS again from scratch. Is there a way I can avoid this and cache the RSS until I call the #CacheEvict
Here is my SubscriptionRssFeedView
#Component("subscriptionRssView")
public class SubscriptionRssFeedView extends AbstractRssFeedView
{
private String base_Url=”http://mycompany.com/”;
private final String feed_title = "My RSS Title ";
private final String feed_desc = "RSS feed desc";
private final String feed_type = "rss_2.0";
#Override
protected void buildFeedMetadata(Map<String, Object> model, Channel feed, HttpServletRequest request)
{
feed.setTitle(feed_title);
feed.setDescription(feed_desc);
feed.setLink(base_Url);
feed.setFeedType(feed_type);
super.buildFeedMetadata(model, feed, request);
}
#Override
protected List<Item> buildFeedItems(Map<String, Object> model, HttpServletRequest request,
HttpServletResponse response) throws Exception
{
List<Message> messageList = new ArrayList(Arrays.asList(getIncidents()));
List<Item> itemList = new ArrayList<Item>(messageList.size());
for (Message message : messageList)
{
itemList.add(createItem(message));
}
return itemList;
}
private Message[] getIncidents()
{
RestTemplate restTemplate = new RestTemplate();
Message[] message = restTemplate.getForObject("http://xxxxx.com/api/message", Message[].class);
return message;
}
private Item createItem(Message message)
{
Item item = new Item();
item.setLink(getFeedItemURL(message));
item.setTitle(prepareFeedItemTitle(message));
item.setDescription(createDescription(message));
item.setPubDate(getLocalizedDateTimeasDate(message.getT()));
return item;
}
}
My SubscriptionEventHandler
#Component("SubscriptionService")
public class SubscriptionEventHandler implements SubscriptionService
{
#Autowired
private SubscriptionRssFeedView subscriptionRssFeedView;
#Override
#Cacheable("rssFeedCache")
public SubscriptionRssFeedView getRssFeed()
{
return subscriptionRssFeedView;
}
}
My SubscriptionService
#Service
public interface SubscriptionService
{
SubscriptionRssFeedView getRssFeed();
}
My SubscriptionController
#Controller
#RequestMapping("/subscription")
public class SubscriptionController
{
#Autowired
private SubscriptionService subscriptionService;
#RequestMapping(value = "/rss", method = RequestMethod.GET)
public SubscriptionRssFeedView getRSS() throws Exception
{
return subscriptionService.getRssFeed();
}
}

When rendering the response of your SubscriptionController the render method of your SubscriptionRssFeedView will always get called. This method is the one triggering the calls to buildFeedMetadata, buildFeedEntries and so and so. The sequence is as follows:
AbstractView.render => AbstractFeedView.renderMergedOutputModel => SubscriptionRssFeedView.buildFeedMetadata and SubscriptionRssFeedView.buildFeedEntries
you can check the parent classes methods AbstractView.render and AbstractFeedView.renderMergedOutputModel if you want to see in more details what triggers the call to those methods.
In you want to avoid recalculating the RSS you can cache the SubscriptionRssFeedView.getIncidents() method instead of the SubscriptionEventHandler.getRssFeed()
I suggest adding a key to your cache otherwise all calls to getIncidents will return always the same value and this will be undesired when you have multiple feeds.