We Keep Coding

How to wrap an external API inside a custom API with Spring Boot

Let's say I have a third party rest api ("https://example.com/write") that allows POST requests with the following body structure:
{
"id": "abc",
"Config": {"Comments":"text"}
}
I am completely new to Java and the Spring Framework, but I want to create a custom API with Spring that only allow users to change the text part of the body. Other parts of the JSON body should have a fixed value (for example id is always "abc"). Basically, when user input a custom text string, my api will compile the input and consume the external api and get the results from it accordingly
I understand the basics of #Getmapping / #RequestMapping after doing some research. Here is what I have so far for my custom API, and I am stuck at the post mapping section.
#RestController
#RequestMapping("/api")
public class ApiController {
#Autowired
private Environment env;
// GET
#RequestMapping(value = "/retrive", method = { RequestMethod.GET })
public ResponseEntity<?> retrive (HttpServletRequest request, HttpServletResponse response) throws IOException {
// URL
URL u = new URL("https://example.com/get");
HttpURLConnection uc = (HttpURLConnection) u.openConnection();
// Create HttpHeaders for ResponseEntity
HttpHeaders responseHeaders = new HttpHeaders();
uc.setRequestProperty ("Authentication", "Bearer "+ env.getProperty("api-key"));
try (InputStream inputStream = uc.getInputStream();
OutputStream outputStream = response.getOutputStream();
)
{IOUtils.copy(inputStream, outputStream);
}
return new ResponseEntity<>(responseHeaders, HttpStatus.OK);
}
// POST
#RequestMapping(value = "/write", method = { RequestMethod.POST },
consumes = {MediaType.APPLICATION_JSON_VALUE})
public ResponseEntity process(#RequestBody Root input) throws IOException {
// Operation goes here...
return new ResponseEntity<>(input, HttpStatus.OK);
}
public class Root{
private String Comments;
// Getters and Setters
}

Create a Custom DTO class which will represent the response from Third party API. What It means is it should have all fields which should map with corresponding required field of response payload of third party API. It will be populated on de-serializing the third party API response. You can take help of feign client here. Its very easy and declarative way to make REST API call.
Create a separate response DTO with 2 fields 1 which will have the static value and second the DTO which we created above (which will be populated with the response of third party API). This DTO will be the response from POST "/write"

Java spring file download proxy with rest calls

I need to create a rest service in java which will in turn connect to another rest service for file download. For now, I just need to transfer the file from the other backend to client but in future some processing/transformations would be done.
For all the web services in my project, we are using spring rest (for providing as well as consuming the services).
My question is what would be the appropriate way of doing it considering that the files would be large and I don't want to run into OutOfMemory errors.
People in some other posts have suggested to use streams on both the ends but is that really possible? For this, do I need to write the file on disk first?
My current code for file download (consumer) -
public BackendResponse<byte[]> callBackendForFile(BackendRequest request) {
String body = null;
ResponseEntity<byte[]> responseEntity = null;
URI uri = createURI(request);
MultiValueMap<String, String> requestHeaders = getHeadersInfo(request.getHttpRequest());
if (HttpMethod.GET.equals(request.getMethod())) {
responseEntity = restTemplate.exchange(uri, request.getMethod(),
new HttpEntity<String>(body, requestHeaders), byte[].class);
} else {
LOG.error("Method:{} not supported yet", request.getMethod());
}
BackendResponse<byte[]> response = new BackendResponse<>();
response.setResponse(responseEntity);
return response;
}
My client code (provider):
#RequestMapping(value = "/file", method = RequestMethod.GET, produces = "application/xml")
#ResponseBody
public void downloadFileWithoutSpring(HttpMethod method, HttpServletRequest httpRequest,
HttpServletResponse httpResponse) {
BackendRequest request = new BackendRequest(method,
httpRequest.getRequestURI(), httpRequest.getQueryString(), httpRequest);
BackendResponse<byte[]> backendResponse = dutyplanService.getFile(request);
ResponseEntity<byte[]> response = backendResponse.getResponse();
httpResponse.addHeader("Content-Disposition", "attachment; filename=\"" + "attachment.zip" + "\"");
httpResponse.getOutputStream().write(response.getBody());
httpResponse.flushBuffer();
}
Note: The code above doesn't work somehow as the attachment downloaded is a corrupt file

I don't think you will need to create that file on server as long as you are having the bytearray content of it received from another server.
You can try changing value of produces annotation to the value application/zip (or application/octet-stream, depending on the target browser) instead of 'application/xml'

you can pass HttpServletResponse#getOutputStream() directly in restTemplate and write it without save file in server.
public void getFile(HttpServletResponse response) throws IOException {
restTemplate.execute(
"http://ip:port/temp.csv",
HttpMethod.GET,
null,
clientHttpResponse -> {
StreamUtils.copy(clientHttpResponse.getBody(), response.getOutputStream());
return null;
}
);
}
note that after call getFile(), you should close outputStream like this
response.getOutputStream().close()

HTTP 406 downloading a file with rest call

I followed this tutorial to implement rest API with Spring Boot for downloading files (xml format).
My controller class is as follows:
#RestController
public class RistoreController {
#Autowired
private RistoreService ristoreService;
#RequestMapping(
value = "/ristore/foundation/{trf}",
method = RequestMethod.GET,
produces = "application/xml")
public ResponseEntity<InputStream> getXMLById(#PathVariable("trf") String trf) throws IOException {
InputStream inputStream = ristoreService.findByTRF(trf);
return ResponseEntity
.ok()
.contentType(MediaType.parseMediaType("application/octet-stream"))
.body(inputStream);
}
}
I have service interface RistoreService autowired in the controller and Bean class for that service looks like this:
#Service
public class RistoreServiceBean implements RistoreService {
public InputStream findByTRF(String trf) throws IOException {
String filePath = "/Users/djiao/Box Sync/Work/Projects/RIStore/foundation/foundation_new/" + trf + ".xml";
File file = new File(filePath);
return new FileInputStream(file);
}
}
I tested the application using the following curl command:
curl -i -H "Accept: application/xml" http://localhost:8080/ristore/foundation/TRF133672_1455294493597
However, I got 406 error, "Not Acceptable". Something wrong with the file format?

Try to change the definition of the controller that way
#RequestMapping(value = "/ristore/foundation/{trf}", method = RequestMethod.GET, produces = "application/xml")
public ResponseEntity<InputStreamResource> downloadXMLFile(#PathVariable("trf") String trf)
throws IOException {
// Optinal headers configuration
HttpHeaders headers = new HttpHeaders();
headers.add("Cache-Control", "no-cache, no-store, must-revalidate");
headers.add("Pragma", "no-cache");
headers.add("Expires", "0");
// get the inputStream
InputStream xmlFileInputStream = ristoreService.findByTRF(trf);
return ResponseEntity
.ok()
.headers(headers)
.contentType(MediaType.parseMediaType("application/octet-stream"))
.body(new InputStreamResource(xmlFileInputStream));
}
Then your service class would be :
#Service
public class RistoreServiceBean implements RistoreService {
public InputStream findByTRF(String trf) throws IOException {
String filePath = "/Users/djiao/Box Sync/Work/Projects/RIStore/foundation/foundation_new/" + trf + ".xml";
File file = new File(filePath);
return new FileInputStream(file);
}
}
406 Not Acceptable
The resource identified by the request is only capable of generating response entities which have content characteristics not acceptable according to the accept headers sent in the request.
That means that the inputstream you return must be considered as a resource as soon as you have a REST controller.

The following two lines in your code contradict each other:
.contentType(MediaType.parseMediaType("application/octet-stream"))
and
produces = "application/xml")

Sending file between systems A->B->C without storing whole file in B

I have 3 separate spring web applications
A uses spring 4.x
B uses spring 3.2.0
C uses spring 4.x
B and C exposes REST controllers for uploading files
A reads file and uploads it to B
B sends the request to C without any need to read file content
and then C does whatever it wants with the file.
So the flow would be A->B->C
My question is - is it possible to setup B in such a way so that B wouldn't store whole file in the memory, but would read incoming stream and forward it to C?
What I managed to do is:
A
public void sendFileFromA() throws FileNotFoundException {
final InputStream fis = new FileInputStream(new File("someFile"));
final RequestCallback requestCallback = new RequestCallback() {
#Override
public void doWithRequest(final ClientHttpRequest request) throws IOException {
request.getHeaders().add("Content-type", "application/octet-stream");
IOUtils.copy(fis, request.getBody());
}
};
final RestTemplate restTemplate = new RestTemplate();
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
requestFactory.setBufferRequestBody(false);
restTemplate.setRequestFactory(requestFactory);
final HttpMessageConverterExtractor<String> responseExtractor = new HttpMessageConverterExtractor<>(
String.class, restTemplate.getMessageConverters());
restTemplate.execute("http://b_url/upload", HttpMethod.POST, requestCallback, responseExtractor);
}
B
#RequestMapping(value = "/upload", method = RequestMethod.POST)
public #ResponseBody String handleFileUpload(HttpServletRequest request) throws IOException {
final ServletInputStream input = request.getInputStream();
final RequestCallback requestCallback = new RequestCallback() {
#Override
public void doWithRequest(final ClientHttpRequest request) throws IOException {
request.getHeaders().add("Content-type", "application/octet-stream");
try (OutputStream body = request.getBody()) {
IOUtils.copy(input, body);
}
}
};
final RestTemplate restTemplate = new RestTemplate();
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
requestFactory.setBufferRequestBody(false);
restTemplate.setRequestFactory(requestFactory);
final HttpMessageConverterExtractor<String> responseExtractor = new HttpMessageConverterExtractor<>(
String.class, restTemplate.getMessageConverters());
restTemplate.execute("http://c_url/upload", HttpMethod.POST, requestCallback, responseExtractor);
return "success";
}
C
#RequestMapping(value = "/upload", method = RequestMethod.POST)
public #ResponseBody String handleFileUpload(HttpServletRequest request) throws IOException {
ServletInputStream input = request.getInputStream();
try (BufferedOutputStream output = new BufferedOutputStream(new FileOutputStream("zibiTest"))) {
IOUtils.copy(input, output);
}
return "success";
}
I can easily copy files over >10GB from A to C using B.
With such a solution we can try to stop A while transferring, B and C should be notified about the error, but sometimes it happens that the error message doesn't reach C - it gets closed with socket timeout exception, any idea why this happens and how to implement it properly?
Is this a valid approach or can it be handled better?

I would try setting smaller socket timeout on C than you have on B. Currently it seems that both have some default value, so if A hangs, both B and C will stop getting data almost the same time. Both start timing out, and maybe it is a race condition, where it depends on the timeout accuracy which one times out first.

REST service sending corrupted file

I'm trying to create a java REST service that will download a word doc. The file downloads but the contents are just garbage hex, not the actual Word doc contents. My sample code is below. What am I missing? The before & after files have the same amount of bytes.
#SuppressWarnings("resource")
#RequestMapping(value = "get/testdoc", method=RequestMethod.GET, produces="application/octet-stream)
public #ResponseBody ResponseEntity<byte[]> getTestDoc() throws Throwable{
File doc = new File("C:\\temp\\file.doc");
InputStream is = new FileInputStream(doc);
byte[] bytes = IOUtils.toByteArray(is);
HttpHeaders responseHeaders = new HttpHeaders();
responseHeaders.setContentType(MediaType.APPLICATION_OCTET_STREAM);
responseHeaders.set("Content-Disposition" , "Attachment; filename=file.doc");
responseHeaders.setContentLength(ProposalDoc.length());
return new ResponseEntity<byte[]>(bytes, responseHeaders, HttpStatus.OK);
}

I think there are two problems:
1. The Length Header:
I my opinion there is at least one very strange line:
responseHeaders.setContentLength(ProposalDoc.length());
I think, it should be:
responseHeaders.setContentLength(bytes.length);
2. #ResponseBody Annotation
If you use return type ResponseEntity<byte[]>, then you must NOT add #ResponseBody.
#RequestMapping(value = "get/testdoc", method=RequestMethod.GET)
public ResponseEntity<byte[]> getTestDoc() throws Throwable{
...
}

try to replace produces="application/octet-stream")
with produces="application/vnd.ms-word")

Thanks for all the help. I ended up bypassing Spring & attaching the file to the response, as listed in the code below. I'm suspecting that sprint was converting the bytes somehow behind the scenes. I looked into configuring the ByteArrayHttpMessageConverter, but that didn't seem to help. This is good enough for me, for now.
#SuppressWarnings("resource")
#RequestMapping(value = "get/doc", method=RequestMethod.GET, produces="application/octet-stream")
public HttpEntity getProposalDocs(HttpServletResponse response) throws Throwable{
File doc = new File("C:\\temp\\file.doc");
InputStream is = new FileInputStream(doc);
response.setHeader("Content-Disposition", "attachment;filename=\"test.doc\"");
response.setHeader("Content-Type", "application/octet-stream;");
StreamUtils.copy(is ,response.getOutputStream());
return new ResponseEntity(HttpStatus.OK);
}

Check this code also, it works fine with me.
#RequestMapping(value = "/get/doc" , method = RequestMethod.GET ,
produces = "application/msword")
public ResponseEntity<InputStreamResource> getProposalDocs() throws IOException{
ClassPathResource docfile = new ClassPathResource("file.doc");
HttpHeaders headers
= new HttpHeaders();
headers.add("Cache-Control", "no-cache, no-store, must-revalidate");
headers.add("Pragma", "no-cache");
headers.add("Expires", "0");
return ResponseEntity.ok()
.headers(headers)
.contentLength(docfile.contentLength())
.contentType(MediaType.parseMediaType("application/msword"))
.body(new InputStreamResource(docfile.getInputStream()));
}
EDITED: the idea that worked with me to return InputStreamResource instead of byte[].
Also specify the content type as produces="application/octet-stream".
This works fine with me without needed to bypass servlet response..