We Keep Coding

Spring Boot Multipart File Upload - Tips to Improve Performance

I am exposing RESTful API to the reactjs front end application which is used to upload a file to Database.
Server Side Controller Code:
#RequestMapping(value = "/api/upload", method = RequestMethod.POST, consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public UploadResponse uploadDocument(#RequestParam("doc") MultipartFile doc,
#RequestParam("metaData") String metaData, HttpServletResponse response) {
// logic to save in DB
return new UploadResponse();
}
Client Side JS Code:
uploadDocument(formData, callback) {
instance.post('/api/upload', formData)
.then((response) => {
callback(response);
})
.catch((error) => {
const errorObj = {
status: error.response.status,
data: {
message: error.response.data.message,
},
};
callback(errorObj);
});
}
application.properties
spring.http.multipart.max-file-size=20MB
spring.http.multipart.max-request-size=20MB
I am trying to upload a 20MB file (CSV or any other) , it is taking too much time to reach the controller side. (~ 1-2 minutes )
Please suggest some good techiniques or tips to improve the performance using same multipart request.
(ex: Chunking or Compressing or Streaming)

I think the easiest way would be to just zip content at javascript side and upload it to you spring boot application.
react js parts: please read upload zip file from reactjs to nodejs
spring boot multipart octet stream handling - necessary classes, test mocks etc. are described at How to go from spring mvc multipartfile into zipinputstream
Using this you should be able to zip content at react side and use it at your spring application.
Or you just zip at react side and upload the file in a normal way without any special octet stream handling in spring boot but just using java zip package classes to unzip files.

Server Side Request Forgery vulnerability

I have a RESTful service controller that requests another RESTful service
#ResponseBody
#RequestMapping(value = "/headerparameters/{instanceId}", method = RequestMethod.DELETE)
public RestContainerFormBean passivizeHeaderParameter(#PathVariable String instanceId) throws GenericException, IOException {
String url = proactiveURL + "/customerheaders/" + instanceId;
if(isSecurityCheckOK(url)){
ResponseEntity<CustomerHeaderParameterBean> response = restTemplate.exchange(url, HttpMethod.DELETE, new HttpEntity<>(new HttpHeaders()), CustomerHeaderParameterBean.class);
CustomerHeaderParameterBean result = response.getBody();
setButtonActivity(result);
l10nOfValue(result);
return new RestContainerFormBean(result);
} else{
throw new IOException();
}
}
This code can not pass SonarQube policy.
Refactor this code to not construct the URL from tainted,
User provided data, such as URL parameters, POST data payloads, or
cookies, should always be considered untrusted and tainted. A remote
server making requests to URLs based on tainted data could enable
attackers to make arbitrary requests to the internal network or to the
local file system.
The problem could be mitigated in any of the following ways:
Validate the user provided data based on a whitelist and reject input
not matching. Redesign the application to not send requests based on
user provided data.
How can I pass the policy by sticking on REST conventions ?

Use UriComponentsBuilder to encode the URL instead of using raw URL.

Spring Boot serving an m3u8 playlist

I'm trying to serve an m3u8 playlist through Spring Boot. I have a running ffmpeg process that is transcoding a multicast in real-time and sending the files to /src/resources/public/output.m3u8. I see the playlist updating and the new .ts files being generated correctly however when trying to watch the stream in a video player, it only plays a certain amount of video. Is there a way to properly serve up a running playlist in Java instead of serving it statically?
EDIT: When starting a basic http server with python python3 -m http.server, I'm able to view the stream perfectly fine. Is there a Spring Boot way to accomplish the same task?

With Spring 4.1 your approach will work there is no issue in it. Here below is another approach in case if you want to look
#RequestMapping(value = "/VMS-49001/playlist/{listName:.+}")
public ResponseEntity<byte[]> testphoto() throws IOException {
InputStream in = servletContext.getResourceAsStream("/images/no_image.jpg");
final HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.parseMediaType("application/vnd.apple.mpegurl"));
headers.setContentDispositionFormData(fileName, fileName);
return new ResponseEntity<byte[]>(IOUtils.toByteArray(in), headers, HttpStatus.CREATED);
}

Stream file download through RestTemplate

I have a large file download that is served by a RestController on one server, that I need to stream through a RestController on another server. When calling the end server directly the result streams fine. However when using RestTemplate to call this server and then write the response to an OutputStream, the response is buffered on the front server until the whole file is ready, and then streamed. Is there a way I can write the file to an OutputStream as it comes in?
At the moment my code on the front server looks similar to this
#ResponseBody
public void downloadResults(HttpServletRequest request, HttpServletResponse response, #RequestParam("id") String jobId, OutputStream stream)
throws IOException
{
byte[] data = restTemplate.exchange("http://localhost/getFile", HttpMethod.POST, requestEntity, byte[].class, parameters).getBody();
stream.write(data);
}
I've set my RestTemplate to not buffer and I've verified that this is working by checking the Request type that is used, (SimpleStreamingClientHttpRequest).
The data all comes back correct, its just only written to the stream all at once, rather than as it comes in

RestTemplate is not meant for streaming the response body, as pointed out in this JIRA issue.

You can use restTemplate.execute. See https://www.baeldung.com/spring-resttemplate-download-large-file

Jersey servlet returns zip file that contains more bytes than response sent

I'm trying to implement a simple servlet that returns a zip file that is bundled inside the application (simple resource)
So I've implemented the following method in the server side:
#GET
#Path("{path}/{zipfile}")
#Produces("application/zip")
public Response getZipFile(
#PathParam("path") String pathFolder,
#PathParam("zipfile") String zipFile) IOException {
String fullPath= String.format("/WEB-INF/repository/%s/%s",
pathFolder, zipFile);
String realPath = ServletContextHolder.INSTANCE.getServletContext()
.getRealPath(fullPath);
File file = new File(realPath );
ResponseBuilder response = Response.ok((Object) file);
return response.build();
}
When I call this method from the borwser, the zip file is downloaded and its size is the same number of bytes as the original zip in the server.
However, when I call this using a simple XMLHttpRequest from my client side code:
var oXHR = new XMLHttpRequest();
var sUrl = "http://localhost:8080/path/file.zip"
oXHR.open('GET', sUrl);
oXHR.responseType = 'application/zip';
oXHR.send();
I can see in the Network tab of the Developer tools in chrome that the content size is bigger, and I'm unable to process this zip file (for instance JSzip doesn't recognize it).
It seems like somewhere between my response and the final response from org.glassfish.jersey.servlet.ServletContainer, some extra bytes are written/ some encoding is done on the file.
Can you please assist?
Best Regards,
Maxim

When you use an ajax request, the browser expects text (by default) and will try to decode it from UTF-8 (corrupting your data).
Try with oXHR.responseType = "arraybuffer"; : that way, the browser won't change the data and give you the raw content (which will be in oXHR.response).
This solution won't work in IE 6-9 : if you need to support it, check JSZip documentation : http://stuk.github.io/jszip/documentation/howto/read_zip.html
If it's not the right solution, try downloading directly the zip file (without any js code involved) to check if the issue comes from the js side or from the java side.