We Keep Coding

How to upload file with utf-8 symbols in name on jira task as attachment using spring boot

I am trying to upload file on jira task as attachment.
for example I am trying to upload file with the name "test-äöü.txt" and it appears on jira like "test-???.txt"
Here is my original code:
RestTemplate restTemplate = restTemplateFactory.getInstance(RestTemplateUtil.getRequestFactory(proxyInfo,useEncryption));
File file = new File(filepath,filename);
MultiValueMap<String, Object> multipartMap = new LinkedMultiValueMap<>();
FileSystemResource var = new FileSystemResource(file);
multipartMap.add("file", var);
HttpEntity<MultiValueMap<String, Object>> request = new HttpEntity<MultiValueMap<String, Object>>(multipartMap,
RestTemplateUtil.buildHttpHeadersForUploadAttachment(jiraUser, jiraPass,useEncryption)
ResponseEntity<String> result = restTemplate.exchange( jiraAdressAttachFile, HttpMethod.POST, request, String.class);
I am setting "Authorization" , "X-Atlassian-Token" headers in buildHttpHeadersForUploadAttachment metgod and "Content-Type" with MediaType.MULTIPART_FORM_DATA value
Like I said file goes on jira with name "test-???.txt", than I searched some solutions and tried them but nothing worked for me.
Here is the solutions I tried to solve my problem:
restTemplate.getMessageConverters().stream()
.filter(FormHttpMessageConverter.class::isInstance)
.map(FormHttpMessageConverter.class::cast)
.findFirst()
.orElseThrow(() -> new IllegalStateException("Failed to find FormHttpMessageConverter"))
.setMultipartCharset(StandardCharsets.UTF_8);
Did not work
restTemplate.getMessageConverters().stream()
.filter(AllEncompassingFormHttpMessageConverter.class::isInstance)
.map(AllEncompassingFormHttpMessageConverter.class::cast)
.findFirst()
.orElseThrow(() -> new IllegalStateException("Failed to find AllEncompassingFormHttpMessageConverter"))
.setMultipartCharset(StandardCharsets.UTF_8);
Did not work
restTemplate.getMessageConverters().add(0, new FormHttpMessageConverter() {
#Override
protected String getFilename(Object part) {
if (part instanceof Resource) {
Resource resource = (Resource) part;
try {
return new String(resource.getFilename().getBytes(StandardCharsets.UTF_8));
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
return null;
}
} else {
return null;
}
}
});
In this example I tried every possible value of StandardCharsets but again nothing worked
I tried this one too
FormHttpMessageConverter converter = new FormHttpMessageConverter();
converter.setMultipartCharset(Charset.forName("UTF-8"));
restTemplate.getMessageConverters().add(0, converter);
and this
List<HttpMessageConverter<?>> messageConverters = restTemplate.getMessageConverters();
StringHttpMessageConverter stringMessageConverter = new StringHttpMessageConverter(Charset.forName("UTF-8"));
ByteArrayHttpMessageConverter byteArrayHttpMessageConverter = new ByteArrayHttpMessageConverter();
byteArrayHttpMessageConverter.setDefaultCharset(Charset.forName("UTF-8"));
ResourceHttpMessageConverter resourceHttpMessageConverter = new ResourceHttpMessageConverter();
resourceHttpMessageConverter.setDefaultCharset(Charset.forName("UTF-8"));
FormHttpMessageConverter formHttpMessageConverter = new FormHttpMessageConverter();
formHttpMessageConverter.setMultipartCharset(Charset.forName("UTF-8"));
AllEncompassingFormHttpMessageConverter allEncompassingConverter = new AllEncompassingFormHttpMessageConverter();
allEncompassingConverter.setCharset(Charset.forName("UTF-8"));
allEncompassingConverter.setMultipartCharset(Charset.forName("UTF-8"));
allEncompassingConverter.setPartConverters(Arrays.asList(stringMessageConverter,byteArrayHttpMessageConverter,resourceHttpMessageConverter));
for (Iterator<HttpMessageConverter<?>> iterator = messageConverters.iterator(); iterator.hasNext(); ) {
HttpMessageConverter conv = iterator.next();
if (conv instanceof AllEncompassingFormHttpMessageConverter) {
iterator.remove();
}
}
messageConverters.add(allEncompassingConverter);
restTemplate.setMessageConverters(messageConverters);
again bad result.
Am I doing something wrong in my original code or in a possible solutions I tried and described?
or is there something I did not tried and I must do to work my code properly ?
Before posting request, in File, FileSystemResource and MultiValueMap<String, Object> objects this file has correct name like "test-äöü.txt"
update:
issue is with spring boot version less than 2.0

415 Unsupported Media Type while sending json file over REST Template

I am trying to send a json file over REST Template. When I send it via POST man as MULTIPART_FORM_DATA, it works fine. The name I am supposed to give is specific (lets say aaa). Attached screenshot of POSTMAN. But when I try same in code as specified in another stackoverflow post, I get 415 Unsupported Media Type error as
org.springframework.web.client.HttpClientErrorException: 415 Unsupported Media Type
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:91) ~[spring-web-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:616) ~[spring-web-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:572) ~[spring-web-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:532) ~[spring-web-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.web.client.RestTemplate.postForObject(RestTemplate.java:332) ~[spring-web-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at
Please do not mark it as duplicate as the specified answer did not work for me. Not sharing code as my code is exactly same as this except
requestParamerterMap.add("attachment", resource);
where as my code is
requestParamerterMap.add("aaa", resource);
After debugging it from the server side, looks like request is reaching out to server. I was able to see below error in the server side:
[{error=Unsupported Media Type, exception=org.springframework.web.HttpMediaTypeNotSupportedException, message=Content type 'application/octet-stream' not supported, status=415, timestamp=1532557180124}] as "application/json" using [org.springframework.http.converter.json.MappingJackson2HttpMessageConverter#74d4827a]
So, from the server side logs, I am not sure where the content type is getting added as application/octet-stream as I have set the content type as
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
Below is the code from server controller. Server side code uses Spring boot.
#RequestMapping(method = RequestMethod.POST, produces = MediaType.APPLICATION_JSON_VALUE,consumes = {"multipart/form-data"})
#ResponseBody
public MyResponse uploadPhoto(#RequestPart(value = "aaa", required = false) Optional<MyRequest> myRequest,
#RequestPart(value = "file", required = false) Optional<MultipartFile> file,
HttpServletRequest request) {
//some logic
return myResponse;
}
The server code has an interceptor where I can see my request has content type as multipart/form-data. It does not reach to RestController
When I debugged the server side code in 2 cases:
POSTMAN request
client code request
One thing I figured out that file iteam has content type as application/json when I post from POSTMAN and the content type was application/octet-stream when the request goes from client side code.
In my client side code, I am creating JSONObject as
JSONObject json = new JSONObject();
json.append("myKey", "myValue");
and convert it to byte array as
json.toString().getBytes("UTF-8")
then I have followed this . The difference in my code is, I am sending my JSONObject as byte stream as I can not create file (performance issues).
And I cant not send JSONObject as string as server is expecting multipart-form-data for both file and aaa
I have created the restTemplate as
public RestTemplate myRestTemplate() {
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
requestFactory.setReadTimeout(HTTP_CLIENT_TIMEOUT);
requestFactory.setConnectTimeout(HTTP_CLIENT_TIMEOUT);
RestTemplate restTemplate = new RestTemplate(requestFactory);
List<HttpMessageConverter<?>> messageConverters = new ArrayList<HttpMessageConverter<?>>();
messageConverters.add(new FormHttpMessageConverter());
messageConverters.add(new StringHttpMessageConverter());
restTemplate.setMessageConverters(messageConverters);
return restTemplate;
Here is the client side code which calls the service:
public Optional<JSONObject> callService(byte[] multipartFile) {
MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
InputStream stream = new ByteArrayInputStream(multipartFile);
MultipartByteArrayResource resource = new MultipartByteArrayResource(multipartFile,fileName);
body.add("aaa", resource);
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<>(body, headers);
try {
response = restTemplate.postForObject(url, requestEntity , String.class);
} catch (Exception exception) {
LOG.error("Error", exception);
return Optional.empty();
}
}
public class MultipartInputStreamFileResource extends InputStreamResource {
private final String filename;
MultipartInputStreamFileResource(InputStream inputStream, String filename) {
super(inputStream);
this.filename = filename;
}
#Override
public String getFilename() {
return this.filename;
}
#Override
public long contentLength() throws IOException {
return -1; // we do not want to generally read the whole stream into memory ...
}
}
And same code works when I send file (note file and aaa are two different things though both are multipart/form-data in server side. file is just a file of any time (image/text/pdf) but aaa is json data file)
After debugging little bit more, what I observed is server side controller is expecting the file content to be json as Jackson try to deserialize that json to MyRequest object. When I send post from POSTMAN, it has the json content so working as expected but from the client side code, the content is byteArray, and its not getting deserialize to MyRequest object. Not sure how to fix this

Finally I solved this issue. As mentioned in question, having different content type of multipart file while sending request from POSTMAN vs code is where I began with. I will explain in details if anyone has any questions.
public Optional<JSONObject> save(byte[] multipartFile, String fileName) {
MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
Resource content = new MultipartByteArrayResource(multipartFile , fileName);
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
HttpEntity<Resource> requestEntityBody = new HttpEntity<Resource>(content, headers);
body.add("aaa", requestEntityBody);
String result = "";
JSONParser parser = new JSONParser();
JSONObject json = null;
HttpHeaders requestHeaders = new HttpHeaders();
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<>(body, requestHeaders);
ResponseEntity<String> response = null;
try {
RestTemplate restTemplate = customizeRestTemplate(); //I have defined this in different config file in my actual code
response = restTemplate.exchange(url , HttpMethod.POST , requestEntity , String.class);
result = (response != null && response.getBody() != null) ? response.getBody().toString() : result;
json = (JSONObject) parser.parse(result);
LOG.info( "Response:", response );
} catch (Exception exception) {
LOG.error("Error , exception);
return Optional.empty();
}
return Optional.ofNullable(json);
}
public class MultipartByteArrayResource extends ByteArrayResource{
private String fileName;
public MultipartByteArrayResource(byte[] byteArray , String filename) {
super(byteArray);
this.fileName = filename;
}
public String getFilename() {
return fileName;
}
public void setFilename(String fileName) {
this.fileName= fileName;
}
}
public RestTemplate customizeRestTemplate() {
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
requestFactory.setReadTimeout(10000);
requestFactory.setConnectTimeout(10000);
RestTemplate restTemplate = new RestTemplate(requestFactory);
List<HttpMessageConverter<?>> messageConverters = new ArrayList<HttpMessageConverter<?>>();
messageConverters.add(new FormHttpMessageConverter());
messageConverters.add(new StringHttpMessageConverter());
restTemplate.setMessageConverters(messageConverters);
return restTemplate;
}
}

The server-side exception is produced by org.springframework.http.converter.json.MappingJackson2HttpMessageConverter. Jackson is a JSON library and MessageConverter are used by Spring to format requests and responses.
Can it be that the client sends an "Accept: application/octet-stream" while the server has a #Produces(APPLICATION_JSON) annotation? That would mean that the server processes the request and only has problems sending the response. You could add some log.info() statements in the server to verify this.

Spring controller image download in byte[] instead base64

I encountered a problem with downloading grid fs stored image via spring controller. When trying to open the downloaded file the image viewer says that it is corrupted, it turns out that the image is in base64 format.
There is the controller part:
#Override
#RequestMapping(value = "/image_download", method = RequestMethod.GET)
public ResponseEntity<byte[]> downloadImage(...) throws IOException {
final GridFSDBFile image = getImageFromGrifFs(...);
final HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.valueOf(image.getContentType()));
headers.setContentDispositionFormData("attachment", image.getFileName());
final byte[] content = IOUtils.toByteArray(image.getInputStream());
return new ResponseEntity<>(content, headers, HttpStatus.OK);
}
Spring version is 4.3.11.
And here are the message converters:
#Override
public void configureMessageConverters(final List<HttpMessageConverter<?>> converters) {
converters.add(new MappingJackson2HttpMessageConverter());
converters.add(byteArrayHttpMessageConverter());
super.configureMessageConverters(converters);
}
#Bean
public ByteArrayHttpMessageConverter byteArrayHttpMessageConverter() {
final ByteArrayHttpMessageConverter arrayHttpMessageConverter = new ByteArrayHttpMessageConverter();
arrayHttpMessageConverter.setSupportedMediaTypes(getSupportedMediaTypes());
return arrayHttpMessageConverter;
}
private List<MediaType> getSupportedMediaTypes() {
final List<MediaType> list = new ArrayList<>();
list.add(MediaType.IMAGE_JPEG);
list.add(MediaType.IMAGE_PNG);
list.add(MediaType.IMAGE_GIF);
list.add(MediaType.APPLICATION_OCTET_STREAM);
return list;
}
I also tried using InputStreamResource the following way in the controller:
return ResponseEntity.ok()
.contentLength(image.getLength())
.contentType(MediaType.parseMediaType(image.getContentType()))
.body(new InputStreamResource(image.getInputStream()));
But got the exception:
Could not write content: No serializer found for class com.mongodb.gridfs.GridFSDBFile$MyInputStream
Any help appreciated. Thank you.

After I'd done some more digging I found good explanation: https://stackoverflow.com/a/44943494/2421204
And indeed adding (produces = "image/jpeg") to RequestMapping solved the issue.
#RequestMapping(value = "/image_download", method = RequestMethod.GET, produces = "image/jpeg")
The images that are downloaded are in binary.

How to send large file using Spring REST multipart in chunks (REST client)

I am using spring REST to write a client which will upload a file to DB.
Following is the server side controller code which I can not change :
#RequestMapping(method = RequestMethod.POST)
public ResponseEntity<UploadResponseDto> uploadFile(#RequestParam("file") MultipartFile file) throws IOException {
String contentType = file.getContentType();
if ( contentType == null || !contentType.equalsIgnoreCase(APPLICATION_OCTET_STREAM)) {
contentType = APPLICATION_OCTET_STREAM;
}
GridFSFile gridFSFile = gridFsTemplate.store(file.getInputStream(), file.getOriginalFilename(), contentType);
MultiValueMap<String, String> headers = new LinkedMultiValueMap<>();
String fileLocation = linkTo(FileAttachmentController.class).slash(gridFSFile.getId()).toUri().toString();
headers.add(LOCATION, fileLocation);
UploadResponseDto uploadResponseDto = new UploadResponseDto(file.getOriginalFilename(), fileLocation);
return new ResponseEntity<>(uploadResponseDto, headers, HttpStatus.CREATED);
}
And my client side code for sending file is :
SimpleClientHttpRequestFactory factory = new SimpleClientHttpRequestFactory();
factory.setBufferRequestBody(false);
RestTemplate restTemplate = new RestTemplate(factory);
HttpHeaders headers = new HttpHeaders();
headers.set(HttpHeaders.AUTHORIZATION, "Bearer " + token);
headers.set("Accept", "application/json");
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
File file = new File(fileToUpload);
MultiValueMap<String, Object> data = new LinkedMultiValueMap<String, Object>();
ByteArrayResource resource = new ByteArrayResource(
Files.readAllBytes(Paths.get(fileToUpload))) {
#Override
public String getFilename() {
return file.getName();
}
};
data.add("file", resource);
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<MultiValueMap<String, Object>>(
data, headers);
ResponseEntity<Map> apiResponse = null;
apiResponse = restTemplate.exchange(
"http://{end_point_url}",
HttpMethod.POST, requestEntity, Map.class);
But when I use this code to send lets say 50 MB file, it throws "413 Request entity too large error"
Can somebody please help me out on how to send a large file in chunks?
Thanks & Regards,
Vikas Gite

You can specify a size of the upload file by using
org.springframework.web.multipart.commons.CommonsMultipartResolver
#Bean(name = "multipartResolver")
public CommonsMultipartResolver multipartResolver() {
CommonsMultipartResolver multipartResolver = new CommonsMultipartResolver();
multipartResolver.setMaxUploadSize(54525952); //...specify your size of file (20971520 - 20 MB) (54525952 - 52 MB)
return multipartResolver;
}

Update
Okay so you have set multipartMaxFileSize, but along with this you also need to set max request size if you have a single file that's greater than 10MB
Seems you are using Spring 4.x
So config goes like
spring.http.multipart.maxFileSize
spring.http.multipart.maxRequestSize
Official Source
Depricated:
By default SimpleClientHttpRequestFactory buffers the request body internally.
Make it false
SimpleClientHttpRequestFactory factory = new SimpleClientHttpRequestFactory();
factory.setBufferRequestBody(false);
Source

Sending file over spring rest service via resttemplate

Title might look common but none of them fit in my issue.
I have a rest service which accept normal parameters and file in form of multipart.
i want to use resttemplate to send data and file to above rest service.
till the time i was sending normal string data there was no issue. once i add code of sending bytes then i start getting 400 Bad request error.
if i comment code to send ByteArrayResource then it start working for normal parameters.
below is sample code
Rest service controller
#RestController
#RequestMapping(value="/ticket")
public class UserTicketController {
#RequestMapping(value="/createTicket.do",method={RequestMethod.POST},
consumes = {MediaType.MULTIPART_FORM_DATA_VALUE},headers={"content-type="+MediaType.MULTIPART_FORM_DATA_VALUE})
public void createTicket(#ModelAttribute ServiceDeskRequest serviceDeskRequest, HttpServletRequest request,HttpServletResponse response) throws Exception{
}
}
Servicedeskrequest model attribute is
public class ServiceDeskRequest implements Serializable{
private String jsonData;
private MultipartFile attachment;
}
application-context.xml
<bean id="multipartResolver" class="org.springframework.web.multipart.commons.CommonsMultipartResolver">
</bean>
Client Side code
RestTemplate restTemplate = new RestTemplate();
MultiValueMap<String, Object> requestParamerterMap = new LinkedMultiValueMap<String, Object>();
requestParamerterMap.add("jsonData", jsonData);
MultipartFile attachment = userRequest.getAttachment();
if(attachment!=null && attachment.getOriginalFilename()!=null) {
ByteArrayResource byteArrayResource = new ByteArrayResource(attachment.getBytes(), attachment.getOriginalFilename());
requestParamerterMap.add("attachment", byteArrayResource);
}
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<MultiValueMap<String, Object>>(requestParamerterMap, headers);
String response = restTemplate.postForObject(targetUrl, requestEntity, String.class);

I figured it out. There are two piece in this puzzle. No change in service code.
Providing right converter to resttemplate. In list of default converts spring doesn't add FormHttpMessageConverter.
FormHttpMessageConverter converter = new FormHttpMessageConverter();
RestTemplate restTemplate = new RestTemplate();
restTemplate.getMessageConverters().add(converter);
overriding bytearrayresource class. plz note you need to override getFilename method so that document name can be received at service side.
public class MultipartByteArrayResource extends ByteArrayResource{
private String fileName;
public MultipartByteArrayResource(byte[] byteArray) {
super(byteArray);
}
public String getFilename() {
return fileName;
}
public void setFilename(String fileName) {
this.fileName= fileName;
}
}
After above changes client code will be
FormHttpMessageConverter converter = new FormHttpMessageConverter();
RestTemplate restTemplate = new RestTemplate();
restTemplate.getMessageConverters().add(converter);
MultiValueMap<String, Object> requestParamerterMap = new LinkedMultiValueMap<String, Object>();
requestParamerterMap.add("jsonData", jsonData);
MultipartFile attachment = userRequest.getAttachment();
if(attachment!=null && attachment.getOriginalFilename()!=null) {
//ByteArrayResource byteArrayResource = new ByteArrayResource(attachment.getBytes(), attachment.getOriginalFilename());
MultipartByteArrayResource resource = new MultipartByteArrayResource(attachment.getBytes());
//pass file name sepratly
resource.setFilename(attachment.getOriginalFilename());
requestParamerterMap.add("attachment", resource);
}
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<MultiValueMap<String, Object>>(requestParamerterMap, headers);
String response = restTemplate.postForObject(targetUrls.get("sdCreateTicketsUrl"), requestEntity, String.class);

First, value="/createTicket.do" is way off the REST convention. Same goes for /ticket.
Creation of a ticket should be done by POST to URL: .../tickets/