I'm trying to use java + camel to read a multipart file and converting it to object to process it.
My multipart file is composed of 2 kind of datas: an application/json and an application/octet-stream
Is there a way to convert it to some kind of object that i can work with?
basically I'm sending a get REST request to an application that respondes with a multipart file. I can't figure it out how to convert the response body: the application/json to my POJO and the application/octet-stream to a DataHandler
To handle multipart in camel, you can use unmarshal().mimeMultipart(). After this line, Camel Exchange contains an AttachmentMessage, the body of it is the first part of the multipart. The next parts can be obtained by calling attachmentMessage.getAttachmentObjects() in Camel processor. For more information on how unmarshal().mimeMultipart() works, see the source code of method unmarshall() in org.apache.camel.dataformat.mime.multipart.MimeMultipartDataFormat.
Further actions depend on your case. For example, if the first part of a multipart always contains a json object of your SimplePojo class, then you can use unmarshal().json(SimplePojo.class) immediately after unmarshal().mimeMultipart(). Then Camel body will contain your pojo. DataHandlers for the next parts can be obtained as follows:
DataHandler dataHandler = attachmentObjects.get("test.txt").getDataHandler();
// test.txt comes from the filename field of the Content-Disposition header in case you have multipart/form-data.
Below is an example of processing a response from a REST service located at {{http.url}}, in a multipart format of the following form:
--Boundary_2411_1961957611_1491990591774
Content-Disposition: form-data; name="part1"
Content-Type: application/json; charset=utf-8
{
"id": 123,
"name": "simple pojo"
}
--Boundary_2411_1961957611_1491990591774
Content-Disposition: form-data; name="part2"; filename="test.txt"
Content-Type: application/octet-stream
test
--Boundary_2411_1961957611_1491990591774--
Camel RouteBuilder:
from("direct:start")
.setHeader(Exchange.HTTP_METHOD, constant(HttpMethods.GET))
.to("{{http.url}}")
.unmarshal().mimeMultipart()
.unmarshal().json(SimplePojo.class)
.process(exchange -> {
AttachmentMessage attachmentMessage = exchange.getMessage(AttachmentMessage.class);
SimplePojo simplePojo = attachmentMessage.getBody(SimplePojo.class);
Map<String, Attachment> attachmentObjects = attachmentMessage.getAttachmentObjects();
DataHandler dataHandler = attachmentObjects.get("test.txt").getDataHandler();
})
.to("mock:end");
SimplePojo:
public class SimplePojo {
private Long id;
private String name;
//...
//getters, setters
}
In order for unmarshal().mimeMultipart() and unmarshal().json(SimplePojo.class) to work, you must have dependencies:
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-mail</artifactId>
<version>${camel.version}</version>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-jackson</artifactId>
<version>${camel.version}</version>
</dependency>
Related
I have an endpoint setup using Apache Camel to receive a multipart/form-data HTTP request. Essentially I am trying to submit a data file and a configuration file for processing. The request is as follows (generated by Postman):
POST /upload HTTP/1.1
Host: localhost:8900
Content-Length: 363
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="data"; filename="data-file.json"
Content-Type: application/json
(data)
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="config"; filename="config-file.json"
Content-Type: application/json
(data)
----WebKitFormBoundary7MA4YWxkTrZu0gW
My route is set up like so:
#Component
public class FileReceiverRoute extends RouteBuilder {
#Override
public void configure() throws Exception {
rest()
.post("/upload")
.consumes(MediaType.MULTIPART_FORM_DATA_VALUE)
.bindingMode(RestBindingMode.off)
.route().routeId("upload-route")
.unmarshal().mimeMultipart()
.setHeader(Exchange.FILE_NAME, () -> UUID.randomUUID().toString())
.to("file:{{temp.input.files.dir}}")
.process(configFileProcessor)
// on to further processing
}
}
And my config file processor:
#Component
public class ConfigFileProcessor implements Processor {
#Override
public void process(Exchange exchange) throws Exception {
Message message = exchange.getIn();
AttachmentMessage attachmentMessage = exchange.getMessage(AttachmentMessage.class);
Map<String, Attachment> attachmentObjects = attachmentMessage.getAttachmentObjects();
// this fails - the key is instead the file name "config-file.json"
Attachment configAttachment = attachmentObjects.get("config");
}
}
I want to be able to retrieve the key-value pairs from the form-data and process the data and config files accordingly. Instead I get the following behaviour:
The first value in the form (in this case data-file.json) is parsed into the Camel message body, and the key seems to be discarded. The rest of the entries are parsed into an AttachmentMessage. This behaviour is documented here https://stackoverflow.com/a/67566273/11248602
The keys in the AttachmentMessage are not the original keys from the form-data request, but the filenames (e.g. config-file.json)
Is there any way to parse this request into a map or similar structure so that all the original keys and values can be accessed?
For the sake of clarity, as the proposed solution here above seems to work, a bit more explanations (although it is more a workaround than a solution):
Starting from:
Map<String, Attachment> attachmentObjects = attachmentMessage.getAttachmentObjects();
It is possible to browse all map entries, and for each found Attachment object, examine the attachment headers using getHeaderNames(), among other the 'Content-Disposition' one:
Content-Disposition: form-data; name="data"; filename="data-file.json"
which can finally be parsed to grab the form name ('data' in this example).
Not straightforward true, but this apparently works...
I have to upload a binary file to my server using UI/Postman client. My backend code for Rest API is:
#POST
#Produces({JSONHeaders.MEDIA_TYPE_JSONAPI, MediaType.APPLICATION_JSON})
#Path("loadLicense")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response loadLicense2(#ApiParam("load a license") File input) {
....
}
But the file I get has header details added to it which I don't need. File content is something like:
----------------------------013134317098674079511595^M
Content-Disposition: form-data; name="file"; filename="license.lic"^M
Content-Type: application/octet-stream^M
^M
^#^#^T.^#^#m.....^#^#^#^K^#^A^#^#^#^#^C^#^#^#(^#.^E^#^#^#^K^#.^#^#^#^G.^#^#^#^K^#.^#^#^#^#^B^#^#^#^K^#.^#^#^#^#^#^#^#^#^K^#.^#^#^#^#^F^#^#^#^K^#.^#^#^
^#^#^#^U^#^N^BGR^#^#^#^#^M^#^G^Bc^#^#^#^#^K^#^O^B1.0^#^#^#^#^Q^#4^Bpermanent^#^#^#^#^G^#.
^G.^V......I..^HC_.^^.^U...Y..G.^K.R.^?^O&..^.{V.Z.......h^B.<^O....w'#bk.^B]..*...8.W93...Z.\..... ..g.a+.....,^M
----------------------------013134317098674079511595--^M
But I just need the binary content. Is there any way to do that?
Note: I tried #FormParam - it doesn't work and I get this error
The #FormParam is utilized when the content type of the request entity is not application/x-www-form-urlencoded]
Tried #FormDataParam - not able to resolve it in code.
Instead of #FormParam you might need to use #FormDataParam, depending on how you send the data via Postman. They have different purposes, i.e. are for use with different MIME types:
#FormParam is intended to be used with MIME type application/x-www-form-urlencoded (constant MediaType.APPLICATION_FORM_URLENCODED)
#FormDataParam is intended to be used with multipart/form-data (constant MediaType.MULTIPART_FORM_DATA)
The following snippet expects the file being sent via form data:
#POST
#Produces({JSONHeaders.MEDIA_TYPE_JSONAPI, MediaType.APPLICATION_JSON})
#Path("loadLicense")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response loadLicense2(
#FormDataParam("file") InputStream istream,
#FormDataParam("file") FormDataContentDisposition disp) {
}
The input stream contains the binary data, the second parameter gives you some information about the uploaded file, e.g. the file name.
You need the jersey-media-multipart artifact for this:
https://mvnrepository.com/artifact/org.glassfish.jersey.media/jersey-media-multipart
<dependency>
<groupId>org.glassfish.jersey.media</groupId>
<artifactId>jersey-media-multipart</artifactId>
<version>3.0.2</version>
</dependency>
Is it possible to send two different Content-Type for POST method using post-man? Like, if the service is used for downloading excel file so in that case, #Consumes(MediaType.APPLICATION_JSON) is used for sending some user detail which is json structure and #Produces(MediaType.APPLICATION_OCTET_STREAM) is used for sending back the response as file.
NOTE: I don't want to use form-data, that is a constraint for this service.
On the client request, the Content-Type header is only for the type of data that is in the entity body. The Accept header is what you send for the type of data you want back.
On the server side, it is perfectly fine to do the following.
#POST
#Path("something")
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_OCTET_STREAM)
public Response post(Model model) throws Exception {
final InputStream in = new FileInputStream("path-to-file");
StreamingOutput entity = new StreamingOutput() {
#Override
public void write(OutputStream out) {
IOUtils.copy(in, out);
out.flush();
}
};
return Response.ok(entity)
.header(HttpHeaders.CONTENT_DISPOSITION,
"attachment;filename=somefile.xls")
.build();
}
The request to this endpoint should look like
POST /api/something HTTP 1.1
Content-Type: application/json;charset=utf-8
Accept: application/octet-stream
{"the":"json", "entity":"body"}
See also:
This post about purpose of #Produces and #Consumes and the role they play in content negotiation.
Aside
As an aside, consider using application/vnd.ms-excel as the Content-Type instead of application/octet-stream. This is already the standard MIME type for Microsoft Excel files1. When using StreamingOutput as the response entity type, you can make the #Produces anything you want, as there is no real conversion needed.
In Postman, when you use the "Send and download" feature, you will notice that when the Content-Type is application/octet-stream, it will suggest the file extension of .bin when saving the file. But if you have the Content-Type as application/vnd.ms-excel, then the suggested extension is .xls, which is what it should be.
1. Incomplete list of MIME types
I am trying to write a REST endpoint that receives multiple files (as input streams) and data about those files (as Json).
Here's my client:
public void doSomething() {
MultiPart parent = new MultiPart(MultiPartMediaTypes.createMixed());
MultiPart child = new MultiPart(MultiPartMediaTypes.createMixed());
child.bodyPart(new BodyPart(someObject, MediaType.APPLICATION_JSON_TYPE));
child.bodyPart(new StreamDataBodyPart(someInputStream));
child.bodyPart(new StreamDataBodyPart(someOtherInputStream));
BodyPart bp = new BodyPart(child, MultiPartMediaTypes.createMixed());
parent.bodyPart(bp);
client.post(Entity.entity(parent, MultiPartMediaTypes.createMixed());
}
Here's my server:
#POST
#Path("something")
#Consumes(MultiPartMediaTypes.MULTIPART_MIXED)
public Response multiPart( #Context HttpServletRequest httpServletRequest,
MultiPart multiPart) {
List<BodyPart> bodyParts = multiPart.getBodyParts();
for(BodyPart bp : bodyParts) {
MultiPart multiPart = (MultiPart)bp.getEntity();
}
}
Ideally, I should be able to receive the read Entity on that bodyPart as a MultiPart. I should be able to access the underlying multipart. But I get a class cast exception (because the Entity is a BodyPartEntity).
However, on the receiving side, I get a MultiPart which contains a bodyPart whose entity is a BodyPartEntity which only provides an InputStream. The input stream contains all of the contents of the nested MultiPart, but ideally I should just be able to receive it as a MultiPart and read directly from there.
The input Stream looks something like this:
--Boundary_3_1589640004_1523988834754
Content-Type: application/json
{"someJson" : "value"}
--Boundary_3_1589640004_1523988834754
Content-Type: application/octet-stream
Content-Disposition: form-data; filename="test.txt"; name="test.txt"
(a bunch of characters representing the image)
--Boundary_3_1589640004_1523988834754
These 2 questions are asking exactly what I am asking, yet neither of them seem to work for me
How to consume a nested multipart/mixed mime type in jersey
Jersey reading nested Multipart (multipart/mixed)
How can I easily access the underlying MultiPart without having to manually parse through the InputStream? What am I missing?
I need to receive an HTTP Post Multipart which contains only 2 parameters:
A JSON string
A binary file
Which is the correct way to set the body?
I'm going to test the HTTP call using Chrome REST console, so I'm wondering if the correct solution is to set a "label" key for the JSON parameter and the binary file.
On the server side I'm using Resteasy 2.x, and I'm going to read the Multipart body like this:
#POST
#Consumes("multipart/form-data")
public String postWithPhoto(MultipartFormDataInput multiPart) {
Map <String, List<InputPart>> params = multiPart.getFormDataMap();
String myJson = params.get("myJsonName").get(0).getBodyAsString();
InputPart imagePart = params.get("photo").get(0);
//do whatever I need to do with my json and my photo
}
Is this the way to go?
Is it correct to retrieve my JSON string using the key "myJsonName" that identify that particular content-disposition?
Are there any other way to receive these 2 content in one HTTP multipart request?
If I understand you correctly, you want to compose a multipart request manually from an HTTP/REST console. The multipart format is simple; a brief introduction can be found in the HTML 4.01 spec. You need to come up with a boundary, which is a string not found in the content, let’s say HereGoes. You set request header Content-Type: multipart/form-data; boundary=HereGoes. Then this should be a valid request body:
--HereGoes
Content-Disposition: form-data; name="myJsonString"
Content-Type: application/json
{"foo": "bar"}
--HereGoes
Content-Disposition: form-data; name="photo"
Content-Type: image/jpeg
Content-Transfer-Encoding: base64
<...JPEG content in base64...>
--HereGoes--