Unable to send `multipart/form-data` request with python requests module - java

I have Java spring server that requires the Content-Type of the requests being sent to the server to be multipart/form-data.
I can correctly send requests to the server with postman:
However, I got The current request is not a multipart request error when trying to send the request with requests module in python3.
My python code is:
import requests
headers = {
'Authorization': 'Bearer auth_token'
}
data = {
'myKey': 'myValue'
}
response = requests.post('http://127.0.0.1:8080/apiUrl', data=data, headers=headers)
print(response.text)
If I add 'Content-Type': 'multipart/form-data' to the header of the request, the error message then becomes Could not parse multipart servlet request; nested exception is org.apache.commons.fileupload.FileUploadException: the request was rejected because no multipart boundary was found.
How can I make the same request as postman sends with python?

requests's author thinks this situation is not pythonic, so requests doesn't support this usage natively.
You need to use requests_toolbelt which is an extension maintained by members of the requests core development team, doc, example:
import requests
from requests_toolbelt.multipart.encoder import MultipartEncoder
m = MultipartEncoder(
fields={'field0': 'value', 'field1': 'value',
'field2': ('filename', open('file.py', 'rb'), 'text/plain')}
)
r = requests.post('http://httpbin.org/post', data=m,
headers={'Content-Type': m.content_type})

Related

400 Bad request on Java Webclient multipart/formdata post request

Im having problems on posting a multipart/formdata request to a REST api. The request returns an 400 Bad Request response.
This is how the request should look like. The link shows you a screenshot captured on a successful request by the web interface.
Successful request
This is the Java code I created.
public void importModel(String projectId, String modelId, MultipartFile file, String fileName) throws IOException {
MultipartBodyBuilder builder = new MultipartBodyBuilder();
builder.part("data", file.getBytes(), MediaType.APPLICATION_OCTET_STREAM)
.header("Content-Disposition", "form-data; name=data; filename=" + fileName);
MultiValueMap<String, HttpEntity<?>> parts = builder.build();
WebClient webClient = WebClient.builder()
.filters(exchangeFilterFunctions -> {
exchangeFilterFunctions.add(logRequest());
exchangeFilterFunctions.add(logResponse());
})
.build();
String request = webClient.post()
.uri(getBaseUriBuilder()
.pathSegment(getTeamSlug())
.path(API_PATH_PROJECTS)
.pathSegment(projectId)
.path(API_PATH_MODEL)
.pathSegment(modelId)
.path("/importasync")
.build())
.contentType(MediaType.MULTIPART_FORM_DATA)
.contentLength(file.getSize())
.header(HttpHeaders.AUTHORIZATION, getPrefixedAuthToken())
.body(BodyInserters.fromMultipartData(parts))
.exchange()
.flatMap(FlatService::apply)
.block();
return;
}
Any help is much appreciated. Thank in advance!
Have you tried to send the request with alternative Software like POSTMAN.
There you can check for the request properties that are being sent with the request
a 400 error can occur due to the following issues with your request
Wrong URL: Same as 404-Error a Bad Request is generated, when the user types in a wrong internet address or he adds special chars to the address.
Error full Cookies: If the Cookie inside your browser is to old or broken it can also be a 400.
Old outdated DNS-Entries: In your DNS-Cache could lie files that point to wrong or outdated IP- addresses
Too big files: when you try to upload very large files, the server can deny the request.
Too long header lines: the communication between the client and server is done with header information about the request. some servers set a limit to the header length.
Also if you can find out the more specific 400 error like this:
400.1: Invalid Destination Header
400.2: Invalid Depth Header
400.3: Invalid If Header
400.4: Invalid Overwrite Header
400.5: Invalid Translate Header
400.6: Invalid Request Body
400.7: Invalid Content
400.8: Invalid Timeout
400.9: Invalid Lock Token
If you are not the server admin you could ask him about specifications of the server. or use tools like postman where you can try to send requests to the server and find out more specific error codes.

How to send body as a Json in RestTemplate to get data from API [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
HTTP GET with request body
I've read few discussions here which do not advocate sending content via HTTP GET. There are restrictions on the size of data that can be sent via clients (web browsers). And handling GET data also depends on servers. Please refer section Resources below.
However, I've been asked to test the possibility to send content via HTTP GET using RestTemplate. I refered few discussions on spring forum but they were not answered. (Please note sending data via http Post works fine). The discussion here suggests using POST instead.
dev env - JBoss AS 5.1, Spring 3.1.3
Client
#Test
public void testGetWithBody()
{
// acceptable media type
List<MediaType> acceptableMediaTypes = new ArrayList<MediaType>();
acceptableMediaTypes.add(MediaType.TEXT_PLAIN);
// header
HttpHeaders headers = new HttpHeaders();
headers.setAccept(acceptableMediaTypes);
// body
String body = "hello world";
HttpEntity<String> entity = new HttpEntity<String>(body, headers);
Map<String, Object> uriVariables = new HashMap<String, Object>();
uriVariables.put("id", "testFile");
// Send the request as GET
ResponseEntity<String> result = restTemplate.exchange(
"http://localhost:8080/WebApp/test/{id}/body",
HttpMethod.GET, entity, String.class, uriVariables);
Assert.assertNotNull(result.getBody());
}
Server #Controller
#RequestMapping(value = "/{id}/body", method = RequestMethod.GET)
public #ResponseBody
String testGetWithBody(#PathVariable String id,
#RequestBody String bodyContent)
{
return id + bodyContent;
}
The problem -
executing this test case returns 500 Internal Server Error. On debugging, I found that the controller is not hit.
Is it correct to understand that the RestTemplate provides the way to send data as request body, but the error occurs because the server could not handle the request body ?
If the request body sent via HTTP Get is not conventional why does RestTemplate provide the APIs to allow sending it ? Does this mean there are few servers capable of handling the Request body via GET ?
Resources - discussions on sending body via HTTP GET using RestTemplate at spring forum
http://forum.springsource.org/showthread.php?129510-Message-body-with-HTTP-GET&highlight=resttemplate+http+get
http://forum.springsource.org/showthread.php?94201-GET-method-on-RestTemplate-exchange-with-a-Body&highlight=resttemplate+http+get
Resources - General discussions on sending body via HTTP GET
get-with-request-body
is-this-statement-correct-http-get-method-always-has-no-message-body
get-or-post-when-reading-request-body
http-uri-get-limit
Is it correct to understand that the RestTemplate provides the way to send data as request body, but the error occurs because the server could not handle the request body ?
You can tell by looking at network traffic (does the request get sent with a request body and a GET method?) and at server logs (the 500 result you receive must have a server-side effect that gets logged, and if not, configure the server to do so).
If the request body sent via HTTP Get is not conventional why does RestTemplate provide the APIs to allow sending it ? Does this mean there are few servers capable of handling the Request body via GET ?
Because it is a generic class that also allows you to craft requests that can include a message body.
As stated in HTTP GET with request body:
In other words, any HTTP request message is allowed to contain a message body, and thus [a server] must parse messages with that in mind. Server semantics for GET, however, are restricted such that a body, if any, has no semantic meaning to the request. The requirements on parsing are separate from the requirements on method semantics.
A body on a GET cannot do anything semantically, because you are requesting a resource. It's like you tell the server: "Give me resource X, oh, and have some apples!". The server won't care about your apples and happily serve resource X - or throw an error because it doesn't like any offers in a request.
However, I've been asked to test the possibility to send content via HTTP GET
Please tell the one who requested this that this is a case that should not have to be tested, because no sensible implementation supports it.

HTTP Status 406 – Not Acceptable in spring MVC Rest Service

Here is my code :
#RequestMapping(value = "/report/download", method = RequestMethod.GET,produces="application/vnd.ms-excel")
public Response getReportFile(#QueryParam("reportid") Long reportId)
{
System.out.println("Param"+reportId);
Long n=(long) 10;
String json=reportService.getReportFile(n);
File file = new File("D:\\Agent Information.xls");
ResponseBuilder response = Response.ok((Object) file);
response.header("Content-Disposition","attachment; filename=Sample.xls");
return response.build();
}
i am getting the below Error in java console: Handler execution resulted in exception: Could not find acceptable representation
Your webservice is saying that the response type it is returning is not provided in the Accept HTTP header in your Client request.
So while making HTTP Request you have to add 'Accept Headers' . If it's JSON request then you will add 'Accept : application/json'. Similarly for your current example it will be
'Accept: text/plain'
'Accept-Charset: utf-8'
Find all accept headers here. And follow this steps to resolve
1) Find out the response (content type) returned by web service.
2) Provide this (content type) in your request Accept header.

File upload in case of HTTP 307

I am writing a Spring controller that handles the HTTP PUT request from client, and generates S3 pre-signed url and issues a HTTP 307 status (Temp redirect) code. So basically I am authenticating the client and if it succeeds then I am asking him to write to a s3 folder. The client is able to write to signed url location.
Now my concern is the client will have to do upload two times. Once to my application server and then to s3, so the operation will take double the time.
Is my understanding correct?Does the client actually does 2 write in this case? Or is the client smart enough and just pushes the part of payload first and if it succeeds then pushes entire payload?
I read about HTTP 100 status code, but looks like the app server/tomcat already issues it and is not in my control.
Here is my spring controller
#RequestMapping("/upload")
public ResponseEntity<Void> execute(HttpServletRequest request) throws IOException, ServletException {
HttpHeaders headers = new HttpHeaders();
String redirectUrl = getRedirectUrl(requestURI, request.getMethod());
headers.setLocation(new URI(redirectUrl));
ResponseEntity<Void> redirectEntity = new ResponseEntity<Void>(null,headers,HttpStatus.TEMPORARY_REDIRECT);
return redirectEntity;
}
How can i prevent clint from uploading the entire payload to my app server?
So my understanding correct?
Answer is YES. Server will send the response of PUT request after reading the full request including body. when you client will repeat the request, in response 307 (Temporary Redirect), it will be like a new http request.
Also an important point on using 307 response code from spec(see below) should be considered for this approach.
If the 307 status code is received in response to a request other
than GET or HEAD, the user agent MUST NOT automatically redirect the
request unless it can be confirmed by the user, since this might
change the conditions under which the request was issued.
On point
How can i prevent client from uploading the entire payload to my app server?
You may do upload to s3 in background from your controller and return the redirect response (301?) point to an URL which will return the status of upload request.
This just isn’t how HTTP works, HTTP has no mechanism to halt a file upload other than closing the connection, but if you close the connection you cant return the redirect information.
If you want the client to upload directly to S3, you will need to do it in two steps.
Have the client request the URL for the file transfer, then have them initiate the transfer with the desired URL.

Spring 4 #RequestMapping -- consumes vs headers?

I'm learning how to build RESTful web services using Spring 4, and one thing I'm not clear on is in #RequestMapping. I've seen examples where one uses headers = "Accept=application/xml" and other examples using consumes (or produces) = "application/xml".
For instance, in my own #RestController class, I have this function...
// POST
#RequestMapping(method = RequestMethod.POST, headers = "Accept=application/xml")
public User create(#RequestBody User user) {
LOG.info("User = " + user.toString());
return userService.create(user);
}
What is the difference between using headers = "Accept=application/xml" vs. using consumes = "application/xml"? Or even using headers = "content-type=application/xml"?
Could someone explain the differences between headers and consumes/produces, and when each is used?
SHORT ANSWER
In the example you have above, using headers = "Accept=application/xml" or produces = "application/xml" will both respond to the client the same way i.e. send a response to the client with XML representation.
LONGER ANSWER
i. Headers
For RESTful web services, the client (e.g. your browser) sends a request (e.g. GET, POST, etc.) to a server, and the server will send a response back. This is an HTTP Transaction. Both the request and response have HTTP header fields ("headers"), which define the operating parameters of an HTTP transaction (I will refer to the headers for client request as "request headers", and these differ from headers from server response "response headers").
As part of the request your browser sends to server, there are different request headers and some examples include Accept, Connection, Content-Length etc. and each of these headers have their own function (see a full list of headers here: https://en.wikipedia.org/wiki/List_of_HTTP_header_fields).
Using your code example, if a client does a POST request, Spring will check the request header(s) and if it finds a header Accept with a value of application/xml, it will map the request to the create method you have above (and in your case the server will return an XML response representation to the client).
Let me modify the headers element in the code you provided:
#RequestMapping(method = RequestMethod.POST, headers = "Connection=keep-alive")
public User create(#RequestBody User user) {
...
}
Notice the headers element now has a value of Connection=keep-alive. If a client does a POST request, Spring will check the request header(s) and if it finds a header Connection with a value of keep-alive, it will map that client request to the create method above.
ii. Produces and Consumes
If you used produces="application/xml" for the create method, this means a client request is only mapped to the create method if the client's Accept header matches application/xml. This essentially is the client saying, "Hey server, I prefer to accept your response in XML representation, so send your response to me in XML". Effectively, the produces="application/xml" is also the server saying, "Hey client, I can only produce responses for you in XML representation, so I will send you that format".
Link to Spring documentation reference.
If you used consumes="application/xml" for the create method, this means a client request is only mapped to the create method if the client's Content-Type header matches application/xml (the Content-Type request header describes the representation the client request is coming in). This essentially is the server saying, "Hey client, I can only consume requests in XML representation, so send that format to me".
SUMMARY
The headers element within the #RequestMapping annotation can take different request headers (Accept, Connection, Cache-Control etc.), but the produces element is only concerned with the Accept request header and the consumes element is only concerned with the Content-Type request header.
As the javadoc of HeadersRequestCondition (which handles the value provided in the headers attribute of a #RequestMapping annotation) states
Expressions passed to the constructor with header names 'Accept' or
'Content-Type' are ignored. See ConsumesRequestCondition and
ProducesRequestCondition for those.
So don't use those headers in headers. Use the produces and consumes attributes for Accept and Content-Type.
As to how to use them, the documentation gives examples: for consumes and for produces.

Categories