We Keep Coding

#RequestMapping handling of special characters

I have a REST API
#RequestMapping(value = "/Save", method = RequestMethod.POST,
consumes = {"application/json;charset=ISO-8859-1" },
produces = {"application/json;charset=ISO-8859-1" })
public SaveResponse save(#RequestBody SaveRequest request) {
//some codes
}
}
My request contains the following lines
{
:
:
"mailingAddress": {
"addressLine1": "Carrera 36E N°",
"addressLine2": "Medellín, Antioquia",
"country": "COL"
}
:
And I received a 400 Bad Request
HTTP/1.1 400 Bad Request
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Date: Wed, 16 Jun 2021 12:26:37 GMT
Pragma: no-cache
Content-Length: 0
Expires: 0
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-RBT-Optimized-By: CANA014IJT (RiOS 6.1.1a #11) SC
And when I removed the characters "°" and "í" from the request, then the request can go through with no error. What is my problem?
Thank you.

You can try changing the character set from ISO-8859-1 to UTF-8, as the former does not support as wide an array of character mappings (What is the difference between UTF-8 and ISO-8859-1?).

Python Requests Emulate a CURL POST sending multipart request with 1 or more files AND JSON body

I've been hacking at this for two days now with no luck!
WORKING CURL request
curl -X POST -v "http://$1:8080/controller/endpoint" -H "Cache-Control: no-cache" -H "Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW" -F "message={ \"id\": \"b3562c86-6ff4-4bf7-9c4a-4c64fff4d0ea\", \"stuff\": [
{
\"id\": \"1ca2d9b1-1d73-432a-b483-be404afff8da\",
.......
\"endTime\": \"\"
}]}};type=application/json" -F "files=#file.zip"
Returns output that looks like:
./rest.sh http://127.0.0.1/anything
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
> POST /anything HTTP/1.1
> User-Agent: curl/7.35.0
> Host: 127.0.0.1
> Accept: */*
> Cache-Control: no-cache
> Content-Length: 493
> Expect: 100-continue
> Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW; boundary=------------------------52912a6946761b42
>
< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
* Server gunicorn/19.9.0 is not blacklisted
< Server: gunicorn/19.9.0
< Date: Tue, 12 Feb 2019 18:18:56 GMT
< Connection: keep-alive
< Content-Type: application/json
< Content-Length: 725
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
<
{
"args": {},
"data": "",
"files": {
"files": "ZIP-CONTENT-GOES-HERE"
},
"form": {
"message": "{ \"runId\": \"1ca2d9b1-1d73-432a-b483-be404a13e8da\", \"reports\": [\n{\n\"executionId\": \"1ca2d9b1-1d73-432a-b483-be404a13e8da\",\n\"endTime\": \"\"\n}]}}"
},
"headers": {
"Accept": "*/*",
"Cache-Control": "no-cache",
"Content-Length": "493",
"Content-Type": "multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW; boundary=------------------------52912a6946761b42",
"Expect": "100-continue",
"Host": "127.0.0.1",
"User-Agent": "curl/7.35.0"
},
"json": null,
"method": "POST",
"origin": "172.17.42.1",
"url": "http://127.0.0.1/anything"
}
* Connection #0 to host 127.0.0.1 left intact
Now, if I added ,scrub2.zip to the curl command (sending 2 zip files, AND JSON data), I get output which looks as follows:
./rest.sh http://127.0.0.1/anything
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
> POST /anything HTTP/1.1
> User-Agent: curl/7.35.0
> Host: 127.0.0.1
> Accept: */*
> Cache-Control: no-cache
> Content-Length: 878
> Expect: 100-continue
> Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW; boundary=------------------------27d684afce904423
>
< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
* Server gunicorn/19.9.0 is not blacklisted
< Server: gunicorn/19.9.0
< Date: Tue, 12 Feb 2019 18:20:36 GMT
< Connection: keep-alive
< Content-Type: application/json
< Content-Length: 1117
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
<
{
"args": {},
"data": "",
"files": {},
"form": {
"files": "--------------------------fd702594c1765b85\r\nContent-Disposition: attachment; filename=\"scrubbed.zip\"\r\nContent-Type: application/octet-stream\r\n\r\nZIP-CONTENT-GOES-HERE\r\n--------------------------fd702594c1765b85\r\nContent-Disposition: attachment; filename=\"scrubbed2.zip\"\r\nContent-Type: application/octet-stream\r\n\r\nZIP-CONTENT-GOES-HERE222222222\n\r\n--------------------------fd702594c1765b85--",
"message": "{ \"runId\": \"1ca2d9b1-1d73-432a-b483-be404a13e8da\", \"reports\": [\n{\n\"executionId\": \"1ca2d9b1-1d73-432a-b483-be404a13e8da\",\n\"endTime\": \"\"\n}]}}"
},
"headers": {
"Accept": "*/*",
"Cache-Control": "no-cache",
"Content-Length": "878",
"Content-Type": "multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW; boundary=------------------------27d684afce904423",
"Expect": "100-continue",
"Host": "127.0.0.1",
"User-Agent": "curl/7.35.0"
},
"json": null,
"method": "POST",
"origin": "172.17.42.1",
"url": "http://127.0.0.1/anything"
}
* Connection #0 to host 127.0.0.1 left intact
Do you see the difference? The 2 files are now embedded inside the form/files instead of files and form/message showing separately!
This sort of CURL requests is accepted on the Java API endpoint
which looks as follows in debugger:
but all my attempts on Python, such as:
multipart_form_data_object = {
'scrubbed.zip': (args.files[0], open(args.files[0], 'rb'), "application/json"),
'files': (args.files[1], open(args.files[1], 'rb'), "application/json"),
'message': (None, open(args.message, 'rb'), 'application/json')
}
response = requests.post(args.url + ':' + str(args.port) + '/' + args.endpoint, files=multipart_form_data_object,
proxies=proxies)
(Which is the closest I got it to work), looks like this:
multipart_form_data_object = {
'scrubbed.zip': (args.files[0], open(args.files[0], 'rb'), "application/json"),
'files': (args.files[1], open(args.files[1], 'rb'), "application/json"),
'message': (None, open(args.message, 'rb'), 'application/json')
}
response = requests.post(args.url + ':' + str(args.port) + '/' + args.endpoint, files=multipart_form_data_object,
proxies=proxies)
Which outputs as:
{'Content-Length': '664', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'User-Agent': 'python-requests/2.21.0', 'Connection': 'keep-alive', 'Content-Type': 'multipart/form-data; boundary=227d4ef5a41db8a690e5cebadf336851'}
{
"args": {},
"data": "",
"files": {
"files": "ZIP-CONTENT-GOES-HERE",
"scrubbed.zip": "ZIP-CONTENT-GOES-HERE22222"
},
"form": {
"message": "{\r\n \"runId\": \"9c4a-4c64f6d4d0ea\",\r\n \"reports\": [\r\n {\r\n \"executionId\": \"d73-432a-b483-be404a13e8da\",\r\n \"endTime\": \"\"\r\n }\r\n ]\r\n}"
},
"headers": {
"Accept": "*/*",
"Accept-Encoding": "gzip, deflate",
"Connection": "keep-alive",
"Content-Length": "664",
"Content-Type": "multipart/form-data; boundary=227d4ef5a41db8a690e5cebadf336851",
"Host": "java.api.host.com",
"User-Agent": "python-requests/2.21.0"
},
"json": null,
"method": "POST",
"origin": "10.0.0.2",
"url": "http://java.api.host.com/anything"
}
Now, attempting to adjust this for sending an array of files (otherwise, if I were to rename scrubbed.zip to files , it gets overwritten), to make it look like:
multipart_form_data_object = {
'files': [(args.files[0], open(args.files[0], 'rb'), "application/json"),
(args.files[1], open(args.files[1], 'rb'), "application/json")],
'message': (None, open(args.message, 'rb'), 'application/json')
}
Causes the error:
Traceback (most recent call last):
File ".\load_stress_test_endpoint.py", line 84, in <module>
post()
File ".\load_stress_test_endpoint.py", line 76, in post
proxies=proxies)
File "C:\Python\Python27\lib\site-packages\requests\api.py", line 116, in post
return request('post', url, data=data, json=json, **kwargs)
File "C:\Python\Python27\lib\site-packages\requests\api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Python\Python27\lib\site-packages\requests\sessions.py", line 519, in request
prep = self.prepare_request(req)
File "C:\Python\Python27\lib\site-packages\requests\sessions.py", line 462, in prepare_request
hooks=merge_hooks(request.hooks, self.hooks),
File "C:\Python\Python27\lib\site-packages\requests\models.py", line 316, in prepare
self.prepare_body(data, files, json)
File "C:\Python\Python27\lib\site-packages\requests\models.py", line 504, in prepare_body
(body, content_type) = self._encode_files(files, data)
File "C:\Python\Python27\lib\site-packages\requests\models.py", line 169, in _encode_files
body, content_type = encode_multipart_formdata(new_fields)
File "C:\Python\Python27\lib\site-packages\urllib3\filepost.py", line 90, in encode_multipart_formdata
body.write(data)
TypeError: 'tuple' does not have the buffer interface
My final attempt was a different data structure (list), as follows:
multiple_files_list = [
('files', (args.files[0], open(args.files[0], 'rb'), "application/json")),
('files', (args.files[1], open(args.files[1], 'rb'), "application/json")),
('message', None, open(args.message, 'rb'), 'application/json')
]
Results in the error:
Traceback (most recent call last):
File ".\load_stress_test_endpoint.py", line 84, in <module>
post()
File ".\load_stress_test_endpoint.py", line 76, in post
proxies=proxies)
File "C:\Python\Python27\lib\site-packages\requests\api.py", line 116, in post
return request('post', url, data=data, json=json, **kwargs)
File "C:\Python\Python27\lib\site-packages\requests\api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Python\Python27\lib\site-packages\requests\sessions.py", line 519, in request
prep = self.prepare_request(req)
File "C:\Python\Python27\lib\site-packages\requests\sessions.py", line 462, in prepare_request
hooks=merge_hooks(request.hooks, self.hooks),
File "C:\Python\Python27\lib\site-packages\requests\models.py", line 316, in prepare
self.prepare_body(data, files, json)
File "C:\Python\Python27\lib\site-packages\requests\models.py", line 504, in prepare_body
(body, content_type) = self._encode_files(files, data)
File "C:\Python\Python27\lib\site-packages\requests\models.py", line 141, in _encode_files
for (k, v) in files:
ValueError: too many values to unpack
Can you please advice, how to make Python request package perform similiarly to the CURL request?
Follows is how the Java endpoint is set up:
public Response index(#RequestPart("message") #Valid
final Message message,
#ApiParam(value = "Multipart File array of compressed archives (zip) ", required = true) #RequestPart("files") #Valid
final MultipartFile[] files)

Your script should look like this:
Note: There is a dependency on requests_toolbelt
send.py
import argparse
import requests
from requests_toolbelt import MultipartEncoder
parser = argparse.ArgumentParser()
parser.add_argument('message')
parser.add_argument('--files', nargs='+')
args = parser.parse_args()
multipart_form_data_object = MultipartEncoder(
fields=(
('files', (args.files[0], open(args.files[0], 'rb'), "application/json")),
('files', (args.files[1], open(args.files[1], 'rb'), "application/json")),
('message', ('message', open(args.message, 'rb'), 'application/json')),
)
)
res = requests.post('http://localhost:8000', data=multipart_form_data_object, headers={'Content-Type': multipart_form_data_object.content_type})
print(res.content)
I tested it using django using this:
urls.py
from django.urls import path
from django.http import JsonResponse
from django.views.decorators.csrf import csrf_exempt
#csrf_exempt
def dump(request):
data = {name: [o.read().decode('utf8') for o in request.FILES.getlist(name)] for name in request.FILES.keys()}
return JsonResponse(data)
urlpatterns = [
path('', dump),
]
Called it using:
curl -s http://127.0.0.1:8000/ -F "message=#$(pwd)/file1" -F "files=#$(pwd)/file2" -F "files=#$(pwd)/file3"
and using python
python send.py file1 --files file2 file3
Same output:
{"files": ["{\\"message\\": \\"hello world\\"}\\n", "something else\\n"], "message": ["hello world\\n"]}

Java HttpURLConnection - enumerate all 302 redirect hops

my goal is to use java.net.HttpURLConnection in order to parse all the 302 redirect hops for a given URL.
This is my snippet code (I'm actually using it in Talend SW):
String url = row2.url;
java.net.HttpURLConnection con = (java.net.HttpURLConnection) new java.net.URL(url).openConnection();
con.setInstanceFollowRedirects(false);
con.connect();
String realURL = con.getHeaderField("Location");
System.out.println(realURL);
It works pretty well if there's only one 302 reply.
For instance if I populate row2.ulr= "https://jigsaw.w3.org/HTTP/300/302.html" the code will output -> https://jigsaw.w3.org/HTTP/300/Overview.html witch is 100% correct.
My problem is that I'm unable to parse several 302 reply, a typical example is the Facebook URL:
www.facebook.com/ID_account ->redirect-> www.facebook.com/ACCOUNT_NAME
Using a redirect checker online I found 2 - 302 response (I need the second one):
This is the CURL output:
> >>> http://www.facebook.com/123456789
>
> > --------------------------------------------
> > 302 Found
> > --------------------------------------------
>
> Status: 302 Found Code: 302
> Location: https://www.facebook.com/123456789 Vary: Accept-Encoding
> Content-Type: text/html; charset=UTF-8
> X-FB-Debug: Muf4PfCP9TRKCO17QUf7SV2vsdnrCu6Gw2+sjWAKe0QPGdAToJPcmgH5LHv3NIAhzsJXfPB3a9/mVtuhiiEihA==
> Date: Mon, 16 Oct 2017 16:02:55 GMT Connection: close
> Content-Length: 0
>
>
>
>
> >>> https://www.facebook.com/123456789
>
> > --------------------------------------------
> > 302 Found
> > --------------------------------------------
>
> Status: 302 Found Code: 302 X-XSS-Protection: 0
> public-key-pins-report-only: max-age=600;
> pin-sha256="WoiWRyIOVNa9ihaBciRSC7XHjliYS9VwUGOIud4PB18=";
> pin-sha256="k2v657xBsOVe1PQRwOsHsw3bsGT2VzIqz5K+59sNQws=";
> pin-sha256="gMxWOrX4PMQesK9qFNbYBxjBfjUvlkn/vN1n+L9lE5E=";
> pin-sha256="q4PO2G2cbkZhZ82+JgmRUyGMoAeozA+BSXVXQWB8XWQ=";
> report-uri="http://reports.fb.com/hpkp/" Pragma: no-cache
> Location: https://www.facebook.com/a_name_account/
> Cache-Control: private, no-cache, no-store, must-revalidate
> X-Frame-Options: DENY Strict-Transport-Security: max-age=15552000;
> preload X-Content-Type-Options: nosniff Expires: Sat, 01 Jan 2000
> 00:00:00 GMT Vary: Accept-Encoding Content-Type: text/html;
> charset=UTF-8
> X-FB-Debug: j2KCBNZ1poIJ0xUeeQYbinpcqq2avoI4z8eWb9Dx/yUUg98uyGYGadydia7en1s5X4DJeaJB7VjxYaRvP+psCw==
> Date: Mon, 16 Oct 2017 16:02:55 GMT Connection: close
> Content-Length: 0
>
>
>
>
> >>> https://www.facebook.com/a_name_account/
Any suggestion on how to find the second 302 "Location" ->https://www.facebook.com/a_name_account/?
Thanks in advance
Regs
S.

This would be great behavior for a Recursive Method.
You could keep calling your method if you determine that there is still a location in the response header.
public void recurseLocation(String url) {
URL url = new URL(location);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setInstanceFollowRedirects(false);
connection.connect();
String newLocation = connection.getHeaderField("Location");
if (newLocation != null) {
newLocation = recurseLocation(newLocation);
} else {
newLocation = location;
}
return newLocation;
}

Angular2 - unable to get Set-Cookie from auth. response (jaas, wf)

Iam trying to authenticate from Angular2.1.0 to JAAS form-based j2ee app on WildFly 8.2
let j_username = 'sb2';
let j_password = 'sb222';
let url: string = 'http://127.0.0.1:8888/prototype-rest/j_security_check';
let body = 'j_username=' + j_username + '&j_password=' + j_password;
let headers = new Headers({
'Content-Type': 'application/x-www-form-urlencoded'
})
;
//let options = new RequestOptions({headers: headers, withCredentials : true});
let options = new RequestOptions({headers: headers});
this.http.post(url, body, options)
.subscribe(
(res: Response) => {
console.log('res = ' + res);
console.log(res.headers.keys());
var headers = res.headers;
var setCookieHeader = headers.get('Set-Cookie');
console.log('setCookieHeader = ' + setCookieHeader);
},
err => {
console.log('err = ' + err);
}
)
;
response from wildfly
HTTP/1.1 200 OK
Expires: 0
Cache-Control: no-cache, no-store, must-revalidate
X-Powered-By: Undertow/1
Set-Cookie: JSESSIONID=iyD6Yz_Tj7xsIM1zRDHaR2bh.sk-za-04702; path=/prototype-rest
Access-Control-Allow-Headers: accept, authorization, content-type, x-requested-with
Server: WildFly/8
Pragma: no-cache
Access-Control-Expose-Headers: Set-Cookie
Date: Thu, 03 Nov 2016 14:01:41 GMT
Connection: keep-alive
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
Content-Length: 0
Access-Control-Allow-Methods: GET, POST, OPTIONS, PUT
Access-Control-Max-Age: 3600
My problem is that I am not able to read Set-Cookie, if I use withCredentials : true (I am not sure if I should), response fails ... I'd like to add JSESSIONID to request, where I register websocket ...

Jersey can't add parameter when post submit

I'm using Jersey 2.4.1
I dont know. Why i can't add parameter when post submit.
Parameter information should print at line number 5 in my think.
my sample code is below
#Test
public void test() {
Client client = ClientBuilder.newClient();
client.register(new LoggingFilter());
WebTarget target = client.target("http://stackoverflow.com/");
Form form = new Form();
form.param("x", "foo");
form.param("y", "bar");
target.request(MediaType.APPLICATION_JSON_TYPE)
.post(Entity.form(form));
}
result logging is..
1 * LoggingFilter - Request received on thread main
1 > POST http://stackoverflow.com/
1 > Accept: application/json
1 > Content-Type: application/x-www-form-urlencoded
2 * LoggingFilter - Response received on thread main
2 < 200
2 < X-Frame-Options: SAMEORIGIN
2 < Date: Mon, 02 Dec 2013 14:13:35 GMT
2 < Vary: *
2 < Content-Length: 195990
2 < Expires: Mon, 02 Dec 2013 14:14:35 GMT
2 < Last-Modified: Mon, 02 Dec 2013 14:13:35 GMT
2 < Content-Type: text/html; charset=utf-8
2 < Pragma: no-cache
2 < Cache-Control: public, max-age=60

To print form parameters to console log you need to instantiate LoggingFilter using other constructor than the default one, see LoggingFilter(java.util.logging.Logger, boolean):
client.register(new LoggingFilter(Logger.getAnonymousLogger(), true));
This behavior might be different from the one present in Jersey 1.x.