I'm using java sockets to communicate with my server. I'm getting http response from the server in a byte[] format. Now I need to parse the headers and content of the byte[]. The sample response I'm getting from server is :
HTTP/1.1 200 OK
alternate-protocol: 443:quic,p=0.01
cache-control: no-cache, must-revalidate
content-disposition: attachment; filename="f.txt"
content-encoding: gzip
content-length: 341
content-type: text/javascript; charset=UTF-8
date: Sat, 15 Nov 2014 18:32:55 GMT
expires: -1
pragma: no-cache
server: gws
status: 200 OK
version: HTTP/1.1
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
1. Either I need an easiest way to parse this in Java (May be a library)
2. Or a way to convert it into HttpResponse object.
Apache HttpComponents is your friend in this case.
Please take a look at http://hc.apache.org/httpclient-3.x/apidocs/org/apache/commons/httpclient/HttpParser.html
Related
I using java Unirest to call my api. but now I need to post an object but with x-www-form-urlencoded, I using this code:
public static void main(String[] args) {
Unirest.config().setObjectMapper(new JacksonObjectMapper());
System.out.println(Unirest
.post("http://192.168.2.157:8082/auth/realms/collatum/protocol/openid-connect/token")
.header("Content-Type", "application/x-www-form-urlencoded")
.body(RequestToken.builder()
.username("user")
.password("1234")
.grant_type("password")
.client_id("front")
.build()).asString().getBody()
);
}
and I get this error:
{"error":"invalid_request","error_description":"Missing form parameter: grant_type"}
When I analise the request the header are ok:(but looks like the object are in json and no x-www-form-urlencoded
POST /auth/realms/collatum/protocol/openid-connect/token HTTP/1.1
Content-Type: application/x-www-form-urlencoded
user-agent: unirest-java/3.1.00
accept-encoding: gzip
Content-Length: 87
Host: 192.168.2.157:8082
Connection: Keep-Alive
{"grant_type":"password","client_id":"front","username":"user","password":"1234"}HTTP/1.1 400 Bad Request
Cache-Control: no-store
X-XSS-Protection: 1; mode=block
Pragma: no-cache
X-Frame-Options: SAMEORIGIN
Referrer-Policy: no-referrer
Date: Thu, 20 Jan 2022 13:56:53 GMT
Connection: keep-alive
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
Content-Type: application/json
Content-Length: 84
{"error":"invalid_request","error_description":"Missing form parameter: grant_type"}
It looks like your body is in the wrong form for application/x-www-form-urlencoded.
It shouldn't be a json object and instead should be something like grant_type=password&client_id=front&username=user&password=1234
An example can be found here
I have a strange problem in WildFly 12. When I post a HTTP request sometimes body is replaced by HTTP response header. Sometimes it doesn't.
the error json data:
HTTP/1.1 200 OK^M Expires: 0^M Cache-Control: no-cache, no-store, max-age=0, must-revalidate^M X-Powered-By: Undertow/1^M Server: WildFly/11^M X-XSS-Protection: 1; mode=block^M Pragma: no-cache^M X-Frame-Options: SAMEORIGIN^M Date: Thu, 08 Nov 2018 13:00:25 GMT^M Connection: keep-alive^M X-Content-Type-Options: nosniff^M Content-Type: text/plain;charset=UTF-8^M Content-Length: 1569^M ^M ationId":"61e1e1b0867c11e8a9a81fbc702bb402","value":"MU5123","startStationName":"上海虹桥国际机场","endAreaName":"北京首都国际机场"},{"shiftId":"6c874935c59e11e8a4f557d52226d05c","startAreaId":"61e1e1b0867c11e8a9a81fbc702bb402","endStationName":"北京首都国际机场","lineId":"336fb55d-cc7c-11e8-9ed0-00163e02896a","label":"MU5159","dispatchTime":"14:30","endAreaId":"61e1e193867c11e8a9a81fbc702bb402","endStationId":"61e1e193867c11e8a9a81fbc702bb402","startAreaName":"上海虹桥国际机场","startStationId":"61e1e1b0867c11e8a9a81fbc702bb402","value":"MU5159","startStationName":"上海虹桥国际机场","endAreaName":"北京首都国际机场"},{"shiftId":"6c874930c59e11e8a4f557d52226d05c","startAreaId":"61e1e1b0867c11e8a9a81fbc702bb402","endStationName":"北京首都国际机场","lineId":"9db7b2a5c5a911e89ed000163e02896a","label":"MU5331","dispatchTime":"15:00","endAreaId":"61e1e193867c11e8a9a81fbc702bb402","endStationId":"61e1e193867c11e8a9a81fbc702bb402","startAreaName":"上海虹桥国际机场","startStationId":"61e1e1b0867c11e8a9a81fbc702bb402","value":"MU5331","startStationName":"上海虹桥国际机场","endAreaName":"北京首都国际机场"}]
Could someone let me know the reason would be for this behavior?
I've written a query tool which at it's heart uses this block to gather info from an external url:
AsyncHttpClientConfig proxiedCF = new DefaultAsyncHttpClientConfig.Builder().setUserAgent(pickUserAgent()).build();
AsyncHttpClient asyncHttpClient = new DefaultAsyncHttpClient(proxiedCF);
Future<Response> f = asyncHttpClient.prepareGet(url).setProxyServer(new ProxyServer.Builder(pickProxyServer(), 80)).execute();
It works fine. However it works even when an invalid proxy is provided, which is a bit suspicious and I feel my elaborate proxy configuration is not used at all.
I stumbled upon this by having pickProxyServer() return a String "1.1.1.1", which is obviously not a valid web proxy.
I use SLF4J for logging and it looks pretty normal:
20:31:47.454 [AsyncHttpClient-7-1] DEBUG o.a.n.channel.NettyConnectListener - Using new Channel '[id: 0x03359938, L:/10.0.0.101:59775 - R:/1.1.1.1:80]' for 'GET' to '[[url removed by me]]'
20:31:47.586 [AsyncHttpClient-7-1] DEBUG o.a.netty.handler.HttpHandler -
Request DefaultFullHttpRequest(decodeResult: success, version: HTTP/1.1, content: EmptyByteBufBE)
GET [[url removed by me]] HTTP/1.1
Host: [[url removed by me]]
Accept: */*
User-Agent: burning_dandelion
Response DefaultHttpResponse(decodeResult: success, version: HTTP/1.1)
HTTP/1.1 200 OK
Date: Fri, 13 Oct 2017 18:31:49 GMT
Expires: Fri, 13 Oct 2017 18:31:49 GMT
Cache-Control: private, max-age=3600
Content-Type: text/xml; charset=ISO-8859-1
P3P: CP="This is not a P3P policy!"
Server: gws
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Set-Cookie: 1P_JAR=2017-10-13-18; expires=Fri, 20-Oct-2017 18:31:49 GMT; path=/; domain=[[url removed by me]]
Set-Cookie: NID=114=qiVBv02cmXYHh2RfLQbhBfESWIoaGlf3d2jlSbAdQ8yWPDsCpOeK9aYbvfq0HWsER68W1oE53jiriM_fivTc1bJi1F2sfCi0wMptKI-9U3ueVKITtFvYYZx2T0rJf1kQ; expires=Sat, 14-Apr-2018 18:31:49 GMT; path=/; domain=[[url removed by me]]; HttpOnly
Accept-Ranges: none
Vary: Accept-Encoding
Transfer-Encoding: chunked
20:31:47.587 [AsyncHttpClient-7-1] DEBUG o.a.netty.channel.ChannelManager - Adding key: ProxyPartitionKey(proxyHost=1.1.1.1, proxyPort=80, secured=false, targetHostBaseUrl=[[url removed by me]]:80 for channel [id: 0x03359938, L:/10.0.0.101:59775 - R:/1.1.1.1:80]
Can someone point me towards my error? Obviously, I want an I/O exception or any kind of notice when an invalid proxy is called upon.
Ok it's the middle of the night but I found something. When the port is anything but :80, the proxy is correctly used. This does the trick for me.
I'm trying to use JMeter to test a restful endpoint for uploading files, but I'm getting a 400 error. The one thing that jumps out at me is the boundary value; it's not the same I provided as the one shown in the request. I'm able to use the endpoint in the browser without issue, and I've replicated the headers revealed in the developer tools in FF.
Here is the relevant info from JMeter:
Result Tab:
Thread Name: Asdf - Load Test 1-1
Sample Start: 2017-06-05 08:47:46 EDT
Load time: 159
Connect Time: 28
Latency: 159
Size in bytes: 438
Sent bytes:821003
Headers size in bytes: 438
Body size in bytes: 0
Sample Count: 1
Error Count: 1
Data type ("text"|"bin"|""):
Response code: 400
Response message: Bad Request
Response headers:
HTTP/1.1 400 Bad Request
Date: Mon, 05 Jun 2017 12:47:46 GMT
Server: Apache/2.4.25 (Win64) OpenSSL/1.0.2k
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-XSS-Protection: 1; mode=block
X-Frame-Options: DENY
X-Content-Type-Options: nosniff
Cache-Control: no-cache, must-revalidate
Content-Length: 0
Connection: close
HTTPSampleResult fields:
ContentType:
DataEncoding: null
Request Tab:
POST https://localhost/my/rest/endpoint
POST data:
--9amm365-gMmimP70lvs9jIvlIxOfkocUN
Content-Disposition: form-data; name="parseable"; filename="asdf.docx"
Content-Type: application/vnd.openxmlformats-
officedocument.wordprocessingml.document
Content-Transfer-Encoding: binary
--9amm365-gMmimP70lvs9jIvlIxOfkocUN--
[no cookies]
Request Headers:
Connection: keep-alive
Content-Type: multipart/form-data; boundary=--Uc_2uLvcVgc7SqvzIJxR3encUKw-
f7w9
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.5
Accept: application/json, text/plain, /
X-Request-OnBehalfOf: some-user-uuid
X-Requested-With: XMLHttpRequest
Content-Length: 820532
Host: localhost
User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_121)
Response Data Tab is blank.
Server side implementation:
#POST
#Override
#EnhancedDetail
#Consumes(MediaType.MULTIPART_FORM_DATA)
#Path("my/rest/endpoint")
public Response uploadProduct(#PathParam("id") final String id, MultiPart multipart) throws IOException {
return processMultiFileUpload(id, multipart, MultiPartUploadType.DRAFT, false);
}
If you are building your request manually like it is described in Testing REST API File Uploads in JMeter article - make sure you have Use multipart/form-data for POST box of the HTTP Request sampler is not checked.
And vice versa, if you tick this box, you won't need to override the Content-Type header
I believe the easiest way would be just recording your file upload event using HTTP(S) Test Script Recorder (just make sure your asdf.docx file is copied to the "bin" folder of your JMeter installation)
I figured it out. I changed the Client Implementation on the HTTP Request to Java. Everything I've read said use HttpClient4, but I tried Java and it worked.
I've got this URL (http://vignette4.wikia.nocookie.net/fantendo/images/6/6e/Small-mario.png/revision/latest?cb=20120718024112)
how determinate the file extension if it isn't at the end of the url?
You'll need to read the response headers. The MIME Type if known is stored in the Content-Type header.
HTTP/1.1 200 OK
Content-Disposition: inline; filename="Small-mario.png"
X-Thumbnailer: Vignette
Content-Type: image/png
Cache-Control: public, max-age=31536000
X-Surrogate-Key: ad1f82ba0cbe38fa60f83c036993a71e05dae492
Server: Jetty(9.2.z-SNAPSHOT)
X-Cacheable: YES
Content-Length: 58457
Accept-Ranges: bytes
Date: Mon, 06 Jul 2015 16:12:31 GMT
Age: 65
Connection: keep-alive
X-Served-By: thumbnailer-s1, cache-wk-sjc3160-WIKIA, cache-lhr6322-LHR
X-Cache: ORIGIN, MISS, HIT
X-Cache-Hits: ORIGIN, 0, 5
X-Timer: S1436199151.564330,VS0,VE0
Vary: Accept-Encoding
Timing-Allow-Origin: *
You're looking for the Content-Type header, which the server ought to send in the HTTP response to tell you this.
Note that it is not guaranteed to be accurate, or present at all.