HTTPClient 4.x - first http request significantly slower - java

I changed my implementation using a newer version of Apache HTTP Client (3.x before, now 4.x)
My problem is, that the first HTTP request of a series of 4 different GET requests takes significantly
longer than the following ones. This problem is reproducable, if a pause of 60 seconds is done between each request-series. below there are my results of a simple benchmark. server code stays the same and also the requests are always the same.
I think the problem is something inside the client is destroyed after not used and re-created
It would be help if somebody has any idea what I could configure to avoid this poor performance.
Apache HTTP Client 3.x
REQUEST A duration in ms >> 25
REQUEST B duration in ms >> 20
REQUEST C duration in ms >> 15
REQUEST D duration in ms >> 15
----->>> 60 sec pause
REQUEST A duration in ms >> 20
REQUEST B duration in ms >> 15
REQUEST C duration in ms >> 15
REQUEST D duration in ms >> 15
Apache HTTP Client 4.2.6
REQUEST A duration in ms >> 4659
REQUEST B duration in ms >> 25
REQUEST C duration in ms >> 20
REQUEST D duration in ms >> 20
----->>> 60 sec pause
REQUEST A duration in ms >> 4510
REQUEST B duration in ms >> 20
REQUEST C duration in ms >> 15
REQUEST D duration in ms >> 10
Thank you very much in advance
Shane

Thanking you very much. To be more exact, i am using HTTP Client 4.2.6 inside the Resteasy 3.0.9 framework.
Some example would be this code:
ProxyFactory.create(ITestService.class, baseUrl, new ApacheHttpClient4Executor(httpClient));
The problem was (means it is reproducable) that i was using a deprecated contrutor of the Scheme class fore registering a new scheme to apache "Scheme Registry"
SchemeRegistry registry = new SchemeRegistry();
WRONG -> using deprecated Scheme constructor
registry.register(new Scheme("http",PlainSocketFactory.getSocketFactory(), usedPort));
correct way
registry.register(new Scheme("http",usedPort,PlainSocketFactory.getSocketFactory()));

Related

First REST call slow on Apache server

I'm facing this issue on some of the requests. All the first calls of a user (or when some time has passed) take up to 60 seconds to be completed. After that it takes milliseconds. It doesn't matter if I call it on an internet browser or with SOAP UI, it is always the same. I'm running a java application on Apache.
I ran it with fiddler to see where is the call spending the most time, and it looks like it is between when the server gets the request, and it started the answer, everytime it takes 61 seconds. Fiddler also shows the SSL handshake taking milliseconds in both cases
I'm not sure where else to look. If anyone can shed some light it will be greatly appreciated.
First request (takes 1:01 mins)
Request Count: 1
Bytes Sent: 683 (headers:683; body:0)
Bytes Received: 846 (headers:347; body:499)
ACTUAL PERFORMANCE
--------------
This traffic was captured on Friday, May 18, 2018.
ClientConnected: 16:21:17.244
ClientBeginRequest: 16:21:17.465
GotRequestHeaders: 16:21:17.465
ClientDoneRequest: 16:21:17.465
Determine Gateway: 0ms
DNS Lookup: 0ms
TCP/IP Connect: 0ms
HTTPS Handshake: 0ms
ServerConnected: 16:21:17.275
FiddlerBeginRequest: 16:21:17.466
ServerGotRequest: 16:21:17.466
ServerBeginResponse: 16:22:18.597
GotResponseHeaders: 16:22:18.597
ServerDoneResponse: 16:22:18.599
ClientBeginResponse: 16:22:18.599
ClientDoneResponse: 16:22:18.599
Overall Elapsed: 0:01:01.133
RESPONSE BYTES (by Content-Type)
--------------
application/json: 499
~headers~: 347
Subsequent requests (Takes 0.120 mins)
Request Count: 1
Bytes Sent: 683 (headers:683; body:0)
Bytes Received: 846 (headers:347; body:499)
ACTUAL PERFORMANCE
--------------
This traffic was captured on Friday, May 18, 2018.
ClientConnected: 16:22:38.582
ClientBeginRequest: 16:22:38.607
GotRequestHeaders: 16:22:38.607
ClientDoneRequest: 16:22:38.607
Determine Gateway: 0ms
DNS Lookup: 0ms
TCP/IP Connect: 0ms
HTTPS Handshake: 0ms
ServerConnected: 16:22:38.589
FiddlerBeginRequest: 16:22:38.607
ServerGotRequest: 16:22:38.607
ServerBeginResponse: 16:22:38.726
GotResponseHeaders: 16:22:38.726
ServerDoneResponse: 16:22:38.727
ClientBeginResponse: 16:22:38.727
ClientDoneResponse: 16:22:38.727
Overall Elapsed: 0:00:00.120
RESPONSE BYTES (by Content-Type)
--------------
application/json: 499
~headers~: 347
This is how it looks another first call made with the browser
Chrome Network Analysis
Thanks!!
So, I finally came with this post about LDAP ->
Debugging a timeout with ldap auth on apache
Which happened to be my case.
Adding LDAPConnectionPoolTTL 0 to my httpd.config solved my issue.

How to check time on all the nodes in hadoop cluster

I am running spark job on hadoop cluster, and the job is failing at few times with the exception :
exception : Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.JavaMain], main() threw exception, begin > end in range (begin, end): (1494159709088, 1494159706071)
the job ran successfully on the rerun.
After searching on google, It might be Clock skew between the Oozie server host and launcher host.
Is there a way i can check if there is clock skew ? or how can i check the time on all the nodes whether they are in sync or not.
Thanks
ntptime command output :
ntp_gettime() returns code 0 (OK)
time dcb9b19b.a2328f64 Sun, May 7 2017 14:45:47.633, (.633584090),
maximum error 434990 us, estimated error 815 us, TAI offset 0
ntp_adjtime() returns code 0 (OK)
modes 0x0 (),
offset 176.871 us, frequency -25.666 ppm, interval 1 s,
maximum error 434990 us, estimated error 815 us,
status 0x2001 (PLL,NANO),
time constant 10, precision 0.001 us, tolerance 500 ppm,
ntpstat command output :
synchronised to NTP server (174.68.168.57) at stratum 3
time correct to within 77 ms
polling server every 1024 s

Tomcat 8 : 100% cpu usage

I am having a problem with tomcat since switching to a different package provider (bitnami -> official debian).
Someone seems to be hitting our servers with a request (with malicious intent):
59.111.29.6 - - [04/Feb/2017:16:17:58 +0000] "-" 400 -
where "-" is the request path, which coincides with
Feb 04, 2017 4:17:58 PM org.apache.coyote.http11.AbstractHttp11Processor process
INFO: Error parsing HTTP request header
Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level.
which coincides with the increased CPU usage.
The server status shows the following:
<h1>JVM</h1><p> Free memory: 355.58 MB Total memory: 833.13 MB Max memory: 2900.00 MB</p><table border="0"><thead><tr><th>Memory Pool</th><th>Type</th><th>Initial</th><th>Total</th><th>Maximum</th><th>Used</th></tr></thead><tbody><tr><td>Eden Space</td><td>Heap memory</td><td>34.12 MB</td><td>229.93 MB</td><td>800.00 MB</td><td>12.47 MB (1%)</td></tr><tr><td>Survivor Space</td><td>Heap memory</td><td>4.25 MB</td><td>28.68 MB</td><td>100.00 MB</td><td>2.22 MB (2%)</td></tr><tr><td>Tenured Gen</td><td>Heap memory</td><td>85.37 MB</td><td>574.51 MB</td><td>2000.00 MB</td><td>462.84 MB (23%)</td></tr><tr><td>Code Cache</td><td>Non-heap memory</td><td>2.43 MB</td><td>7.00 MB</td><td>48.00 MB</td><td>6.89 MB (14%)</td></tr><tr><td>Perm Gen</td><td>Non-heap memory</td><td>128.00 MB</td><td>128.00 MB</td><td>512.00 MB</td><td>52.57 MB (10%)</td></tr></tbody></table><h1>"http-nio-8080"</h1><p> Max threads: 200 Current thread count: 10 Current thread busy: 3 Keeped alive sockets count: 1<br> Max processing time: 301 ms Processing time: 71.068 s Request count: 10021 Error count: 2996 Bytes received: 0.00 MB Bytes sent: 3.18 MB</p><table border="0"><tr><th>Stage</th><th>Time</th><th>B Sent</th><th>B Recv</th><th>Client (Forwarded)</th><th>Client (Actual)</th><th>VHost</th><th>Request</th></tr><tr><td><strong>F</strong></td><td>1486364749526 ms</td><td>0 KB</td><td>0 KB</td><td>185.40.4.169</td><td>185.40.4.169</td><td nowrap>?</td><td nowrap class="row-left">? ? ?</td></tr><tr><td><strong>F</strong></td><td>1486364749526 ms</td><td>0 KB</td><td>0 KB</td><td>185.40.4.169</td><td>185.40.4.169</td><td nowrap>?</td><td nowrap class="row-left">? ? ?</td></tr><tr><td><strong>R</strong></td><td>?</td><td>?</td><td>?</td><td>?</td><td>?</td><td>?</td></tr><tr><td><strong>S</strong></td><td>36 ms</td><td>0 KB</td><td>0 KB</td><td>106.51.39.130</td><td>106.51.39.130</td><td nowrap>104.197.119.177</td><td nowrap class="row-left">GET /manager/status?org.apache.catalina.filters.CSRF_NONCE=072F9F6884D94C5D7B30D1D34CE61BD9 HTTP/1.1</td></tr><tr><td><strong>R</strong></td><td>?</td><td>?</td><td>?</td><td>?</td><td>?</td><td>?</td></tr></table><p>P: Parse and prepare request S: Service F: Finishing R: Ready K: Keepalive</p><hr size="1" noshade="noshade">
<center><font size="-1" color="#525D76">
So it doesn't seem like an out of memory issue (but I could be wrong).
How can I stop someone from making the request in the first place to avoid the issues I'm facing? My webapp running on tomcat restricts HTTP methods to GET/POST, but how can I configure tomcat as a whole to restrict them?
I would advise you to obtain a thread dump of your server :
Isolates the PID of the tomcat server using :
jps -l
Obtains a thread dump using :
kill -3 PID
or jstack PID
Then checks the Thread dump, you should find the reason of the hogging thread

Restlent returns nothing to a partial request

I building a REST-API with restlet to serve a mp3 file. But the File is created and served at the same time. More informations about that here.
It work perfectly but I only tested it in a desktop environment. When I fired up my iPad to test the API, it starts playing the File but after some seconds it stops, send a new Request and starts playing the File from the beginning.
After some researcher I find out that the iPad sends a Partial Request and therefore expects a partial response.
So I modified the Response-Header to fulfill the requirements.
I use curl to test the API :
curl -v -r 0-1 http://localhost:12345/api/path/file.mp3
Request-Header
GET /api/path/file.mp3 HTTP/1.1
Range: bytes=0-1
User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
Host: localhost:12345
Accept: */*
Response-Header
HTTP/1.1 206 Partial Content
Date: Tue, 29 Oct 2013 12:18:11 GMT
Accept-Ranges: bytes
Server: Restlet-Framework/2.1.0
Vary: Accept-Charset, Accept-Encoding, Accept-Language, Accept
Content-Length: 2
Content-Range: bytes 0-1/19601021
Content-Type: audio/mpeg; charset=UTF-8
Expires: Tue, 29 Oct 2013 12:18:07 GMT
Last-Modified: Tue, 29 Oct 2013 12:18:07 GMT
Error-Message
But there is no Data that comes back from the server. Curl keeps the connection open to the server and the iPad sends a new Request.
When I shut the server down curl gives me:
transfer closed with 1 bytes remaining to read
Closing connection #0
curl: (18) transfer closed with 1 bytes remaining to read
Code
And here is the code I use to return the data. As you can see, this method is just for testing and should always return 2 bytes.
private InputRepresentation rangeGetRequest() throws IOException {
final byte[] bytes = new byte[2];
bytes[0] = 68;
bytes[1] = 68;
final InputRepresentation inputRepresentation = new InputRepresentation(new ByteArrayInputStream(bytes), MediaType.AUDIO_MPEG);
return inputRepresentation;
}
I have no idea what to do. I try to write my own InputStream that returns the 2 bytes, but without success.
Or is the InputRepresentation not suitable for this area of application?
Thanks in advance
Ok, I solved the problem, it was a stupid mistake.
To modify the response-header, I used
getResponse().setEntity(new StringRepresentation(" "));
getResponse().getEntity().setSize(19601021);
getResponse().getEntity().setRange(new Range(0, 2));
getResponse().getEntity().setModificationDate(new Date());
getResponse().getEntity().setExpirationDate(new Date());
getResponse().getEntity().setMediaType(MediaType.AUDIO_MPEG);
The first line was a relic from an old test. If I just use the actual InputRepresentation that holds the two bytes and it work just fine.
What do I learn from that, first tidy up your code, then ask questions ^^

buffering issue with java communicating vlc

I'm a newbie to Socket communication, so I may be wrong, but please advice or at least give the direction!
I'm implementing an RTSP server according to http://www.csee.umbc.edu/~pmundur/courses/CMSC691C/lab5-kurose-ross.html#appendix taking a look to the similar code from http://www.java2s.com/Open-Source/Android/UnTagged/mynpr/com/webeclubbin/mynpr/RTSPserver.java.htm
At the moment I'm implementing responce to the OPTIONS request. To make it easy in the first approach, I decided to hardcode the answer according to the sample RTSP request/response log done for some real communication between vlc and gstreamer rtsp.
So, the log recorded with vlc URL -vvv says:
Sending request: OPTIONS rtsp://localhost:8554/test RTSP/1.0
CSeq: 2
User-Agent: LibVLC/2.0.8 (LIVE555 Streaming Media v2013.04.30)
Received 183 new bytes of response data.
Received a complete OPTIONS response:
RTSP/1.0 200 OK
CSeq: 2
Public: OPTIONS, DESCRIBE, GET_PARAMETER, PAUSE, PLAY, SETUP, SET_PARAMETER, TEARDOWN
Server: GStreamer RTSP server
Date: Tue, 10 Sep 2013 19:56:53 GMT
Sending request: DESCRIBE rtsp://localhost:8554/test RTSP/1.0
CSeq: 3
User-Agent: LibVLC/2.0.8 (LIVE555 Streaming Media v2013.04.30)
Accept: application/sdp
i.e.
RTSP/1.0 200 OK
CSeq: 2
Public: OPTIONS, DESCRIBE, GET_PARAMETER, PAUSE, PLAY, SETUP, SET_PARAMETER, TEARDOWN
Server: GStreamer RTSP server
Date: Tue, 10 Sep 2013 19:56:53 GMT
part is 183 bytes length
I'm writing to the buffer right according to the example:
try{
System.out.println("S -> C");
System.out.println("RTSP/1.0 200 OK");
System.out.println("CSeq: "+RTSPSeqNb);
//System.out.println("Session: "+RTSP_ID);
if (responceType==OPTIONS) {System.out.println("Public: OPTIONS, DESCRIBE, GET_PARAMETER, PAUSE, PLAY, SETUP, SET_PARAMETER, TEARDOWN");};
if (responceType==OPTIONS) {System.out.println("Server: GStreamer RTSP server"); };
if (responceType==OPTIONS) {System.out.println("Date: Tue, 10 Sep 2013 19:56:53 GMT");};
RTSPBufferedWriter.write("RTSP/1.0 200 OK"+CRLF);
RTSPBufferedWriter.write("CSeq: "+RTSPSeqNb+CRLF);
//RTSPBufferedWriter.write("Session: "+RTSP_ID+CRLF);
if (responceType==OPTIONS) {RTSPBufferedWriter.write("Public: OPTIONS, DESCRIBE, GET_PARAMETER, PAUSE, PLAY, SETUP, SET_PARAMETER, TEARDOWN"+CRLF);};
if (responceType==OPTIONS) {RTSPBufferedWriter.write("Server: GStreamer RTSP server"+CRLF); };
if (responceType==OPTIONS) {RTSPBufferedWriter.write("Date: Tue, 10 Sep 2013 19:56:53 GMT"+CRLF); };
RTSPBufferedWriter.write("Session: "+RTSP_ID+"\r"+CRLF);
RTSPBufferedWriter.flush();
//RTSPBufferedWriter.newLine();
System.out.println("RTSP Server - Sent response to Client.");
}
catch(IOException ex)
{
System.out.println("Exception caught: "+ex.getStackTrace());
// System.exit(0);
}
and the vlc log says
Opening connection to 127.0.0.1, port 6666...
...remote connection opened
Sending request: OPTIONS rtsp://127.0.0.1:6666/autostream.mjpg RTSP/1.0
CSeq: 2
User-Agent: LibVLC/2.0.8 (LIVE555 Streaming Media v2013.04.30)
Received 193 new bytes of response data.
[0x7fd01c001178] live555 demux debug: connection timeout
[0x7fd01c001178] live555 demux error: Failed to connect with rtsp://127.0.0.1:6666/autostream.mjpg
Where CRLF is '\n'. Before I tried CRLF="\r\n" (and no +"\r"+ in the last line) with
Received 198 new bytes of response data.
So, what is wrong there? What vlc is waiting for? Why default delimeters from the example doesn't work for it?
I looks like I always find the answer to my questions after I publish them to stackoverflow...
DOUBLE CRLF should stand after the last header for the usual RTSP protocol 9not the customized one used in the example).

Categories