I am trying to send multiple HTTP POST over single socket connection , but EPIPE broken once I read inputstream.
mBufferedWriter.write("POST " + "/base/mypath" + " HTTP/1.1" + "\r\n");
mBufferedWriter.write("Host: " + mIp + "\r\n");
mBufferedWriter.write("Content-Length: " + json.toString().length()"\r\n");
mBufferedWriter.write("Content-Type: application/json\r\n");
mBufferedWriter.write("Connection: Keep-Alive\r\n\r\n");
mBufferedWriter.write(json.toString() + "\r\n");
mBufferedWriter.flush();
My first POST will reach server. Am trying to achieve pipelining. Set keep-alive for socket also
HTTP Pipelining involves more than just re-using an existing connection (HTTP Keep-Alive). Pipelining involves having multiple HTTP requests in flight over a single connection at the same time, where a client sends an HTTP request but does not wait for its response before sending the next HTTP request on the same connection. Both client and server have to implement support for pipelining in order to use it effectively. Even then, there are issues related to pipelining POST requests, so they should nver be pipelined. HTTP Pipelining is discussed in RFC 2616 Sections 8.1.1 and 8.1.2.2.
If you are not actually implementing Pipelining, only Keep-Alive, then you have to pay attention to the server's Connection response header, as it tells you whether you are allowed to send a new HTTP request over the same connection or not. For an HTTP 1.1 response, if the server sends a Connection: close header, then the socket is going to be closed by the server after it sends the response, so you will have to create a new connection before sending the next HTTP request.
You can use Apache HTTP client (http://hc.apache.org/httpcomponents-client-ga/) or http://developer.android.com/reference/org/apache/http/client/package-summary.html and set an Thread safe client connection manager.
Related
I have a Java/Spring project where I am using Oauth2RestTemplate and making it use HttpClient (org.apache.http.client.Httpclient) instead of the default SimpleClient
HttpClient httpClient = HttpClients.custom().setSSLSocketFactory(socketFactory).build();
HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory(httpClient);
oAuth2RestTemplate.setRequestFactory(requestFactory);
With respect to this, I wanted to know/understand if the keep-alive header is always sent for all the requests?
If it is always sent, is there a way to disable sending it? I see a post - Disable Keep Alive in Apache HttpClient that talks about disabling it but it proposes a setting on httpMethod. I am not sure how to get access to this httpMethod in my code setup I described above.
Implement a ConnectionReuseStrategy with a keepAlive() method that simply returns false. See setConnectionReuseStrategy() in HttpClientBuilder.
You may also want to send a Connection header with the value of close.
https://hc.apache.org/httpcomponents-core-ga/httpcore/apidocs/org/apache/http/ConnectionReuseStrategy.html
Example:
List<Header> headers = new ArrayList<>();
headers.add(new BasicHeader(HttpHeaders.CONNECTION, "close"));
HttpClientBuilder builder = HttpClients.custom().setDefaultHeaders(headers)
.setConnectionReuseStrategy(
new ConnectionReuseStrategy() {
#Override
public boolean keepAlive(HttpResponse httpResponse, HttpContext httpContext) {
log.info("**** keepAlive strategy returning false");
return false;
}
});
CloseableHttpClient httpClient = builder.build();
HttpGet httpGet = new HttpGet("https://google.com");
CloseableHttpResponse response = httpClient.execute(httpGet);
log.info("Response status: " + response.getStatusLine());
response.close();
Some additional information:
1. The Keep-Alive header
When most people say keep-alive header, they usually mean a different header called Connection. The two headers work together:
HTTP/1.1 200 OK
...
Connection: Keep-Alive
Keep-Alive: timeout=5, max=1000
...
The Connection header is a hint that the connection should be re-used. The Keep-Alive header specifies the minimum amount of time the connection should remain open, and the maximum number of requests the connection may be re-used for.
Common values for the Connection header are keep-alive and close. Both the server and the client can send this header. If the Connection header is set to close, the Keep-Alive header is ignored.
2. HTTP/1.1 and HTTP/2
With HTTP/1.1, connections are persistent by default. The Keep-Alive header was deprecated (no longer defined in the HTTP specification), although many servers still send them for backwards compatibility.
Clients which cannot handle HTTP/1.1 persistent connections should set a Connection header with the value close.
HTTP/2 uses multiplexing; neither the Connection nor the Keep-Alive header should be used with HTTP/2.
3. Effect of proxies and caches
Generally speaking, persistent connections do not work through non-transparent proxies. They will silently drop any Connection or Keep-Alive headers.
4. Connection handling
Since persistent connections are now the default with HTTP/1.1, we need a mechanism to control when/how they are used. With the Apache http client, the ConnectionReuseStrategy determines whether or not the connection should be persistent, while the ConnectionKeepAliveStrategy specifies the maximum idle time for a connection to be re-usable.
I am learning Java and was given a task to implement a simplistic version of HTTP server that is able to:
accept /GET requests and route them based on the uri
accept /POST requests and route them based on the uri
respond with a proper HttpResponse
Here is where I am so far:
ServerSocket server = new ServerSocket(7777);
Socket socket = server.accept();
String httpResponse = "Http/1.1 200/OK\r\n\r\n " + new Date();
socket.getOutputStream().write(httpResponse.getBytes("UTF-8"));
and I am stuck. I understand how to return a "Hello World" HTTP response to everything, but I have no idea how to deal with HTTP requests.
Could you give me a possible direction or a couple of next steps I can take to move forward?
If you're looking for a basic example of an HTTP request you could use a tool like Postman (Google Chrome extension) or HttpRequester (Firefox extension) which allows you to construct requests and view the code.
For example here is a basic request:
GET /news/headlines?hl=en HTTP/1.1
Host: news.google.com
Cache-Control: no-cache
This request gets the Google News headlines page.
If you're looking for how to use sockets you could start with the Java Lesson: All About Sockets
I want to retrieve the server's response as is, with all headers. The first thing that comes to mind is to use raw sockets. As I have learned from the search, there are 3 ways to indicate the end of response:
(1) closing the connection;
(2) examining Content-Length;
(3) getting all chunks in the case of Transfer-Encoding: Chunked.
There is also
(4) the timeout method: assume that the timeout means end of data, but the latter is not really reliable.
I want a general-case solution and do not want to
add a Connection: close line to the request itself.
In addition, it is recommended to use an existing library rather than re-invent the wheel.
Question:
How do I use an existing package, preferably, something already present in Android, to detect the end of HTTP response while having access (without interference) to the raw data stream?
UPD: forgot to mention that the HTTP request is given to me as a sequence of bytes. Yes, it is for testing.
PS
relevant reading:
End of an HTTP Response
Detect the end of an HTTP Request in Java
Detect end of HTTP request body
How HTTP Server inform its clients that the response has ended
Proper handling of chuncked Http Response within Socket
Detect the end of a HTTP packet
Android socket & HTTP response headers
Java HTTP GET response waits until timeout
I suggest to use a the Apache HTTP client package (http://hc.apache.org/httpclient-3.x/ ) so you don't need to implement all the finicky details of the HTTP protocol.
The Apache Http Client will give you access to the headers and their content, which may be enough for you.
If you really need access to the actual character sequence sent by the server (e.g. for debugging purposes), you could then intercept the communication by replacing the connection socket factory with your own to create "intercepting" sockets which store all data transferred in a buffer where your code can access it later on. See http://hc.apache.org/httpcomponents-client-4.3.x/tutorial/html/connmgmt.html#d5e418
I would like to send an HTTP request and get the response body, but I have URL that is stored on multiple servers. Let's say I have this list:
www.mysite.com 192.168.1.31
www.mysite.com 192.168.1.32
and I want to make the request to all the the different servers (different IP's) but same URL
Is there any option to do that in Java?
Yes, you can do that. Define the URL as being what you want and using the IP address in place of a domain name. I.E. http://192.168.1.31/path/to/index.html Then add the "Host: www.mysite.com" header before issuing the request. Any HTTP/1.1 compliant server will use the value of that header as the domain with which it was accessed.
Exactly how you accomplish this depends on whatever library you're using to make the connection but they should all have the ability to set arbitrary headers -- just make sure it doesn't overwrite your custom "Host" header with one of its own from the URL. See this other StackOverflow question for examples of how to implement an HTTP request.
This works because on the wire it's all IP. You can try it yourself using nc, socket, or even telnet.
(open TCP connection to 192.168.1.31 port 80)
GET /path/to/index.html HTTP/1.1
Host: www.mysite.com
<--blank line signals end of headers
I have a middlet that sends data to a servlet.
Anyone knows how can I calculate the size of request of the middlet in a servlet?
I'm tring to sum of request header + request size + certificate size
Is it correct?
With this I'm trying to recreate the request:
http://wikis.sun.com/display/WebServerdocs/Analyzing+SSL+Requests+and+Responses
I'm pretty sure that there is no simple way.
In fact, I'm not even sure this is a meaningful thing to measure. The request header and body are part of the HTTP application protocol. The certificate is sent as part of SSL / TLS setup ... before the HTTP stuff starts. And once SSL / TLS has started, the HTTP protocol is run "on top of" the SSL / TLS channel.
Even just measuring the amount of data in a HTTP request is tricky. A typical HTTP stack does not assemble the entire request message in one place, and does not keep a running total of the amount of data sent. Depending on the HTTP stack that you are using, you could (in theory) arrange to use a custom socket factory and socket streams that count the bytes sent.
Read more about SSL handshake:
I think the conversation round-trip may worry you as well as packet sizes.
Once handshake phase is finished and connection is reused (keepalive) only HTTP data is sent, encrypted of course.
like this:
$value) {
$size += (strlen($key) + strlen($value) + 3);
}
printf("size=%d",$size);
exit;
?>