I'd need to send an HTTP/2 request via a TCP socket from my Java classes. I've adapter a piece of code which works for plain HTTP/1.1. However it does not output any response nor error code when using HTTP/2.
Can you see anything wrong in it? The server I'm trying to reach it's on https://localhost:8443
Socket s = new Socket(InetAddress.getByName("localhost"), 8443);
PrintWriter pw = new PrintWriter(s.getOutputStream());
pw.print("GET / HTTP/2.0\r\n");
pw.print("Host: localhost:8443\r\n\r\n");
pw.flush();
BufferedReader br = new BufferedReader(new InputStreamReader(s.getInputStream()));
String t;
while((t = br.readLine()) != null) System.out.println(t);
br.close();
Thanks!
That will not work.
HTTP/2 is a binary protocol, not a textual protocol, so in order to use a raw socket you have to generate the proper bytes that form a HTTP/2 request.
This is quite complicated as it requires that you implement HPACK to compress the headers, so you will be far better off using a Java library that does HTTP/2 for you, with a higher level API (rather than using raw sockets).
[Disclaimer: I'm the HTTP/2 implementer in Jetty].
Jetty offers a low-level HTTP/2 client that allows you to deal with HTTP/2 frames, and a high-level HTTP client that can send generic HTTP request using the HTTP/2 format.
For the first you can find an example here: https://github.com/eclipse/jetty.project/blob/jetty-9.4.18.v20190429/jetty-alpn/jetty-alpn-java-client/src/test/java/org/eclipse/jetty/alpn/java/client/JDK9HTTP2ClientTest.java
For the second one there is this section of the documentation: https://www.eclipse.org/jetty/documentation/9.4.x/http-client-transport.html#_http_2_transport
Related
I'm trying to build a HTTP server in Java out of curiosity.
I know that HTTP uses sockets underneath(correct me if i'm wrong). So started programming initially using ServerSocket class.
public class Server
{
public static void main(String[] args) throws IOException
{
System.out.println("Listening.....");
ServerSocket ss = new ServerSocket(80);
while(true)
{
Socket s = ss.accept();
Scanner sc = new Scanner(s.getInputStream());
while(sc.hasNextLine())
{
String line = sc.nextLine();
if(line.equals(""))
break;
else
System.out.println(line);
}
System.out.println("-------------------------------");
PrintStream ps = new PrintStream(s.getOutputStream());
ps.println("Hello from Server");
s.close();
ps.close();
sc.close();
}
}
}
(I'm using Thread in my actual code to serve multiple users. I've just provided the basic code.)
I'm getting all the headers from the web browser. But how can I send the files and images?
For, simple HTML I can read the file and use PrintStream to print it on the web browser.
But how can I send JavaScript, Images etc to the browser?
HTTP has a protocol to it, you need to follow that protocol. The HTTP 1.1 protocol spec still in wide use is RFC 2616 (though it has officially been replaced with newer RFCs 7230, 7231, 7232, 7233, 7234, and 7235).
In my answer to another question, I show the correct way to read an inbound HTTP request from a Java Socket directly.
When sending a reply back, you can use a PrintStream or PrintWriter to send the response HTTP headers. However, the body content is sent as raw bytes, based on the format specified by the Content-Type and Transfer-Encoding response headers. Typically, you would just send the raw bytes directly to the socket's OutputStream, or at least to a BufferedOutputStream attached to it. If you are sending a pre-existing file from disk, regardless of its type, you could just open an InputStream for the file and then copy its data directly to the socket's OutputStream. If you are generating data dynamically, then you would send the data to the socket's OutputStream using whatever intermediate classes are appropriate. Print... classes are only appropriate for textual data, not binary data, like images.
That being said, Java has its own HttpServer and HttpsServer classes. You should consider using them.
Basically the same way. You should "print" the raw bytes to the socket's OutputStream.
However, for the browser to be able to understand it, you need to shape your response according to the HTTP/1.1 protocol. Specifying a Content-Type header will tell the browser what it is receiving from you. Specifying a Content-Length header will tell the browser how many bytes it is receiving from you. Etc.
I am attempting to connect to an HTTPS endpoint in Java. Every method I have tried (more details below) ends up generating this stack trace:
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:168)
at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)
at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:798)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1138)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:753)
at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)
I have tried:
Connecting with the javax SOAP libs and a new URL("https://...")
Connecting with new URL("https://...").openConnection()
Creating an SSL connection by hand:
Security.addProvider(new com.sun.net.ssl.internal.ssl.Provider());
SSLSocketFactory factory = (SSLSocketFactory) SSLSocketFactory.getDefault();
SSLSocket socket = (SSLSocket) factory.createSocket("...", 443);
Writer out = new OutputStreamWriter(socket.getOutputStream());
// https requires the full URL in the GET line
//
out.write("GET / HTTP/1.0\r\n");
out.write("\r\n");
out.flush();
// read response
BufferedReader in = new BufferedReader(
new InputStreamReader(socket.getInputStream()));
int c;
while ((c = in.read()) != -1) {
System.out.write(c);
}
out.close();
in.close();
socket.close();
A few more details:
Every method I have tried has worked against other SSL servers, it's this particular server (I am not at liberty to discuss what server, it's a business partner)
I can connect to this server both with a web browser, and with a faked up SOAP request with curl; This is something Java-specific.
So, it seems pretty clear that there is some disagreement between Java and the HTTPS server over how the handshake should go down, which probably means the server has some strange SSL configuration. However, I don't have direct access to the server, and the people who do are halfway around the world, so communication is a little strained due to very different timezones.
If my assumptions there are correct, what possible SSL problems could there be? What might cause something like this? Where can I ask the people in control of the server to look for issues? When I do the request with curl, I get back these server configuration headers:
Server: Apache/2.2.9 (Debian) mod_jk/1.2.26 PHP/5.2.6-1+lenny10 with Suhosin-Patch mod_ssl/2.2.9 OpenSSL/0.9.8g mod_perl/2.0.4 Perl/v5.10.0
X-Powered-By: PHP/5.2.6-1+lenny10
X-SOAP-Server: NuSOAP/0.7.3 (1.114)
It is an SSL version problem. The server only supports SSLv3, and Java will start at v2, and attempt to negotiate upwards, but not all servers support that type of negotiation.
Forcing java to use SSLv3 only is the only solution I'm aware of.
Edit, there are two ways to do this that I'm aware of:
If you are creating the socket by hand, you can set the enabled protocols
socket.setEnabledProtocols(new String[] { "SSLv3" });
If you are using a higher level library, you probably need to set all SSL requests to use v3 only, which is accomplished with the "https.protocols" system property:
java -Dhttps.protocols=SSLv3
Maybe also try setting the HTTP version to 1.1 instead of 1.0, as there's some real advantages to the newer standard.
I'm trying to build a "full-duplex" HTTP streaming request using Apache HTTPClient.
In my first attempt, I tried using the following request code:
URL url=new URL(/* code goes here */);
HttpPost request=new HttpPost(url.toString());
request.addHeader("Connection", "close");
PipedOutputStream requestOutput=new PipedOutputStream();
PipedInputStream requestInput=new PipedInputStream(requestOutput, DEFAULT_PIPE_SIZE);
ContentType requestContentType=getContentType();
InputStreamEntity requestEntity=new InputStreamEntity(requestInput, -1, requestContentType);
request.setEntity(requestEntity);
HttpEntity responseEntity=null;
HttpResponse response=getHttpClient().execute(request); // <-- Hanging here
try {
if(response.getStatusLine().getStatusCode() != 200)
throw new IOException("Unexpected status code: "+response.getStatusLine().getStatusCode());
responseEntity = response.getEntity();
}
finally {
if(responseEntity == null)
request.abort();
}
InputStream responseInput=responseEntity.getContent();
ContentType responseContentType;
if(responseEntity.getContentType() != null)
responseContentType = ContentType.parse(responseEntity.getContentType().getValue());
else
responseContentType = DEFAULT_CONTENT_TYPE;
Reader responseStream=decode(responseInput, responseContentType);
Writer requestStream=encode(requestOutput, getContentType());
The request hangs at the line indicated above. It seems that the code is trying to send the entire request before it gets the response. In retrospect, this makes sense. However, it's not what I was hoping for. :)
Instead, I was hoping to send the request headers with Transfer-Encoding: chunked, receive a response header of HTTP/1.1 200 OK with a Transfer-Encoding: chunked header of its own, and then I'd have a full-duplex streaming HTTP connection to work with.
Happily, my HTTPClient has another NIO-based asynchronous client with good usage examples (like this one). My questions are:
Is my interpretation of the synchronous HTTPClient behavior correct? Or is there something I can do to continue using the (simpler) synchronous HTTPClient in the manner I described?
Does the NIO-based client wait to send the whole request before seeking a response? Or will I be able to send the request incrementally and receive the response incrementally at the same time?
If HTTPClient will not support this modality, is there another HTTP client library that will? Or should I be planning to write a (minimal) HTTP client to support this modality?
Here is my view on skim reading the code:
I cannot completely agree with the fact that a non-200 response means failure. All 2XX responses are mostly valid. Check wiki for more details
For any TCP request, I would recommend to receive the entire response to confirm that it is valid. I say this because, a partial response may mostly be treated as bad response as most of the client implementations cannot make use of it. (Imagine a case where server is responding with 2MB of data and it goes down during this time)
A separate thread must be writing to the OutputStream for your code to
work.
The code above provides the HTTPClient with a PipedInputStream.
PipedInputStream makes bytes available as they are written to the corresponding OutputStream.
The code above does not write to the OutputStream (which must be done by a separate thread.
Therefore the code is hanging exactly where your comment is.
Under the hood, the Apache client says "inputStream.read()" which in the case of piped streams requires that outputStream.write(bytes) was called previously (by a separate thread).
Since you aren't pumping bytes into the associated OutputStream from a separate thread the InputStream just sits and waits for the OutputStream to be written to by "some other thread."
From the JavaDocs:
A piped input stream should be connected to a piped output stream;
the piped input stream then provides whatever data bytes are written
to the piped output stream.
Typically, data is read from a PipedInputStream object by one thread
and data is written to the corresponding PipedOutputStream by some
other thread.
Attempting to use both objects from a single thread is not
recommended, as it may deadlock the thread.
The piped input stream contains a buffer, decoupling read operations
from write operations, within limits. A pipe is said to be "broken"
if a thread that was providing data bytes to the connected piped
output stream is no longer alive.
Note: Seems to me, since piped streams and concurrency were not mentioned in your problem statement, that it's not necessary. Try wrapping a ByteArrayInputStream() with the Entity object instead first for a sanity check... that should help you narrow down the issue.
Update
Incidentally, I wrote an inversion of Apache's HTTP Client API [PipedApacheClientOutputStream] which provides an OutputStream interface for HTTP POST using Apache Commons HTTP Client 4.3.4. This may be close to what you are looking for...
Calling-code looks like this:
// Calling-code manages thread-pool
ExecutorService es = Executors.newCachedThreadPool(
new ThreadFactoryBuilder()
.setNameFormat("apache-client-executor-thread-%d")
.build());
// Build configuration
PipedApacheClientOutputStreamConfig config = new
PipedApacheClientOutputStreamConfig();
config.setUrl("http://localhost:3000");
config.setPipeBufferSizeBytes(1024);
config.setThreadPool(es);
config.setHttpClient(HttpClientBuilder.create().build());
// Instantiate OutputStream
PipedApacheClientOutputStream os = new
PipedApacheClientOutputStream(config);
// Write to OutputStream
os.write(...);
try {
os.close();
} catch (IOException e) {
logger.error(e.getLocalizedMessage(), e);
}
// Do stuff with HTTP response
...
// Close the HTTP response
os.getResponse().close();
// Finally, shut down thread pool
// This must occur after retrieving response (after is) if interested
// in POST result
es.shutdown();
Note - In practice the same client, executor service, and config will likely be reused throughout the life of the application, so the outer prep and close code in the above example will likely live in bootstrap/init and finalization code rather than directly inline with the OutputStream instantiation.
I am attempting to connect to an HTTPS endpoint in Java. Every method I have tried (more details below) ends up generating this stack trace:
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:168)
at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)
at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:798)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1138)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:753)
at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)
I have tried:
Connecting with the javax SOAP libs and a new URL("https://...")
Connecting with new URL("https://...").openConnection()
Creating an SSL connection by hand:
Security.addProvider(new com.sun.net.ssl.internal.ssl.Provider());
SSLSocketFactory factory = (SSLSocketFactory) SSLSocketFactory.getDefault();
SSLSocket socket = (SSLSocket) factory.createSocket("...", 443);
Writer out = new OutputStreamWriter(socket.getOutputStream());
// https requires the full URL in the GET line
//
out.write("GET / HTTP/1.0\r\n");
out.write("\r\n");
out.flush();
// read response
BufferedReader in = new BufferedReader(
new InputStreamReader(socket.getInputStream()));
int c;
while ((c = in.read()) != -1) {
System.out.write(c);
}
out.close();
in.close();
socket.close();
A few more details:
Every method I have tried has worked against other SSL servers, it's this particular server (I am not at liberty to discuss what server, it's a business partner)
I can connect to this server both with a web browser, and with a faked up SOAP request with curl; This is something Java-specific.
So, it seems pretty clear that there is some disagreement between Java and the HTTPS server over how the handshake should go down, which probably means the server has some strange SSL configuration. However, I don't have direct access to the server, and the people who do are halfway around the world, so communication is a little strained due to very different timezones.
If my assumptions there are correct, what possible SSL problems could there be? What might cause something like this? Where can I ask the people in control of the server to look for issues? When I do the request with curl, I get back these server configuration headers:
Server: Apache/2.2.9 (Debian) mod_jk/1.2.26 PHP/5.2.6-1+lenny10 with Suhosin-Patch mod_ssl/2.2.9 OpenSSL/0.9.8g mod_perl/2.0.4 Perl/v5.10.0
X-Powered-By: PHP/5.2.6-1+lenny10
X-SOAP-Server: NuSOAP/0.7.3 (1.114)
It is an SSL version problem. The server only supports SSLv3, and Java will start at v2, and attempt to negotiate upwards, but not all servers support that type of negotiation.
Forcing java to use SSLv3 only is the only solution I'm aware of.
Edit, there are two ways to do this that I'm aware of:
If you are creating the socket by hand, you can set the enabled protocols
socket.setEnabledProtocols(new String[] { "SSLv3" });
If you are using a higher level library, you probably need to set all SSL requests to use v3 only, which is accomplished with the "https.protocols" system property:
java -Dhttps.protocols=SSLv3
Maybe also try setting the HTTP version to 1.1 instead of 1.0, as there's some real advantages to the newer standard.
I am decoding http packets.
And I faced a problem that chunk problem.
When I get a http packet it has a header and body.
When transefer-encoding is chunked I don't know what to do ?
Is there a useful API or class for dechunk the data in JAVA ?
And if someone , experienced about http decoding , please show me a way how to do this ?
Use a fullworthy HTTP client like Apache HttpComponents Client or just the Java SE provided java.net.URLConnection (mini tutorial here). Both handles it fully transparently and gives you a "normal" InputStream back. HttpClient in turn also comes with a ChunkedInputStream which you just have to decorate your InputStream with.
If you really insist in homegrowing a library for this, then I'd suggest to create a class like ChunkedInputStream extends InputStream and write logic accordingly. You can find more detail how to parse it in this Wikipedia article.
Apache HttpComponents
Oh, and if we are talking about the client side, HttpUrlConnection does this as well.
If you are looking for a simple API try Jodd Http library (http://jodd.org/doc/http.html).
It handles Chunked transfer encoding for you and you get the whole body as a string back.
From the docs:
HttpRequest httpRequest = HttpRequest.get("http://jodd.org");
HttpResponse response = httpRequest.send();
System.out.println(response);
Here is quick-and-dirty alternative that requires no dependency except Oracle JRE:
private static byte[] unchunk(byte[] content) throws IOException {
ByteArrayInputStream bais = new ByteArrayInputStream(content);
ChunkedInputStream cis = new ChunkedInputStream(bais, new HttpClient() {}, null);
return readFully(cis);
}
It uses the same sun.net.www.http.ChunkedInputStream as java.net.HttpURLConnection does behind the scene.
This implementation doesn't provide detailed exceptions (line numbers) on wrong content format.
It works with Java 8 but could fail in with next release. You've been warned.
Could be useful for prototyping though.
You can choose any readFully implementation from Convert InputStream to byte array in Java.