We Keep Coding

Connecting Akka HTTP directly to an Akka Stream Flow

I've looked through the examples at https://doc.akka.io/docs/akka-http/current/introduction.html for Akka HTTP routing and strangely for something built on top of Akka Streams none of the examples connect to a stream.
Can somebody show a simple example of creating a Java DSL flow (not Scala please), and then connecting a Route directly to that flow?
Or am I missing the point and it's not possible but requires some CompletionStage code within the Route to wait for a result of glue code that calls a Flow?
Edit: to clarify the flow can do something like append a string to a posted request body.

Using akka streams to complete a route is definitely possible. It involves either:
a web socket route, see examples in the docs, or
a chunked http response (since you typically do not know the size of the response if it's fed from a stream). You can create a Chunked Entity from an akka stream Source of ByteStrings
you can also use other response types if the response size is known in advance, see docs for HttpEntity about their specifics

Edit: to clarify the flow can do something like append a string to a posted request body.
Michał's answer contains good links, so please give them a read. Akka HTTP is by default and always streaming with its data -- e.g. the entities. So for example to do a streaming "echo" which at the same time adds a suffix, you could do something like this:
path("test", () ->
// extract the request entity, it contains the streamed entity as `getDataBytes`
extractRequestEntity(requestEntity -> {
// prepare what to add as suffix to the incoming entity stream:
Source<ByteString, NotUsed> suffixSource =
Source.single(ByteString.fromString("\n\nADDS THIS AFTER INCOMING ENTITY"))
// concat the suffix stream to the incoming entity stream
Source<ByteString, Object> replySource = requestEntity.getDataBytes()
.concat(suffixSource);
// prepare and return the entity:
HttpEntity.Chunked replyEntity = HttpEntities.create(ContentTypes.TEXT_PLAIN_UTF8, replySource);
return complete(StatusCodes.OK, replyEntity);
})
);
Having that said, there is numerous ways to make use of the streaming capabilities, including framed JSON Streaming and more. You should also give the docs page about implications of streaming a read.

An easy way to detect the end of http response (raw socket, java)?

I want to retrieve the server's response as is, with all headers. The first thing that comes to mind is to use raw sockets. As I have learned from the search, there are 3 ways to indicate the end of response:
(1) closing the connection;
(2) examining Content-Length;
(3) getting all chunks in the case of Transfer-Encoding: Chunked.
There is also
(4) the timeout method: assume that the timeout means end of data, but the latter is not really reliable.
I want a general-case solution and do not want to
add a Connection: close line to the request itself.
In addition, it is recommended to use an existing library rather than re-invent the wheel.
Question:
How do I use an existing package, preferably, something already present in Android, to detect the end of HTTP response while having access (without interference) to the raw data stream?
UPD: forgot to mention that the HTTP request is given to me as a sequence of bytes. Yes, it is for testing.
PS
relevant reading:
End of an HTTP Response
Detect the end of an HTTP Request in Java
Detect end of HTTP request body
How HTTP Server inform its clients that the response has ended
Proper handling of chuncked Http Response within Socket
Detect the end of a HTTP packet
Android socket & HTTP response headers
Java HTTP GET response waits until timeout

I suggest to use a the Apache HTTP client package (http://hc.apache.org/httpclient-3.x/ ) so you don't need to implement all the finicky details of the HTTP protocol.
The Apache Http Client will give you access to the headers and their content, which may be enough for you.
If you really need access to the actual character sequence sent by the server (e.g. for debugging purposes), you could then intercept the communication by replacing the connection socket factory with your own to create "intercepting" sockets which store all data transferred in a buffer where your code can access it later on. See http://hc.apache.org/httpcomponents-client-4.3.x/tutorial/html/connmgmt.html#d5e418

HTTPClient never leaves socketRead() when executing GET on stream - workaround?

I am using Apache HttpClient (from Apache HTTP Components 4.3) in order to execute a GET against a ShoutCast stream:
CloseableHttpClient client = HttpClients.createDefault();
HttpGet request = new HttpGet("http://relay3.181.fm:8062/");
CloseableHttpResponse response = client.execute(request);
The call to client.execute() never returns, and according to the debugger it is a nested invocation to java.net.SocketInputStream#socketRead0() which is the last node in the call stack. From profiling the code, my only conclusion (based on a steadily rising number of char[] allocations) is that it simply "latches on" to the stream and keeps pulling bytes from the socket indefinitely.
What I would like is for the client to simply work normally and give me a HTTPResponse which I can use to pull what I want from the stream. As a matter of fact, I have been able to do so with other ShoutCast streams, but not this one.
Is there any way to work around this? Could I for example tell the client to break off after a certain number of bytes?

That site is very particular. If you don't specify a supported User-Agent (like Mozilla), the server keep streaming bytes. I don't know what these bytes are meant to represent, audio perhaps.
If you print out the bytes that you receive, you will see
ICY 200 OK
icy-notice1:<BR>This stream requires Winamp<BR>
icy-notice2:SHOUTcast Distributed Network Audio Server/Linux v1.9.8<BR>
icy-name:181.FM - The Beatles Channel
icy-genre:Oldies
icy-url:http://www.181.fm
content-type:audio/mpeg
icy-pub:1
icy-br:128
which indicates that the response is not a valid HTTP response. It is an ICY response from the ICY protocol.
Now the default HttpClient you are using uses a DefaultHttpResponseParser which is a
Lenient HTTP response parser implementation that can skip malformed
data until a valid HTTP response message head is encountered.
In other words, it keeps reading the bytes the server is sending until it finds a valid HTTP response header, which will never happen, thus the infinite read.
I don't think you will be able to accomplish what you want with the Http Components library. Either look for an ICY client implementation in Java or spin your own.

How to build HTTP DELETE request with JSON encoded body using AsyncHttpClient

I need to write a HTTP client which to communicate with Floodlight OpenFlow controller via its REST API.
For testing I did it in python, and it worked OK. But now I'm in a situation where it has to be done in Java, of which I'm admittedly still at the beginner's level. One of my apps uses AsyncHttpClient to dispatch async GET requests, and works just fine. Now as a Floodlight's REST client, it has to do POST and DELETE with JSON encoded body. My code for an async POST request works very much as expected.
But no luck with DELETE.
Somehow it doesn't write JSON string into its request body.
The code is almost identical with POST. For debugging, I don't feed an AsyncCompletionHandler instance to execute() method.
System.out.println(ofEntry.toJson()); // this returns {"name": "xyz"} as expected.
Future<Response> f = httpClient.prepareRequest(new RequestBuilder("DELETE")
.setUrl("http://" + myControllerBaseUrl + urlPathFlowPostDelete)
.setHeader("content-type", "application/json")
.setBody(ofEntry.toJson())
.build()).execute();
System.out.println(f.getStatusCode()); // returns 200.
System.out.println(f.getResponseBody()); // returns {"status" : "Error! No data posted."}.
Just to make sure, I peeped into packet dump with wireshark, and found out the server isn't lying :)
The author of the library has written an extensive amount of relevant, valuable information, but unfortunately I can't find example code specifically for building a DELETE request.
I'd very much appreciate any suggestions, pointers, and of course pinpoint solutions!

Not sure that replying to my own question is appropriate here, but I have just found a related thread at the floodlight-dev Google group.
Problem with Static Flow Pusher DELETE REST method
So this could be a problem with Floodlight REST API which requires message body for a DELETE request to identify what to be deleted, whereas AHC is simply compliant with RFC2616.
I will follow the thread at Google group, and see how it will conclude among developers.

How to configure HTTPServer to use content length and not transfer encoding: chunked?

I'm using java's HTTP Server object with web service implemeted by WebServiceProvider.
I see that no matter of the client request, the answer is chunked and i need it to be with content length.
so i'm assuming the problem is in the server and not the web server provider, right?
and how can i configure the http header to use content length and not chunked?
HttpServer m_server = HttpServer.create();
Endpoint ep= Endpoint.create(new ep());
HttpContext epContext = m_server.createContext("/DownloadFile");
ep.publish(downloadFileContext);

I assume you're talking about the com.sun.net.httpserver HTTPServer. I further assume that you're connecting the server to the service with a call to Endpoint.publish, using some service provider which supports HTTPServer.
The key is in the HttpExchange.sendResponseHeaders method:
If the response length parameter is greater than zero, this specifies an exact number of bytes to send and the application must send that exact amount of data. If the response length parameter is zero, then chunked transfer encoding is used and an arbitrary amount of data may be sent. The application terminates the response body by closing the OutputStream.
So, as long as the handler is passing a positive value for responseLength, Content-Length is used. Of course, to do that, it will have to know how much data it is going to send ahead of time, which it might well not. Whether it does or not depends entirely on the implementation of the binding, i'm afraid. I don't believe this is standardised - indeed, i don't believe that the WebServiceProvider/HTTPServer is standardised at all.
However, even if your provider is uncooperative, you have a recourse: write a Filter which adds buffering, and add it to the HttpContext which you are using to publish the service. I think that to do this, you would have to write an implementation of HttpExchange which buffers the data written to it, pass that down the filter chain for the handler to write its response to, then when it comes back, write the buffered content, setting the responseLength when it does so.