We Keep Coding

Streaming [Random Access] encrypted (AES-CTR) video on the fly using web proxy (nanoHTTPD)

I have an encrypted (128-AES-CTR-NoPadding) video residing on a server which I need to decrypt as it downloads, so that user can stream it (in normal players/web).
I understand the components of this solution and how they should be put together to make this work. It partially works but for the rest I just can't implement streaming right. I have been reading and learning from examples (most of which is playing a file on disk, which is not the case here) on this for the past week and have come to conclusion this is beyond me and I need some help.
Details
I am using a Lightweight webserver (nanoHttpd) acting as a proxy to download the encrypted data from remote server and serve decrypted data. Below are the main codes inside my NanoHTTPD.serve method.
//create urlConnection to encrypted video file with proper headers (ie range headers) as request received by the proxy server
InputStream inputStream = new CipherInputStream(cipher, urlConnection.getInputStream());
return newChunkedResponse(status, contentType,inputStream);
So now if I go to my NanoHttpd webserver (http://localhost:9000), the file starts downloading and after the download completes, the file is fully decrypted and playable as expected.
So this ensures that getting encrypted data from the server and serving decrypted data is working correctly.
But when any video player (html5, vlc) is asked to stream the video from that url, it simply does not work.
If the above code in NanoHTTPD.serve changed to
//create urlConnection to cleardata video file with proper headers (ie range headers) as request received by the proxy server
InputStream inputStream = urlConnection.getInputStream();
return newChunkedResponse(status, contentType,inputStream);
And then try to stream from the aforementioned players, it'll work just fine.
So this ensures that the web proxy is correcting retrieving and feeding data.
Potential problem
To support range requests from the video player we will need to correctly skip to block boundary that is a multiple of cipher block size. So it's possible that when video player is requesting data with header (range: bytes 34-44), the CipherInputStream is probably failing to decrypt the data since the inputstream has data from 34-44. But I am at a loss on how to do this with urlConnection.getInputStream() and CipherInputStream.
But even without this, it should at least start playing the first few seconds because the first request video player sends is (range: 0-) which means inputStream is starting from index 0 so CipherInputStream should be able to decrypt and serve those initial bytes and the video should continue playing.
I am at a complete loss because I don't know how to debug this. Any ideas, sample codes are welcome, I'll try them out and post the results here.

I have figure this out. I'll post the solution here for others.
The problem here was the ranged requests. If the proxy does not send proper responses to these range requests, the playback will fail. This can fail due to a number of reasons.
Your requests to remote server is missing proper range headers.
Your requests to remote server is returning proper ranged data, but you are not decrypting it correctly. This was my case. Of course this decryption process will vary cipher to cipher. For me, I used (AES/CRT/NOPADDING), I was supplying correct iv for the offset. How to calculate iv for offset is described here.
As far as code samples go, I only had to add one line before
InputStream inputStream = new CipherInputStream(cipher, urlConnection.getInputStream());
return newChunkedResponse(status, contentType,inputStream);
which is
jumpToOffset(cipher,....);
After this everything was working correctly including seeking of the video.

Does the http response return a stream or the actual response for large responses?

Does a http client return a stream as the response or does it reads the bytes from the stream and returns us the response ?
If it returns a stream , the user will have to read from that stream to get the response right ?
For example - We are using ApacheAsyncHttpClient to get blobs from Azure Storage.
The REST response should only contain the stream from where to read the blob right ? But what I observed is that the client itself is reading all the bytes from the stream before invoking our completed Callback.
And unless the data from the stream is read , the HTTP connection is kept open ?
Edit - We cant use sdk for reasons beyond the scope of this question.
My question is specifically for ApacheAsyncHttpClient.
It reads the entire stream before calling the completed callback. Ideally, what I would want is that it returns me the stream and my application will read from that stream.
Is there a way to do that ?
And are all http clients behave similarly ?

Parse JSON packet input in java

I have a TCP socket connected to a server communicating via JSON. Since it's a stream the packets often merge into one larger through the inputstream, and I've been trying to parse it correctly but haven't found a successful way to read the input.
The input looks something like this
+#$3{"data1":"datahere","moredata":25}2*${"data1":"differentdata","otherdata":001}
This I have parsed to be
{"data1":"datahere","moredata":25}{"data1":"differentdata","otherdata":001}
Using a reader only returns the first json object when it is set to lenient, otherwise it throws an exception. The JSON comes from a socket, and is a result of packet merging. What can be done in a JSON library to parse these separately?

Messed up string value received on the server side - multilingual

I have written client side rest services on php and the server side is on Java.
I am sending a value அகம்.com.my using the client side webservice and trying to perform some operation on the server side. The received value is à® à®à®®à¯.com.my
what can I do to receive the string in the same format. I tried to encode the value into base64 and decoded on the server side but still the received value is different.

Make sure you add a 'content-type' header in your requests.
You need to inform the server what kind of data you're sending so it will know how to process it correctly.
For example, if you're sending plain text in UTF8, you can add the following line in your client code :
header('Content-Type: text/html; charset=utf-8')

Fixed it so posting the solution.
on php end:
urlencode($domainName);
Decoded and used UTF-8 on the server side (java):
domainName= java.net.URLDecoder.decode(domainName, "UTF-8");

sending JSON data to client using writeandflush() from netty showing Invalid JSON type

I'm using java code for sending JSON data to flash for that I'm using netty server 4.0.23.
By that I'm using two different data to send to the same client frequently. I used channel.writeAndFlush(). So far everything happens good, while receiving data in client both the data are merged together and gives Invalid Json.
I traced the sending data in java there it is in correct format. It is getting invalid while receiving in client side. The two different JSON data are merged into a single JSON Data, so that the Invalid Json type error is occuring. If I put timer delay in between the two data, then the data are correctly received in client side. How can I receive proper data without applying the delay inbetween the data while sending.....
I used the following pipelines
pipeline.addLast(new DelimiterBasedFrameDecoder(8192, Delimiters.lineDelimiter()));
pipeline.addLast(new StringDecoder());
pipeline.addLast(new StringEncoder());
pipeline.addLast("handler", new ServerHandler());

It sounds like your client (the receiving part) does not handle the received JSON messages properly. TCP is streaming protocol, which means two writes on one side can be seen as one read on the other side. For example, when a server sends "A" and "B", a client will not always see "A" and "B" but will sometimes see "AB", and it's fully fine.
To deal with this sort of situation, you need to 'frame' a message. It is usually done in Netty by inserting a framing decoder.
In your case, the client is written in Flash, so you'd better simply use XMLSocket and let your server delimit each JSON message with NUL (0) byte.
If you are interested in framing, please read the Netty user guide.