Hi I'm currently creating a proxy server and I need to implement a cache within the proxy server.
The cache is a hashmap with the URL being the key and the byte array of the URL will be the value.
Currently I am running into a problem when I am trying to write the byte array into the output stream of the client.
try{
if(!myCache.containsKey(urladdr))
{
IOUtils.copy(serverInput,toClient);
byte[] byteArray = IOUtils.toByteArray(connection);
myCache.addToCache(urladdr, byteArray);
}
else{
toClient.write(myCache.getValue(urladdr));
System.out.println("got from cache");
}
}
So basically if the URL is not in the cache yet, I will load the page using IOUtils.copy and then add the url + byte array to my cache.
If the URL is in my cache, I will get the byte array of the URL from the hashmap and write to the client. That is where my programme gets stuck, it just doesn't load the byte array into the client output stream.
Is it maybe a problem due to not sending any headers?
Help appreciated thanks
Related
I'm working on an issue where I've to pass some data from one spring boot application to another. The data size can range between 10-100MB. I get this byte array data from GCP blob storage as
public byte[] downloadFileFromGCS(String objectName, String bucketName) throws IOException {
val storage = StorageOptions.getDefaultInstance().getService();
val blob = storage.get(BlobId.of(bucketName, objectName));
return blob.getContent();
}
The problem I'm facing is that I've to send it to different spring boot application for some processing, I've tried many different approaches like:
encoding into base64: which is not working because of huge size of byte files) by sending as Content-type:application/json
sending directly as byte array (not working because teh byte array values get changed/data loss)
Can someone suggest me what should I do in order to achieve this or any different approach like not sending byte[] array directly but using something different.
I have an encrypted (128-AES-CTR-NoPadding) video residing on a server which I need to decrypt as it downloads, so that user can stream it (in normal players/web).
I understand the components of this solution and how they should be put together to make this work. It partially works but for the rest I just can't implement streaming right. I have been reading and learning from examples (most of which is playing a file on disk, which is not the case here) on this for the past week and have come to conclusion this is beyond me and I need some help.
Details
I am using a Lightweight webserver (nanoHttpd) acting as a proxy to download the encrypted data from remote server and serve decrypted data. Below are the main codes inside my NanoHTTPD.serve method.
//create urlConnection to encrypted video file with proper headers (ie range headers) as request received by the proxy server
InputStream inputStream = new CipherInputStream(cipher, urlConnection.getInputStream());
return newChunkedResponse(status, contentType,inputStream);
So now if I go to my NanoHttpd webserver (http://localhost:9000), the file starts downloading and after the download completes, the file is fully decrypted and playable as expected.
So this ensures that getting encrypted data from the server and serving decrypted data is working correctly.
But when any video player (html5, vlc) is asked to stream the video from that url, it simply does not work.
If the above code in NanoHTTPD.serve changed to
//create urlConnection to cleardata video file with proper headers (ie range headers) as request received by the proxy server
InputStream inputStream = urlConnection.getInputStream();
return newChunkedResponse(status, contentType,inputStream);
And then try to stream from the aforementioned players, it'll work just fine.
So this ensures that the web proxy is correcting retrieving and feeding data.
Potential problem
To support range requests from the video player we will need to correctly skip to block boundary that is a multiple of cipher block size. So it's possible that when video player is requesting data with header (range: bytes 34-44), the CipherInputStream is probably failing to decrypt the data since the inputstream has data from 34-44. But I am at a loss on how to do this with urlConnection.getInputStream() and CipherInputStream.
But even without this, it should at least start playing the first few seconds because the first request video player sends is (range: 0-) which means inputStream is starting from index 0 so CipherInputStream should be able to decrypt and serve those initial bytes and the video should continue playing.
I am at a complete loss because I don't know how to debug this. Any ideas, sample codes are welcome, I'll try them out and post the results here.
I have figure this out. I'll post the solution here for others.
The problem here was the ranged requests. If the proxy does not send proper responses to these range requests, the playback will fail. This can fail due to a number of reasons.
Your requests to remote server is missing proper range headers.
Your requests to remote server is returning proper ranged data, but you are not decrypting it correctly. This was my case. Of course this decryption process will vary cipher to cipher. For me, I used (AES/CRT/NOPADDING), I was supplying correct iv for the offset. How to calculate iv for offset is described here.
As far as code samples go, I only had to add one line before
InputStream inputStream = new CipherInputStream(cipher, urlConnection.getInputStream());
return newChunkedResponse(status, contentType,inputStream);
which is
jumpToOffset(cipher,....);
After this everything was working correctly including seeking of the video.
Please consider the below information:
I am working in Powerbuilder technology.
An OLE web browser object is defined in a window of the Powerbuilder.
Expected scenerio: Data from the Powerbuilder window will be passed to the server where the data will be parsed and a report will be generated which will be viewed in the OLE object in the powerbuilder window.
Steps taken:
While sending data from the Powerbuilder window to the server, I do not want to append the data in the URL and send it to the Server. Therefore, I cannot use the Get method and so have tried with Post method.
Used powerbuilder's PostURL method to send the data to the server.
Below is an example:
Int li_return
String ls_url1, ls_url2
Blob lb_blob
Long ll_bloblen
n_internetresult lu_data
If GetContextService("Internet", iinet_base) = 1 Then
ls_url1 = "http:// ip-address-of-the-server:port no./xxx?"
ls_url2 = "here the data, that will be passed to the server, are stored"
//*the data are converted into blob data*
lb_blob = blob(ls_url2)
ll_bloblen = len(lb_blob)
ls_header = "Content-Type: " + "application/x-www-form-urlencoded~nContent-Length: " + string(ll_bloblen) + "~n~n"
li_return = iinet_base.PostURL(ls_url1, lb_blob, ls_header, 8080, lu_data)
End if
The above code is working fine. We can send data to the server successfully.
Problem
The data is converted to Blob and sent to the server. In the server side, its getting difficult to convert the blob data to the data that we were passing from the Powerbuilder window. In the server side, the language used is java.
So is there any way to send the data via post method without converting the arguments to blob data type in powerbuilder?
Powerbuilder version used: PowerBuilder Classic 12.5
First of all, you can also pass the encoding to use when converting a string into a blob. I would recommend that so that you know what you are receiving on java side. If you don't, the systems default encoding will be used.
The main difference between a string and a blob in Powerbuilder is the missing termination null at the end of the blob, so converting shouldn't be that hard. Just do
new String (myBlob,"encoding")
Where myBlob is the received blob as an array of Bytes and encoding is the encoding used on Powerbuilder side.
I have not tried this myself, but I think it should work.
I'm having a problem to post a image to my wcf rest service. I'm posting some parameters of which one of them is a base64 utf-8 encoded string (the image).
My problem is that every time I post I get "bad request". Here is the code
public String PostImage(Drawable img) throws Exception
{
HttpPost httpost = new HttpPost("http://10.0.2.2:1374/uploaditem");
JSONStringer json = JSONStringer()
.object()
.key("ipm")
.object()
.key("name").value("test")
.key("description").value("asfa")
.key("categoryid").value(1)
.key("data").value(ConvertImgToBase64Str(img))
.key("imagetype").value(2)
.key("tags").value("test;test")
.endObject()
.endObject();
StringEntity entity = new StringEntity(json.toString());
entity.setContentType("application/json;charset=UTF-8");//text/plain;charset=UTF-8
entity.setContentEncoding(new BasicHeader(HTTP.CONTENT_TYPE,"application/json;charset=UTF-8"));
httpost.setEntity(entity);
return ExcecutePostRequest(httpclient,httpost);
}
//Method to convert the image to base64encoded string
private String ConvertImgToBase64Str(Drawable img) {
Bitmap bitmap = ((BitmapDrawable)img).getBitmap();
ByteArrayOutputStream stream = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.PNG, 100, stream);
byte[] bitmapdata = stream.toByteArray();
return Base64.encodeToString(bitmapdata, Base64.URL_SAFE);
}
It is something with the encoded string, but what?
I don't see why it should be a problem with the encoded string.
Firstly, try removing the data attribute in the string object and see if you get the same problem. This way you can eliminate the possibility it is due to the the encoding and ensure you are making the request correctly.
Secondly print the http message being sent and format check it. If you have access to the server log the message being received and any details that may elaborate on the Bad Request. I would have thought WCF will be printing something to stderr if it is responding with a bad request so try just checking the existing logs first.
EDIT
I don't think there should be problems with strange characters because the character used in base64 encoding don't fall outside the ASCi range.
You should check the size of the request being sent. If you images are big you will get a big base64 encoded string which might exceed the server post limit.
If you can't get to the server logs which I think would clarify this if it was the problem you could test it by sending a smaller image.
I want to send a file to the Browser via the REST Interface.
Can you suggest the most efficient way to do it, Keeping in mind the following?
Not much traffic.
I am fetching the file from HBase which means when I fetch it from HBase I get it in Byte Array.
The files are not in any folder in the server. The files can only be fetched from the HBase table.
The Front end is PHP and I do not know PHP.
In the REST api you can just pass the byte array to Response and it takes care of itself.
Using the following code -
#Produces("image/jpg")
public Response getImage() {
<Fetch it from where ever you have it>
Response.ok(<byteArrayOfTheFile>).build();
}
I am giving case study of WebService by which i send file:
It is always good to encode the file content and send it to the destination where they will be decode it and read the content.
Sending as an attachment is always open to the world becasue it is not encrypted.And if the network having high trafic chances of failure is high.