I have a requirement to load big images into chrome browser, in pieces from the server and after it finishes downloading the file completely, if the user wants to look at the file again, then the browser should load it from its cache.
I have done the following to achieve this:
I have written a HTML file and I will be displaying the big image using 'img' tag.
Whenever the user requests for the image file, I'll make a HEAD request to the server enquiring about the content-length and last-modified-date.
The server replies back with the information based on the request and I divide the size of the file into 10 pieces and then make 10 sequential requests to fetch the image file. (Using for loop).
After chrome downloads the entire file and the next time user wants to look at the file, then I am making chrome to send If-Modified-Since query to the server.
If the server responds back with the code 304, then chrome will understand that it has to fetch the file from its cache.
I am facing the following problems:
1. I am getting about 100 bytes per response. Where should I store it and how should I make my img tag understand that it the client is still downloading the file.3
2. How should I recognize the file from the pool of files present in chrome's cache ?
3. The next time someone comes to view the same image, from where should I display the content from ?
Please Help. Can anyone also provide me with some example code or a link to some sample code.
Thanks and Regards,
Akshay Sahu
I got the way to perform this operation.
I am downloading all the bytes of a file in range and using File API of HTML 5 I am creating a file in the user HDD. Therefore, experiencing a complete resume download experience.
Thanks and Regards,
Akshay Sahu.
Related
I have the most basic problem ever. The user wants to export some data which is around 20-70k records and can take from 20-40 seconds to execute and the file can be around 5-15MB.
Currently my code is as such:
User clicks a button which makes an API call to a Java Lambda
AWS Lambda Handler calls a method to get the data from DB and generate excel file using Apache POI
Set Response Headers and send the file as XLSX in the response body
I am now faced with two bottlenecks:
API Gateway times out after 29 seconds; if file takes longer to
generate it will not work and user get 504 in the browser
Response from lambda can only be 6MB, if file is bigger the user will
get 413/502 in the browser
What should be my approach to just download A GENERATED RUNTIME file (not pre-built in s3) using AWS?
If you want to keep it simple (no additional queues or async processing) this is what I'd recommend to overcome the two limitations you describe:
Use the new AWS Lambda Endpoints. Since that option doesn't use the AWS API Gateway, you shouldn't be restricted to the 29-sec timeout (not 100% sure about this).
Write the file to S3, then get a temporary presigned URL to the file and return a redirect (HTTP 302) to the client. This way you won't be restricted to the 6MB response size.
Here are the possible options for you.
Use Javascript skills to rescue. Accept the request from browser/client and immediately respond from server that your file preparation is in progress. Meanwhile continue preparing the file in the background (sperate job). Using java script, keep polling the status of file using separate request. Once the file is ready return it back.
Smarter front-end clients use web-sockets to solve such problems.
In case DB query is the culprit, cache the data on server side, if possible, for you.
When your script takes more than 30s to run on your server then you implement queues, you can get help from this tutorial on how to implement queues using SQS or any other service.
https://mikecroft.io/2018/04/09/use-aws-lambda-to-send-to-sqs.html
Once you implement queues your timeout issue will be solved because now you are fetching your big data records in the background thread on your server.
Once the excel file is ready in the background then you have to save it in your s3 bucket or hard disk on your server and create a downloadable link for your user.
Once the download link is created you will send that to your user via email. In this case, you should have your user email.
So the summary is Apply queue -> send a mail with the downloadable file.
Instead of some sophisticated solution (though that would be interesting).
Inventory. You will split the Excel in portions of say 10 k rows. Calculate the number of docs.
For every Excel generation called you have a reduced work load.
Whether e-mail, page with links, using a queue you decide.
The advantage is staying below e-mail limits, response time-outs, denial of service.
(In Excel one could also create a master document, but I have no experience.)
I have a Java Web application that generates a report and I have the ability to export that report to an excel file, problem is whenever I generate it as an excel file a "Connection Timed Out" page is being displayed on a firefox web browser.
Basically I have no idea why is this happening, I see no problems in my code could it be server issues or the amount of data I'm generating? Also no error logs are being displayed.
Any advise, suggestions would be of great help thanks.
It sounds like the request is taking too long, and being timed out. Basically it's taking too long to generate the report. This could be too long for the client, the app server or the webserver (if you have a separate webserver). You have a few options:
Find out where the timeout settings are in the Application Server and increase them
Speed up your report writing code so it doesn't take as long
Make the report writer an asyncronous job (eg by kicking of the report generation in a new thread), and have the client pole the server until it's finished, then request the file.
Update based on OP comment:
Regarding the last suggestion:
If the report's generated by another thread, the current request will return before the report is generated, so the browser won't have to wait at all. However, this is quite a large amount of work because you have to have a way for the client-side code to find out when the report is finished. Also, you are not supposed to launch your own threads from a Servlet.
Maybe you can make the original request via AJAX, or in an iFrame? This way the restrictive timeout threshold may not be in effect.
I have done a backup and restore application for java phone including nokia, it works fine but pictures larger than 1 MB cannot be uploaded is that possible to upload a file larger than 1 MB, if so please suggest me whether it is possible on HTTP or FTP.
Thank you.
Have a look at this step by step tutorial. What you need is to send files in multiple parts over a persistent HTTP connection.
Uploading files to HTTP server using POST on Android.
I have a regular JSP/Servlet/Java web application that is used for uploading pictures from a mobile device. I am using Apache Commons library for the upload. Application is hosted on WebSphere Application Server 7.0.
Everything is working fine and the user can upload several images totaling 8MB or more if he has a really good/strong signal/connection or on a good WiFi.
The problem arises when the user is at a location with poor 3G/4G signal/connection. He gets errors like "Illegal state exception" or some time-out error, and in some cases the mobile browser just stays on the submit page with the progress bar no longer moving.
Any suggestions on how to "gracefully" handle this? Like is there a way to intervene after a set amount of time and give the the user an option to submit the form without the file attachment (i.e. just submit the form text fields)? Any other suggestions are welcome too.
UPDATE: The setTimeout solution below worked for me. The other missing piece was that I have to issue a "browser stop" command to stop the original submission that's in progress before I can issue a re-submit. Otherwise, my re-submit command will just be ignored by the browser.
The usecase here is simple - if the upload didn't finish in N minutes, remove/clear the field using javascript and resend the form.
You don't need to control the upload in the basic implementation, just safely asume that if you set a timeout to resend, it won't happen if the first attempt was successful and the page reloaded.
jQuery pseudocode:
setTimeout(function(){
$imageFieldNode.remove();
$form.trigger('submit');
},30000);//after 30 seconds
The more advanced way is to use a ready solution for controlled upload. They work like that:
upload starts
js prompts the server in intervals with a GET query to get the size of content that was already received.
everytime it gets the info - it reports progress.
You can do a lot with these libs.
You can think about the approach used in popular webmail clients (when attaching files to a message):
The files are uploaded independently (i.e. before) of the form submit, using javascript. Each of the files are stored in a temporary directory, and after the upload succeeds the user can proceed with the action.
The upload status is displayed to the user, and if it fails the main action (form fill/submit) does not get interrupted.
we will have large files (up to 2 Gb) files on a web page, and want to have the functionality that the user can continue a download if it gets interrupted.
At the moment, the only solution i can come up with is a java applet, i have tried searching for any existing open source projects with this functionality but havent found any so far.
Would be thankful for any tips how to achieve this, or pointers to existing projects or documentation that can be useful.
I am open to any solutions, so it does not have to be java applet (important is that it works in the most common browsers)
You could serve it as a torrent instead of a simple file and let the user's BitTorrent client figure it out.
Assuming you don't want to do that, HTTP lets the client specify a range of bytes to download. The user's browser has to be able to recognize that the file the user wants to download is the same as the one that already exists in partial form and send the proper range headers to the server, and server has to honor them. You can probably take care of the server side, but the browser will have to hold up its end. Some "download manager" programs do this too.
For more information:
http://www.west-wind.com/Weblog/posts/244.aspx
What you need is a Java Download Manager
1. An open Source project exist in google
http://code.google.com/p/jdownloadmon/
HTTPDownloadConnection.java#method connect
2. What partial content range types this server supports
Accept-Ranges: bytes
http://en.wikipedia.org/wiki/List_of_HTTP_header_field
Accept Range will allow the user to pause and resume download.
3.http://stackoverflow.com/questions/3414438/java-resume-download-in-urlconnection