I have spring boot application with basic REST API.
My question is what shall we use to download some bulk data? What is preferable way how to download bulk data without memory leak? Let's suppose we have 10 million records.
Here are some approaches but not sure:
download with PipedInputStream when data are written with PipedOutputStream in separated thread. Is it fine or it is not good choice?
download with ByteArrayOutputStream when data are written into temp file in separated thread and after finish it is ready to download. We can mark this operation with some flags for end user eg. DOWNLOAD_ACTIVE, DOWNLOAD_DONE. The user is just initiating download with result flag DOWNLOAD_ACTIVE and trying to ping server for response flag DOWNLOAD_DONE. When it is done then the user is going to send request to download data.
Summary 2)
1. initiate request to download data - ACTIVE state
2. ping server and server returns current state - ACTIVE or DONE
3. if final state is DONE then user initiate final request to download data
Thanks
You can use the second approach. Which can prepare data in the background and once it's ready you can download it.
Send a request to prepare data. The server responds with a UUID.
Server starts preparing files in the background. The server has a Map that has the key with a new UUID and value as status ACTIVE.
Client saved UUID and checks the server after a certain interval by passing the UUID.
Once the server finishes the task it will update the Map for the given UUID value as status DONE.
As the status is DONE next request will provide the status DONE and UI and send another request to download the file.
The above approach will only work if you don't refresh the page. As page refresh will clear the UUID and you have to proceed again.
To achieve this after refresh/cross-logins then you need to use a database table instead of Map. Store the username along with other information and inform the user once it's ready.
Related
A client sends a request and catches a timeout exception. However the server is still processing the request and saving it to the database. Before that happening, the client already sent a second request which doubles the record on the database. How do I prevent that from happening? Im using java servlets and javascript.
A few suggestions:-
1) Increase the client timeout.
2) Make the server more efficient so it can respond faster.
3) Get the server to respond with an intermediate "I'm working on it" response before returning with the main response.
4) Does the server need to do all the work before it responds to the client, or can some be offloaded to a seperate process for running later?
A client sends a request and catches a timeout exception. However the server is still processing the request
Make the servlet generate some output (can be just blank spaces) and flush the stream every so often (every 15 seconds for example).
If the connection has been closed on the client side, the write will fail with a socket exception.
Before that happening, the client already sent a second request which doubles the record on the database
Use the atomicity of the database, for example, a unique key. Start the process by creating a unique record (maybe in some "unfinished" status), it will fail if the record already exists.
I have a form that creates an account and a servlet that handles the request.
However, the process to create this account is a long process and I want to create something like a status bar or a progress bar. Heres the POST:
$.post("createAccount.jsp", function(data) { $("#status").text(data);
});
And the servlet would continuously print data like "creating x..." then "creating y" as the servlet runs. Is there a way to accomplish this or maybe another way to tackle this issue?
Thanks
Http works on a request-response model. You send a request, and server responds back. After that Server doesn't know who are you?!
It's like Server is a post-office that doesn't know your address. You
go to it and get your letters.It doesn't come to your home for
delivering letters.
If you want constant notifications from server, You can either use Web Sockets(Stack Overflow also uses Web Sockets) or use `AJAX Polling' mechanisms,
which sends an AJAX request to the server and waits for server to
respond. On retrieval of response,it generates another AJAX request
and keep on doing the same until server stops generating new data.
Read this for an explanation of AJAX Polling techniques
You could have your account creation servlet update a database or context attribute as it creates the account.
You could have a separate AJAX request to a different servlet that sends back to the webpage the most recent development found in the database or context attribute. You would then poll your server with that AJAX request every so many fractions of a second(or relevant time interval depending on how long of a task it is to create an account) to get all the updates.
I am making a module for a server software that is allowing support for facebook.
The problem is with the callback URL. If one client start the authorization proccess, then another client starts the proccess at the same time, or before the first user finish. How could I check what user finished first?
I need a way to check what client's callback I'm getting. One solution would be to lock other from register until the first one has finished, but I don't want to do that. Is there another way? I have thought about including ?client=clientid at the end of the callback, but I heard facebook only allows the exact url specified in the app on facebook.
UPDATE
It didn't work to add client="clientid" to the callback. Any other ideas?
After some more searchig I figured facebook will allow a parameter: state. (thanks to #jacob https://stackoverflow.com/a/6470835/1104307)
So I just did ?state=clientId.
For anyone using scribe the code is this:
service.getAuthorizationUrl(null) + "&state=" + clientId;
I think there is no problem on adding and GET parameter like client=clientID. Facebook will redirect you to the URL you have specified and using the REQUEST parameters you can check who completed the request. The problem exist if you have specified URL as http://yoursite.com and pass redirect to http://some-sub-domain.yoursite.com or entirely different location.
if you are using the server-side flow then the oauth 2 flow will be:
redirect user to facebook
facebook then rediects the user to your specified callback
your server uses something like curl to get the access token
your server does some more curl to get maybe more user data or update the user's data
my recommendation would be to set a session cookie in step 1 and simultaneously store this session id on your server. then the session cookie will automatically be sent to the callback url in step 2 and you can identify the session in the database this way.
this will work for all service providers (google, twitter, linkedin, etc) and is the preferred way of maintaining session continuity.
I'm new to server side.
I'm creating a database app for my company that stores links to all our marketing videos. Each entry is a url(to video), description, industry etc.
I already have the front end somewhat set up in HTML/JavaScript. Using a local XML source file, it populates a list with video names, and has text fields for all props of the video item.
Here's my question:
How do I handle updating my view when I send the form data (new entry) to the back end?
Should I insert a new entry based on local data?
Should I wait for the response from the server, and if success then update view based on local data?
Or, same as above, but update view based on back end data?
The goal is to make sure my view always reflects the state of data on the back end.
(Back end is Java / Google App Engine)
When using AJAX:
There is a callback function included in it, which triggers asynchronously when the response of the request comes back from the server.
In this function you can call your page update functions to execute on the page updating processes.
In Spring/Hibernate application. user may send more than on request with the same data to banking account. here is the case:
1st request reaches first, bank serve it, return OK, set local account data status OK;
2nd request reaches second, bank serve it, return NOK, set local account data status NOK;
Now our local account data status is NOK, although its served well.
Any suggestions for this problem.
A. Make sure the recieving web service queues the requests.
B. Perform not more than a maximum of two threads executions at a time at the web service side. This will provide you clues on why the status is set to NOK.