We Keep Coding

Spring boot app and what approach to use to download bulk data

I have spring boot application with basic REST API.
My question is what shall we use to download some bulk data? What is preferable way how to download bulk data without memory leak? Let's suppose we have 10 million records.
Here are some approaches but not sure:
download with PipedInputStream when data are written with PipedOutputStream in separated thread. Is it fine or it is not good choice?
download with ByteArrayOutputStream when data are written into temp file in separated thread and after finish it is ready to download. We can mark this operation with some flags for end user eg. DOWNLOAD_ACTIVE, DOWNLOAD_DONE. The user is just initiating download with result flag DOWNLOAD_ACTIVE and trying to ping server for response flag DOWNLOAD_DONE. When it is done then the user is going to send request to download data.
Summary 2)
1. initiate request to download data - ACTIVE state
2. ping server and server returns current state - ACTIVE or DONE
3. if final state is DONE then user initiate final request to download data
Thanks

You can use the second approach. Which can prepare data in the background and once it's ready you can download it.
Send a request to prepare data. The server responds with a UUID.
Server starts preparing files in the background. The server has a Map that has the key with a new UUID and value as status ACTIVE.
Client saved UUID and checks the server after a certain interval by passing the UUID.
Once the server finishes the task it will update the Map for the given UUID value as status DONE.
As the status is DONE next request will provide the status DONE and UI and send another request to download the file.
The above approach will only work if you don't refresh the page. As page refresh will clear the UUID and you have to proceed again.
To achieve this after refresh/cross-logins then you need to use a database table instead of Map. Store the username along with other information and inform the user once it's ready.

How to combine two servlets response without having any direct connection between them (Just like between two servlet there is a payment gateway )

Just like between two servlet there is a payment gateway , servlet1 response goes to servlet2 and from servlet2 it redirected to servlet3 I need to combine the response of servlet1 and servlet3 for generating output.
Its somthing like we are using a payment gateway between two servlet and generating response by combining both servlet response, Please help

I'm not sure I 100% understand your question. For payments, you usually create a token/reference and store that in a DB on the first request/session. Later, after the user pays, the payment gateway will do a HTTP POST to your service, and one of the parameters will be that token/reference. Now you just need to validate the id/token and mark whatever you need as paid.
Try to avoid storing data in the session for any payment related data, as the node where the payment is taking place might go down for any reason while the client is paying... and you don't want him/her complaining that the money is gone, but he didn't get the product/service.
 Edit
You have a few options about how to notify the customer. The more traditional answer is (and this works fine for low volumes of transactions):
Customer starts a payment, so the server creates and stores a transaction reference.
Customer is sent to the PSP site.
The customer completes the payment on the device.
The device finds out the payment is 'complete' and starts polling the server with the payment reference.
(time passes)
server receives the async notification from the PSP with the outcome of the transaction (and the payment reference). The server updates the transaction on the DB.
(the device is still polling)... pools once more and finds the outcome of it.
Job done
There are a few more advanced techniques, like async requests, but I would say to not use them unless you are already using them somewhere else in your application OR you have a considerable volume of transactions.

You can add the response of servlet1 to the session and get it in session2
Servlet1:
HttpSession session = request.getSession();
session.setAttribute("sessionResponse", object);
Servlet2:
Object value = request.getSession().getAttribute("sessionResponse");

Session is getting overwritten in Java

I am facing a strange issue:
I have a page with an email field in it when I submit the page the control goes to a servlet where I am saving the email value in session by using
request.getSession().setAttribute("email_Value", request.getParameter("email_Value"));
Now, on the basis of this email value I lookup the database and extracts the information for this user if information found then remove the session attribute by
request.getSession().removeAttribute("email_Value");
if not then redirect the request to same page with an error message and prefilled email value which I am extracting from session using
if(null!= request.getSession().getAttribute("email_Value")){
String Email = (String)(request.getSession().getAttribute("email_Value"));
request.getSession().removeAttribute("email_Value");
}
It works fine on our deleopment, UAT environments but problem is coming only on PROD where we have load balancer.
The issue is that while coming back to the same page it change the email address field witch some different email value which I have not even entered on my machine i.e. it is accessing someone else session.
Could someone provide any pointer to resolve this issue. As this is Production issue, any help would be appreciated.
Thanks

looks like you need to use sticky-sessions. This must be configured in the apache

Http is a stateless protocol meaning, the server doesnt know to identify a client over a period of time.
When a client makes a call to the server (load balanced, say server_1 & server_2), it could reach either server_1 or server_2, assume the request reaches the server_1, now your code creates a session and adds the email to the session.
When the same client makes another call to the server, this time it hits server_2, the email which is in server_1 session is not available to server_2 and server_2 might have email from another session thats why you are seeing another email address.
Hope its clear.
Solution:
URL Rewriting
Cookies

If your application is deployed on multiple servers, chances are there that your sessions may get transferred between servers. Also, in such scenarios, if you are storing any objects in sessions, they HAVE TO implement Serializable interface. If they don't, then the data will not be persisted when the session gets migrated.
Also, it seems that the session gets interchanged with another one. Are you storing anything at Application level?
I would also advice you to look into HttpSessionActivationListener for your case.

How to create Ajax request that gets information as the servlet runs?

I have a form that creates an account and a servlet that handles the request.
However, the process to create this account is a long process and I want to create something like a status bar or a progress bar. Heres the POST:
$.post("createAccount.jsp", function(data) { $("#status").text(data);
});
And the servlet would continuously print data like "creating x..." then "creating y" as the servlet runs. Is there a way to accomplish this or maybe another way to tackle this issue?
Thanks

Http works on a request-response model. You send a request, and server responds back. After that Server doesn't know who are you?!
It's like Server is a post-office that doesn't know your address. You
go to it and get your letters.It doesn't come to your home for
delivering letters.
If you want constant notifications from server, You can either use Web Sockets(Stack Overflow also uses Web Sockets) or use `AJAX Polling' mechanisms,
which sends an AJAX request to the server and waits for server to
respond. On retrieval of response,it generates another AJAX request
and keep on doing the same until server stops generating new data.
Read this for an explanation of AJAX Polling techniques

You could have your account creation servlet update a database or context attribute as it creates the account.
You could have a separate AJAX request to a different servlet that sends back to the webpage the most recent development found in the database or context attribute. You would then poll your server with that AJAX request every so many fractions of a second(or relevant time interval depending on how long of a task it is to create an account) to get all the updates.

Send multiple tweets(mentions) with multiple accounts Twitter4j

I am designing an application using which the users would be able to send multiple tweets(mentions) from multiple accounts and am using Twitter4j for the same. Regarding this, I have a few questions:
Authentication: The application has an authentication part.I am able to authenticate using the PIN based authentication(oob).But I want to know if it is possible to persist the access token such that the user only needs to authenticate once at the time of installing the app and never thereafter.
Exception Handling: The application would be sending multiple tweets.So I wish to know that what would be the way to check for exceptions, particularly the suspension of user account.
Limit: What would be a safe rate(I mean minimum interval between two tweets) to send the tweets hourly or per minute. Does twitter also check for the IP of the machine for suspending the accounts? I mean suppose I need to send 60 tweets.If I am using 4 accounts to do the same with 15 tweets per account and run 4 threads for the same, will that allow me to send more tweets form my app?
EDIT
Kindly note that I am making a desktop app, not a web app.So a PIN based authentication is necessary. Also I am not sure about the validity lifetime of an access token.Will the access token once generated by using OOB authentication be valid forever so that I can save it in a database and hence enable user to authenticate only once?

Some points to note:
Authentication : It is possible to get the access token without using pin based authentication, for that when you create an application you have to specify a callback url, and so you dont have to copy and paste the pin, instead a parameter oauth_verifier is appended with the callback url which you can fetch from the url itself. With this oauth_verifier , you can get access_token and token_secret as mentioned here
Exception Handling : Nothing other than simple try catch can be more helpful here.
Limit: There are two initial buckets available for GET requests: 15 calls every 15 minutes, and 180 calls every 15 minutes. Refer here for more details.
Edit: For desktop app there is no other alternative, so you have to do a pin based authentication and since twitter works on Oauth1, so as of now, the validity of the access_token and token_secret is lifetime. So authentication is required only once and the api calls can be made using same tokens.