We have an external SOAP based web-service which provides information regarding customer's gift card balance when presented with an Id. This Id is stored in our database.
The requirement is to find out the balance for all such customers who has this Id flagged and then send them an email. This logic is supposed to be run as a scheduled job once every alternate day.
When we queried the DB, we found out that there are more than 5000 such customers who have this Id flagged. Unfortunately, the web-service will NOT accept a list of Ids, and can only give information about a single customer in one network call.
Now, our doubt is whether it will be a good idea to loop through 5000 Ids and call the web-service in this loop as many times.
As a test run, when we called the web-service for 500 Ids, it completed in 3.7 minutes and 1000 Ids 7.25 minutes. By this measure, we can guesstimate that for 5000 Ids, it should roughly take 40 minutes.
Our web-application is JavaEE 6 stack and DB is Oracle.
Is there a better way to do this ? Any suggestions are welcome.
Thanks.
If you could write a deterministic function that takes the input of the customer id and that gives you a number from 0 to 47 representing the number of hours in the 2 day cycle of sending these email alerts, you could shard the email sending and convert it to a job that runs every hour.
I know that is changing the requirements a bit, but there isn't much difference between sending a batch every 2 days and a smaller batch every hour. Each customer who remains on your list would continue to get emails every 2 days.
Another possibility is to send queries to the web service in a multi-threaded manner.
The web service provider should really think about changing their interface.
Unfortunately, the web-service will NOT accept a list of Ids, and can
only give information about a single customer in one network call.
You should really take contact with the service provider to get a suitable solution.
As workaround, if making multiple concurrent invocations is allowed by the SOAP WS, you could make multiple invocations of the WS by multiple Threads.
To achieve that, create a Runnable or a Callable implementation that performs the invocation to the WS with a specific id.
For example to perform concurrently 10 invocations of the WS, with Callable and ExecutorService, you could do something as :
MyWs myWs = ...; // web service stub
List<Long> ids = ...; // ids to search
List<Callable<Double>> callables = ids.stream()
.map(id -> (Callable<Double>) () -> myWs.getBalance(id))
.collect(Collectors.toList());
ExecutorService executorService = Executors.newFixedThreadPool(10)
List<Future<Double>> balanceFutures = executorService.invokeAll(callables);
Of course adjust the number of invocations according to the CPU of the machine that runs the JVM.
Related
1. cron job started
2. create Entity1 and save to DB
3. Fetch transactionEntity from DB
4. using transactions as transactionIds.
for (Transaction id : transactionIds) {
a. create Entity2 and save to db
b. fetch paymentEntity from DB.
c. response = post request Rest API call
d. udpate Entity2 with response
}
5. udpate Entity1.
Problem statement - I am getting 5000+ transaction from db in transactionIds using cron jobs which need to process as given above. With the above approach while my previous loop is going on, next 5000+ transactions come in the loop as cron job runs in 2 minutes.
I have checked multiple solutions(.parallelStream() with ForkJoinPool / ListenableFuture, but am unable to decide which is the best solution to scale the above code. Can I use spring batch for this, if yes, how to do this? What are the steps comes in reader, process and writer from above steps.
One way to approach this problem will be to use Kafka for consuming the messages. You can increase the number of pods (hopefully you are using microservices) and each pod can be part of a consumer group. This will effectively remove the loop in your code and consumers can be increased on demand to process any scale.
Another advantage of message based approach will be that you can have multiple delivery modes(at least once, at most once etc) and there are a lot of open source libraries available to view the stats of the topic (Lag between consumption and production of messages in a topic).
If this is not possible,
The rest call should not happen for every transaction, you'll need to post the transactions as a batch. API calls are always expensive to do, so the lesser roundtrips will give you a huge difference in time taken to complete the loop.
Instead of directly updating DB before and after API call, you can change the loop use
repository.saveAll(yourentitycollection) // Only one DB call after looping, can be batched
Suggest you to move to producer-consumer strategy in near future.
In a microservice architecture, suppose there is a business scenario where a user purchases something that will expire after two years, and the system needs to notify the user a little bit in advance.
In this case, how should we handle the situation so that the users can be notified on time even if there are many users who need to be notified?
For example, using a delayed queue of message queue will cause the messages to pile up when there are many users; using a timed task, too many users will overload the server CPU.
Is there a good way to do this?
While "microservices" do not inherently mean "REST", they usually are. And in REST you shouldn't store in memory anything that needs to survive more than one request. Two years is an extreme case, but even if it is for just 10 minutes, it should probably go to the DB.
Building up a queue for two years will just be very impractical and likely to fail if the queue contents are not persisted somewhere. Since you mention purchases I am assuming you have some sort of data store to record them either in sql or no-sql.
You can simply add purchase date/time column(s) to the table to make life easier. If you volumes are low enough for daily purchases then I would start with date based lookup only. You will need a scheduled execution of some service method say at 6am everyday that looks up purchases close to expiry i.e 7 days before 2 years purchase_date = now - 723days and then send rest request somewhere or publishes an event or jms message with order number and purchase_date as content for each purchase order. This will then be picked up by event/message listener somewhere and processed accordingly i.e. send a notification to customer. To avoid sending duplicate notifications you should also persist the expiry notifications in a database and ensure you check that notification has been sent for purchase id before sending it again.
If you ever reach a situation where you are processing thousands of orders a day and don't want to publish large number of events in one go then extend the functionality to filter by purchase timestamp and process chunks of purchases multiple times a day by changing the lookup condition.
This is just general idea of such requirement and you will have to fine-grain a lot of implementation details such as what happens if your email server is down.
You can use quartz job and configure it to use persistent mode in database (JDBC JobStore) to not loose information and also it is suitable for clustering mode.
Quartz checks periodically the database for the nearest task (configurable parameter) if the time comes, it will process the notification.
You can configure the thread pool size in order to avoid overload.
I have an application, call it Service 1, that potentially makes a lot of the same requests to another application, call it Service 2. As an example, x number of people use Service 1 and that results in x requests (which are the exact same request) to Service 2. Each response is cached in Service 1.
Currently, we have a synchronized method that checks whether or not the same request has been made within a certain time threshold. The problem we are having is that when the server is under a heavy load that synchronized method locks up the threads, kubernetes can't perform liveness checks, so kubernetes restarts the service. The reason we want to prevent duplicate requests is two fold: 1) we don't want to hammer service 2, and 2) if we already are making the request we don't want to make it again, just wait for the result that will already be coming back.
What is the fastest, most scalable solution to not making duplicate requests without locking up and taking down the server?
FWIW, my experience with rx-java specifically is very limited, so I'm not entirely confident how applicable this is for your case. This is a solution I've used several times with Scala and I know Java itself does have analogous constructs that would allow the same approach.
A solution I have used in the past that has worked very well for me involves using Futures. It does not reduce duplication entirely, but it does remove duplication per requesting server. The approach involves using a TTL Cache in which we stored the Future object that does or will contain the result of a request we want to deduplicate on. It is stored under a key that can determine uniqueness of the request such as the different parameters that might be applicable.
So let's say you have a method that you call to fetch the response from Service 2 and returns it as a Future. As an example we'll say getPage which has one parameter, an integer, which is the page you'd like to fetch.
When a request begins and we're about to call getPage with the page number of 2, we check the cache for a key like "getPage:2". This won't contain anything for the first request, so we call getPage(2) which returns a Future[SomeResponseObject]. We set "getPage:2" in the TTL Cache to the Future object. When another request comes in that may spawn a duplicate request, the same cache check happens, however, there's a Future object already in the cache. We get this future and add a response listener to be invoked when the response is available, or in Scala, simply .map() on it.
This has a few advantages. If your request is slow or there's highly duplicative requests even in a small time frame, many requests to Service 1 are serviced by a single response from Service 2.
Secondarily, once the request to Service 2 has come back, assuming you have a window in which the response is still valid, the response is already available immediately and no request is necessary at all.
If your Service 2 request takes 50ms, and your response can be considered valid for 5 seconds, all requests happening to the same server in the first 50ms are serviced at ms 50 when the response is returned, and from that point forward for the remaining 4950 ms already have access to the response.
As I alluded earlier to the effectiveness here is tied to how many instances of Service 1 are running. The number of duplicate requests at any time is linear to the number of Servers running.
This is a mostly lock free way to achieve this. I saw mostly because some synchronization is necessary the TTL Cache itself to make sure the request is only started once, but has never been an issue for performance in my experience.
As an extension of this, you can potentially use something like redis to cache responses from Service 2 if it has long-ish response times, and have your getPage equivalent first check a redis cache for the serialized response (and write an expiring value if one wasn't there). This allows you to further reduce requests to Service 2 by having a more global value cached, but having a second caching layer does add some complexity and potential for issues.
We have web service APIs to support clients running on ten millions devices. Normally clients call server once a day. That is about 116 clients seen per second. For each client (each with unique ID), it may make several APIs calls concurrently. However, Server can only process those API calls one by one from the same client. Because, those API calls will update the same document of that client in the backend Mongodb database. For example: need to update last seen time and other embedded documents in the document of this client.
One solution I have is to put synchronized block on an "intern" object representing this client's unique ID. That will allow only one request from the same client obtains the lock and be processed at the same time. In addition, requests from other clients can be processed at the same time too. But, this solution requires to turn on load balancer's "stickiness". That means load balancer will route all requests from the same ip address to a specific server within a preset time interval (e.g. 15 minute). I am not sure if this has any impact to the robustness in the whole system design. One thing I can think of is that some clients may make more requests and make the load not balanced (create hotspots).
Solution #1:
Interner<Key> myIdInterner = Interners.newWeakInterner();
public ResponseType1 processApi1(String clientUniqueId, RequestType1 request) {
synchronized(myIdInterner.intern(new Key(clientUniqueId))) {
// code to process request
}
}
public ResponseType2 processApi2(String clientUniqueId, RequestType2 request) {
synchronized(myIdInterner.intern(new Key(clientUniqueId))) {
// code to process request
}
}
You can see my other question for this solution in detail: Should I use Java String Pool for synchronization based on unique customer id?
The second solution I am thinking is to somehow lock the document (Mongodb) of that client (I have not found a good example to do that yet). Then, I don't need to touch load balancer setting. But, I have concerns on this approach as I think the performance (round trips to Mongodb server and busy waiting?) will be much worse compared to solution #1.
Solution #2:
public ResponseType1 processApi1(String clientUniqueId, RequestType1 request) {
try {
obtainDocumentLock(new Key(clientUniqueId));
// code to process request
} finally {
releaseDocumentLock(new Key(clientUniqueId));
}
}
public ResponseType2 processApi2(String clientUniqueId, RequestType2 request) {
try {
obtainDocumentLock(new Key(clientUniqueId));
// code to process request
} finally {
releaseDocumentLock(new Key(clientUniqueId));
}
}
I believe this is very common issue in a scalable and high concurrent system. How do you solve this issue? Is there any other option? What I want to achieve is to be able to process one request at a time for those requests from the same client. Please be noted that just controlling the read/write access to database does not work. The solution need to control the exclusive processing of the whole request.
For example, there are two requests: request #1 and request #2. Request #1 read the document of the client, update one field of a sub-document #5, and save the whole document back. Request #2 read the same document, update one field of sub-document #8, and save the whole document back. At this moment, we will get an OptimisticLockingFailureException because we use #Version annotation from spring-data-mongodb to detect version conflict. So, it is imperative to process only one request from the same client at any time.
P.S. Any suggestion in selecting solution #1 (lock on single process/instance with load balancer stickiness turned on) or solution #2 (distributed lock) for a scalable, and high concurrent system design. The goal is to support tens of millions clients with concurrently hundreds of clients access the system per second.
In your solution, you are doing a lock split based on customer id so two customers can process the service same time. The only problem is the sticky session. One solution can be to use distributed lock so you can dispatch any request to any server and the server gets the lock process. Only one consideration is it involves remote calls. We are using hazelcast/Ignite and it is working very well for average number of nodes.
Hazelcast
Why not just create a processing queue in Mongodb whereby you submit client request documents, and then another server process that consumes them, produces a resulting document, that the client waits for... synchronize the data with clientId, and avoid that activity in the API submission step. The 2nd part of the client submission activity (when finished) just polls Mongodb for consumed records looking for their API / ClientID and some job tag. That way, you can scale out the API submission, and separately the API consumption activities on separate servers etc.
One obvious approach is simply to implement the full optimistic locking algorithm on your end.
That is, you get sometimes get OptimisticLockingFailureException when there are concurrent modifications, but that's fine: just re-read the document and start the modification that failed over again. You'll get the same effect as if you had used locking. Essentially you are leveraging the concurrency control already built-in to MongoDB. This also has the advantage of getting several transactions go through from the same client if they don't conflict (e.g., one is a read, or they write to different documents), potentially increasing the concurrency of your system. On other hand, you have to implement the re-try logic.
If you do want to lock on a per-client basis (or per-document or whatever else) and your server is a single process (which is implied by your suggested approach) you just need a lock manager that works on arbitrary String keys, which has several reasonable solutions including the Interner one your mentioned.
Recently I was creating an auction site. I want to make it like when user bid the item, there is a AI bidder to upbid the user. For say user bid on item1 after 5 seconds the AI bidder will auto bid the item1 as well. Any idea how can I execute it automatically after 5 seconds?
A simple and efficient solution could be to store all future bids with a "due date" and all the information to bid in a list. Then every 5 seconds or so you could loop through the list and make all bids if they are due. This system would be extensible and would work for a large amount of bids. Of course, ideally this would run in a different thread.
It's a bit like re-implementing a "cron-like" job management in your servlet but I can't see any solution that would fit your needs out of the box.
I am not sure I answered your question, hope so.
Regards,
Stéphane
Depends on what technology you actually use, you can use EJB timers for that for example, just start the timer ejb when a new bid occurs, on timer timeout (after some time) the method executes and updates the bid.
Standard servlet solution
Create a Filter, map it to the url pattern of your bid servlet.
In your doFilter(), after your filterChain.doFilter() call (ie, after the request has been processed by the servlet/JSP), schedule an action for 5 seconds in the future (you can use the standard java ScheduledExecutorService)
In the Runnable implementation you schedule (your task), place the AI bid.
In my opinion:
If user bid, and after 5 secs, it sends the request to the server, i prefer JS with setTimeout(). (Of course it required Browser's JS - read more abt this in W3School).
Otherwise, you can use an array (or smt like that) act as an queue (in server side), after each 5 secs, it's lock the queue (sync), and check for which inserted 5 secs ago, and process it (or use an Thread for each time an event requests to server). Basically, you can use a thread to do that trick? (Did u mean this?).