I have integration with video stream provider. The flow is following: user requests a stream url, next we on behalf of user, request it from stream provider and return to the user. Next, we should prolong the stream id (session) every 10 secs. To minimize interaction with the client and because of slow network, we want to do this session prolongation on behalf of the user. So let's say, user will trigger one request per 2-5 mins, at the same time server will trigger session prolongation requests every 10 secs.
The question is in possible design of such service. I have not found better solution other than just simply iterate over all available session keys periodically and call prolongation service.
But this approach has disadvantages when user count will be really big it could slow down processing. Also, it is hard to scale with such an approach.
Maybe you have ideas about how to overcome this? Or please propose a better solution
I would write the keep alive as a single self contained piece of code, that will call the keep alive every x number for seconds for y amount of time before ending itself, where x, y and the keep alive endpoint are startup parameters.
Each time the user triggers a request - kick one of these off in the background. How you package that is determined on your deployment environment and how you intend to manage scaling out (background thread, new process, server-less function, etc.).
You may need to maintain some state info in a cache for management purposes (don't start a new one if one is already running, hung process states, etc.).
Related
In a microservice architecture, suppose there is a business scenario where a user purchases something that will expire after two years, and the system needs to notify the user a little bit in advance.
In this case, how should we handle the situation so that the users can be notified on time even if there are many users who need to be notified?
For example, using a delayed queue of message queue will cause the messages to pile up when there are many users; using a timed task, too many users will overload the server CPU.
Is there a good way to do this?
While "microservices" do not inherently mean "REST", they usually are. And in REST you shouldn't store in memory anything that needs to survive more than one request. Two years is an extreme case, but even if it is for just 10 minutes, it should probably go to the DB.
Building up a queue for two years will just be very impractical and likely to fail if the queue contents are not persisted somewhere. Since you mention purchases I am assuming you have some sort of data store to record them either in sql or no-sql.
You can simply add purchase date/time column(s) to the table to make life easier. If you volumes are low enough for daily purchases then I would start with date based lookup only. You will need a scheduled execution of some service method say at 6am everyday that looks up purchases close to expiry i.e 7 days before 2 years purchase_date = now - 723days and then send rest request somewhere or publishes an event or jms message with order number and purchase_date as content for each purchase order. This will then be picked up by event/message listener somewhere and processed accordingly i.e. send a notification to customer. To avoid sending duplicate notifications you should also persist the expiry notifications in a database and ensure you check that notification has been sent for purchase id before sending it again.
If you ever reach a situation where you are processing thousands of orders a day and don't want to publish large number of events in one go then extend the functionality to filter by purchase timestamp and process chunks of purchases multiple times a day by changing the lookup condition.
This is just general idea of such requirement and you will have to fine-grain a lot of implementation details such as what happens if your email server is down.
You can use quartz job and configure it to use persistent mode in database (JDBC JobStore) to not loose information and also it is suitable for clustering mode.
Quartz checks periodically the database for the nearest task (configurable parameter) if the time comes, it will process the notification.
You can configure the thread pool size in order to avoid overload.
I am using the default heartbeat interval of 30 seconds. Has anybody experience any load issues using that default interval?
It depends on the number of reads your app does per given window. If you are sure that your app is going to keep knocking the database with queries then you can go with a longer interval or even disable it by setting it to 0. If you have a on and off kind of load pattern ( like your get request for 2mins and no request for 2 next two mins) then having the heart beat is critical in order to keep the connection active. Otherwise making a new connection would be costly.
I face a scenario where everything works fine with keeping session.setMaxInactiveInterval(15*60) //15 minutes
Now, there is one request in application where number of users are getting saved, which takes more then 15 minutes at server side as internally many operations are involved while saving each user.
When saveUser operation is performed from browser, request received at server, starts processing, after 15 minutes client see "Session Time out" but user saving operation still going on.
I want to avoid this scenario, and need is when request is received by server at that time till the time it responds back to client don't considered in-between time as inactive time.
I can very well do it by making setMaxInactiveInterval(-1) as first line of my method saveUser and just before giving response as setMaxInactiveInterval(15*60).
I am looking for standard approach or this is the standard approach to follow?
Is there any inbuilt Tomcat configuration provided for such scenario?
The standard Java EE approach for this would be as follows:
Instead of doing all the processing in the web-layer, put all the details on a JMS queue and return the request straight away.
The data would be taken off the JMS queue by a worker (possibly on a different server, as you wouldn't want to have the load of creating something for 15 minutes on your web layer)
Once the data is create for the user a notification would be generated (e.g. the browser could query every 30 seconds whether the user generation has finished).
N.B. blocking a thread in a server for 15 minutes is a very bad idea, because essentially it means that your servlet container wouldn't be able to do anything else with that thread. Imagine if a large number of those requests came in at the same time, your web layer would probably buckle under the pressure...
N.B.2: 15*50 does not make 15 minutes
The application we are building has a very simple concept: it receives incoming events from a Database and for each event it opens an interactive session with clients (in the event) by showing a menu. Based on client response, we move to the next state or take some concrete action (e.g. transferring funds).
Sessions are independent of one another. For example, suppose we get two events from the database saying clients A and B have reached a zero account balance state. In response to this event, we establish two connections to A and B show a menu which looks like the following:
Please select an option:
1. Get $5
2. Get $10
3. Ignore
For options 1 and 2, we ask for confirmation in the form of second menu.
Are you sure?
1. yes
2. no
In this case, we'll have two sessions. Client A might choose option 1 (1. Get $5), whereas Client B chooses option 3 [in the first menu]. In the case of Client A, we'll present the second menu (above) and if the response is 1. yes, we'll take some concrete action such as transferring funds and closing the session.
All client communication is done by a 3rd party system which takes JSON including client address, menu text and returns a response back to us. It takes care of actually maintaing the session on the wire, whereas we only need to do response correlation and dealing with session states.
We're expected to handle 50,000 of such sessions in parallel.
Earlier, we designed the system in Java using SEDA model. Having heard of Actors, we are willing to check them out and write a quick PoC project (Java/AKKA). My questions are:
Has anyone had experience in building such kind of an application? Is 50,000 simultaneous sessions too much for AKKA to handle? (Note, we are only waiting for the response. When the response comes, based on the answer, we jump to the next stage, so it should be possible).
Which architectural stye/paradigm which best suit this problem in AKKA? Are there any frameworks out there for this kind of problem?
This is actually a reasonably easy use case with Akka's clustering. 50K sessions represented as an Actor instance for each is not very high load. The reason to use clustering is only for fault tolerance.
The idea behind the architecture would be to have a web tier for handling RESTful requests that correspond to the sessions. These requests would be sent to the Akka cluster and routed to the appropriate session Actor by session ID, or a new one would be created. When a session is done, you stop the actor that is associated with it.
Note that the session actors should send themselves timeout messages via the scheduler. Upon completion of handling a new message, the actor should schedule itself a message via the ActorSystem scheduler for 15 minutes (or whatever your timeout is). When a new session message is received, that scheduled task should be cancelled, the new update handled, and then a new timeout scheduled. There is a plausible race condition here, in that a timeout message may be in your session actor's mailbox queue AFTER a session message, but if your timeout message includes a time of when it was scheduled (the 15 minutes ago), you can check that and ignore it and reschedule another (just as a safety mechanism to avoid a memory leak). If the time is greater than 15 minutes ago, then you stop the actor.
To see how the distribution of work to the session actors would be implemented, please see the "Distributed Workers with Akka and Java" template in Typesafe's Activator. You will have a fully running clustered Akka application that you can tailor to do the session management as I've described above. You can then export the project and work on it in Eclipse/IntelliJ/Sublime/TextMate/etc. To download Activator, see here.
I have an app that runs over several instances and all requests come through one servlet.
I need to run a cron job which executes once a week for about 3 minutes. During that cron call some kind of flag/boolean will be modified somewhere so that the servlet can pick up and send an "server temporarily unavailable" type message back instead of processing the request. Once the cron job is complete it will flag it back to true.
I cannot use a singleton or a static boolean as the app will be in multiple instances. Nor do I want the servlet to have to fetch a value from the datastore on every request, as it will mean hundreds of thousands of extra datastore reads.
What can I do? Any ideas?
I think you may be able to store boolean in memcached. GAE has a Cache API for Memcached. However note that cache values are not persistent and may not be survived for even 3 minutes. I think you should have a firm time to start cron task hardcoded in one of your Java classes or .properties file and then when your task finishes, it should look at that hard-coded time and schedule itself for next round according to that time.
And by this way your servlet can also look at that time and do not serve requests in the interval you are going to specify. Yeah, that will be very fast but your jobs will be scheduled to a fixed time periodically and you won't be able to change this unless you re-deploy application.
I think the better solution is you should keep the boolean in the datastore and make use of cache. See the following algorithm:
is my boolean in the cache?
yes:
[alright, then choose to serve or not to serve request using it.]
no:
[fetch variable from datastore and put it on the cache.] (cache miss)
Again, cache will be fast, but not as much as hard-coding the schedule in the program.
EDIT: Another solution. (however not possible to implement)
If you want to serving pages during the task execution, you should use a task api
First of all you should be familiar with using countdown for your task (in this case next week) http://code.google.com/appengine/docs/java/javadoc/com/google/appengine/api/taskqueue/TaskOptions.html#countdownMillis(long)
Then you can use size() method of Queue – which I was expecting it to be there but apparently Google didn't implement it– to see if task queue size is 0, then it means it is processed right now because when the task finishes, it submits itself again to 1 week later.
One approach would be to have the cron job publish a message to a JMS topic to which all the servlet instances were listening. The messages could inform the servlet instances to set a value in the static boolean you mentioned to true or false.