On Tomcat 6, I have a servlet running which accepts requests and passes these onto an external system.
There is a throttling limitation on the external system - if the number of requests exceed a certain number per second, then the external system responds with a Http 503.
No further requests may hit the external system for at least 2 seconds or else the external system will restart its throttling timer.
Initially, I detected the 503 HttpResponse and did a Thread.sleep(2000) but that is wrong as it doesn't prevent the servlet servicing other requests using other threads - once a 503 response is detected, I need to block all threads for at least the 2 seconds.
Ideally, I would prefer the blocked threads not to wake up all at the same time but say a 100ms apart so that requests would be handled in order.
I've looked at the Condition and ReentrantLock but unsure if these are appropriate.
Just create a global (static) date variable in the servlet. When you get a 503, change this variable from null to the local time. The servlet should always check this variable before contacting the external system. If the variable is null, or more than 2 seconds have passed, then you can proceed. Otherwise block the thread (or throw an exception).
Looks like calling Amazon services to me, and it can be managed so easy.
You need a central and managed module for doing it, and it comes like a single module.
The important thing is you should not reach the throttling limitation at all, and if you get too much requests which would reach this value, so you should respond to your client check the result later(as async work).
If the request is kinda important business(such as capturing a payment), so you have to implement a failover with the module too, simply by persisting the request data into the database, so if there is any fail, you will have the data from the database.
If you are familiar with MQ arch, so it would be the best solution where they are designed for this kind of stuffs, but you like to have your own, you may accept and process all requests to call teh external system by the module manage.
first you may have a entity class which carries the request info like
class entity{public String id,srv,blah_blah;}
Second, a stand-alone module for accepting and processing the requests, which would be the context for the requests too. like following
class business{private business(){}// fan of OOP? K, go for singleton
private static final ArrayList<entity> ctx=new ArrayList<entity>();
static public void accept_request(entity e){_persist(e);ctx.add(e);}
static private void _persist(entity e){/*persist it to the db*/}
static private void _done(entity e){_remove(e);/*informing 3rd. parties if any*/}
static private void _remove(entity e){/*remove it from the db, it's done*/}
final private static int do_work(e){/*do the real business*/return 0;}//0 as success, 1, fail, 2....
}
But it's not completed yet, now you need a way to call the do_work() guy, so I suggest a background thread(would be daemon too!)
So clients just push the requests to this context-like class, and here we need the thread, like following
class business{...
static public void accept_request(entity e){_persist(e);ctx.add(e);synchronized(ctx){ctx.notify();}}
...
private static final Runnable r=new Runnable(){public void run(){try{
while(!Thread.currentThread().interrupt()){
if(ctx.size()==0){synchronized(ctx){if(ctx.size()==0){ctx.wait();}}}
while(ctx.size()>0){entity e=ctx.get(0);ctx.remove(0);
if(do_work(e)==0){_done(e);}else{ctx.add(e);/*give him another chance maybe!*/}end-else
Thread.Sleep(100/*appreciate sleep time*/);}//end-loop
}
}catch(Throwable wt){/*catch signals, maybe death thread*/}}};
static private Thread t;
void static public start_module(){t=new Thread(r);t.start();}
void static public stop_module(){t.interrupt();t.stop();}
...}
Tip: try not start the thread(calling start_module()) out of container initiation process, or you will have memory leak! best solution would call the thread by init() method of servlet(s) would call this module(once), and of course halting the the thread by application halt (destroy())
Related
I implement AWS Lambda function with Java and face with the question - how to release used resources correctly? In my function I make different calls of some resources: execute queries to DB, make REST-calls to third-party services (send StatsD metrics, invoke Slack webhooks, etc), interact with Kinesys stream.
Not going into details, my function looks like this:
public class RequestHandler {
private StatisticsService statsService; //Collect StatsD metrics
private SlackNotificationService slackService; //Send Slack notifications
private SearchService searchService; //Interact with DB
//Simplified version of constructor
public RequestHandler() {
this.statsService = new StatisticsService();
this.slackService = new SlackNotificationService();
this.searchService = new SearchService();
}
public LambdaResponse handleRequest(LambdaRequest request, Context context) {
/**
* Main method of function
* where business-logic is executed
* and all mentioned services are invoked
*/
}
}
And my main question is - where is more correctly release resources which are used in my services, in the end of handleRequest() method (in such case I'll need to open them all again in each next invocation of Lambda-function) or in finalize() method of RequestHandler class?
According to Lambda best practices you should :
Keep alive and reuse connections (HTTP, database, etc.) that were
established during a previous invocation.
So your current code is right.
Regarding the finalize() function, I don't think it is relevant. Lambda execution context will be deleted at some point freeing automatically every open resources.
https://docs.aws.amazon.com/lambda/latest/dg/best-practices.html#function-code
I am looking for a rule engine for my web application and I found Easy Rules. However, in the FAQ section, it states that the limitation on thread safety.
Is a Web Container considered as a multi-threaded environment? For HTTP request is probably processed by a worker thread created by the application server.
How does thread safety comes into place?
How to deal with thread safety?
If you run Easy Rules in a multi-threaded environment, you should take into account the following considerations:
Easy Rules engine holds a set of rules, it is not thread safe.
By design, rules in Easy Rules encapsulate the business object model they operate on, so they are not thread safe neither.
Do not try to make everything synchronized or locked down!
Easy Rules engine is a very lightweight object and you can create an instance per thread, this is by far the easiest way to avoid thread safety problems
http://www.easyrules.org/get-involved/faq.html
http://www.easyrules.org/tutorials/shop-tutorial.html
Based on this example, how will multi-threading affects the rule engine?
public class AgeRule extends BasicRule {
private static final int ADULT_AGE = 18;
private Person person;
public AgeRule(Person person) {
super("AgeRule",
"Check if person's age is > 18 and
marks the person as adult", 1);
this.person = person;
}
#Override
public boolean evaluate() {
return person.getAge() > ADULT_AGE;
}
#Override
public void execute() {
person.setAdult(true);
System.out.printf("Person %s has been marked as adult",
person.getName());
}
}
public class AlcoholRule extends BasicRule {
private Person person;
public AlcoholRule(Person person) {
super("AlcoholRule",
"Children are not allowed to buy alcohol",
2);
this.person = person;
}
#Condition
public boolean evaluate() {
return !person.isAdult();
}
#Action
public void execute(){
System.out.printf("Shop: Sorry %s,
you are not allowed to buy alcohol",
person.getName());
}
}
public class Launcher {
public void someMethod() {
//create a person instance
Person tom = new Person("Tom", 14);
System.out.println("Tom:
Hi! can I have some Vodka please?");
//create a rules engine
RulesEngine rulesEngine = aNewRulesEngine()
.named("shop rules engine")
.build();
//register rules
rulesEngine.registerRule(new AgeRule(tom));
rulesEngine.registerRule(new AlcoholRule(tom));
//fire rules
rulesEngine.fireRules();
}
}
Yes, a web application is multithreaded. As you expect, there is a pool of threads maintained by the server. When the serversocket gets an incoming request on the port it's listening to, it delegates the request to a thread from the pool.Typically the request is executed on that thread until the response is completed.
If you try to create a single rules engine and let multiple threads access it, then either
the rules engine data is corrupted as a result of being manipulated by multiple threads (because data structures not meant to be threadsafe can perform operations in multiple steps that can be interfered with by other threads as they're accessing and changing the same data), or
you use locking to make sure only one thread at a time can use the rules engine, avoiding having your shared object get corrupted, but in the process creating a bottleneck. All of your requests will need to wait for the rules engine to be available and only one thread at a time can make progress.
It's much better to give each request its own copy of the rules engine, so it doesn't get corrupted and there is no need for locking. The ideal situation for threads is for each to be able to execute independently without needing to contend for shared resources.
I've been assuming that static initialization (e.g. for my persistence library) will persist between requests for a given instance? Suddenly it occurred to me that maybe I'm wrong - maybe my app's init is redone for each request even on a warm instance.
Here's why I'm asking:
I have a request handler (it happens to be a Google Endpoint, but I don't think that is relevant) that receives a list of entities and saves them.
The entities can be any of 20 different types, so my static initialization 'registers' all 20 different entity types. I happen to be using objectify, so it looks like this:
#Api(name = "myendpoint")
public class MyEndpoint {
static {
ObjectifyService.register( EntityOne.class );
ObjectifyService.register( EntityTwo.class );
... x20
}
If it is doing all this 'registering' for each request then I had better change this to only register for the entities needed for the request.
(On the other hand, if I'm right and my static init is only done when creating a new instance then I should put as much initialization as I can into the static init.)
Static initializer blocks are run once when the class is being loaded.
Yes, static initialization blocks are only run when the class is being loaded. That means that it happens only once per GAE instance.
Something else worth considering: It is often better to register these classes with Ofy via your own Objectify service class, and then use that to access Objectify functionality. See https://code.google.com/p/objectify-appengine/wiki/BestPractices ... That ensures that the blocks are run before any datastore access happens.
I am using Apache TomEE 1.5.2 JAX-RS, pretty much out of the box, with the predefined HSQLDB.
The following is simplified code. I have a REST-style interface for receiving signals:
#Stateless
#Path("signal")
public class SignalEndpoint {
#Inject
private SignalStore store;
#POST
public void post() {
store.createSignal();
}
}
Receiving a signal triggers a lot of stuff. The store will create an entity, then fire an asynchronous event.
public class SignalStore {
#PersistenceContext
private EntityManager em;
#EJB
private EventDispatcher dispatcher;
#Inject
private Event<SignalEntity> created;
public void createSignal() {
SignalEntity entity = new SignalEntity();
em.persist(entity);
dispatcher.fire(created, entity);
}
}
The dispatcher is very simple, and merely exists to make the event handling asynchronous.
#Stateless
public class EventDispatcher {
#Asynchronous
public <T> void fire(Event<T> event, T parameter) {
event.fire(parameter);
}
}
Receiving the event is something else, which derives data from the signal, stores it, and fires another asynchronous event:
#Stateless
public class DerivedDataCreator {
#PersistenceContext
private EntityManager em;
#EJB
private EventDispatcher dispatcher;
#Inject
private Event<DerivedDataEntity> created;
#Asynchronous
public void onSignalEntityCreated(#Observes SignalEntity signalEntity) {
DerivedDataEntity entity = new DerivedDataEntity(signalEntity);
em.persist(entity);
dispatcher.fire(created, entity);
}
}
Reacting to that is even a third layer of entity creation.
To summarize, I have a REST call, which synchronously creates a SignalEntity, which asynchronously triggers the creation of a DerivedDataEntity, which asynchronously triggers the creation of a third type of entity. It all works perfectly, and the storage processes are beautifully decoupled.
Except for when I programmatically trigger a lot (f.e. 1000) of signals in a for-loop. Depending on my AsynchronousPool size, after processing signals (quite fast) in the amount of about half of that size, the application completely freezes for up to some minutes. Then it resumes, to process about the same amount of signals, quite fast, before freezing again.
I have been playing around with AsynchronousPool settings for the last half hour. Setting it to 2000, for instance, will easily make all my signals be processed at once, without any freezes. But the system isn't sane either, after that. Triggering another 1000 signals, resulted in them being created allright, but the entire creation of derived data never happened.
Now I am completely at a loss as to what to do. I can of course get rid of all those asynchronous events and implement some sort of queue myself, but I always thought the point of an EE container was to relieve me of such tedium. Asynchronous EJB events should already bring their own queue mechanism. One which should not freeze as soon as the queue is too full.
Any ideas?
UPDATE:
I have now tried it with 1.6.0-SNAPSHOT. It behaves a little bit differently: It still doesn't work, but I do get an exception:
Aug 01, 2013 3:12:31 PM org.apache.openejb.core.transaction.EjbTransactionUtil handleSystemException
SEVERE: EjbTransactionUtil.handleSystemException: fail to allocate internal resource to execute the target task
javax.ejb.EJBException: fail to allocate internal resource to execute the target task
at org.apache.openejb.async.AsynchronousPool.invoke(AsynchronousPool.java:81)
at org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(EjbObjectProxyHandler.java:240)
at org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(EjbObjectProxyHandler.java:86)
at org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(BaseEjbProxyHandler.java:303)
at <<... my code ...>>
...
Caused by: java.util.concurrent.RejectedExecutionException: Timeout waiting for executor slot: waited 30 seconds
at org.apache.openejb.util.executor.OfferRejectedExecutionHandler.rejectedExecution(OfferRejectedExecutionHandler.java:55)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:132)
at org.apache.openejb.async.AsynchronousPool.invoke(AsynchronousPool.java:75)
... 38 more
It is as though TomEE would not do ANY queueing of operations. If no thread is free to process in the moment of the call, tough luck. Surely, this cannot be intended..?
UPDATE 2:
Okay, I seem to have stumbled upon a semi-solution: Setting the AsynchronousPool.QueueSize property to maxint solves the freeze. But questions remain: Why is the QueueSize so limited in the first place, and, more worryingly: Why would this block the entire application? If the queue is full, it blocks, but as soon as a task is taken from it, another should pop in, right? The queue appears to be blocked until it is completely empty again.
UPDATE 3:
For anyone who wants to have a go: http://github.com/JanDoerrenhaus/tomeefreezetestcase
UPDATE 4:
As it turns out, increasing the queue size does NOT solve the problem, it merely delays it. The problem remains the same: Too many asynchronous operations at once, and TomEE chockes so bad, that it cannot even undeploy the application on termination anymore.
So far, my diagnosis is that the task cleanup does not work properly. My tasks are all very small and fast (see the test case on github). I was already afraid that it would be OpenJPA or HSQLDB slowing down on too many concurrent calls, but I commented out all em.persist calls, and the problem remained the same. So if my tasks are quite small and fast, but still manage to block out TomEE so bad that it could not get any further task in after 30 seconds (javax.ejb.EJBException: fail to allocate internal resource to execute the target task), I would imagine that completed tasks linger, clogging up the pipe, so to speak.
How could I resolve this issue?
Basically BlockingQueues use locks to ensure the consistency of data and avoid data loss, so in too highly concurrent environment it will reject a lot of tasks (your case).
You can play on trunk with the RejectedExecutionHandler implementation to retry to offer the task. One implementation can be:
new RejectedExecutionHandler() {
#Override
public void rejectedExecution(final Runnable r, final ThreadPoolExecutor executor) {
for (int i = 0; i < 10; i++) {
if (executor.getQueue().offer(r)) {
return;
}
try {
Thread.sleep(50);
} catch (final InterruptedException e) {
// no-op
}
}
throw new RejectedExecutionException();
}
}
It even works better with random sleep (between min and max).
The idea is basically: if the queue is full, wait some short time to reduce the concurrency.
configurable through WEB-INF/application.properties https://issues.apache.org/jira/browse/TOMEE-1012
I have implemented an Actor system using Akka and its Java API UntypedActor. In it, one actor (type A) starts other actors (type B) dynamically on demand, using getContext().actorOf(...);. Those B actors will do some computation which A doesn't really care about anymore. But I'm wondering: is it necessary to clean up those actors of type B when they have finished? If so, how?
By having B actors call getContext().stop(getSelf()) when they're done?
By having B actors call getSelf().tell(Actors.poisonPill()); when they're done? [this is what I'm using now].
By doing nothing?
By ...?
The docs are not clear on this, or I have overlooked it. I have some basic knowledge of Scala, but the Akka sources aren't exactly entry-level stuff...
What you are describing are single-purpose actors created per “request” (defined in the context of A), which handle a sequence of events and then are done, right? That is absolutely fine, and you are right to shut those down: if you don’t, they will accumulate over time and you run into a memory leak. The best way to do this is the first of the possibilities you mention (most direct), but the second is also okay.
A bit of background: actors are registered within their parent in order to be identifyable (e.g. needed in remoting but also in other places) and this registration keeps them from being garbage collected. OTOH, each parent has a right to access the children it created, hence no automatic termination (i.e. by Akka) makes sense, instead requiring explicit shutdown in user code.
In addition to Roland Kuhn's answer, rather than create a new actor for every request, you could create a predefined set of actors that share the same dispatcher, or you can use a router that distributes requests to a pool of actors.
The Balancing Pool Router, for example, allows you to have a fixed set of actors of a particular type share the same mailbox:
akka.actor.deployment {
/parent/router9 {
router = balancing-pool
nr-of-instances = 5
}
}
Read the documentation on dispatchers and on routing for further detail.
I was profiling(visualvm) one of the sample cluster application from AKKA documentation and I see garbage collection cleaning up the per request actors during every GC. Unable to completely understand the recommendation of explicitly killing the actor after use. My actorsystem and actors are managed by SPRING IOC container and I use spring extension in-direct actor-producer to create actors. The "aggregator" actor is getting garbage collected on every GC, i did monitor the # of instances in visual VM.
#Component
#Scope(ConfigurableBeanFactory.SCOPE_PROTOTYPE)
public class StatsService extends AbstractActor {
private final LoggingAdapter log = Logging.getLogger(getContext().getSystem(), this);
#Autowired
private ActorSystem actorSystem;
private ActorRef workerRouter;
#Override
public void preStart() throws Exception {
System.out.println("Creating Router" + this.getClass().getCanonicalName());
workerRouter = getContext().actorOf(SPRING_PRO.get(actorSystem)
.props("statsWorker").withRouter(new FromConfig()), "workerRouter");
super.preStart();
}
#Override
public Receive createReceive() {
return receiveBuilder()
.match(StatsJob.class, job -> !job.getText().isEmpty(), job -> {
final String[] words = job.getText().split(" ");
final ActorRef replyTo = sender();
final ActorRef aggregator = getContext().actorOf(SPRING_PRO.get(actorSystem)
.props("statsAggregator", words.length, replyTo));
for (final String word : words) {
workerRouter.tell(new ConsistentHashableEnvelope(word, word),
aggregator);
}
})
.build();
}
}
Actors by default do not consume much memory. If the application intends to use actor b later on, you can keep them alive. If not, you can shut them down via poisonpill. As long your actors are not holding resources, leaving an actor should be fine.