I need my application to submit data to a web service and wait for 90 sec for the response. But if there's a no response in 60 secs I need to redirect the user to a different page and continue to wait for the the response for another 30 secs if it come then process it.
I know I need to use thread for this but not sure how to integrate the treads in this case so threads can exchange data between themselves.
Any ideas?? I'm using JSF for UI.
The requirment is follows : The web service will send response in 90 secs (That's the maximum response time for it). But the user will be given a response(A dummy response in case the response does not come within 60 sec) in 60 sec. So even if the user has been given a dummy response (after 60 sec) my application will continue to wait for another 30 sec for the response
Don't know much about JSF, but it sounds like you want a timer, probably java.util.Timer. If the answer comes back before the timer goes off, shut down the timer. If the timer goes off, reset it for 30 seconds and redirect the user. The next time it goes off, give up waiting for the correct answer.
That much you seem to understand. But you've got at least two interacting threads threads here. How to communicate?
Just use instance fields. All references to them should be done with code in synchronized methods or blocks. Do that and you should be fine. You'll have to figure it out, but I would imagine you'd have an int timerPhase, that would indicate the timer was not started, in the first 60 seconds, in the next 30, or timed out. Also a boolean answerReceived, something with the answer in it, and perhaps a few others.
(Too much synchronization can slow your program down. I don't think you will have this problem. But if you do, split the synchronize blocks up, with each field synchronized separately unless they interact. Remove synchronization and use the volatile keyword. Read up on multithreading. (Read up on the volatile keyword.) Think real hard about how parallel threads can interact. And prepare for things to get real weird.)
Related
In my application, there are roughly 15 threads that each send an http request to an api endpoint once every 15 seconds; meaning about 1 request a second. These threads should be running indefinitely and only need to be created once. I am unsure how to continuously receive the responses on the main thread so that they can be parsed and dealt with. In trying to research this problem I found several frameworks that look like they could help; ScheduledExecutorService, NIO, Grizzly, AHC. But, I'm paralyzed by the amount of options and am unsure of what to implement.
My main goal is, for each of the 15 requests, to have the request sent off on its own every 15 seconds and receive the response on the main thread as it comes in.
No special frameworks are required for such a simple task. Just create an instance of BlockingQueue (ArrayBlockingQueue looks like the best choice). Each network thread calls queue.put(response) and the main thread makes response=queue.take() in a loop.
I am testing a certain "functionality" that happens after log in.
The test case is 500 users exercising that functionality within 5 minutes.
I can add a synchronising timer after the log in, to ensure all 500 threads have logged in but then it will do all 500 "functionality" tasks at once, rather than 5 minutes, which will crash the app (it thinks there's a DDoS attack and shuts down).
Right now, I am handling this by giving some think time after login, to slow down login to a stable figure that I can predict and then start "functionality" at each thread's turn, as scheduled by: the main scheduler + the the log in response time + the think time...
But that's a bit fuzzy.
Is there a way to "ramp up" tasks once already running?
I can think in two options.
The first one is two use random times. You would use the range from 0 seconds to 300 - 1 that is [0-300) or using millis [0-300000). Then sleep the thread basesd on this ramdon time.
This approach can be a little more realist, because for instance, in a specific second of the given interval you don't have any threads starting and in other particular second you have 2-3. This still should be well balanced in general, since you won't make all petitions at start.
The second one is to start the threads uniformly. During your configuration time (login and before firing the threads) you can use something like an AtomicInteger, initializing it with new AtomicInteger(0) and calling getAndIncrement() to assign the possition of the thread, in the range [0-500) and then when you fire the threads sleep 300.0 * id / 500.0 milliseconds to execute the task/petition.
By default JMeter executes requests as fast as it can, you can "throttle" the execution to desired throughput (request per minute) rate using Constant Throughput Timer.
Example Test Plan would look like:
Thread Group
Login
Synchronizing Timer
Functionality
Constant Throughput Timer
Constant Throughput Timer follows JMeter Scoping Rules so you can apply it either to single sampler or to a group of samplers.
I'm making a Java server application. The application would comsume alot of resources if it just ran when possible.
As far as I know if I added a sleep method, it would run like this:
Do task (Might take 10ms to do. Can also take longer or less)
Sleep 50ms
Do task (Might take 10ms to do. Can also take longer or less)
Sleep 50ms
So how can I make it run every 50ms (20 tick)?
Thanks
You can use a ScheduledExecutorService
ScheduledExecutorService service = Executors.newScheduledThreadPool(10);
service.scheduleAtFixedRate(() -> {
System.out.println("whatever");
}, 0, 50, TimeUnit.MILLISECONDS);
// ^ rate
The scheduledAtFixedRate() method will schedule the given task for execution at a fixed rate, regardless of the time the task took. You could possibly have one execution take longer than 50ms, and the next one would still run (assuming you have enough threads).
Without knowing what your application does (you could've included it in your question), you could use a scheduler (Quartz, java.util.Timer). Which task are you trying to perform every 50ms?
Edit:
While the "game loop" is all well and good in games, servers rarely have them. Receiving data is a continuous action, and the state should change accordingly. This is a larger design issue in the server. With proper design you don't need to create artificial pauses.
For example a simple design would be having threads waiting to receive input from the clients, and when a message is received, it's processed, and a message is sent to all clients to inform of the changes. No busy waiting, nothing will happen unless a message arrives from a client.
Status: solved
I had to make a pastebin as I had to point out line numbers.
note: not using executorsService or thread pools. just to understand that what is wrong in starting and using threads this way. If I use 1 thread. the app works Perfect!
related links:
http://www.postgresql.org/docs/9.1/static/transaction-iso.html
http://www.postgresql.org/docs/current/static/explicit-locking.html
main app, http://pastebin.com/i9rVyari
logs, http://pastebin.com/2c4pU1K8 , http://pastebin.com/2S3301gD
I am starting many threads (10) in a for loop with instantiating a runnable class but it seems I am getting same result from db (I am geting some string from db, then changing it) but with each thread, I get same string (despite each thread changed it.) . using jdbc for postgresql what might be the usual issues ?
line 252
and line 223
the link is marked as processed. (true) in db. other threads of crawler class also do it. so when line 252 should get a link. it should be processed = false. but I see all threads take same link.
when one of the threads crawled the link . it makes it processed = true. the others then should not crawl it. (get it) is its marked processed = true.
getNonProcessedLinkFromDB() returns a non processed link
public String getNonProcessedLink(){ line 645
public boolean markLinkAsProcesed(String link){ line 705
getNonProcessedLinkFromDB will see for processed = false links and give one out of them . limit 1
each thread has a starting interval gap of 20 secs.
within one thread. 1 or 2 seconds (estimate processing time for crawling)
line 98 keepS threads from grabbing the same url
if you see the result. one thread made it true. still others access it. waaaay after some time.
all thread are seperate. even one races. the db makes the link true at the moment the first thread processes it
This is a situation of not a concise question being asked. There is lots of code in there and you have no idea what is going on. You need to break it down so that you can understand where it is going wrong, then show us that bit.
Some things of potential conflict.
You are opening a database connections for almost every process. The normal flow of an application is to open a few connections, do some processing, then close them.
Are you handling database commits? I don't remember what the default setting is for a postres database, you'll have to look into it.
There are 3 states a single url is in. Unprocessed, being processed, processed. I don't think you are handling the 'being processed' state at all. Because being processed takes time and may fail, you have to account for those situations.
I did not read the logs because they are useless to me.
-edit for comment-
Databases generally have transactions. Modifications you make in one transaction are not seen in other transactions until they are committed. Transaction can be rolled back. You'll need to look into fetching the row you just updated and see if the value has really changed. Do this in another transaction or on another connection.
The gap of 20 seconds looks like it is only when the process is started. Imagine a situation where Thread1 processes URL1 and Thread2 processes URL2. They both finish at about the same time. They both look for the next unprocessed URL (say URL3). They would both start processing this Url because they don't know another thread has started it. You need one process handing out the Url, possibly a queue is what you'd want to look at.
Logging might be improved if you knew which threads were working on which URLs. You also need a smaller sample size so that you can get your head around what is going on.
Despite the comments and response by helpers in this post were also correct.
at the start of crawl() method body.
synchronized(Crawler.class){
url = getNonProcessedLinkFromDB();
new BasicDAO().markLinkAsProcesed(url);
}
and at the bottom of crawl() method body (when it has done processing):
crawl(nonProcessedLinkFromDB);
actually solved the issue.
It was the gap between marking a link processed true and fetching a new one and letting other threads get the same link while the current was working on it.
Synchonized block helped further.
Thanks to helper. "Fuber" on IRC channels. Quakenet servers #java and Freenode servers ##javaee
and ALL who supported me!
I have an application that checks a resource on the internet for new mails. If there is are new mails it does some processing on them. This means that depending on the amount of mails it might take just a few seconds to hours of processing.
Now the object/program that does the processing is already a singleton. So right now I already took care of there really only being 1 instance that's handling the checking and processing.
However I only have it running once now and I'd like to have it continuously running, checking for new mails more or less every 10 minutes or so to handle them in a timely manner.
I understand I can take care of this with Timer/Timertask or even better I found a resource here: http://www.ibm.com/developerworks/java/library/j-schedule/index.html that uses Scheduler/SchedulerTask. But what I am afraid of.. is if I set it to run every 10 minutes and a previous session is already processing data it will put the new task in a stack waiting to be executed once the previous one is done. So what I'm afraid of is for instance the first run running for 5 hours and then, because it was busy all the time, after that it will launch 5*6-1=29 runs immediately after each other checking for mails and/do some processing without giving the server a break.
Does anyone know how I can solve this?
P.S. the way I have my application set up right now is I'm using a Java Servlet on my tomcat server that's launched upon server start where it creates a Singleton instance of my main program, then calls some method to do the fetching/processing. And what I want is to repeat that fetching/processing every "x" amount of time (10 minutes or so), making sure that really only 1 instance is doing this and that really after each run 10 minutes or so are given to rest.
Actually, Timer + TimerTask can deal with this pretty cleanly. If you schedule something with Timer.scheduleAtFixedRate() You will notice that the docs say that it will attempt to "make up" late events to maintain the long-term period of execution. However, this can be overcome by using TimerTask.scheduledExecutionTime(). The example therein lets you figure out if the task is too tardy to run, and you can just return instead of doing anything. This will, in effect, "clear the queue" of TimerTask.
Of note: TimerTask uses a single thread to execute, so it won't spawn two copies of your task side-by-side.
On the side note part, you don't have to process all 10k emails in the queue in a single run. I would suggest processing for a fixed amount of time using TimerTask.scheduledExecutionTime() to figure out how long you have, then returning. That keeps your process more limber, cleans up the stack between runs, and if you are doing aggregates, ensures that you don't have to rebuild too much data if, for example, the server is restarted in the middle of the task. But this recommendation is based on generalities, since I don't know what you're doing in the task :)