We Keep Coding

Prevent from slow job taking over a thread pool

I have a system where currently every job has it's own Runnable class and I pre defined a fixed number of threads for every job.
My understanding is that it is a wrong practice, because:
You have to tailor the number of threads with respect to the machine running the process.
Each threads can only take one type of job.
Would you agree on that? (current solution is wrong)
So, I'd like to use something like Java's ThreadPool instead. I was conflicted with an argument claiming that by doing so, slow jobs will take over most of the thread pool, leaving no place to the other jobs. Whereas, with the current solution, a fixed number of threads were assigned to the slow worker and it won't hurt the others.
(Notice that you can't know a-priori if a job will be "slow")
How can a system be both adaptive in the number of threads it uses, but at the same time not be bounded to the most slow job?

You could try getting the time it takes for the job to complete (With a hand-made Timer class of sorts. Then you normalize this value by dividing this time by the maximum time any given thread has taken. Finally, you multiply this number by a fixed number which varies depending on how many threads you want running per job per second. This will be the requested amount of threads this process should be using. You can adjust that according.
Edit: You can set minimum and maximum values that regulate how many threads a job is entitled to. You could alternatively request threads from a very spacious job when another thread enters the system.
Hope that helps!

It's more of a business problem. Let's say I am a telecom operator. I bar my subscribers from making outgoing calls when they don't clear their dues. When they make payment I clear a flag and in a second the subscriber can make calls. But a lot of other activities go on in my system like usage processing, billing, bill formatting etc.
Now let's assume I have a system wide common pool of threads and I started the billing of 50K subscribers. All my threads are now processing the relatively long running billing jobs and a huge queue is building up.
A poor customer now makes a payment and wants to make an urgent call. But I have no thread left in my pool to clear the flag. The customer had to wait for an hour before he can make the call. That's SLA breach.
What I should have done is create separate thread pools. If the call unblocking jobs are not very frequent and short, I can create a separate pool for it with core size 5 maybe. For billing jobs I'd rather create a pool with core size 25 and max-size 30.
So, my system limits won't anyway exceed because I know in even the worst situation I won't have more than 30 threads.
This will also make it easy to debug. If I have a different thread name pattern for each pool amd my system has some issues. I can easily take a thread dump and understand if the billing or the payment stuff is the culprit.
So, I think the existing design is based on some business use case which you need to thoroughly understand before proposing a solution.

How to simulate CPU bound job

I want to simulate CPU bound jobs in my simulator and i need a calculation or code that run for 1 second in the cpu ...how i will do it...
i am using the folllowing code
long Time1 = System.currentTimeMillis();
///calculation or loop that spends 1 second in cpu
long Time2 = System.currentTimeMillis();
System.out.println(Time2-Time1);
Now i need the calculation that take 1 second...I also need to simulate for 2 ,3 to 4 seconds
what code i should put in line 2.?

If you really meant binding a job to the CPU for 1 second, I don't think it is possible only with pure Java. The reason is that the OS will still remove the process from the CPU, schedule it again and so on for a number of times within 1 second. So you need to make a special kind of request to the OS to do this. The request should go and affect the process scheduling algorithm of the OS. But this is not how we want our applications to consume the CPU. We want the CPU to honor the interrupts and so on. So may be your intension is not clearly mentioned in the question or you might be trying to test something special and uncommon.
If you are just simulating something, may be you might just use a sleep() call as suggested in the comments which actually would not be consuming the CPU for 1 second but allows you to assume so for the simulation purpose.

Scheduling tasks, making sure task is ever being executed

I have an application that checks a resource on the internet for new mails. If there is are new mails it does some processing on them. This means that depending on the amount of mails it might take just a few seconds to hours of processing.
Now the object/program that does the processing is already a singleton. So right now I already took care of there really only being 1 instance that's handling the checking and processing.
However I only have it running once now and I'd like to have it continuously running, checking for new mails more or less every 10 minutes or so to handle them in a timely manner.
I understand I can take care of this with Timer/Timertask or even better I found a resource here: http://www.ibm.com/developerworks/java/library/j-schedule/index.html that uses Scheduler/SchedulerTask. But what I am afraid of.. is if I set it to run every 10 minutes and a previous session is already processing data it will put the new task in a stack waiting to be executed once the previous one is done. So what I'm afraid of is for instance the first run running for 5 hours and then, because it was busy all the time, after that it will launch 5*6-1=29 runs immediately after each other checking for mails and/do some processing without giving the server a break.
Does anyone know how I can solve this?
P.S. the way I have my application set up right now is I'm using a Java Servlet on my tomcat server that's launched upon server start where it creates a Singleton instance of my main program, then calls some method to do the fetching/processing. And what I want is to repeat that fetching/processing every "x" amount of time (10 minutes or so), making sure that really only 1 instance is doing this and that really after each run 10 minutes or so are given to rest.

Actually, Timer + TimerTask can deal with this pretty cleanly. If you schedule something with Timer.scheduleAtFixedRate() You will notice that the docs say that it will attempt to "make up" late events to maintain the long-term period of execution. However, this can be overcome by using TimerTask.scheduledExecutionTime(). The example therein lets you figure out if the task is too tardy to run, and you can just return instead of doing anything. This will, in effect, "clear the queue" of TimerTask.
Of note: TimerTask uses a single thread to execute, so it won't spawn two copies of your task side-by-side.
On the side note part, you don't have to process all 10k emails in the queue in a single run. I would suggest processing for a fixed amount of time using TimerTask.scheduledExecutionTime() to figure out how long you have, then returning. That keeps your process more limber, cleans up the stack between runs, and if you are doing aggregates, ensures that you don't have to rebuild too much data if, for example, the server is restarted in the middle of the task. But this recommendation is based on generalities, since I don't know what you're doing in the task :)

Java NIO Selector Hang (jdk1.6_20)

I'm using jdk1.6_20 on Linux 2.6. I am observing a behavior where the NIO Selector, after calling Selector.select(timeout), fails to wake-up within the timeout(timeout=5 sec). It returns much later, couple of seconds delay(2~10 seconds) . This seems to be happening frequently during initial couple of minutes of application start-up time and stabilizes later on. Since our server is heart-beating with the client, the selector failing to wake-up on time causes it miss heartbeat and the peer disconnecting us.
Any help appreciated. Thanks.

From the Javadoc for Selector.select(long):
This method does not offer real-time guarantees: It schedules the
timeout as if by invoking the Object.wait(long) method.
Since startup time for an application might put a lot of stress on a system, this may lead to wakeup-delays.
For a solution: Switch to Selector.selectNow() as a non-blocking operation and handle retries in your application code.

It doesn't matter what the timeout is, as soon as a client is connecting, the selector should wake up immediately. Therefore you have some more serious bugs.

fails to wake-up within the timeout(timeout=5 sec).
It's not supposed to 'wake-up within the timeout'. It is supposed to wakeup after the timeout expires. If you're supposed to send heartbeats within 5 seconds, a timeout of 5 seconds is too long. I would make it 2.5s in this case.

hmm... actually the story doesnt stop there ..we are not using incremental cms ..hence during the concurrent phase it is not relinquishing the cpu ... we are having 2 application servers on the same host with 16 cores and each is having 4 Parallel CMS threads besides the application threads of which there are about roughtly 45 to 60. Hence chances of CPU starvation are the most likely especially since we see that every time the selector gets delayed it is 100~200 milliseconds immediately after the concurrent-mark phase..

Java concurrency - Should block or yield?

I have multiple threads each one with its own private concurrent queue and all they do is run an infinite loop retrieving messages from it. It could happen that one of the queues doesn't receive messages for a period of time (maybe a couple seconds), and also they could come in big bursts and fast processing is necessary.
I would like to know what would be the most appropriate to do in the first case: use a blocking queue and block the thread until I have more input or do a Thread.yield()?
I want to have as much CPU resources available as possible at a given time, as the number of concurrent threads may increase with time, but also I don't want the message processing to fall behind, as there is no guarantee of when the thread will be reescheduled for execution when doing a yield(). I know that hardware, operating system and other factors play an important role here, but setting that aside and looking at it from a Java (JVM?) point of view, what would be the most optimal?

Always just block on the queues. Java yields in the queues internally.
In other words: You cannot get any performance benefit in the other threads if you yield in one of them rather than just block.

You certainly want to use a blocking queue - they are designed for exactly this purpose (you want your threads to not use CPU time when there is no work to do).
Thread.yield() is an extremely temperamental beast - the scheduler plays a large role in exactly what it does; and one simple but valid implementation is to simply do nothing.

Alternatively, consider converting your implementation to use one of the managed ExecutorService implementations - probably ThreadPoolExecutor.
This may not be appropriate for your use case, but if it is, it removes the whole burden of worrying about thread management from your own code - and these questions about yielding or not simply vanish.
In addition, if better thread management algorithms emerge in future - for example, something akin to Apple's Grand Central Dispatch - you may be able to convert your application to use it with almost no effort.

Another thing that you could do is use the concurrent hash map for your queue. When you do a read it gives you a reference of the object you were looking for, so it is possible you my miss a message that was just put into the queue. But if all this is doing is listening for a message you will catch it the next iteration. It would be different if the messages could be updated by other threads. But there doesn't really seem to be a reason to block that I can see.