I am working on a bittorrent client. While communicating with the peers the easiest way for me to communicate with them is to spawn a new thread for each one of them. But if the user wants to keep connections with large number of peers that my cause me to spawn a lot of threads.
Another solution i thought of is have one thread to iterate through peer objects and run them for e period.
I checked other libraries mostly in ruby( mine is in java ) and they spawn one thread for each new peer. Do you think spawning one thread will degrade performence if user sets the number of connections to a high number like 100 or 200?
It shouldn't be a problem unless you're running thousands of threads. I'd look into a compromise, using a threadpool. You can detect the number of CPUs at runtime and decide how many threads to spin up based on that, and then hand out work to the threadpool as it comes along.
You can avoid the problem altogether by using Non-blocking IO (java.nio.*).
I'd recommend using an Executor to keep the number of threads pooled.
Executors.newFixedThreadPool(numberOfThreads);
With this, you can basically add "tasks" to the pool and they will complete as soon as threads become available. This way, you're not exhausting all of the enduser's computer's threads and still getting a lot done at the same time. If you set it to like 16, you'd be pretty safe, though you could always allow the user to change this number if they wanted to.
No.....
Once I had this very same doubt and created a .net app (4 years ago) with 400 threads....
Provided they don't do a lot of work, with a decent machine you should be fine...
A few hundred threads is not a problem for most workstation-class machines, and is simpler to code.
However, if you are interested in pursuing your idea, you can use the non-blocking IO features provided by Java's NIO packages. Jean-Francois Arcand's blog contains a lot of good tips learned from creating the Grizzly connector for Glassfish.
Well in 32bit Windows for example there is actually a maximum number of native Threads you can create (2 Gigs / (number of Threads * ThreadStackSize (default is 2MB)) or something like that). So with too many connections you simply might run out of Virtual Memory address space.
I think a compromise might work: Use a Thread Pool with e.g. 10 Threads (depending on the machine) running and Distribute the connections evenly. Inside the Thread loop through the peers assigned to this Thread. And limit the maximum number of connections.
Use a thread pool and you should be safe with a fairly large pool size (100 or so). CPU will not be a problem since you are IO bound with this type of application.
You can easily make the pools size configurable and put in a reasonable maximum, just to prevent memory related issues with all the threads. Of course that should only occur if all the threads are actually being used.
Related
I have 1,00,00 elements to use in my web crawling app. I am confused to choose the number of threads to be used with Executor Service (Java 6).
Actually I am getting Out Of Memeory Error if I use more threads which makes me confused to choose the number of threads. Also as many threads hit the server, the internet gets halted and I have to restart my PC everytime.
Could someone help me choose the number of threads for this case?
Thanks!
It is difficult to say, because the limiting factors in terms of throughput is probably your system's network bandwidth and latency. If you try to go beyond that limit, you are likely to reduce throughput due to various secondary effects: e.g. congestion, thrashing, throttling (by the server), etc.
The correct approach is to make the number of threads in the pool a configuration parameter ... and tune it. Start small, and when increasing the thread count doesn't improve throughput significantly, stop increasing it.
I need to process 80 files of info, and I'm doing it via groups of 8 threads, what i would like to do is to to always have 8 threads running (right now i have 8 threads, and after those 8 finish their job, another 8 are generated, and so on).
So I Would like to know if there is a way to do his:
launch 8 threads.
after 1 thread finishes its job, launch another thread (so all the
time I have 8 threads running, until the job is done)
Why not use a thread pool, and in particular a fixed size thread pool ? Configure your thread pool size to be 8 threads, and then submit all your work items as Runnable/Callable objects. The thread pool will execute these using the 8 configured threads.
So, everyone is quick to jump in and tell you to use a thread pool. Sure, that's the right way to achieve what you want. The question is, is it the right thing to want? It's not as simple as throw a bunch of threads at the problem, and magically everything is solved.
You haven't told us the nature of the processing. Are the jobs I/O bound, or CPU bound1? If they are CPU bound, the threads do nothing. If they are I/O bound, the threading might help.
You haven't told us if you have eight cores (or compute units). If you can't guarantee that you'll have that, it might not be best to have eight threads running.
There's a lot to think about. You're increasing the complexity of your solution. Maybe it's getting you what you want, maybe not.
1: Yes, you said you're processing files, but that doesn't tell us enough. Maybe the processing is intensive (think: rendering a video file). Or maybe you're reading the files from a very fast disk (think: SSD or memory-mapped files).
I have a program which runs (all day) tasks in parallel (no I/O in the task to be executed) so I have used Executors.newFixedThreadPool(poolSize) to implement it.
Initially I set the poolSize to Runtime.getRuntime().availableProcessors(), but I was a bit worried to use all the available cores since there are other processes running on the same PC (32 cores).
In particular I have ten other JVM running the same program (on different input data), so I'm a bit worried that there might be a lot of overhead in terms of threads switching amongst the available cores, which could slow down the overall calculations.
How shall I decide the size of the pool for each program / JVM?
Also, in my PC, there are other processes running all the time (Antivirus, Backup, etc.). Shall I take into account these as well?
Any advice is going to be dependent upon your particular circumstances. 10 JVMs on 32 cores would suggest 3 threads each (ignoring garbage collection threads, timer tasks etc...)
You also have other tasks running. The scheduler will ensure they're running, but do they have to be responsive ? More responsive than the JVM ? If you're running Linux/Unix then you can also make use of prioritisation (via nice) to ensure particular processes don't hog the CPU.
Finally you're running 10 JVMs. Will that cause paging ? If so, that will be slow and you may be better off running fewer JVMs in order to avoid consuming so much memory.
Just make sure that your key variables are exposed and configurable, and measure various scenarios in order to find the optimal one.
How shall I decide the size of the pool for each program / JVM?
You want the number of threads which will get you close to 99% utilisation and no more.
The simplest way to balance the work is to have the process running once, processing multiple files at concurrently and using just one thread pool. You can set up you process as a service if you need to start files via the command line.
If this is impossible for some reason, you will need to guesstimate how much the thread pools should be shrunk by. Try running one process and look at the utilisation. If one is say 40% then I suspect ten processes is over utilised by 400%. i.e then you might reduce the pool size by a factor of 4.
Unfortunately, this is a hard thing to know, as programs don't typically know what else is or might be going on on the same box.
the "easy" way out is to make the pool size configurable. this allows the user who controls the program/box to decide how many threads to allocate to your program (presumably using their knowledge of the general workload of the box).
a more complex solution would be to attempt to programmatically determine the current workload of the box and choose the pool size appropriately from that. the efficacy of this solution depends on how accurately you can determine the workload and potentially adapt as it changes over time.
Try grepping the processes, check top/task manager and performance monitors to verify if this implementation is actually affecting your machine.
This article seems to contain interesting info about what you are trying to implement:
http://www.ibm.com/developerworks/library/j-jtp0730/index.html
I am writing a server side application using Java.
The server holds a number of users of the system. For each user, I want to synchronize its disk space with a remote network storage. Because synchronizations are independent, I am thinking to do them concurrently.
I am thinking to create one thread for each user and let the synchronization tasks to fire at the same time.
But the system can have tens of thousands of users. This means creating tens of thousand thread at one time and fire at the same time. I am not sure if this is something JVM can handle.
Even if it can handle this, will that be memory efficient because each thread have its own stack and this could be a big memory hit!
Please let me know your opinion.
Many thanks.
You could look at a fixed size thread pool giving a pool of threads to execute your task. This would give the benefit of multithreading with a sensible limit.
Check out Executors.newFixedThreadPool()
You should look into Non-blocking IO.
Here is a "random" article about it from courtesy of google:
http://www.developer.com/java/article.php/3837316/Non-Blocking-IO-Made-Possible-in-Java.htm
Personally I wouldn't have tens of thousands of users on a single machine. You won't be able to much per user with this many users active. You should be able to afford more than one machine.
You can have this many thread in Java but as you say this is not efficient. You can use an NIO library to manage multiple connection with each thread.
Libraries like
http://mina.apache.org/
http://www.jboss.org/netty
Are suitable.
Also interesting http://code.google.com/p/nfs-rpc/
I have a program that starts up and creates an in-memory data model and then creates a (command-line-specified) number of threads to run several string checking algorithms against an input set and that data model. The work is divided amongst the threads along the input set of strings, and then each thread iterates the same in-memory data model instance (which is never updated again, so there are no synchronization issues).
I'm running this on a Windows 2003 64-bit server with 2 quadcore processors, and from looking at Windows task Manager they aren't being maxed-out, (nor are they looking like they are being particularly taxed) when I run with 10 threads. Is this normal behaviour?
It appears that 7 threads all complete a similar amount of work in a similar amount of time, so would you recommend running with 7 threads instead?
Should I run it with more threads?...Although I assume this could be detrimental as the JVM will do more context switching between the threads.
Alternatively, should I run it with fewer threads?
Alternatively, what would be the best tool I could use to measure this?...Would a profiling tool help me out here - indeed, is one of the several profilers better at detecting bottlenecks (assuming I have one here) than the rest?
Note, the server is also running SQL Server 2005 (this may or may not be relevant), but nothing much is happening on that database when I am running my program.
Note also, the threads are only doing string matching, they aren't doing any I/O or database work or anything else they may need to wait on.
My guess would be that your app is bottlenecked on memory access, i.e. your CPU cores spend most of the time waiting for data to be read from main memory. I'm not sure how well profilers can diagnose this kind of problem (the profiler itself could influence the behaviour considerably). You could verify the guess by having your code repeat the operations it does many times on a very small data set.
If this guess is correct, the only thing you can do (other than getting a server with more memory bandwidth) is to try and increase the locality of your memory access to make better use of caches; but depending on the details of the application that may not be possible. Using more threads may in fact lead to worse performance because of cores sharing cache memory.
Without seeing the actual code, it's hard to give proper advice. But do make sure that the threads aren't locking on shared resources, since that would naturally prevent them all from working as efficiently as possible. Also, when you say they aren't doing any io, are they not reading an input or writing an output either? this could also be a bottleneck.
With regards to cpu intensive threads, it is normally not beneficial to run more threads than you have actual cores, but in an uncontrolled environment like this with other big apps running at the same time, you are probably better off simply testing your way to the optimal number of threads.