I am developing a application for benchmarking purposes, for which I require to create large number of http connection in a short time, I created a program in java to test how much threads is java able to create, it turns out in my 2GB single core machine, the limit is variable between 5000 and 6000 with 1 GB of memory given to JVM after which it hits outofmemoryerror with heap limit reached.
It is suggested that erlang will be able to generate much more concurrent processes, I am willing to learn erlang if it is capable of solving the problem , can erlang be able to generate somewhere around 100000 processes which are essentially http requests waiting for responses, in a matter of few seconds without reaching any limit like memory error etc.,
According famous Richard Jones blog you can handle 100k connection almost out of the box. You have to increase process limit, see +P parameter and it needs little bit memory management trickery e.g. gc or hibernate. To achieve significantly more you have to do more hacking with C.
It can handle pretty much anything you throw at it. Erlang processes are extremely light weight.
See http://www.sics.se/~joe/apachevsyaws.html for a benchmark in concurrency between Yaws and Apache. It gives you a good idea of what's possible.
I thought an interesting thing for this is that a guy was able to get a million comet connections with mochiweb http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-1
As previously stated, a lot is a good answer. And also given your requirement is to write a tsunami client, you can/should distribute code over to several erlang nodes.
Even better, you might want to check out Tsung. It is a distributed load testing application written in erlang.
And if you don't want to use it, I am pretty sure there's code you want to read in there.
Related
I'm interested in executing about 50 HTTP requests/second from a single machine. I don't care so much about latency but I do care about throughput.
I'm trying to decide whether to use Apache HttpAsyncClient or use Netty. Could someone shed some light about the advantages of each regarding my problem?
I've found this comparison but I was hoping for a little bit more detailed explanation on which one is better and for what use case. Also, does the comparison means that using the synchronous apache HTTP client with 200 threads be better than the other options? Isn't 200 threads a bit too much (assuming I'm using a normal computer with 4 cores, 2 threads per core and 12GB of RAM)?
Thanks in advance
The main problem with these benchmarks is that in real life you have more threads and much more noise, so you can't really expect to get similar results in production unless you go for the async IO option.
You're looking into getting more throughput, and as expected Netty based clients wins big time in their benchmark. So it's probably your best bet.
We're using Netty very successfully for a wide array of applications, and it never fails us. You can use ning async-http-client, and then you don't have to implement a client all by your self.
Do note however, as I stated in the comments, I base my answer on my personal experience, and on our production metrics. Never believe a random benchmark post you see posted on the internet, nor a StackOverflow answer. Test it your self ;)
I'm beginning to work on a game server written in Java. Granted, Java is not the best solution to server development, but it is the language that I am most comfortable with.
I'm trying to design this game server to handle a lot of connections, and I'm not sure what to focus more on: storing data in memory or keeping memory usage down and using more raw CPU power?
Certainly depends on what your game is like. I recently benchmarked my server with 5,000 clients at like 20MB of ram. You may keep much more state data than me, and it could be a concern.
Your serious issue with many connections is setting up the sockets to handle it properly, using certain manner of socket handling becomes very bogged down or breaks #1024 connections etc. I'm not sure how much optimization you can do in java.
Look at this link for what I'm talking about.
And, good luck! And also, switch languages as soon as possible to a language offering comparable features to java but without the awful drawbacks (meaning Objective-C or C#). I'm not just "slamming java" but the problem you're going to reach when you talk about doing things that are performant is that java will abstract you too far from the Operating System and you will not be able to take advantage of optimizations when you need it.
I wouldn't suggest you design the server for far more than you really need to. If you suddenly find you have 10,000s of clients, you can re-design the system.
I would start with a basic server e.g. i5 with 8 GB of memory for less than £500, or an i7 with 24 GB for less than £1000.
As your number of connections grows you are likely to run out of bandwidth before you run out of resources unless you use a cloud solution.
BTW: You can implement a high frequency trading system with less than 100 micro-second latency in Java. I haven't heard of any Objective-C high frequency trading systems. C# might be able to perform as well or better on Windows, but I prefer Linux for server system.
I am building a complex HTML 5 application that takes advantage of Websockets. I am getting to the point where I have a lot of different types of data that gets updated in real time on the screen.
I want to know if it is going to be better for me to have fewer Websockets that are more complex, or a lot of simple Websockets open per page.
I added http://github.com/TooTallNate/Java-WebSocket web socket server to my Grails Application.
Right now I am going down the path of using a lot of simple web sockets for each task. I know using more sockets will use more memory on the server side but also more sockets means more concurrent processing.
Does anyone have any advice on how I can balance this.
Thanks for any tips in advance. Keith Blanchard
I think it is hard to make any reasonable statements about websockets without measuring the actual performance in specific browsers.
My inclination would be to have a single websocket per client.
There are some pretty hard limits on capacity server-side when doing IO ... relatively easily to saturate the channel when you have many connections (something that can bite heavily ajaxified systems as well).
Again, need to really measure to make intelligent statements about this.
Websocket-per-client would also make the application much more manageable ... depends on your actual use-case, but "more concurrency" is not necessarily better and can make managing state incredibly complex.
I personally did some benchmark on this one, and the results are:
10 websockets on a single page will cause page a little unresponsive when data coming in from each socket.
50 websockets on a single page will cause an unbearable freeze on the web.
So somewhere around 10 or less than 10 would be your upper limit.
What's a good method for assigning work to a set of remote machines? Consider an example where the task is very CPU and RAM intensive, but doesn't actually process a large dataset. The language of choice would be Java. I was thinking Hadoop would be a good option, but the dataset passed between remote machines is fairly small, and Hadoop seems to focus mainly on the distribution of data rather than distribution of work.
What are some good technologies that can help?
EDIT: I'm mainly interested in load balancing. There will be a series of jobs with a small (< 3MB) dataset, but significant processing and memory needs.
MPI would probably be a good choice, there's even a JAVA implementation.
MPI may be part of your answer, but looking at the question, I'm not sure if it addresses the portion of the problem you care about.
MPI provides a communication layer between processing components. It is low level requiring you to do a fair amount of work, but from what I saw in an introduction presentation, it also comes with some common matrix data manipulation functions.
In your question, you seem to be more interested in the load balancing/job processing aspects of the problem. If that really is your focus, maybe a small program hosted in a Servlet or an RMI server might be sufficient. Let each program go to the server for their next unit of work and then submit the results back (you might even be able to use a database/file share, but pay attention to locking issues). In other words, a pull mechanism versus a push mechanism.
This approach is fairly simple to implement and gives you the advantage of scaling up by just running more distributed clients. Load balancing isn't too important if you intend to allow your process to take full control of the machine. You can experiment with running multiple clients on a machine that has multiple cores to see if you can improve overall through-put for the node. A multi-threaded client would be more efficient, but can increase complexity depending on the structure of the code you are using to solve the problem.
So for some research work, I need to analyze a ton of raw movement data (currently almost a gig of data, and growing) and spit out quantitative information and plots.
I wrote most of it using Groovy (with JFreeChart for charting) and when performance became an issue, I rewrote the core parts in Java.
The problem is that analysis and plotting takes about a minute, whereas loading all of the data takes about 5-10 minutes. As you can imagine, this gets really annoying when I want to make small changes to plots and see the output.
I have a couple ideas on fixing this:
Load all of the data into a SQLite database.
Pros: It'll be fast. I'll be able to run SQL to get aggregate data if I need to.
Cons: I have to write all that code. Also, for some of the plots, I need access to each point of data, so loading a couple hundred thousand files, some parts may still be slow.
Java RMI to return the object. All the data gets loaded into one root object, which, when serialized, is about 200 megs. I'm not sure how long it would take to transfer a 200meg object through RMI. (same client).
I'd have to run the server and load all the data but that's not a big deal.
Major pro: this should take the least amount of time to write
Run a server that loads the data and executes a groovy script on command within the server vm. Overall, this seems like the best idea (for implementation time vs performance as well as other long term benefits)
What I'd like to know is have other people tackled this problem?
Post-analysis (3/29/2011): A couple months after writing this question, I ended up having to learn R to run some statistics. Using R was far, far easier and faster for data analysis and aggregation than what I was doing.
Eventually, I ended up using Java to run preliminary aggregation, and then ran everything else in R. R was also much easier to make beautiful charts than using JFreeChart.
Databases are very scalable, if you are going to have massive amounts of data. In MS SQL we currently group/sum/filter about 30GB of data in 4 minutes (somewhere around 17 million records I think).
If the data is not going to grow very much, then I'd try out approach #2. You can make a simple test application that creates a 200-400mb object with random data and test the performance of transferring it before deciding if you want to go that route.
Before you make a decision its probably worth understanding what is going on with your JVM as well as your physical system resources.
There are several factors that could be at play here:
jvm heap size
garbage collection algorithms
how much physical memory you have
how you load the data - is it from a file that is fragmented all over the disk?
do you even need to load all of the data at once - can it be done it batches
if you are doing it in batches you can vary the batch size and see what happens
if your system has multiple cores perhaps you could look at using more than one thread at a time to process/load data
if using multiple cores already and disk I/O is the bottleneck, perhaps you could try loading from different disks at the same time
You should also look at http://java.sun.com/javase/technologies/hotspot/vmoptions.jsp if you aren't familiar with the settings for the VM.
If your data have a relational properties, there are nothing more natural than storing it at some SQL database. There you can solve your biggest problem -- performance, costing "just" to write your appropriate SQL code.
Seems very plain to me.
I'd look into analysis using R. It's a statistical language with graphing capabilities. It could put you ahead, especially if that's the kind of analysis you intend to do. Why write all that code?
I would recommend running a profiler to see what part of the loading process is taking the most time and if there's a possible quick win optimization. You can download an evaluation license of JProfiler or YourKit.
Ah, yes: large data structures in Java. Good luck with that, surviving "death by garbage collection" and all. What java seems to do best is wrapping a UI around some other processing engine, although it does free developers from most memory management tasks -- for a price. If it were me, I would most likely do the heavy crunching in Perl (having had to recode several chunks of a batch system in perl instead of java in a past job for performance reasons), then spit the results back to your existing graphing code.
However, given your suggested choices, you probably want to go with the SQL DB route. Just make sure that it really is faster for a few sample queries, watch the query-plan data and all that (assuming your system will log or interactively show such details)
Edit,(to Jim Ferrans) re: java big-N faster than perl (comment below): the benchmarks you referenced are primarily little "arithmetic" loops, rather than something that does a few hundred MB of IO and stores it in a Map / %hash / Dictionary / associative-array for later revisiting. Java I/O might have gotten better, but I suspect all the abstractness still makes it comparitively slow, and I know the GC is a killer. I haven't checked this lately, I don't process multi-GB data files on a daily basis at my current job, like I used to.
Feeding the trolls (12/21): I measured Perl to be faster than Java for doing a bunch of sequential string processing. In fact, depending on which machine I used, Perl was between 3 and 25 times faster than Java for this kind of work (batch + string). Of course, the particular thrash-test I put together did not involve any numeric work, which I suspect Java would have done a bit better, nor did it involve caching a lot of data in a Map/hash, which I suspect Perl would have done a bit better. Note that Java did much better at using large numbers of threads, though.