I would like to test a sevlet that I 've made with simultaneous requests (100 or 1000 or more). I was searching about it and I found JMeter but I am not quite sure if it does what I want.
Do you know how I could do something like that. Do you know any tutorial or guide that it could help me? (I am not experienced in programming)
p.s.
I run my servlet on Jetty because I am using the jetty continuations. This is also what I want to test.
JMeter is rather easy to use. Also consider installing JMeter plugins that enable richer set of graphs and sampling methods. See my blog post for some instructions, also have a look at a sample performance test plan for JMeter.
JMeter is a good choice, it can do this job. See the user manual, it explains in detail how to set up the test.
Btw: Running the test tool and the application on the same machine is not a relistic performance/throughput test scenario and can only provide an indication on how your servlet behaves in the real world.
You can just use any HTTP performance tester, for example apache bench:
ab -c 100 -n 100000 http://localhost/
# Hit the http://localhost/ with 100000 requests, 100 at a time
This will output something like:
Requests per second: 4497.18 [#/sec] (mean)
JMeter is a good choice - easy to use and sufficient for most cases.
If you want to do simple tests it should be enough. However, you are interested in writing more complex tests scenarios, I'd recommend HP LoadRunner (but it's commercial software).
You may want to rethink he preciseness of your test. Unless your connecting agents are defined by a synchronized clock the odds of a simultaneous event connection are pretty low. Humans are pretty chaotic, organic computing units tied to imprecise clocks dictating the interval between requests to a service. You actually have to get a very high number of chaotic requests before you get some behavior of natural simultaneous incidents of some number of users within the same section of code having made the request and the same time mark. Now, it is highly likely that you can have a high number coincident within a short window, such as 200 per second, but true simultaneous behavior is quite rare in real world conditions.
Food for thought....
Related
We have a costumer that have 3 stores with different databases. In every store has a wildfly running some webservices which communicate between them. Each json request with 10/30 rows spends 1 seconds in average. Every wildfly uses 1,5 gb of RAM. I know that memory is always a problem in Java, but can I be more economic using some microframework like Javalin or microservices rather than a java ee app server? And node.js would be a option for better performance?
Before you start looking into a different architecture, which would probably make for a major rewrite, find out where all that time is going. Set up profiling on the WildFly servers. Start by doing that on one, then have some calls coming in. Check how much time is spent in various parts of the stack. Is one call to the web service handled rather slowly? Then see where that time goes. It might be the database access. Is one such call handled pretty quickly on the server itself once it comes in? Then your best bet is your losing time on the network layer.
Check the network traffic. You can use Wireshark or a similar tracing tool for this. See how much time actually passes between a request coming in and the response going out. Is that slow but the processing on Wildfly itself seems fast enough? Maybe there's some overhead going on (like security). Is the time between request and response very fast? You're definitely looking at the network as the culprit.
Eventually you may need to have profiling and network tracing active on all three servers simultaneously to see what's going on, or for each combination of two servers. It may turn out only one of them is the bottleneck. And if you have servers A, B and C, from the sound of it your setup might cause a call from A to B to also require a call from B to C before some result can be returned to A. If that is the case, it's little wonder you may see some serious latency.
But measure and find the root of the problem before you start deciding to change the entire framework and a different programming language. Otherwise you may put a lot of time into something for no improvement at all. If the architecture is fundamentally flawed you need to think of a different approach. If this is still in the prototyping phase that would be substantially easier.
Well, first you may prune your WildFly installation or try Quarkus :)
I'm interested in executing about 50 HTTP requests/second from a single machine. I don't care so much about latency but I do care about throughput.
I'm trying to decide whether to use Apache HttpAsyncClient or use Netty. Could someone shed some light about the advantages of each regarding my problem?
I've found this comparison but I was hoping for a little bit more detailed explanation on which one is better and for what use case. Also, does the comparison means that using the synchronous apache HTTP client with 200 threads be better than the other options? Isn't 200 threads a bit too much (assuming I'm using a normal computer with 4 cores, 2 threads per core and 12GB of RAM)?
Thanks in advance
The main problem with these benchmarks is that in real life you have more threads and much more noise, so you can't really expect to get similar results in production unless you go for the async IO option.
You're looking into getting more throughput, and as expected Netty based clients wins big time in their benchmark. So it's probably your best bet.
We're using Netty very successfully for a wide array of applications, and it never fails us. You can use ning async-http-client, and then you don't have to implement a client all by your self.
Do note however, as I stated in the comments, I base my answer on my personal experience, and on our production metrics. Never believe a random benchmark post you see posted on the internet, nor a StackOverflow answer. Test it your self ;)
I would like to run as many requests as possible against a URL and would like to know what the best approach would be.
Requirements trigger GET requests and read response.
I started by simply creating a runnable that synchronously does a get and blocks. Then I dumped those with a loop into a fixed thread pool executor. Running it with 100 Threads will result in 200 requests per second.
I also looked into async libraries like http://hc.apache.org/httpcomponents-asyncclient-4.0.x/ but running those will result in lots of futures that never return.
Can things like NIO or async libraries help to improve throughput or how should I tackle the problem?
There is a free and open source tool from Apache called JMeter. It's designed for HTTP load testing and provides
record-and-replay capabilities via embedded HTTP Proxy Server for rapid tests development
Assertions - to assure that response time does not exceed reasonable values, check response content, return code, message, etc.
Listeners - to build tables and graphs which help to analyze load test results.
Already has a number of plugins which extend base functionality
Supports scripting using a number of supported languages like JavaScript or Beanshell
Hope this helps you in your testing and saves your valuable time from re-inventing the wheel to produce a better product instead.
I am developing a application for benchmarking purposes, for which I require to create large number of http connection in a short time, I created a program in java to test how much threads is java able to create, it turns out in my 2GB single core machine, the limit is variable between 5000 and 6000 with 1 GB of memory given to JVM after which it hits outofmemoryerror with heap limit reached.
It is suggested that erlang will be able to generate much more concurrent processes, I am willing to learn erlang if it is capable of solving the problem , can erlang be able to generate somewhere around 100000 processes which are essentially http requests waiting for responses, in a matter of few seconds without reaching any limit like memory error etc.,
According famous Richard Jones blog you can handle 100k connection almost out of the box. You have to increase process limit, see +P parameter and it needs little bit memory management trickery e.g. gc or hibernate. To achieve significantly more you have to do more hacking with C.
It can handle pretty much anything you throw at it. Erlang processes are extremely light weight.
See http://www.sics.se/~joe/apachevsyaws.html for a benchmark in concurrency between Yaws and Apache. It gives you a good idea of what's possible.
I thought an interesting thing for this is that a guy was able to get a million comet connections with mochiweb http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-1
As previously stated, a lot is a good answer. And also given your requirement is to write a tsunami client, you can/should distribute code over to several erlang nodes.
Even better, you might want to check out Tsung. It is a distributed load testing application written in erlang.
And if you don't want to use it, I am pretty sure there's code you want to read in there.
We currently facing some stability issues with our develop web application product. This product was built in part by our partner contractor, and we want to have a good standard metric for stability. The issues we have been facing is constant crashing. The web application is unable to identify when there are more request than it can handle, it builds up memory (like a memory leak) and later it dies without any type of possible recovery.
We would like to write a very simple measurement for our partner contractor to meet. We thought about a few ideas:
A system that is able to identify high loads of request and serve server unavailable try again pages, until it recovers from the high load.
A set number of users concurrent or pageviews that will allow us to have a clear metric of when to use scalability options like Load Balancer and Caching.
At this moment we have to use caching and load balancing to be able to recycle the web applications every x hours (depending on the load) so they don't die constantly.
Thanks for your help.
"High load" is really hard to define.
It's much, much easier for you to determine what the minimally acceptable service levels are.
Minimum number of concurrent requests.
Maximum time to serve a request.
Minimum number of requests per hour.
Simple service levels like that are easy to measure, easy to simulate and easy to write into a contract. Any lawyer or accountant can review the results of a load test and say they did or did not meet the minimums. No deep technical knowledge needed at all.
Note that when you do this, "minimums become maximums". If you say they must serve a minimum of 10,000 requests per hour, your testing will often reveal that's the maximum, also.
So define your minimums and maximums from your business model. How many do you need to keep people happy and productive? Asking for more is silly. Asking for fewer means unhappy or unproductive users.
Typically, I've seen these performance type requirements built into the specification... "the system should support x amount of concurrent users" or "x number of requests per hour".
These things can be easily tested and verified using something like LoadRunner or you can roll your own similar load tester using something like HttpUnit.
What you'd need are following parameters:
Load Test:
i) Estimate your application usage behaviour.. (i.e.) no. of concurrent users expected, typical user activity
ii) Load the application gradually and look at parameters like cpu usage, response times, throughput, etc.
Sustainability:
Load the application (at optimum load) for a considerable amount of time (12-24 hrs) and look at the same parameters cpu usage, error levels, etc.
You can also try scalability, where you would add hardware incrementally and monitor the behaviour of the application.
This should give you a decent understanding of the behaviour of your system.