I am currently using HtmlUnit and Selenium to drive it (WebDriver) within my production code.
I am scaping and interacting with various websites programmatically with these libraries and am having some success and not experiencing memory issues (ensuring sessions are always cleaned up).
I am wondering if these libraries are okay for a production environment or recommended against. This is difficult to find via Google due to the enormous amount of information about automated testing rather than how I am using them.
I realise this is a fairly generic question, but I am seeking advice on these libraries and potentially better alternatives.
WebDriver and Selenium are perfectly suited for production environnement. I use them quite extensively for 2 years now on a multi-machines/multi-datacenters distributed grid and had absolutely no performance nor stability problems we couldn't have coped with.
Our preferred driver is the Firefox one (heavier than HTMLUnit, and harder to configure), and we had to tweak the grid to understand how many instances we can run. Our maximum for stability was 1 per core
Our selenium/webdriver instances have run 24/7 for 2 years now (1 year with selenium 1, and the other migrating selenium 2/ WebDriver incrementally) and with an appropriate monitoring (you should monitor Memory Usage/CPU Usage) and a bunch of load testing, we had reached the good level where we have experienced several monthes without restarting a process
We've used HTMLUnit extensively too, and are equally satisfied with this library
The essential point of my post is : YES, these library are production-ready. But, as all production software, you'll have to benchmark their use to find the appropriate configuration for the optimal stability. I recommend you to use the Selenium Grid in production, which is a great way to parallelize process
I'm using HtmlUnit for something similar in production and have had quite a bit of issues - mostly performance related. Currently I switched to snapshot version of HtmlUnit 2.10 where some important for me performance improvements were implemented (e.g. replacing ArrayList.contains() with HashSet.contains() on DomNode.addDomChangeListener()).
Still, the CPU load is quite high on JavaScript-heavy pages. Typically, I can't run more than 10 of them simultaneously on dual core Linux box. I believe HtmlUnit using Rhino (JavaScript engine) in interpreter mode only, which is pretty slow. Also, you need to be careful with releasing all resources used by HtmlUnit to avoid memory leaks.
All in all, it certainly noticeable that HtmlUnit was designed to run relatively short lived test cases and not long running server applications. It's possible to tweak it enough so it's manageable but certainly it could have been better.
Another approach I found promising is phantom-js, which is headless version of WebKit browser, native app which is much faster on running JavaScript.
Generally, use your testing "gut feeling" about that. What WebDriver and HTMLUnit does is, that it simulates real user performing some actions in the webpage.
My personal gut feeling says, that I should do as less production testing, as possible. So I personally would use these tools only for verification, if my webapp is still alive.
Yes, its generic answer for generic question, but try this:
Gather around people responsible for the webapp and ask them:
Should be it tested on production? (so there is always slight chance, that some customers will see those test data)
If yes, what should be tested on production?
If yes, should it be automated?
And then you have answer ;)
Related
I wrote a performance test using JMeter intended to measure the speed of loading a home page. The website is built using Spring MVC. The average speed is 8462 ms and the median is 8271 ms, which are pretty good. I would like to determine what component in the application that consume the most time to load the home page. What is the best technique to do this ?
For the server side, use Visual VM to see what's happening. You can see heap memory, CPU, threads, and lots more.
For the client side, I'd recommend using YSlow and Chrome developer tools. You'll be able to see each element on the page and its load time.
I think the right anwser would be to go with spring AOP.
Here a link to a strong article that explain how to do it.
http://www.javaworld.com/community/node/3763
If you have performance requirement in your application, I would suggest that you let some of those indicators in production.
Try JVM Monitor http://www.jvmmonitor.org/. If you are using eclipse as an IDE, it's got an eclipse plugin. I found it easy to install and use. It helped me identify bottlenecks in a web application I was working on.
We would be in near future implementing a solution to modernize our iSeries applications
written as RPG programs with some stored procedures, and our preferred way is leveraging the latest and greatest of what Java has to offer in this space.
From googling and checking other questions here on STOVFlow, JTOpen seems to be the defacto
library/toolset which has worked for most and I was encouraged to see that Tomcat runs on an I-series box with out any issues.
With this as the background, I am thinking of the following as the high level sol arch
Install IBM JRE and use JTOpen's capabilities to invoke RPG Programs and in some cases directly call the stored procedures running on DB2
Have Tomcat host a modern web application built using Grails and other frameworks (Camel, Smooks) to provide an application logic layer which would fill any mediations, transformations required for the old functionality to be offered to the user from a browser
Questions-
If any one of you has been involved in such an exercise, please share the pitfalls with this approach
Is there a significant performance drop with respect to response times for the end user?
Would it be better to some how expose the JT400 code as web services and run the web app on a different machine altogether consuming these web services?
Be very careful with calling RPG from Java because RPG is not threadsafe without some changes.
When I was at COMMON, the best product I felt on the market was Profound UI. There are several others from a variety of vendors. Most of these products do not use Java. Java on the i tends to be slow. (There are things that can be done to make it faster, but native is always faster.) You'll pay the price for these products, but just imagine how much time it would take you to do this yourself. For the above, I was quoted in the $20+ thousand range. But like all i products prices vary greatly based on system.
To directly answer your questions:
I have been doing research on modernization as time allows, the products weren't quite there yet (at the time I looked) to use it for what we wanted to use it for (before COMMON 2011). Now it looks like it might work.
This really depends on your system. A newer system will have less problems than an older system. Web will always be slower than the green-screen. Hands-down entry people won't like it. Executives and younger people will love it.
Your slow point is running the business logic. It wouldn't matter which server the HTML is coming from.
I've found that for all practical purposes an AS/400 behaves like an AIX box seen from Java code, and you must use jt400 (jtOpen) to communicate with the AS/400 specific features like data queues, files etc. This works pretty well, but the slowness of invoking the JVM pressures Java based solutions to be long running.
Note also that QTEMP is generally unavailable as a mechanism to keep state due to the nature of prestarted jobs.
Under V6R1 Java 6 is available and runs pretty well in the "new technology" edition. You can then run almost all Java based solutions, including web servers like Jetty in it. Note that Java defaults to code page 819 when accessing IFS files directly. Windows clients using AS/400 as a network drive uses a compatible code page.
Can anyone point me out to a project out there that I can download and run it and it would load / stress test itself and then provide me with reports? I want the project to be as big as it can and to involve as many components in java as it can, also i need it free... or to some very good already made results over the web that I can already take a look and get decision. Thanks!
main issue to benchmark is which would run it faster / better, solaris or linux
Linux and Solaris are not much different seen from your perspective, and I do not believe that the benchmark you ask for exists. A much better approach is to take the application you want to run - which hopefully should be platform independent already - and deploy to the architectures you want to test and then attach with jvisualvm and apply your standard test suite.
This will give you quite a good look at the performance without skewing with heavy profiling.
My guess is that for identical configurations you will see that Linux is slightly better than Solaris as the amount of unused memory available for disk caching will strongly influence the performance of the system. Also note that expert system tuning can also make a big difference, but that I believe that you are most interested in the "out of the box" performance.
It really depends on the aspects of usage you want to benchmark.
I did this for database applications, in this area TPC could be helpful.
I would recommend google: benchmark java numeric|transaction|rendering|olap
Depending on the characteristics of your use case.
Edit: regarding your comment of a java app running on an applicationserver, check from the backend db server whats the maximal throughput? TPC, then write a multithreaded benchmark client to checkout whats the business logics performance. The last step would be to involve webservers using Apache JMeter. This procedure allows you to tune all relevant parameters from OS over DB-Poolsizes etc.
I plan to start learning a Java web framework (I love the Java API) I have already used Rails and Django.
I want something close to Java but without all the complexity of J2EE.
I've found 2 frameworks that could be good for me:
Grails
Grails looks great, it uses Groovy which is better than Java for web application (I think..) but it's slower than pure-java based frameworks (Hibernate, Strut, Spring) It looks pretty simple to deploy (send .war and it's ok!), the GSP is great! It's a bit harder to debug (need to restart the server at each modification and stacktraces contain a mix of Java and Groovy traces which is not always the easiest to understand)
Play!
This framework also looks great; it's faster than Grails (It uses Java) but I don't really like how it uses Java, it modifies the source code to transform the property calls as setXXX/getXXX, I do not like that... The framework also has a caching function that Grails does not have. I don't really like the Template Engine.
It's also easer to debug (no need to restart the server, stacktraces are clearer)
What do you recommend?
I am looking for something easy to learn (I have a lot of Ruby experience, not so much Java experience but I love the Java API), fully featured (That's no a problem with all the Java Library available, but if it's bundle and integrated I prefer), has good scalability and is not too slow (faster than Ruby) Ideally I would like to use a framework with a decent community to easily find support.
PS: I am not interested in JRuby on Rails
I switched from Grails to Play and I never looked back. My biggest problem with Grails was overall robustness and developer usability. Most of the time I got bitten by the fact that Grails glues together the usual stack of Spring MVC and Hibernate while trying to hide this fact and giving you a Rails-like API (personal opinion of mine). The problem with this is, once something goes beyond the trivial samples, it easily broke and didnt work for me. Developing with it was like walking on eggs (for me). Whenever I googled for documentation of a feature I needed, I was not redirected to samples, tutorials, blogs, but to the Grails JIRA explaining me why the feature wouldnt work for my use case and that the bug was unresolved since two versions before the one I was using.
While that may not be the overall experience for every developer (I am not writing this to bash Grails, but to give my experiences with it here), I needed something that helped me and would not stand in my way or break down on me when I needed it the most. Thats when I found Play and I have quickly migrated my app to it after I found out about it (around the ~1.0 release).
So far it has been a great ride and for the first time in my web development career, I have stopped looking at other frameworks trying to find something that I would like better.
If I had to close with one thing that Play did better than Grails - at least for me - it would be the fact the Play is built from the ground up with developer usability in mind. It does not sacrifice ease of use for enterprise buzzwords. It has the guts to throw away what does not fit into this paradigm (e.g. ditchting Servlet-based runtimes during development for faster turnaround). It is willing to make compromises in order to guarantee awesomeness. And that is something I have only seen in communities like Rails or Django before I found Play.
I'd suggest Grails. It has a bigger community than the play framework does (~350 plugins covering pretty much every basic need). Also, grails is written almost completely in Java, it just lets you use Groovy for your domain specific implementation.
If you do run into a performance issue where the groovy pages that you've created are the bottleneck, you can always just switch to a Java implementation. Then you're in the same boat that you would have been with the Play framework all the time. You've optimized your development time by putting off the coding of things in Java till you know that you actually need to do it (which, in my experience is very rare).
I'm also not sure where you heard that you need to restart your server for each modification, but that's actually not true. Grails supports reloading of controllers/gsps/services/domain objects, etc without restarting your server.
The mixed stacktraces can get a little long, but tool vendors (like Intellij) have made some recent improvements that strip out all the stacktrace portions that you don't care about.
I've been using grails since the .5 days and have been very happy with the platform.
Take note that the Play! framework now supports using Scala as of 1.1
From my experience with Play it's a great framework. My favorite features are the cool controller system and the template system - both are simple but feature-rich and powerful.
However the most important benefit of Play is definitely the rapid development cycle, where virtually no reloading is needed on code changes. But if you're not careful, this greatness won't last much, and slowness will eventually creep into your code.
Why is that?
With Play there is common use of some plugins with pretty heavy initialization, notably EJB (Hibernate) and Spring. The initialization of these plugins is re-run on every code change before the new code is loaded. As a result of this, as your model and your system configuration grow, this heavy initialization starts to seriously slow down your development. In the system I used 20 seconds were a typical startup time on a virtual machine running on a kickass laptop.
What you can do to avoid this depends on your application, e.g. if you're building a NoSQL application then then EJB plugin should not give you trouble. Spring can be replaced with a custom hard-coded Java plugin, which IMHO is also easier to maintain, or run a Groovy script if scriptability is that important. In any case, watch out for these problems and kill them while the're young - and be sure not to be running your own bulky initializations on every refresh.
If you have used Ruby and Python before, you will probably enjoy Grails better than Play. It very hard to get back to Java once you are used to these dynamic languages.
There is also Lift on Scala.
Imho scala is the best static typed language and lift is a pretty nice framework (for a static typed language).
I'm currently using Selenium for testing our web application's interface. However, it isn't very reliable (it's hard to set a good waiting timeout, absolutely a nightmare to work with in any webpages involving frames) and lacks many features (popups/downloads).
I took a look at http://sikuli.csail.mit.edu/ and quite like it, but again, it isn't very reliable as in very often it fails to identify the correct buttons/links.
What is a good approach for testing web applications?
PS: I'm after the ones that would allow for testing even if the tester has ZERO knowledge about the internal of the codes (not even the method signatures).
Unfortunately, UI testing is not very reliable in general. Having to use XPath, because IDs aren't set on elements on the page. Having to use frames (I hate frames and glad its being deprecated in HTML5).
It's the nature of the beast unfortunately and a major issue I find with Selenium, which is my prefered UI testing tool for browsers, is that people start on the Selenium IDE which tries to record everything you do, but it can only do so much making tests unreliable.
Record/Replay tools should be called Record/Tweak/Replay, so that it can handle tests correctly.
Writing your tests for Selenium RC or WebDriver (these are currently being merged to create Selenium 2) allows you to handle the issues that you may see in Selenium IDE. It makes you write tests using programming languages like Java, Python, C# and many more. Since you can add conditionals to your tests.
The way that I have got less technical people to use Selenium is to create a DSL for them to use, so that it is a more natural language when writing tests.
Have a look at Sahi.
It does not need to wait for Ajax or page loads.
It does not use XPaths, using DOM relations (_near, _in) instead.
It traverses frames/iframes automatically
The recorder works for frames, iframes, and popup windows
It handles auto generated IDs well
The recorder and playback works on almost all browsers and operating systems.
It does not use special privileges to run tests.
Over the last year, Sahi has had seven public releases and the support is prompt on the forums.
Disclaimer: I am the author of Sahi.
Instead of using selenium.waitForPageToLoad("30000"); use like this
selenium.waitForPageToLoad(Timeout);
and don't forget to set the timeout using settimeout option
selenium.setTimeout(Timeout);
I too faced the same issues with the timeout. After setting the timeout, things worked well.