I'm not trying duplicate questions such as this one:
Unit testing framework for a Swing UI
What I'd like to know is, does anyone have any good comparisons for the various Swing Unit testing libraries such as:
WindowTester Pro
FEST
etc...
We've never done any GUI testing so we're not familiar with the gotchas that may lie ahead.
Thanks in advance.
I have had some rather good experience with Abbot and FEST, both open source libraries for Swing UI testing.
Abbot seems not supported anymore; it was a bit hard to get into, because the recorder wasn't generating scripts "good enough". Actually, I've used the recorder to "learn" the script language (XML tags), and I finally wrote the scripts myself directly with a simple text editor. This worked quite well.
FEST takes another approach where you have to code (in Java) your UI tests. That makes it a tool reserved for Java developers, whereas Abbot could be used by other people (e.g. QA Team testers).
The main issues with both tools, and probably with any UI testing tool, are:
to find a way to identify components uniquely without using their position or text content (which can change from one revision to another or makes it difficult to test the same application in a different Locale)
to use correct timing in scripts: those testing tools can run your UI much faster than a human user, thus your UI may not be fast enough for them (e.g. it may take several dozens ms to open a dialog, even more to populate a table from a database)
For both issues, there is a solution though.
For components identification, I strongly advise to name all Swing Components (using Component.setName()) in the UI and use a naming strategy for that, which can ensure that there are never 2 components with the same name visible at the same time. In the guts-gui library, I have even developed a strategy that automatically names Swing components that are stored as fields in panels, this helps adding component names after the application has been coded.
For script timing, both frameworks accept a timeout value while waiting for a dialog to appear; it's up to you to choose the best value, considering the facts that your tests may run on different kinds of machine with more or less available power. You should use a timeout that is large enough to ensure that the script won't report false negatives (e.g. a dialog that appears after 1sec, whereas the script waits for only 500ms), but also not too long so that when there is a real error (e.g. an expected dialog never appears). I suggest to use timeouts ranging from 2 to 5 seconds, that should fit most testing platforms and most applications.
Hope this helps.
Jemmy provides a reasonably good capabilities for UI testing. Although, it wouldn't be a staight out of the box solution for JUnit testing, it could be easily extended to suit your purposes.
I am not sure about other UI testing tools, but when compared with RFT it provides you a handle of the actual UI object (RFT returns a proxy object). This could be handy from my experience.
It is an open-source project (licensed under CDDL) and is actively under development.
I think other popular (or used to be??) was jfcUnit. Though I don't think this is under active development.
There are many factors to consider. Record /Replay, Unit test support, Nature of code change, Licensing, Cost, multi-platform support, Testing with multiple look and feel, support for i18n testing ... what is your list look like?
Some comments on the tools we used.
IBM Rational Functional Tester :
This has ability to record scripts, and play back. It supports verification points. One of the biggest plus point is no code change is required. RFT modifies the JVM and uses java accessibility extensions to record and test. We use it mainly for Java (swing/awt with lot of 2D and dialog operations). It works with Browsers as well.
RFT expose two mechanisms to identify GUI elements. One uses object map. This very weak and has trouble in long term maintainability. Using "find" API is more programmer friendly, though it requires code change. Having all objects with proper name helps too.
Not at all suitable for unit testing.
Works with Windows and Linux.
Very costly floating license is in the range of 12000USD, fixed licenses will cost half of that. All nodes (recording the tests and running the tests) need license. Pricing is approximate and old, but it will give an idea.
Needs real GUI session, on windows. (It may be OK on linux with VNC)
Jemmy:
We shifted to jemmy for new testing. Supports windows, linux. It used to be free, not sure what Oracle's plan on it. We added our test layer on top of jemmy - for assertions and other verification mechanisms. This presentation on 'jemmy testing toolkit' has some more details on jemmy.
In 2012 #AlexRuiz, the main developer behind FEST decided to stop development of FEST
The FEST codebase was later forked into AssertJ-Swing.
Skip FEST and start with AssertJ-Swing.
You may want to try Swing Testing Toolkit. It use a different approach: It does not record all events. Test instructions are reduced to the bare minimum.
I'm currently using Selenium for testing our web application's interface. However, it isn't very reliable (it's hard to set a good waiting timeout, absolutely a nightmare to work with in any webpages involving frames) and lacks many features (popups/downloads).
I took a look at http://sikuli.csail.mit.edu/ and quite like it, but again, it isn't very reliable as in very often it fails to identify the correct buttons/links.
What is a good approach for testing web applications?
PS: I'm after the ones that would allow for testing even if the tester has ZERO knowledge about the internal of the codes (not even the method signatures).
Unfortunately, UI testing is not very reliable in general. Having to use XPath, because IDs aren't set on elements on the page. Having to use frames (I hate frames and glad its being deprecated in HTML5).
It's the nature of the beast unfortunately and a major issue I find with Selenium, which is my prefered UI testing tool for browsers, is that people start on the Selenium IDE which tries to record everything you do, but it can only do so much making tests unreliable.
Record/Replay tools should be called Record/Tweak/Replay, so that it can handle tests correctly.
Writing your tests for Selenium RC or WebDriver (these are currently being merged to create Selenium 2) allows you to handle the issues that you may see in Selenium IDE. It makes you write tests using programming languages like Java, Python, C# and many more. Since you can add conditionals to your tests.
The way that I have got less technical people to use Selenium is to create a DSL for them to use, so that it is a more natural language when writing tests.
Have a look at Sahi.
It does not need to wait for Ajax or page loads.
It does not use XPaths, using DOM relations (_near, _in) instead.
It traverses frames/iframes automatically
The recorder works for frames, iframes, and popup windows
It handles auto generated IDs well
The recorder and playback works on almost all browsers and operating systems.
It does not use special privileges to run tests.
Over the last year, Sahi has had seven public releases and the support is prompt on the forums.
Disclaimer: I am the author of Sahi.
Instead of using selenium.waitForPageToLoad("30000"); use like this
selenium.waitForPageToLoad(Timeout);
and don't forget to set the timeout using settimeout option
selenium.setTimeout(Timeout);
I too faced the same issues with the timeout. After setting the timeout, things worked well.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
I'm at the beginning/middle of a project that we chose to implement using GWT. Has anyone encountered any major pitfalls in using GWT (and GWT-EXT) that were unable to be overcome? How about from a performance perspective?
A couple things that we've seen/heard already include:
Google not being able to index content
CSS and styling in general seems to be a bit flaky
Looking for any additional feedback on these items as well. Thanks!
I'll start by saying that I'm a massive GWT fan, but yes there are many pitfalls, but most if not all we were able to overcome:
Problem: Long compile times, as your project grows so does the amount of time it takes to compile it. I've heard of reports of 20 minute compiles, but mine are on average about 1 minute.
Solution: Split your code into separate modules, and tell ant to only build it when it's changed. Also while developing, you can massively speed up compile times by only building for one browser. You can do this by putting this into your .gwt.xml file:
<set-property name="user.agent" value="gecko1_8" />
Where gecko1_8 is Firefox 2+, ie6 is IE, etc.
Problem: Hosted mode is very slow (on OS X at least) and does not come close to matching the 'live' changes you get when you edit things like JSPs or Rails pages and hit refresh in your browser.
Solution: You can give the hosted mode more memory (I generally got for 512M) but it's still slow, I've found once you get good enough with GWT you stop using this. You make a large chunk of changes, then compile for just one browser (generally 20s worth of compile) and then just hit refresh in your browser.
Update: With GWT 2.0+ this is no longer an issue, because you use the new 'Development Mode'. It basically means you can run code directly in your browser of choice, so no loss of speed, plus you can firebug/inspect it, etc.
http://code.google.com/p/google-web-toolkit/wiki/UsingOOPHM
Problem: GWT code is java, and has a different mentality to laying out a HTML page, which makes taking a HTML design and turning it into GWT harder
Solution: Again you get used to this, but unfortunately converting a HTML design to a GWT design is always going to be slower than doing something like converting a HTML design to a JSP page.
Problem: GWT takes a bit of getting your head around, and is not yet mainstream. Meaning that most developers that join your team or maintain your code will have to learn it from scratch
Solution: It remains to be seen if GWT will take off, but if you're a company in control of who you hire, then you can always choose people that either know GWT or want to learn it.
Problem: GWT is a sledgehammer compared to something like jquery or just plain javascript. It takes a lot more setup to get it happening than just including a JS file.
Solution: Use libraries like jquery for smaller, simple tasks that are suited to those. Use GWT when you want to build something truly complex in AJAX, or where you need to pass your data back and forth via the RPC mechanism.
Problem: Sometimes in order to populate your GWT page, you need to make a server call when the page first loads. It can be annoying for the user to sit there and watch a loading symbol while you fetch the data you need.
Solution: In the case of a JSP page, your page was already rendered by the server before becoming HTML, so you can actually make all your GWT calls then, and pre-load them onto the page, for an instant load. See here for details:
Speed up Page Loading by pre-serializing your GWT calls
I've never had any problems CSS styling my widgets, out of the box, custom or otherwise, so I don't know what you mean by that being a pitfall?
As for performance, I've always found that once compiled GWT code is fast, and AJAX calls are nearly always smaller than doing a whole page refresh, but that's not really unique to GWT, though the native RPC packets that you get if you use a JAVA back end are pretty compact.
We have been working with gwt for almost 2 years. We have learned a lot of lessons. Here is what we think:
Dont use third party widget libraries especially gwt-ext. It will kill your debugging, development and runtime performance. If you have questions about how this happens, contact me directly.
Use gwt to only fill in the dynamic parts of your apps. So if you have some complex user interactions with lots of fields. However, don't use the panels that come with it. Take your existing stock designer supplied pages. Carve out the areas that will contain the controls for your app. Attach these controls to the page within onModuleLoad(). This way you can use the standard pages from your designer and also do all the styling outside the gwt.
Don't build the entire app as one standard page that then dynamically builds all the pieces. If you do what I suggest in item 2, this won't happen anyway. If you build everything dynamically you will kill performance and consume huge amounts of memory for medium to large apps. Also, if you do what I am suggesting, the back button will work great, so will search engine indexing etc.
The other commenters also had some good suggestions. The rule of thumb i use is to create pages like you were doing a standard web page. Then carve out the pieces that need to be dynamic. Replace them with elements that have id's and then use RootPanel.get( id ).add( widget ) to fill those areas in.
Pitfalls that we've run into:
While you can get a lot of mileage from using something like GWT EXT, any time you use this sort of thin veneer on top of a JavaScript library, you lose the ability to debug. More than once I've bashed my head on the desk because I cannot inspect (inside my IntelliJ debugger) what's happening in the GWT EXT table class... All you can see is that it's a JavaScriptObject. This makes it quite difficult to figure out what's gone wrong...
Not having someone on your team who knows CSS. From my experience, it didn't matter that the person wasn't expert...it's enough that he has some good working knowledge, and knows the right terms to google when necessary.
Debugging across browsers. Keep an eye on Out of Process Hosted Mode[1][2][3], hopefully coming in GWT 1.6... For now, you just have to get things good with hosted mode, then use the "Compile/Browse" button, where you can play with other browsers. For me, working on Windows, this means I can view my work in FireFox, and use FireBug to help tweak and make things better.
IE6. It's amazing how different IE 6 will render things. I've taken the approach of applying a style to the outermost "viewport" according to the browser so that I can have CSS rules like:
.my-style { /* stuff that works most everywhere */ }
.msie6 .my-style { /* "override" so that styles work on IE 6 */ }
Finally, make sure you use an editor that helps you. I use IntelliJ -- it's got lots of GWT smarts. E.g., If I try to use a class that isn't handled by the JRE emulation, it lets me know; if I specify a style for a widget, and I haven't defined that style yet, the code gets the little red squiggly... Or, when looking at the CSS, it will tell me when I've specified conflicting attributes in a single rule. (I haven't tried it yet, but I understand that version 8 has even better GWT support, like keeping the "local" and "async" RPC interfaces and implementations in sync.)
GWT 2.0, which is supposed to come out sometime in the next few months, solves a lot of the issues discussed.
Create layouts using an html/xml like syntax
Dynamic Script Loading - only the essential JS will be downloaded initially. The rest will be downloaded as needed
In-Browser Hosted Mode - This might take care of the hosted mode speed issues discussed, among other benefits
"Compiler Optimizations" - Faster compilation, hopefully
GWT 2.0 Preview Video at Google I/O
Not "unable to be overcome" but a bit of a pain for something basic.
Date handling:
GWT uses the deprecated java.util.Date which can lead to unexpected behaviour when dealing with dates on the client side. java.util.Calendar is not supported by GWT. More info here.
Related problem examples:
GWT java.util.Date serialization bug
Get Date details (day, month, year) in GWT
Client side time zone support in GWT
I'll add some points to the ones already mentioned:
Databinding/validation. GWT doesn't have a databinding/validation support out of the box, although there are some projects on this area starting to emerge. You'll find yourself writing alot of this:
TextField fname, faddress;
...
fname.setText(person.getName());
faddress.setText(person.getAddress());
...
Lazy loading. Since gwt is on the client side, lazy loading is really not an option. You'll have to design your RPCs and Domain Objects carefully in order to
send all your object data that is needed
avoid eager fetching all of your data
You'll have also to make sure that you will not send proxies/non serializable objects. hibernate4gwt can help you with these points.
UI design. It is harder to visualize an UI in java (Panels, Buttons, etc) than in html.
History support. GWT does not ship with a History subsystem, nor does it ship with any subsystem for nice urls or statefull bookmarking. You'll have to roll your own (although it has support for History tokens, which is a start). This happens with all AJAX toolkits AFAIK.
IMHO, GWT is missing a framework that has out of the box support for all of the issues mentioned on this 'thread'.
I'm working on a project right now that uses EXT GWT (GXT) not to be confused with GWT EXT. There is a difference, EXT GWT is the one that is actually produced by the company that wrote ExtJS the javascript library. GWT EXT is a GWT wrapper around the ExtJS library. GXT is native GWT.
Anyways, GXT is still somewhat immature and lacks a solid community that I feel GWT EXT has. However, the future is with GXT, as it's native GWT and actually developed by the company that made ExtJS. GWT EXT is somewhat crippled as the license changed on the ExtJS library, thus slowing the development of GWT EXT.
Overall, I think GWT/GXT is a good solution for developing a web application. I actually quite like hosted mode for development, it makes things quick and easy. You also get the benefit of being able to debug your code as well. Unit testings with JUnit is pretty solid as well. I haven't yet seen a great JavaScript unit testing framework that I felt was mature enough for testing an enterprise application.
For more information on GWT EXT:
http://gwt-ext.com/
For more information on EXT GWT (GXT):
http://extjs.com/products/gxt/
No major pitfalls that I haven't been able to overcome easily. Use hosted mode heavily.
As you are using GWT-ext you will almost never need to touch CSS yourself unless you want to tweak the out of the box look.
My recommendation is to use a GWT "native" widget over a library one where they are close in features.
Re search engine indexing: yes the site will not have navigable URLs normally (unless you are only adding widgets to elements of a regular web site). You can do history back/forward functionality though.
I used GWT and GWT-ext together on a project a while ago. I found the experience quite smooth as web development goes, but my advice would be this:
Don't mix GWT native widgets with EXT widgets. It's confusing as hell, since usually the names are the same (GWT.Button or GWText.Button?)
One thing that happened to me that really made the code more complex than I'd like, was that I wanted a Panel that was
a) dynamically updatable
b) cascadable
GWT native panels are dynamic, Ext panels are cascadable. Solution? A GWT.VerticalPanel wrapping a GWTExt Panel... Chaos. :)
But hey, it works. ;)
I second the comment from ykagano, the biggest disadvantage is losing the V in MVC. Although you can separate the true ui class from the rest of your client side code, you cannot easily use an HTML page generated by a graphic/web designer. This means you need a developer to translate HTML into java.
Get a wysiwyg ui editor, it will save you lots of time. I use GWTDesigner.
The biggest upside of GWT is being able to forget about cross browser issues. Its not 100% but takes almost all that pain away. Combined with the benefit of hosted mode debugging (as opposed to Firebug which is excellent but not the same as a java debugger) it gives the developer a huge advantage in generating complex ajax apps.
Oh and its fast at runtime, especially if you use a gzip filter.
Slightly off-topic, but the #gwt channel on irc is very helpful, in-case you have a persistent problem.
GWT is pretty straight-forward and intuitive.
Especially with the release of UIBinder to allow GWT widgets to be laid out in XML and then coded-behind in Java.
So if you have used other Ajax or Flash design tools, or Silverlight, etc, GWT is very easy to learn.
The major hurdle, if not pitfall, is GWT RPC. The very reason you wish to use GWT is because of GWT async RPC. Otherwise, why not just rely on css to format your page?
GWT RPC is that element that allows your server to refresh data on your server without having to refresh the page. This is an absolute requirement for pages such as stock performance monitoring (or the current national and public debt of the US or the number of unborn babies aborted worldwide by the second).
GWT RPC takes some effort to understand but given a few hours, it should come all clear.
Above that, after putting in some effort to learn GWT RPC, you finally discover that you cannot use JSPs as the service component for RPC, unless ... I have an 8 part (I think) series on my blog on how to use JSP as the GWT RPC servicer. However, since you had not asked for answers but just issues, I shall desist from advertising my blog.
So. I very much believe that the worst roadblocks/pitfalls to using GWT is finding out how to properly deploy GWT async RPC and how to enable it to use JSP servicers.
We've had a very hard time marrying our GWT codebase with HTML web templates that we got from a web designer (static HTML pages with specific div ids that we wanted GWT to manage). At least back when we used it, we couldn't get GWT to integrate with parts of our website that were not coded in GWT. We had it working eventually, but it was a big hack.
The Async interface you have to write for each service interface looks like something that could have been automatically generated by the GWT compiler.
Compile times become long for large projects
But for a large Javascript project it's the best choice
GWT 2.4 has fixed many of the aforementioned issues and a great widget library is just coming out of Beta (Ext GWT 3.0.4 a.k.a. GXT), which is written completely in GWT, not a wrapper of a JS lib.
Remaining pain:
Lack of CSS3 selector support, you can use "literal()" in some cases to get around it.
Lack of support for CSS3 and modern browser events like transitionEnd.
Lack of Java Calendar class support (many years later).
Lack of JUnit4 support (5 years and counting).
Lack of clear road map and release schedule from Google GWT team.
Regarding GWT 2.4, Use Firefox when debugging GWT, it alot more faster then using chrome.
And if you'll using only firefox, consider putting this line in your project.gwt.xml file
<set-property name="user.agent" value="gecko1_8" />
Also, If you're using eclipse, then add the following under arguments -> VM arguments:
-Xmx512m -XX:MaxPermSize=1024m -XX:PermSize=1024m
You can divide your server and client, and use the following under arguments -> Program arguments:
-codeServerPort 9997 -startupUrl http://yourserver/project -noserver
Also, to prevent refreshing your server on each change, use JRebel
http://zeroturnaround.com/blog/how-to-rock-out-with-jrebel-and-google-web-toolkit-gwt/
And here's a live demo
http://www.youtube.com/watch?feature=player_embedded&v=4JGGFCzspaY
One major pitfall is that sometimes you need to explicitly assign an id to what ultimately becomes an HTML element to be able to use certain CSS styles. For instance: a GWT TabPanel will only do :hover over tabBarItems when the tabBar of the tabPanel has been assigned an id and you specify a :hover on that elementId.
I wrote about some other disadvantages of GWT elsewhere, but they are already covered by rustyshelfs answer :).
I have done a lot of work on GWT recently, and this is wht i have to say:
CSS styling is tricky only sometimes, use IE developer tool in IE and firebug in Firefox to figure out what exactly is happening and you will get a clear idea of what css needs to be changed
You can use tricks to get google to index it. A very famous site is http://examples.roughian.com/ check its ratings at google. A far less famous site is www.salvin.in (couldnt resist to mention that), i optimised it to words: salvin home page (search google for these three words)
I do not know much about GWT-EXT, But i too am of the belief that there is no need to include Third party libraries.
Best of luck on your decision :)
GWT does Browser Sniffing instead of Feature Detection and your application will not work on some browsers (specially new ones)
Here are some references of the problem:
google-web-toolkit Issue 2938: RFE: improve the user.agent property-provider to cope for userAgent string "masking"
Iceweasel no longer supported? - Google Docs Help
GWT implementations for every browser
Here are some references to Feature Detection:
Browser Detecting (and what to do Instead)
Feature Detection: State of the Art Browser Scripting
Browser Feature Detection
Extracted from Comparison of JavaScript frameworks - Wikipedia
The GWT team make a lot of great improvements in to last year releasing GWT 2.7. One major weakness of GWT was that compilation takes to much time in GWT 2.6 and below. This is now gone GWT has not incremental compile which is super fast and compiles only the changes.
GWT 2.7 now has (Source):
Incremental builds now just seconds
More compact, more accurate SourceMaps
GSS support
JSInterop
Great JavaScript Performance
Smaller Code Size
The best way to get reliable facts are from the gwt survey. One of the biggest issues with GWT has always been a long compile time. Fortunately, it's improving very quickly so it won't be a significant issue in the near future. Another pitfall is that GWT is dramatically more complicated because Java is a more complicated language that resists bad coders every step of the way. In addition, compiling adds a layer. For example, js interop requires a little boilerplate. The fundamental issue is that GWT wasn't designed to be simple. It was designed from the ground up for extremely complicated web apps and the entire community consistently prioritizes, performance, code quality, architecture etcetera over easy coding.
Remember that you can use js in GWT at any point so if you are struggling with GWT consider using js. At the end of the day GWT is js so you can do anything in GWT that you can in js. In fact, most GWT projects use js. The problem is that GWT is drastically more complicated. Nevertheless, it's sometimes worth the extra complexity.
It's worth noting that GWT 3.0 will bring massive improvements.
Re-using RPC service objects.
It causes race conditions with symptoms that look like the app hanging.
Pitfalls I ran into
1. Different behaviour in superdev mode. E.g. Someclass.class.getName() works absolutely fine in Superdev mode and returns the fully qualified name of the class. In productive mode this does not work.
addWidget(widget) will call widget's removefromparent()
GWT is a technology masterpiece. It unites client and server programming making it one coherent application - the way software was written before "layering", and the way it should be written. It eliminates different skills sets, miscommunication between team members, and generally the whole Web Design phase: both the artistic and programming. And it is the closest you'd get to mobile e.g. Android development. In fact GWT was designed to generate different native UIs, not just HTML. Though it requires enormous discipline to ensure such decoupling - to keep your inner layers presentation-agnostic.
The first mistake you should avoid, which took me four years to realize, is using third-party extensions like EXT-GWT aka GXT and SmartGWT. It is very tempting to start using their pretty desktopish widgets instead of investing in your own styling, but I cannot tell how many problems I had with SmartGWT until I finally got fed up. In short it freezes the core GWT feature set at the certain (pretty outdated) level and then builds on top of it. Also keep in mind, that chiseled desktop look and feel looks silly nowadays, not to mention the sluggish performance, tons of bugs, and compatibility features - especially on mobile devices. You want to stay as close to the native browser controls, as possible i.e. dropdowns rendered as native <select> elements, not some custom-painted controls.
Thanks to mobile trends the whole UX is becoming simpler and flatter, so you don't need to do much to style a sharp-looking application. Though if you want "3D" look, there are also gradients. CSS3 made everything easy, and GWT wraps it an elegant object-oriented manner unlike the raw CSS. So don't be discouraged by looking at rather ugly barebones controls in the GWT Showcase. The GWT team intentionally didn't offer any styling, because it it the developer's job.
The rest is pretty much conventional browser programming in strongly typed Java with beautiful concise APIs. But of course never forgetting your code runs inside the browser, so all of the calls are asynchronous e.g. you cannot call GWT-RPC methods in a loop (to populate some list), but need to recursively chain them if you ever come to to this situation.
There are some self-proclaimed "anti-patterns" like don't use GWT-RPC. It's been good to me so far: for 10 years. Simplicity is key. I wouldn't think even a second to sacrifice some marginal performance for code elegance and maintainability. besides this is not where your bottlenecks would be - in the database. Of course mind how much data you are sending to the client.
And if you cannot find or style the existing gadget - read rich HTML5 element set, you can always wrap a third-party one. I did it with a popular jQuery FullCalendar. Not rocket science at all. Everything else like Google Maps and Google Charts has semi-official GWT wrappers.
GWT is perfect. The only reason it doesn't get enough love is because early Internet adopters who still influence the industry didn't come from Computer Science and object-oriented languages to appreciate them. They have either artistic (Photoshop/WordPress) or network (Perl/Python) background.