We Keep Coding

Functional testing a JavaEE server

I'm looking for a way to test the different functionality of my JavaEE server application. I first tried to do it with Junit and TestNG, but building the different scenario was too tedious.
What i'm searching is a tools that will simulate an user using my server application trough his browser, and then once this scenario is established i could check the output and verify if it's working.
Here an exemple:
An user connects to the server, the server application interface is displayed.
He executes the functionality to modify his personal datas
He modifies the different that he want to change.
He saves the modifications.
Then what i want to check is the actual state of the data in the database.
What i need is something that will simulate the action he did with his browser like i said above...
I've read different article about different tools that could do the works but i'm not sure because i don't really know what to type in google.
I discovered Jmeter (that is not working with my application because of web socket) and the Grinder.
The Grinder seems to be interesting but most of the things i've read about refers to it as load testing tools, which is not what i'm looking for.
Can someone experienced tells me if i can do what i want with the Grinder ?

You can use scripting tools like: Sikuli(Record Playback & Scripting) or Automa (Component Identification Scripting)

The most popular tool currently is Selenium. It will certainly do the job. I would also mention Geb because it provides more convenient API when compared to Web Driver.
I'll also let myself to give an advice, although it doesn't relate directly to your question.
If your project is big enough (4-5 teams over a couple of years can deliver a lot of code), you should think what to automate.
These tests can be very heavy both in terms of CPU load and in terms of time.
So if you'll rely ultimately on these tests, your build will run ages and will be potentially unstable.
So these tools should be used only to complement unit/component and integration testing that should exist anyway and they will use completely different tools.
Also in UI, consider using various testing techniques that would test only the UI side (mock the server endpoints and so on).

Rapid Test Driven Development in Java

I have been using the guard framework in rails for quiet a while and I have just fallen in love with it. I also work in Java a lot and I find it surprising that such a tool is not available for rapid test driven development. Here's what guard does for you (form the original website)-
File system changes handled by our awesome Listen gem.
Support for visual system notifications.
Huge (more than 120) guard extensions eco-system.
Tested against Ruby 1.8.7, 1.9.2, 1.9.3, REE and the latest versions of JRuby & Rubinius.
In essence, it helps me keep an eye on test cases while I am making changes or adding stuff to my codebase. The following are the benefits of such an approach-
Unobtrusive test case driven development
Instant acknowledgement of the impact of a code change
High quality code
Minimized regression ripples
Does anyone know how to achieve similar goals in Java?
Note- Automated test tools like Hudson is not an answer I am looking for. I need something that can be used on local development machines/environment so there is an instant test case feedback on a code change.
Thanks

Not sure what you are looking for ... but if you search for a tool that runs your unit tests in the background during development have a look at Infinitest or JUnit Max.

If you work with eclipse, and you dock your JUnit view in a convenient place, it's one button to click to rerun your tests, and see the report immediately in the same view.
It's a very efficient way to perform test driven developpement.

You could use Sonar with Hudson to do this for you. In the company I work in, we use this combination to solve your issue. There's also a Sonar plugin for Eclipse. But, if you look for something like an IDE-based solution I can't help you.

Swing UI Testing Library Comparisons: FEST, WindowTester Pro, etc

I'm not trying duplicate questions such as this one:
Unit testing framework for a Swing UI
What I'd like to know is, does anyone have any good comparisons for the various Swing Unit testing libraries such as:
WindowTester Pro
FEST
etc...
We've never done any GUI testing so we're not familiar with the gotchas that may lie ahead.
Thanks in advance.

I have had some rather good experience with Abbot and FEST, both open source libraries for Swing UI testing.
Abbot seems not supported anymore; it was a bit hard to get into, because the recorder wasn't generating scripts "good enough". Actually, I've used the recorder to "learn" the script language (XML tags), and I finally wrote the scripts myself directly with a simple text editor. This worked quite well.
FEST takes another approach where you have to code (in Java) your UI tests. That makes it a tool reserved for Java developers, whereas Abbot could be used by other people (e.g. QA Team testers).
The main issues with both tools, and probably with any UI testing tool, are:
to find a way to identify components uniquely without using their position or text content (which can change from one revision to another or makes it difficult to test the same application in a different Locale)
to use correct timing in scripts: those testing tools can run your UI much faster than a human user, thus your UI may not be fast enough for them (e.g. it may take several dozens ms to open a dialog, even more to populate a table from a database)
For both issues, there is a solution though.
For components identification, I strongly advise to name all Swing Components (using Component.setName()) in the UI and use a naming strategy for that, which can ensure that there are never 2 components with the same name visible at the same time. In the guts-gui library, I have even developed a strategy that automatically names Swing components that are stored as fields in panels, this helps adding component names after the application has been coded.
For script timing, both frameworks accept a timeout value while waiting for a dialog to appear; it's up to you to choose the best value, considering the facts that your tests may run on different kinds of machine with more or less available power. You should use a timeout that is large enough to ensure that the script won't report false negatives (e.g. a dialog that appears after 1sec, whereas the script waits for only 500ms), but also not too long so that when there is a real error (e.g. an expected dialog never appears). I suggest to use timeouts ranging from 2 to 5 seconds, that should fit most testing platforms and most applications.
Hope this helps.

Jemmy provides a reasonably good capabilities for UI testing. Although, it wouldn't be a staight out of the box solution for JUnit testing, it could be easily extended to suit your purposes.
I am not sure about other UI testing tools, but when compared with RFT it provides you a handle of the actual UI object (RFT returns a proxy object). This could be handy from my experience.
It is an open-source project (licensed under CDDL) and is actively under development.
I think other popular (or used to be??) was jfcUnit. Though I don't think this is under active development.

There are many factors to consider. Record /Replay, Unit test support, Nature of code change, Licensing, Cost, multi-platform support, Testing with multiple look and feel, support for i18n testing ... what is your list look like?
Some comments on the tools we used.
IBM Rational Functional Tester :
This has ability to record scripts, and play back. It supports verification points. One of the biggest plus point is no code change is required. RFT modifies the JVM and uses java accessibility extensions to record and test. We use it mainly for Java (swing/awt with lot of 2D and dialog operations). It works with Browsers as well.
RFT expose two mechanisms to identify GUI elements. One uses object map. This very weak and has trouble in long term maintainability. Using "find" API is more programmer friendly, though it requires code change. Having all objects with proper name helps too.
Not at all suitable for unit testing.
Works with Windows and Linux.
Very costly floating license is in the range of 12000USD, fixed licenses will cost half of that. All nodes (recording the tests and running the tests) need license. Pricing is approximate and old, but it will give an idea.
Needs real GUI session, on windows. (It may be OK on linux with VNC)
Jemmy:
We shifted to jemmy for new testing. Supports windows, linux. It used to be free, not sure what Oracle's plan on it. We added our test layer on top of jemmy - for assertions and other verification mechanisms. This presentation on 'jemmy testing toolkit' has some more details on jemmy.

In 2012 #AlexRuiz, the main developer behind FEST decided to stop development of FEST
The FEST codebase was later forked into AssertJ-Swing.
Skip FEST and start with AssertJ-Swing.

You may want to try Swing Testing Toolkit. It use a different approach: It does not record all events. Test instructions are reduced to the bare minimum.

Control TPTP profiling programmatically

I need to profile a server-client Java application (based on Jersey/REST FYI).
I have learned around that profiling remote servers is a real pain with TPTP, so I want to keep it simple :
I have written a test case (a "main") that runs a standalone server (Grizzly) and a client (within a separate thread) and process some load tests.
I can run it in profiling mode (with Eclipse/TPTP), but I would like to isolate my core methods from all the noise of the server and client setup.
I there a mean to programmatically enable / disable the TPTP profiling in my code, at some check points?

I would recommend using Netbeans for profiling your app, irrespective if you're using it for development. It can filter by package, class, or method so you only see what you want, as well as having nifty profiling points (things like stopwatches but way more powerful). You can do things like:
Sorry for all the screenshots, but IMHO they help people who have never profiled before and are wondering what it can do for them.

Reliable web application GUI testing

I'm currently using Selenium for testing our web application's interface. However, it isn't very reliable (it's hard to set a good waiting timeout, absolutely a nightmare to work with in any webpages involving frames) and lacks many features (popups/downloads).
I took a look at http://sikuli.csail.mit.edu/ and quite like it, but again, it isn't very reliable as in very often it fails to identify the correct buttons/links.
What is a good approach for testing web applications?
PS: I'm after the ones that would allow for testing even if the tester has ZERO knowledge about the internal of the codes (not even the method signatures).

Unfortunately, UI testing is not very reliable in general. Having to use XPath, because IDs aren't set on elements on the page. Having to use frames (I hate frames and glad its being deprecated in HTML5).
It's the nature of the beast unfortunately and a major issue I find with Selenium, which is my prefered UI testing tool for browsers, is that people start on the Selenium IDE which tries to record everything you do, but it can only do so much making tests unreliable.
Record/Replay tools should be called Record/Tweak/Replay, so that it can handle tests correctly.
Writing your tests for Selenium RC or WebDriver (these are currently being merged to create Selenium 2) allows you to handle the issues that you may see in Selenium IDE. It makes you write tests using programming languages like Java, Python, C# and many more. Since you can add conditionals to your tests.
The way that I have got less technical people to use Selenium is to create a DSL for them to use, so that it is a more natural language when writing tests.

Have a look at Sahi.
It does not need to wait for Ajax or page loads.
It does not use XPaths, using DOM relations (_near, _in) instead.
It traverses frames/iframes automatically
The recorder works for frames, iframes, and popup windows
It handles auto generated IDs well
The recorder and playback works on almost all browsers and operating systems.
It does not use special privileges to run tests.
Over the last year, Sahi has had seven public releases and the support is prompt on the forums.
Disclaimer: I am the author of Sahi.

Instead of using selenium.waitForPageToLoad("30000"); use like this
selenium.waitForPageToLoad(Timeout);
and don't forget to set the timeout using settimeout option
selenium.setTimeout(Timeout);
I too faced the same issues with the timeout. After setting the timeout, things worked well.