I am writing integration tests for a Java EE Servlet using Arquillian + JUnit. I need to be able to execute code before the server launches.
So is it possible to execute code before #Deployment? I tried #BeforeClass with no luck.
The reason I need to do this, is because trust and keystores for ssl needs to exists before the server starts. I am creating the stores problematically and is saving them to files afterwards.
I know a possible workaround would be to have static trust and keystores, but I prefer to create them programmatically before the test starts for full flexibility when writing tests.
There is not really a need to have your own specialization of Arquillian JUnit runner. This solution would be only for JUnit 4.x in that case which you are using for writing your tests.
Arquillian let you hook through extensions mechanism to its runtime and this way you can have some custom logic executed before server startup to provide your keystores. I believe this is more elegant and portable solution.
Please have a look at sample extensions on Github (especially lifecycle would be a good starting point). If you feel like implementing it this way I'm more than happy to help you. The event you might want to observe on is either BeforeSetup or BeforeStart.
You have two other options for executing code before and after your test:
Rules or ClassRules are executed around and before/after
Using a custom Testrunner (extending the default 'Arquillian' runner)
But as the static deployment method is not invoked by a rule, I assume you have to go for the testrunner.
Related
Some time ago, I implemented a remote test execution feature on JUnit 4 for the z2-environment (a Java server development and execution environment for large Java applications). Possibly similar to the Teleporter Rule of Apache Sling (for which I failed to find a JUnit 5 version).
This worked essentially like this:
A custom Runner (Z2UnitTestRunner) is declared on the test class using #RunWith
Z2UnitTestRunner passes a test invocation (actually a test description) to the remote side
On the remote side the test description is executed by a TestExecutor
A registered RunListener logs all test events back to the client side
On the client side any registered RunNotifier will be passed the test events received from the remote side
So it is rather simple actually: It just establishes a man-in-the-middle between Runner and TestExecuter. The cool thing is: While all test execution is performed in the "native" server environment of the application, tests can be triggered from the IDE or ANT/Jenkins, as if running locally. We use that quite a lot.
I am now trying to implement support for JUnit 5. I had a deeper look lately at the various extension and configuration tweaks supported by JUnit 5 but haven't really found a complete solution yet.
The most robust solution, I think, would be to integrate with the DefaultLauncher (as that is used by IDEs and ANT as far as I can tell) or via a custom launcher. The altered behavior would make sure that selectors and filters are sent to the remote side while all TestExecutionListener events would be conveyed to be client side.
Neither approach seems to be supported currently. At least, as far as I can tell, there is no way to provide a custom Launcher nor a way to change the behavior of the DefaultLauncher. But there is a LauncherFactory and a DefaultLauncher - which looks like there IS the intention to support custom launchers (are they?)!
I also looked into implementing a custom engine that would somehow take over test execution delegated to the remote side. But that seems to be the wrong level of interception. Plus I haven't found a way to "suppress" execution via the Jupiter Engine anyway.
Currently I am looking for any good idea or example that would help me move forward. Any suggestion welcome!
I am trying to create the following setup :
A Selenium (Java) project that has a set of 10 automated test cases.
When this project is executed, it generates an HTML test execution report.
This project should be 'hosted' on an internal network.
Anyone who has access to the network should be able to 'invoke' this project, which in turn executes the test cases and passes the HTML report to the person who invoked it.
The project should be accessible ONLY for execution and the code should NOT be accessible.
My goal is that this implementation should be executable by any framework irrespective of the technology that the framework uses. I was thinking of creating the project as a WebService using Java (servlet).
My question is:
Can this implementation be accessed by any external automation framework ?
Are there any limitations to this implementation?
Is there a better way to implement this requirement?
Thanks in advance.
You can create a maven project and have your automated tests under maven test folder.Configure your tests to run through POM.xml(use maven surefire plugin).Configure a jenkins job to run the maven test.Anybody with access the jenkins can build/run this task!
Below link should give you a headstart
http://learn-automation.com/selenium-integration-with-jenkins/
As a matter of fact, it is something we did on one of our projects. As I cannot share specifics, I will give you overall architectural view of the project.
The core of all things was a service that could run JUnit tests on requests. It was a Soap web-service, but nothing stops you from making it REST. To implement this you need to implement your version of JUnit test runners (see for example: http://www.mscharhag.com/java/understanding-junits-runner-architecture or https://github.com/junit-team/junit4/wiki/Test-runners)
If you use JUnit as test framework for running your Selenuim tests this may be a great solution for you - JUnit will generate HTML reports for you if you configure it properly, it will hide actual test suite implementation from users and run test suite on demand. This solution is also great because it operates on JUnit level and does not care about what kind of tests it actually runs, so it can be also reused for any other kind of automated tests.
So to answer all your questions:
Can this implementation be accessed by any external automation
framework ? -> yes, it can be accessed by anybody who able send http
requests
Are there any limitations to this implementation? -> none that I am
aware of
Is there a better way to implement this requirement? -> well, I
didn't actually work with TestNG much so I don't know if it is
easier or more difficult to do it on Junit level. You can use
Jenkins or other CI tool as well to achieve same results - they can
run JUnit tests for you and almost always have API ready for this,
although those APIs may be not perfect.
So I'd say that if you need this only for one thing you can use CI tools for this purpose, if you don't have CI tools available then choice has been made for you. However, from our experience, having this kind of service was a great asset for a company and I really wonder why there's no such products available elsewhere yet.
What I am doing: Using Jenkins to run the same test suites and test cases against various environments - dev / staging / production. I'm using WebDriver with a Java implementation and TestNG.
What I'd like to do: Selectively disable some tests, but not entire test suites, from running depending on the environment. Rather than maintain separate codebases between environments, I'd like to know of a way to accomplish this.
Initial thoughts: I was thinking setting a system property in Jenkins for each job in each environment and each test decorator would have to pull this piece of information out to determine if it should be ran or not. I think it's clunky, I'm not sure how to do it, and I'm not sure if this is the right approach.
Can someone tell me the best way to accomplish this? I'm hoping this isn't the best way.
Thanks,
Joe
Have you looked at TestNG listeners?
You can write a listener which just before the berfore test suite is run BUT after the tests to run have been identifier, to iterate around the list of tests and remove tests you do not want to run.
Because this is programatic you can write any java to achieve what you want.
Also, you could create annotations to identify which tests run in which environment; e.g annotate those tests with something like #RunInEnvironment({"UAT", INT"}); Your listener could then use those annotations to remove tests from the list which are not required.
I think that groups is your answer. With TestNG you can include/exclude groups. You will just need to define which tests are in which groups.
http://testng.org/doc/documentation-main.html#exclusions
#MrTi this solution is rather static and I believe he wants more dynamicity.
If detecting your environment can be done at startup, you may be able to try one of the solution described in that thread. This framework might be useful as well: https://github.com/wolfs/testng-rules
Note: on JUnit you would use Rules https://github.com/lacostej/web-validators/commit/2e1af8e1d9d1bf206849702d4231961563457815 (implementation uses old API)
JUnit,TestNG, and Spring testing have test execution listeners as an extension mechanism. Test execution listeners seem to be a low level feature of interest to framework developers and tool developers.
What are the main use cases in which Test execution listeners are useful? Can they be useful to application developers?
In Case of testng the listners are the most remarkable thing for me. The testNg listners allows me to simplyfy the test method content and manage each test with server startup , user registration, populating artifacts need for tests . And also clear up the environment for each test cycle.
At the each listener level im performing following operations so i don't have to bother about them in side my tests
IExecutionListener
- onExecutionStart()
Emma instrumentations
Server start
- onExecutionFinish()
Server Sutdown
Emma report generation
ISuiteListener
Set environment properties ex- Key Store Paths
Populate Users.
ITestListener
On Start
Artifact Deployment
On Finish
Artifact Clean up
IReporter
Generate TestNg Report,
Generate sure-fire report,
Export data for Dashboard
For JUnit, the listeners are designed to be a method of reporting, and are used like this internally. So, you can display counts of tests executed, success, failure, error, that sort of thing. This is used externally as well, such as in maven-surefire. See JUnit4RunListener.java as an example.
Another use would be to output in a different format, such as XML. I think the main use cases for the other frameworks are the same.
In JUnit, the listener class is not meant to be used in the manner that Dharshana uses his test listeners in testng, that is as a setup/teardown. The objects used in the Listeners (Description, Failure, Result) are immutable and don't encourage direct access to the test class itself. I'm not sure about TestNG, Cedric would be a better person to ask about that.
Are they useful for an application developers? They may well be, depending upon how your tests are set up. They would only be used in the context of tests, so if they can improve them, then go ahead, use them. One use case would be JUnit test report enrichment with JavaDoc, see my answer. To recap the answer, if the developers add a test for a specific bug, then they can add an annotation to that test linking it back to the bug. There is a custom RunListener, which collects all of the information in the annotations together and produces a report for the final customer.
Is it possible to run JUnit Tests from a java web application (on a Tomcat server) ?
How can I with a click on a link, launch the instantiation of an object ? or the call to a method ?
Thank you very much.
Best Regards.
Sure, although I'm not sure why you'd want to.
See the cookbook example, nutshell:
org.junit.runner.JUnitCore.runClasses(TestClass1.class, ...);
Obviously you'd need to deploy your test classes so they're available on the classpath, and convert from a string class name to an actual Class.
You'd likely also need to scan for annotations or convention-based class names; there are a variety of ways to do that including things like the reflections library.
IMO it'd be easier to set up a continuous integration (CI) server like Jenkins or CruiseControl etc. and get a complete package, but it depends on what your needs actually are.
There a number of JUnit-based frameworks that can assist with web testing.
Jwebunit
Canoo Webtest
HttpUnit
HtmlUnit