I've developed some testcases localy with this setup:
Eclipse,
Selenium webdriver 2.0,
Maven 2,
TestNG,
Git
When I run the test via Maven test it works just fine. But now my next task is to move My tests to Bamboo. I've searched at Atlassians HomePage but not found any good information on how to do it.
I figure I must install webdriver on the Bamboo server? And then?
Could someone please tell me how to set up Bamboo so IT would run My tests?
Best regards, P
I'm using Ruby Selenium, so my setup is going to be different. I'll explain it anyways, in case it helps you go find your solution.
My selenium tests are written using Ruby, wrapped in RSpec (test framework). The Rspec tests are called via Rake tasks (like ANT).
On the Bamboo side, each job looks like this:
Source code checkout
Bundle install (get GEMs & dependencies)
Rake task : Bamboo Rake task that runs the named Rake task (which results in RSpec-Selenium tests running)
JUnit Parser (rake is configured to output the results of RSpec in JUnit format using CI_Reporter) parses results of tests, so Bamboo knows what passed/failed. I have this in the job config -> tasks -> final tasks section in bamboo; anything in this section will run even if a previous task failed.
Related
I have written selenium java code to do the UI automation test in eclipse. Now, I want those code to be runed in VSTS. Problem is that there are lots of components which I am not clear how they are used under the tab "build & release". So I am looking for masters who can share with me the steps to achive my target. Thank you.
If the test need to be run in interactive way, you need to setup a build agent with Interactive mode. (e.g. Deploy an agent on Windows), then run test through this build agent.
There is the Maven task that can run java test.
A related article: Testing Java applications with VSTS
I'm finding it difficult to phrase this question well, as there are quite a few generic terms (run, configuration, launch, etc.). Here goes:
You can save run configurations in a .launch file. (in the Run Configuration Dialog, under the Common tab, Save as a shared file.
We check these in to SVN. The developers can pass them around, and it helps getting new devs running a working application quicker.
I'd like to check these out as part of our build and use them to programatically run the application, the tests, etc, without spinning up the whole IDE.
What would be the best way to run a .launch file outside of the UI?
Edit: I am trying to unify the tests run on the build server and the IDE. I do not
particularly want to give up integrated debugging, which would be the case with an ant script to run the tests .
This is probably more a problem for integration testing with multiple bundles, or unit testing a whole bundle, where you'd like to mock up extensions.
there is an eclipse plugin built over JUnit, called TPTP. It provides an automation client which can be used to launch the test from eclipse with no gui. maybe it helps
Ant4Eclipse may provide a good starting point on how to do this.
Unfortunately, this is limited to Java Applications and JUnit configurations; I am more interested in PDE applications and Plugin JUnit tests.
I have recently had alot of success building an Eclipse RCP app inside a Hudson CI server using Eclipse Buckminster. It took a bit of doing, but once I setup both features, made my RCP product be based on features, and added the Buckminster query files and the like, it worked. There is a Hudson/Jenkins Buckminster plugin that allowed me to have hudson build the application.
After saving the launch configurations for each test fragment, I created hudson commands to invoke them (yes one line per test fragment unfortunately), but after that I got the automated CI build that I wanted.
You could also use the shell command Eclipse uses. To get it:
Run your program in Eclipse
Go to the "Debug" view
Right-click on the process (probably the second item in the tree) and select "Properties"
Copy shell command and delete the agentlib flag to run in bash
I think you don't need to use the .launch configurations to run the tests. If you build an application using the Eclipse Build System, then you can use the AntRunner application from Eclipse to run your units tests. This doesn't start the whole IDE.
This article describes how to run the tests during your build process. With this process, you use a special "Test" Eclipse and load the plugins you want to test.
Perhaps running the configurations the way you would run your own custom run configurations would help here. It is described in this article.
friends. I try to understand how does selenium work. I written some tests with JUnit 4. These tests works fine when my app is running and I put as URL localhost:8077 where my app is started. All tests pass as I need. But now I want that my war build will depend by these tests. After these I want that the automated release (Jenkins) will also depend by selenium tests.
I don't understand some things:
Should the selenium test be included in my application or they should be assembled in a standalone application? (I read about Selenium Grid but I am not sure that I need it)?
I have one developer machine, one server with jenkins and the production server. What are the needed steps to implement Selenium test? What Url should be used in test? Is it needed to install Tomcat Server on the same server with Jenkins?
Can I invalidate the build and drop the release on Jenkins if selenium tests are not passed?
Should the selenium test be included in my application or they should be assembled in a standalone application? (I read about Selenium Grid but I am not sure that I need it)?
It depends on how you want to run your tests. Best practice is to call the test configuration inside your maven project and tests will run during your build job. You can also call your maven target after the build is completed.
Selenium Grid is useful but only if you have dedicated machines having proper setup to run those tests, else your tests may often fail. You should consider running your tests remotely on cloud using services like Saucelabs, etc.
I have one developer machine, one server with jenkins and the production server. What are the needed steps to implement Selenium test? What Url should be used in test? Is it needed to install Tomcat Server on the same server with Jenkins?
Selenium tests can be implemented with Maven on Jenkins following the step from here, here and here.
Dev or Test URLs should be used for running the test.
I think you would need Tomcat Server if you are hosting your application on the Jenkins machine or else it should not be needed.
Can I invalidate the build and drop the release on Jenkins if selenium tests are not passed?
When you include the test configuration in your project configurations, the build may fail during the build job if your test fails. You can set how you want to proceed with your build when you test fails like failing the build or pass the build with warning. You may want to get notifications for failed build also.
Ex. Jenkins marks good build as failure because of test failed
Jenkins plugins can be used based on the needs.
You may like to read more about How to do Integration tests here.
I'll do my best to answer your questions, and hopefully clear a few things up for you.
From my experience each test should be created as a new item and then configured to run after your app is deployed or periodically. No you dont need Selenium Grid. It is used for parallel execution of tests.
You'll need to configure your machines as nodes on jenkins and install the jenkins agent on the slave machines. Setting up selenium tests differs depending on how you have written the tests, for example i use maven as my build tool. So for each new test i only need to point towards svn and give the maven launch commands. The url is whatever url you use to access your app. There is a very good article on how to setup jenkins on their site, you can find it here
Yes.
I have installed the Jenkins on my windows machine.
I am currently doing the functional testing of a website and i have created all the test cases using selenium . I ran all these testcases using eclipse with testng plugin.
Now is there any way where i can run these testcases in eclipse from Jenkins itself i.e Can i control the eclipse from Jenkins.
I am trying hard now and able to run it using batch command( i have written all the commands in a batch file to run the eclipse and selenium driver and that batch file is executed from Jenkins ) but i am in search of a plugin where i can run directly the eclipse , selenium web-driver using that plugin.
Please see to this issue.
Thanks.
You are on the right lines with a batch file, but I recommend you use Ant or Maven instead of Eclipse - they are the standard way of scripting builds in Java, and they have good Jenkins support.
Here is a plugin for Maven to start up Selenium before the tests: http://mojo.codehaus.org/selenium-maven-plugin/
Here is how to run your TestNG tests using Maven: http://maven.apache.org/plugins/maven-surefire-plugin/examples/testng.html
apologies for the somewhat confusing title :-) Basically, I've configured my Selenium tests to run through JUnit when I build the project through maven. If I start my Selenium Server in a command prompt and then run the maven build goal in another cmd prompt then the tests runs as expected.
I figured that this process would be easy to implement in Hudson but it has proven quite problematic. I can start my selenium server as before and then configure the build to kick off via a hudson job. However, hudson doesn't seem to be picking up the selenium server - I get an error message which says the following:
Unable to obtain goal [test:test]
The reason I think that hudson isn't contacting the selenium server is because when I run the junit tests through my own command prompt with my server stopped, I get the same error message.
Its also important to note that I used to use the seleniumHQ plugin for hudson to run my tests using html files - but wish to change to JUnit as its more flexible.
Any help on this is very much appreciated!
Thanks,
Gearoid.
I recommend that you run selenium tests separately from the test:test goal.
You can create two hudson builds for your project, with the second build being triggered by the successful completion of the first build.
Why would you do this?
Unit tests complete faster, so your feedback time is mush shorter
the second build can run a shell script or ant instead of maven, allowing you easier control of your remote server
The selenium tests won't be triggered by mvn test on a developer machine, which could cause conflicts.
You can run two hudson nodes in a "farm" configuration, the second instance can run on the same server as selenium. Server affinity can be set up for specific builds.
Is there a config file with the server URL that is not being picked up running inside Hudson? Can you print the server URL to make sure it's pointing at the right place?
You could use the selenium-maven-plugin to start and stop the selenium server automatically as part of the maven build process.