I need to build a datadriven framework using selenium webdriver with java binding. i am bit confused with regards to choosing the right tool for building framework
Could somebody please recommend which framework (i.e. Junit, TestNG, Cucumber) works better with large set of test data
Also, please suggest which data file format i.e. .xls or .tsv to use for executing automated tests from Jenkins(deployed on Linux box)
As fare as I know and what I have earlier used Selenium for is to implement automated test scenarios of a web-application. Would typically be accept-test og end-to-end test (e2e).
These kind of test are what you call Black-Box testing. You give a input and expect a output, but does not know what is going on inside the black-box (you application).
To implement and execute your tests scenarios you would use a test-framework as you also mentioned (JUnit, TestNG, Cucumber etc.).
All test are able to be executed on a build server running Jenkins whether it is unit/integration or e2e-test (which could be implemented with Selenium).
Selenium test are expensive to maintain and often you would run into timing issues which causes test to fail even though the code does not contain any errors.
Therefore it is important to implement the test correct, otherwise you would run into a maintenance hell.
I am not sure if you are new to testing?? Maybe you could benefit from reading a bit about software testing.
The following is a good link: https://www.softwaretestingmaterial.com/software-testing/
Let me know, if I have misunderstood you question totally :)
Related
I am trying to create the following setup :
A Selenium (Java) project that has a set of 10 automated test cases.
When this project is executed, it generates an HTML test execution report.
This project should be 'hosted' on an internal network.
Anyone who has access to the network should be able to 'invoke' this project, which in turn executes the test cases and passes the HTML report to the person who invoked it.
The project should be accessible ONLY for execution and the code should NOT be accessible.
My goal is that this implementation should be executable by any framework irrespective of the technology that the framework uses. I was thinking of creating the project as a WebService using Java (servlet).
My question is:
Can this implementation be accessed by any external automation framework ?
Are there any limitations to this implementation?
Is there a better way to implement this requirement?
Thanks in advance.
You can create a maven project and have your automated tests under maven test folder.Configure your tests to run through POM.xml(use maven surefire plugin).Configure a jenkins job to run the maven test.Anybody with access the jenkins can build/run this task!
Below link should give you a headstart
http://learn-automation.com/selenium-integration-with-jenkins/
As a matter of fact, it is something we did on one of our projects. As I cannot share specifics, I will give you overall architectural view of the project.
The core of all things was a service that could run JUnit tests on requests. It was a Soap web-service, but nothing stops you from making it REST. To implement this you need to implement your version of JUnit test runners (see for example: http://www.mscharhag.com/java/understanding-junits-runner-architecture or https://github.com/junit-team/junit4/wiki/Test-runners)
If you use JUnit as test framework for running your Selenuim tests this may be a great solution for you - JUnit will generate HTML reports for you if you configure it properly, it will hide actual test suite implementation from users and run test suite on demand. This solution is also great because it operates on JUnit level and does not care about what kind of tests it actually runs, so it can be also reused for any other kind of automated tests.
So to answer all your questions:
Can this implementation be accessed by any external automation
framework ? -> yes, it can be accessed by anybody who able send http
requests
Are there any limitations to this implementation? -> none that I am
aware of
Is there a better way to implement this requirement? -> well, I
didn't actually work with TestNG much so I don't know if it is
easier or more difficult to do it on Junit level. You can use
Jenkins or other CI tool as well to achieve same results - they can
run JUnit tests for you and almost always have API ready for this,
although those APIs may be not perfect.
So I'd say that if you need this only for one thing you can use CI tools for this purpose, if you don't have CI tools available then choice has been made for you. However, from our experience, having this kind of service was a great asset for a company and I really wonder why there's no such products available elsewhere yet.
I am an entry level tester, mainly been doing manual testing for a company in the UK following scripts on a spreadsheet which I have written in the BDD format, however, I have been learning some automation on the side as that's what I want to move into full time. I have some questions though which are as follows.
I've been using Selenium web driver + java bindings to make simple tests such as logging in or filling out a registration form, i've also set up log4j but only basic to record low level recording. I have now come across testNG. My main question is this framework used by testers? or developers? Is testNG only for unit tests? or UI tests?
From what i've learnt so far the developer does the unit and component tests and the tester does the services/ui tests is this correct?
Unfortunately I was put into a team of developers and not testers as this is my first job outside of university. So I haven't had the chance to learn from other testers. There was no plan for me when I started just that I was going to be the first tester in this development team without any prior testing knowledge.
Which is why I need a bit of guidance on these issues.
My main question is this framework used by testers? or developers? Is
testNG only for unit tests? or UI tests?
TestNG can be used for both, developers and automation testers, it is a tool that can operate over and together with Junit, basically in some cases is being used to create the concept of test suite, that allows to split all the test cases based on specific criteria (time, module, complexity). Also this framework can be used in unit testing and integration testing as well as ui-testing.
TestNG also in some cases replaced Junit entirely, whit this approach you will have a framework with some out of the box capabilities as DataProviders, Multi threading support and other, you could check this link, consider this as and powerful option for Junit.
From what i've learnt so far the developer does the unit and component
tests and the tester does the services/ui tests is this correct?
Unit testing which I consider very similar as "component test" is being done by the developers. If you have web services or a REST API, developers sometimes are in charge of create some test using integration testing, basically verify that services are working as we expected, returning JSON/XML with the correct format and other kind of validations.
Testers also could check services, using tools such as Jmeter, SOAP-UI, they check more things related to the business logic.
Finally I would said UI test is being done in most of the places by the manual and automation testing team, in places where is no QA department this tasks also belongs to the DEV team.
In order to run tests you need to have a test runner it could be anything, most common in java world is JUnit and TestNG, with those frameworks you can run the tests which annotated by #Test tag, also you can group the tests the way you want it and run them in parallel.
Testers use it to run Selenium tests and do assertions, even though for assertions it is good to have knowledge of hamcrest matchers. Also it providing you reports after tests been completed.
Developers would use same frameworks for unit testing purposes.
Check out guys from toolsqa.com they have pretty comprehensive tutorials on using Selenium with TestNG.
TestNG is basically used by developers for doing unit testing, I agree. But it is also widely used by system test automation using Selenium. This framework is inspired by JUnit framework, and most of the automation test developers use this framework because of its advantages and more added features to support reporting.
I can say following advantages I got by using this framework:
1.Support for parameters.
2.Supports dependent methods testing.
3.Test configuration flexible. Supports powerful execution model.
4.Embeds BeanShell for further flexibility.
5.TestNG has a more elegant way of handling parameterized tests with the data-provider concept.
6.For the same test class TestNG support for multiple instances.
7.Extendibility of using different Tools and plug-ins like Eclipse, Maven, IDEA etc.
8.Default JDK functions for runtime and logging (no dependencies).
9.Supported different Annotations like #BeforeSuite, #AfterSuite, #BeforeClass, #AfterClass, #BeforeTest, #AfterTest, #BeforeGroups, #AfterGroups, #BeforeMethod, #AfterMethod, #DataProvider, #Factory, #Listeners, #Parameters, #Test.
The most beautiful part I found in testNG is, using data provider, i can easily read test inputs and expected results from excel. And I can able to see the Results of Pass/Fail and skip test cases in an emailable format.
For testing a system, we don't need any training/extra classes. Just if we know the system requirements, and this as a end user what they want from the system and start testing. If any deviations found in the system behavior and are not as per the expectations of user. Then mark it as an issue and raise a defect and track it until it get resolved. Retest the same and confirm that the system is working as per the expectations. even at the Unit test level this principle holds the same. But only the difference is that we can do Structure based testing there.
To your questions ..
1.My main question is this framework used by testers? or developers? Is testNG only for unit tests? or UI tests?
Answer = Test NG can be used for unit testing as well as UI testing. the advantage of test NG over JUNIT is that you dont need to write code for test result reporting.
I am really new to Fit/Fitnesse and, in general, to test automation.
I am trying to use them from Eclipse.
I have several question about it:
is there a way to obtain the html tables that Fitnesse pass to Fit?
once I write several tests with Fitnesse, is there a way to call them several times from Java without clicking on the Test button of the wiki?
About passing objects from one table to another in a flow. I read about symbols but it seems that, in java, they works only with ColumnFixturewhile I would like to use DoFixture. how to do this?
Finally,is if there is any plugin for eclipse you suggest to use with Fit/Fitnesse?
Regarding you question 2: I would recommend using the JUnit integration (#RunWith(FitNesseRunner.class) to run the test page (or a suite) as a unit test from Eclipse. This also gives you the ability to debug inside your fixture code.
It takes a bit of configuration to get it running 'just right'. In my pre-packaged FitNesse I provide a unit test FixtureDebug where you only have to enter the test name (and you can also use that to run your tests on a build/continuous integration server).
I have a doubt. Say I have a web application which is big and relies on Java/Java EE (JSP/Servlets).
Every time before a drop we test each and every functionality on GUI so that everything is working properly. Previously it was easy but now as the number of modules has increased exponentially, manually testing each and every GUI with required functionality is no more a feasible option.
I am on lookout for tools in which I can write my entire test case say about 1000 and then just run it once before the drop and it will list down all the test cases that have failed.
The tool preferably must be free to download and use it.
I dont know whether using
Arquilian
or
JUnit
in this regard will help or not but automating testing before the drop is really needed..
Please guide.
Use Junit together with a mock framework i.e Mockito to test units (service methods)
Use Arquillian to test on an integration level ( how different services, modules work together )
Use a database testing tool (i.e dbunit) to test your database / persistence layer)
Use Selenium to test your frontend
Test as much as possible.
Use Jenkins and Sonar to track your build process and your quality of tests and code
You should always test your application on different level. There is not just one solution.
Use unit testing to test small pieces of your application and to make refactoring as easy as possible.
Use integration test to check your modules still work together as expected.
Use GUI testing to check if your customers can work with your software.
If its relevant, think about performance testing (i.e. jmeter )
Definitively Selenium. Couple it with maven cause you will probably need to package your project specifically for testing purpose. Moreover maven allow you to launch a container during the integration-test phase and to close it automatically at the end. You can also configure this as a nightly build on jenkins / hudson so you will be quicly notified of any regression.
This may sound like a vague question but I am looking for some specif pointers.
Our J2EE app is built on Struts2 + Plain Servlets + JSP + iBatis + Oracle
I would prefer to write unit tests in Scala so that I can learn the language on the side as well
What would I need to be able to verify that a spcific column is displayed in the JSP following some specific steps
Click on a link. select some parameters and submit the page to the servlet
Verify that the next page has a specific column inside its <table> tag.
What would I need to create mock requests for the serlvet?
I am trying to write tests like above in addition to core business functionality tests however, the problem is that I am trying to wrap legacy code in unit tests and the code of course is not designed for unit testing.
I wouldn't call this unit testing. As you are trying to test integration of several units. Also it's rather hard to create a unit test for a JSP becuase it has many context dependencies available only when you are in the container.
Instead I would advice writing some automated functional tests that are executed against running (deployed) application.
Frameworks like Selenium may be of great help here as they allow to simulate real user behaviour and make asserts against produced HTML code.
EDIT: Another approach here may be to:
start an embedded servlet container like Jetty within your test code
deploy all your plain servlets and JSPs to that
replace Oracle database with in-memory database like HSQL or Derby
populate it with some test data using DBUnit
and then again use either Selenium (which has Java binding) or HttpUnit to make requests and asserts against generated HTML code.
But again it will not be a unit test, but rather an integration test.
Like everyone said, your not really talking about unit testing. You're talking about functional testing. I'd think hard about what your real goals are. What is driving this push for automated testing? Does the application have configuration issues(i.e. its hard to configure so some parts work and others don't). This might justify building a smoke test suite in selenium targeting your pain pages and test cases. This will also help detect regression bugs.
As for the legacy concerns. No application is beyond help. If you are running front end tests in selenium then it doesn't matter how the code is written as long as its parseable HTML.
As for your actual server side code. You just gotta roll Andy Dufresne style. As you fix bugs and add functionality code with Test Driven Development principles in mind. Rework code that relates to your changes and add unit tests. You'd be surprised at how fast a legacy app can come around if you keep chipping away at it.