Automated Testing Framework - java

I am trying build an automated testing framework for some of our internal web services (java) at work. Each service has a set of APIs (3-5), although this could be relatively easy to achieve, the problem comes with some APIs which do not behave as pure functions, ex: something like persistX, this could store something in a database and returns an exception incase of failure.
There is no easy way to validate since there is no output here.
So I was wondering if this could a a bit generalized, say while testing the API, the user could provide a simple plugin or script to the framework of some sort that could validate the test. This is just one idea would be great if someone can tell me some better ones or any resources about the same.
Thanks

I recommend the robot framework. It is a keyword driven framework written in python. Because of that, you can run it in the JVM with jython which means you can extend it with java code (or python, of course). I've sucessfully used it to call APIs, then verify the result by peeking into a database, or querying the file system.
It also works on the .NET platform, has a selenium module for testing the front end, a jenkins plugin, and several other tools. It's very extensible and flexible.

What you are looking at is the application of black-box and white-box testing and the tools that support both.
For the web services that return a proper response you can perform black-box testing by verifying the data in returned response. SoapUI is the best tool for this.
For the APIs which do not behave as pure functions, you do white-box testing by verifying its side-effects like persistence, event generation, logging etc. For this you like programmable tools and SoapUI may or may not be the correct option.
We do both at our work and after evaluating multiple tools/frameworks (SoapUI, RSSPec, Robotframework), I chose Spock. Why spock?
It allows you to write intention revealing tests in BDD style
We are Java shop and we want to use the same familiar language for automation as well but with simplified syntatic sugar. And spock is all groovy based.
Excellent Webdriver/Selenium 2 support (including PageFactory) with Geb
It is built on top of JUNIT so all the plugins of JUNIT can be leveraged (code coverage, hudson/jenkins integration, etc)
Lot of webservice APIs and XML DSLs (no need to work with XPATH for simple scenarios)
Simplified setup (unlike robotframework it doesnt require python, jython setup)
etc....

Related

Testing APIs with Gherkin/Java and IntelliJ IDEA

I have been asked to create some automation tests to test an API for a project. I am fairly new to automation testing with limited knowledge of java. The instructions I have been given include; create a BDD style test suite using cucumber/gherkin syntax, utilizing Apache fluent-hc to interrogate the API. I have also been asked to create the test suite using Java. How can I utilize 'Apache fluent-hc' to test my API and can this be done via a Java IDE like 'IntelliJ IDEA'? Thanks
That's indeed a very general question. I can just say in really short "yes you can use IntelliJ IDEA" for all Java related development.
From the other point, it is something that you have to learn and understand your own, before starting any development, as a sample, in which way your BDD tests will be developed based on the API technology and as you said that you have limited knowledge of Java I would suggest you read first how to create API in Java, of course, it all based on your requirements, but I prefer API development using Spring Boot.
And if you are interested in that, I will post my blog post URL here later on where I'm describing from 0 to end how to cover APIs with BDD (currently it is under preparation, next week I will post it).
Meanwhile, I will suggest you read this blog and earn some knowledge of APIs and their testing.
UPDATE: Here is the blog post about BDD that I promised - https://www.blazemeter.com/blog/api-testing-with-cucumber-bdd-configuration-tips

Is Citrus Framework suitable for standalone deployment

I'm a developer of a BPM application. Our team has a simple hand-written tool, that allows to mock external systems over JMS and SOAP, also this tool allows us to call our own integrations.
However, we want to use something more extensible and bulletproof for integration testing purposes and Citrus Framework seems like a good alternative, but it is designed primarily to be executed like JUnit/TestNG tests, which is not suitable for the case.
We need an application that runs as a standalone one and allows simple configuration without redeployment.
Is this possible with Citrus Framework?
If not, what approach would you recommend for the purpose?
Thank you!
There is a side project from Citrus called Citrus Simulator that is a standalone simulator for multiple message transports including JMS. The project is available on github: https://github.com/christophd/citrus-simulator
User manual (still under construction) can be found here: https://christophd.github.io/citrus-simulator/

Testing rest services by using the client

I have been an old school type of programmer and didn't really use tests in my projects. I want to change that now but since it's not an industrial project but my own I would like to ask:
Is there any problem with using the client to test my Rest service?
Why would I need to write a unit test to test the rest backend and then write a client when I can just extend the client?
Is there any convention that the testing language should be the same as the implementing language ? I am talking about Java rest and JS client.
I think this will make my approach much more agile and fast.
Why would I need to write a unit test to test the rest backend and then write a client when I can just extend the client?
Basically in Test Cases you test everything while a client may use only a subset of the interface and will never hit a possible bug.
Is there any convention that the testing language should be the same as the implementing language ?
No. But there are very good tools (even automatic generation) that snaps into the Java IDE, Selecting java Automation testing tools for web application .

Selecting java Automation testing tools for web application [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I am very new to web automation testing tools. I want to automate my functional tests of web.
Please suggest me a few number of web automated testing tools. I want to use Java to write my automated testing tool.
I have visited the following site to select a good tool, but as I have no idea it is difficult to select a suitable tool for me.
http://java-source.net/open-source/web-testing-tools
Can any one suggest me a good tool for web testing?
Web Application Testing Tools
Selenium is widely used portable software testing framework for web applications. Selenium provides a record/playback tool for authoring tests without learning a test scripting language.
Selenium has the support of some of the largest browser vendors who have taken (or are taking) steps to make Selenium a native part of their browser. It is also the core technology in countless other browser automation tools, APIs and frameworks.
Anteater is a
testing framework designed around Ant, from the Apache Jakarta Project.
It is basically a set of Ant tasks for the functional testing of Web
sites and Web services (functional testing being: hit a URL and ensure
the response meets certain criteria). One can test HTTP parameters,
response codes, XPath, regexp, and Relax NG expressions. Anteater also
includes HTML reporting (based on junitreport) and a hierarchical
grouping system for quickly configuring large test scripts. When a Web
request is received, Anteater can check the parameters of the request
and send a response accordingly. This makes it useful for testing SOAP
and XML applications.
The ability to wait for incoming HTTP messages is something unique to
Anteater, which makes it especially useful when building tests for
applications that use high level SOAP-based communication, like ebXML or
BizTalk. Applications written using these protocols usually receive SOAP
messages and send back a meaningless response. It is only later that
they inform the client, using an HTTP request on the client, about the
results of the processing. These are the so-called asynchronous SOAP
messages, and are the heart of many high-level protocols based on SOAP
or XML messages.
Written in Java, HttpUnit emulates the
relevant portions of browser behavior, including form submission,
Javascript, basic HTTP authentication, cookies, and automatic page
redirection, and allows Java test code to examine returned pages either
as text, an XML DOM, or containers of forms, tables, and links.
jWebUnit is a Java
framework which facilitates creation of acceptance tests for Web
applications. It provides a high-level API for navigating a Web
application combined with a set of assertions to verify the
application's correctness. This includes navigation via links, form
entry and submission, validation of table contents, and other typical
business Web application features. It utilizes HttpUnit behind the
scenes. The simple navigation methods and ready-to-use assertions allow
for more rapid test creation than using only JUnit and HttpUnit.
Bugkilla is a tool set to
create, maintain, execute, and analyze functional system tests of Web
applications. Specification and execution of tests is automated for both
the Web frontend and business logic layers. One goal is to integrate
with existing frameworks and tools (an Eclipse Plugin exists)
The Grinder, a Java
load testing framework freely available under a BSD-style Open Source
license, makes it easy to orchestrate the activities of a test script in
many processes across many machines, using a graphical console
application. Test scripts make use of client code embodied in Java
plugins. Most users of The Grinder do not write plugins themselves; they
use one of the supplied plugins. The Grinder comes with a mature plugin
for testing HTTP services, as well as a tool which allows HTTP scripts
to be automatically recorded.
Jameleon is an
automated testing tool that separates applications into features and
allows those features to be tied together independently, creating test
cases. These test cases can then be data-driven and executed against
different environments. Jameleon breaks applications into features and
allows testing at any level, simply by passing in different data for the
same test. Because Jameleon is based on Java and XML, there is no need
to learn a proprietary technology.
It's an acceptance testing tool for testing the functionality provided
by applications, and currently supports the testing of Web applications.
It differs from regular HttpUnit and jWebUnit in that it separates
testing of features from the actual test cases themselves. If I
understand it correctly, you write the feature tests separately and then
script them together into a reusable test case. Incidentally, you can
also make these test cases data-driven, which gives an easy way of
running specific tests on specific environments.
The framework has a plugin architecture, allowing different functional
testing tools to be used, and there is a plugin for testing Web
applications using HttpUnit/jWebUnit. The test case scripting is done
with XML and Jelly.
Jameleon combines XDoclet, Ant and Jelly to provide a
potentially powerful framework for solid functional testing of your
Webapp. It strikes a good balance between scripting and coding, and
allows you to set up multiple inputs per test by providing input via CSV
files. Along with the flexibility come a complexity and maintenance
overhead, but you are getting your Webapp tested for you.
LogiTest is the
core application in the LogiTest suite. LogiTest is designed to aid in
the testing of Web site functionality. It currently supports HTTP and
HTTPS protocols, GET and POST methods, multiple document views, custom
headers, and more. The LogiTest application provides a simple graphical
user interface for creating and playing back tests for testing
Internet-based applications.
Solex is a set of
Eclipse plugins providing non-regression and stress tests of Web
application servers. Test scripts are recorded from Internet browsers,
thanks to a built-in Web proxy. For some Web applications, a request
depends on a previous server's response. To address such a requirement,
Solex introduces the concept of extraction and replacement rules. An
extraction rule tied to an HTTP message's content will bind an extracted
value with a variable. A replacement rule will replace any part of an
HTTP message with variable content.
The tool therefore provides an easy way to extract URL parameters,
Header values, or any part of a request or a response, bind their values
with variables, and then replace URL parameters, Header values, or any
part of a request with the variable content. The user has the ability to
add assertions for each response. Once a response has been received, all
assertions of this response will be called to ensure that it is valid.
If not, the playback process is stopped. Several kinds of rules and
assertions are provided. The most complicated ones support regular
expressions and XPath.
Tclwebtest is a
tool for writing automated tests of Web applications in Tcl. It
implements some basic HTML parsing functionality to provide comfortable
commands for operations on the HTML elements (most importantly forms) of
the result pages.
TagUnit is a framework
through which custom tags can be tested inside the container and in
isolation from the pages on which they will ultimately be used. In
essence, it's a tag library for testing tags within JSP pages. This
means that it is easy to unit test tags, including the content that they
generate and the side effects that they have on the environment, such as
the introduction of scripting variables, page context attributes,
cookies, etc.
Web Form Flooder
is a Java console utility that analyzes a Web page, completes any forms
present on the page with reasonable data, and submits the data. It
crawls links within the site in order to identify and flood additional
forms that may be present. It is great for load testing of Web forms,
checking that all links work and that forms submit correctly.
XmlTestSuite
provides a powerful way to test Web applications. Writing tests requires
only knowledge of HTML and XML. The authors want XmlTestSuite to be
adopted by testers, business analysts, and Web developers who don't have
a Java background. XmlTestSuite supports "test-driven development". It
lets you separate page structure from tests and test data. It can also
verify databases. It's like JWebUnit, but has simple XML test
definitions and reusable pages.
For web testing, use Selenium (WebDriver) It's probably best tool for this purpose out there on GoogleCode
You can find lot of examples on google, and documentation on seleniumhq.org its quite helpful for starters.
Selenium is the most mature and widely used tool for web UI testing (open source anyway, though I don't think there are much better proprietary tools either). Its biggest advantage is that it uses the actual browser engine and can therefore be used to test AJAX-heavy pages as well.

How to conduct blackbox testing on an AJAX application?

What's the best, crossplatform way to perform blackbox tests on AJAX web applications?
Ideally, the solution should have the following attributes:
Able to integrate into a continuous integration build loop
Cross platform so I you can run it on Windows laptops and Linux continuous integration servers
Easy way to script the interactions
Free-as-in-freedom so you can adapt it into your tool chain if necessary
I've looked into HttpUnit but I'm not conviced it can handle AJAX-heavy websites.
Selenium might be what you're looking for: http://selenium.openqa.org/
It allows you to script actions and evaluate the results. It's open-source (Apache 2.0), cross platform, and has nice tools.
I have used Selenium for exactly this task, but found it to be brittle.
Check out this talk by two Googlers: Does my button look big in this? Building testable AJAX applications
They isolate the testable javascript (non DOM-interaction) and test that using the Rhino javascript engine.

Categories