I have a test method inside a test class where I want to verify a couple things, only fail after I soft assert in this specific test method.
But, I feel my test method is getting messy with failure handling. I haven't been able to find any best practices on this. Any ideas? If I move the asserts into the page object class, it will be a bit messy there too.
#Test
public void test() {
// steps here
// then asserts here
SoftAssert soft = new SoftAssert();
String expectedHeaderText = "foo";
soft.assertTrue(pageObjectClass.isHeaderPresent(), "Unable to find the Header page object.");
soft.assertTrue(pageObjectClass.getHeader().contains(expectedHeaderText),
String.format("Expected to find '%s'. Page actually shows '%s'", expectedHeaderText, pageObjectClass.getHeader()));
// more asserts
sa.assertAll();
}
Check below convention
#Test
public void test() {
// steps here
// then asserts here
SoftAssert soft = new SoftAssert();
String expectedHeaderText = "foo";
Boolean checkHeader=pageObjectClass.isHeaderPresent() //Change the method on POM pageObjectClass such that it returns the true or false
soft.assertTrue(checkHeader,true);
String checkHeaderContent=pageObjectClass.getHeader()//change method on POM pageObjectClass to return a string
soft.assertTrue(checkHeaderContent.contains(expectedHeaderText), String.format("Expected to find '%s'. Page actually shows '%s'", expectedHeaderText, checkHeaderContent));
// more asserts
sa.assertAll();
}
You can check QMetry Automation Framework which provided assertion and verification methods. For example:
//verify element present
firstName.verifyPresent();
firstName.assertPresent();
//verify Text of Element
firstName.verifyText("First User");
firstName.assertText("First User");
//verify Text of element with StringMatchers conditions
firstName.verifyText(StringMatcher.contains("First User"));
firstName.assertText(StringMatcher.contains("First User"),"Username Validation");
In case of assert method, your test will not continue on assert fail.
In case of any verify method, your test will continue even if verification failed and the final status of test will be failed if one or more verification failed.
It's always a dilemma: to have explicit checks and readable error messages, or omit something in order to make the code shorter or more generic to be reused.
Your example is a plain SoftAssert usage which is recommended by many tutorials.
It's a best practice to keep all assertions on Test Level, not in Page Objects.
But how to deal when you see, some assertions are huge and duplicated within several test methods?
I suggest following next rules:
Similar(duplicated) assert constructions can be moved to a new helper method within a current Test Class.
Try to split the tests into classes in the way, test classes contain only similar test methods.
If several test classes contain the same code, you can create a common parent for a group of classes and move some reusable code on this test-class-group level.
Do not try to fully avoid duplications on Test Level, tests are something that is changed and outdated rapidly. They should be still readable and easy to understand what happens in the code.
I don't use SoftAsserts, but can suggest some of this as a point for the extension:
import org.testng.asserts.SoftAssert
import static java.lang.String.format
public class ProjectSoftAssert extends SoftAssert {
public void assertElementVisibleAndContainsText(
boolean isVisible, String actualText, String expectedText, String elementName
) {
assertTrue(isVisible, format("Unable to find the '%s' page object.", elementName));
assertTrue(
actualText.contains(expectedText),
format(
"Wrong '%s' page object text. Expected to find '%s'. Page actually shows '%s'",
elementName, expectedText, actualText
)
);
}
}
And in your scenario:
#Test
public void test() {
// steps here
// then asserts here
ProjectSoftAssert soft = new ProjectSoftAssert();
soft.assertElementVisibleAndContainsText(
pageObjectClass.isHeaderPresent(), pageObjectClass.getHeader(), "foo", "Header"
);
// more asserts
soft.assertAll();
}
Related
I've read in lot of places that test methods should/must be void, but no one says what is the reason for this.
I found in MethodValidator the following check without comments/javadocs.
if (each.getReturnType() != Void.TYPE) {
errors.add(new Exception("Method " + each.getName()
+ " should be void"));
}
So why it should be void?
Ask you the reverse question : why JUnit test methods would need to be not void ?
No reason : because a test method is not designed to return a result that will be exploited by a client class.
The goal of an unit test is validating some assertions. A test runner invokes the test methods and and this is the runner that interprets any assertion failure or any exception thrown during the test execution to produce the test result.
We could wonder why tests don't return the assertion result.
But it would be a bad idea as it would be very cumbersome to write unit tests :
#Test
public AssertionResult foo(){
Bar actualBar = foo.doThat(...);
if (actualBar == null){
return AssertionResult.fail("actualBar == null");
}
}
Write something like that is really readable and straight :
#Test
public void foo(){
Bar actualBar = foo.doThat(...);
Assert.assertNotNull(actualBar);
}
We could also wonder why test methods cannot be invoked by other test methods such as :
#Test
public int foo(){
Bar actualBar = foo.doThat(...);
//...
return intValue;
}
#Test
public void fooWithComputedInt(){
Bar actualBar = foo.doThat(foo());
//...
}
But this also would be not a good idea as this would couple the tests execution while unit tests execution has to be isolated from others. And it also would make the tests be executed multiple times and unit tests have to be executed as fast as possible.
So really no value to make a test method return other thing than void.
This is purely a design choice. JUnit does not know about your code, so it could not do anything if your method would return something.
So either it should discard a return value, or require you to use "void" methods. The authors chose the latter option - you could argue that this slightly better because it's not confusing the reader.
Note that non-#Test methods are free to do whatever they want - they don't have this limitation.
I am currently stuck trying to create a unit test for this piece of code I have. I honestly can't figure out at all how to create a unit test for these lines of code. I have looked multiple places online and couldn't find anything. Its probably just because I don't understand unit test so I can't figure out how to create this one but could someone help me please?
public List<Overview> findOverviewByStatus(String status) throws CustomMongoException {
List<Overview> scenarioList = new ArrayList<Overview>();
LOGGER.info("Getting Scenario Summary Data for - {}", status);
Query query = new Query(Criteria.where("status").is(status));
if (mongoTemplate == null)
throw new CustomMongoException("Connection issue - Try again in a few minutes",
HttpStatus.FAILED_DEPENDENCY);
LOGGER.info("Running Query - {}", query);
scenarioList = mongoTemplate.find(query.with(new Sort(Sort.Direction.DESC, "lastUpdatedDate")), Overview.class);
return scenarioList;
}
So you want to unit test the method. Start with pretending you don't know what the code looks like (black box testing).
What happens if you call it with status of null, and then status of empty string?
What are some status string that return expected values?
Add all these as asserts to your test method to make sure that if someone changes this method in the future the unit test makes sure that it returns the expected result.
That is all a unit test usually does, makes sure that the code behaves in a predictable way and safeguard against change that violates a contract you created for the method when you wrote it.
For example:
import org.junit.Assert;
import org.junit.Test;
public class MyObjectTest {
#Test
public void testMyObjectMethod() {
// Create the object that contains your method (not in the sample you provided)
MyObjectToTest obj = new MyObjectToTest();
// Check that for a null status you get some result (assuming you want this)
Assert.assertNotNull(obj.findOverviewByStatus(null));
// Lets assume that a null status returns an empty array, add a check for it
Assert.assertTrue("null parameter size should be 0", obj.findOverviewByStatus(null).size() == 0);
//etc...
}
}
When using DataProviders, on TestNG, my test method has asserts that will fail since the data passed in navigates to a different url. Is there a way to work around this, i.e. a way for the data to only be injected to certain/specific asserts?
Instead of testing one scenario with different data, I am instead testing multiple scenarios with different data which is where my conflict arises.
#DataProvider(name = "VINNumbers")
public String[][] VINNumbers() {
return new String[][] {
{"2T1BU4ECC834670"},
{"1GKS2JKJR543989"},
{"2FTDF0820A04457"}
};
}
#Test(dataProvider = "VINNumbers")
public void shouldNavigateToCorrespondingVinEnteredIn(String VIN) {
driver.get(findYourCarPage.getURL() + VIN);
Assert.assertTrue(reactSRP.dealerListingMSRPIsDisplayed());
}
The assert test whether or not the page has an MSRP displayed, but not all dataproviders will have an MSRP displayed so it will fail. The only dataprovider that has it is the first array. Is there a way for dataproviders to be called to specific asserts?
If depending on the VIN, MSRP is displayed or not (boolean), you could for example create a provider the way it provides VIN and expected result:
#Test(dataProvider = "VINNumbers")
public void shouldNavigateToCorrespondingVinEnteredIn(String VIN, boolean isMSRPDisplayed) {
// act
// assert
assertThat(reactSRP.dealerListingMSRPIsDisplayed()).is(isMSRPDisplayed);
}
This way you end up with an provider like below:
{
{"2T1BU4ECC834670", true},
{"1GKS2JKJR543989", false},
{"2FTDF0820A04457", true},
}
In my opinion this is acceptable for simple cases. To make assertion more readable, I would add a custom message to it that is also parameterized.
I hope this helps.
By 'output steam' i mean any object which receives a sequence of bytes, or characters or whatever. So, java.io.OutputStream, but also java.io.Writer, javax.xml.stream.XMLStreamWriter's writeCharacters method, and so on.
I'm writing mock-based tests for a class whose main function is to write a stream of data to one of these (the XMLStreamWriter, as it happens).
The problem is that the stream of data is written in a series of calls to the write method, but what matters is not the calls, but the data. For example, given an XMLStreamWriter out, these:
out.writeCharacters("Hello, ");
out.writeCharacters("world!");
Are equivalent to this:
out.writeCharacters("Hello, world!");
It really doesn't matter (for my purposes) which happens. There will be some particular sequence of calls, but i don't care what it is, so i don't want to write expectations for that particular sequence. I just want to expect a certain stream of data to be written any which way.
One option would be to switch to state-based testing. I could accumulate the data in a buffer, and make assertions about it. But because i'm writing XML, that would mean making some fairly complex and ugly assertions. Mocking seems a much better way of dealing with the larger problem of writing XML.
So how do i do this with a mock?
I'm using Moxie for mocking, but i'm interested in hearing about approaches with any mocking library.
A fairly elegant strategy to test output or input streams is to use PipedInputStream and PipedOutputStream classes. You can wire them together in the set up of the test, and then check what has been written after the target method is executed.
You can work the other direction preparing some input and then let the test read this prepared data from the input stream as well.
In your case, you could just mock that "out" variable with a PipedOutputStream, and plug a PipedInputStream to it this way:
private BufferedReader reader;
#Before
public void init() throws IOException {
PipedInputStream pipeInput = new PipedInputStream();
reader = new BufferedReader(
new InputStreamReader(pipeInput));
BufferedOutputStream out = new BufferedOutputStream(
new PipedOutputStream(pipeInput))));
//Here you will have to mock the output somehow inside your
//target object.
targetObject.setOutputStream (out);
}
#Test
public test() {
//Invoke the target method
targetObject.targetMethod();
//Check that the correct data has been written correctly in
//the output stream reading it from the plugged input stream
Assert.assertEquals("something you expects", reader.readLine());
}
I'll admit that I'm probably partial to using a ByteArrayOutputStream as the lowest level OutputStream, fetching the data after execution and peforming whatever assertions that are needed. (perhaps using SAX or other XML parser to read in the data and dive through the structure)
If you want to do this with a mock, I'll admit I'm somewhat partial to Mockito, and I think you could accomplish what you're looking to do with a custom Answer which when the user invokes writeCharacters on your mock, would simply append their argument to a Buffer, and then you can make assertions on it afterwards.
Here's what I have in my head (hand written, and haven't executed so syntax issues are to be expected :) )
public void myTest() {
final XMLStreamWriter mockWriter = Mockito.mock(XMLStreamWriter.class);
final StringBuffer buffer = new StringBuffer();
Mockito.when(mockWriter.writeCharacters(Matchers.anyString())).thenAnswer(
new Answer<Void>() {
Void answer(InvocationOnMock invocation) {
buffer.append((String)invocation.getArguments()[0]);
return null;
}
});
//... Inject the mock and do your test ...
Assert.assertEquals("Hello, world!",buffer.toString());
}
(Disclaimer: I'm the author of Moxie.)
I assume you want to do this using logic embedded in the mock so that calls that violate your expectation fail fast. Yes, this is possible - but not elegant/simple in any mocking library I know of. (In general mock libraries are good at testing the behavior of method calls in isolation/sequence, but poor at testing more complex interactions between calls over the lifecycle of the mock.) In this situation most people would build up a buffer as the other answers suggest - while it doesn't fail fast, the test code is simpler to implement/understand.
In the current version of Moxie, adding custom parameter-matching behavior on a mock means writing your own Hamcrest matcher. (JMock 2 and Mockito also let you use custom Hamcrest matchers; EasyMock lets you specify custom matchers that extend a similar IArgumentMatcher interface.)
You'll want a custom matcher that will verify that the string passed to writeCharacters forms the next part of the sequence of text you expect to be passed into that method over time, and which you can query at the end of the test to make sure it's received all of the expected input. An example test following this approach using Moxie is here:
http://code.google.com/p/moxiemocks/source/browse/trunk/src/test/java/moxietests/StackOverflow6392946Test.java
I've reproduced the code below:
import moxie.Mock;
import moxie.Moxie;
import moxie.MoxieOptions;
import moxie.MoxieRule;
import moxie.MoxieUnexpectedInvocationError;
import org.hamcrest.BaseMatcher;
import org.hamcrest.Description;
import org.junit.Assert;
import org.junit.Rule;
import org.junit.Test;
import javax.xml.stream.XMLStreamException;
import javax.xml.stream.XMLStreamWriter;
// Written in response to... http://stackoverflow.com/questions/6392946/
public class StackOverflow6392946Test {
private static class PiecewiseStringMatcher extends BaseMatcher<String> {
private final String toMatch;
private int pos = 0;
private PiecewiseStringMatcher(String toMatch) {
this.toMatch = toMatch;
}
public boolean matches(Object item) {
String itemAsString = (item == null) ? "" : item.toString();
if (!toMatch.substring(pos).startsWith(itemAsString)) {
return false;
}
pos += itemAsString.length();
return true;
}
public void describeTo(Description description) {
description.appendText("a series of strings which when concatenated form the string \"" + toMatch + '"');
}
public boolean hasMatchedEntirely() {
return pos == toMatch.length();
}
}
#Rule
public MoxieRule moxie = new MoxieRule();
#Mock
public XMLStreamWriter xmlStreamWriter;
// xmlStreamWriter gets invoked with strings which add up to "blah blah", so the test passes.
#Test
public void happyPathTest() throws XMLStreamException{
PiecewiseStringMatcher addsUpToBlahBlah = new PiecewiseStringMatcher("blah blah");
Moxie.expect(xmlStreamWriter).anyTimes().on().writeCharacters(Moxie.argThat(addsUpToBlahBlah));
xmlStreamWriter.writeCharacters("blah ");
xmlStreamWriter.writeCharacters("blah");
Assert.assertTrue(addsUpToBlahBlah.hasMatchedEntirely());
}
// xmlStreamWriter's parameters don't add up to "blah blah", so the test would fail without the catch clause.
// Also note that the final assert is false.
#Test
public void sadPathTest1() throws XMLStreamException{
// We've specified the deprecated IGNORE_BACKGROUND_FAILURES option as otherwise Moxie works very hard
// to ensure that unexpected invocations can't get silently swallowed (so this test will fail).
Moxie.reset(xmlStreamWriter, MoxieOptions.IGNORE_BACKGROUND_FAILURES);
PiecewiseStringMatcher addsUpToBlahBlah = new PiecewiseStringMatcher("blah blah");
Moxie.expect(xmlStreamWriter).anyTimes().on().writeCharacters(Moxie.argThat(addsUpToBlahBlah));
xmlStreamWriter.writeCharacters("blah ");
try {
xmlStreamWriter.writeCharacters("boink");
Assert.fail("above line should have thrown a MoxieUnexpectedInvocationError");
} catch (MoxieUnexpectedInvocationError e) {
// as expected
}
// In a normal test we'd assert true here.
// Here we assert false to verify that the behavior we're looking for has NOT occurred.
Assert.assertFalse(addsUpToBlahBlah.hasMatchedEntirely());
}
// xmlStreamWriter's parameters add up to "blah bl", so the mock itself doesn't fail.
// However the final assertion fails, as the matcher didn't see the entire string "blah blah".
#Test
public void sadPathTest2() throws XMLStreamException{
PiecewiseStringMatcher addsUpToBlahBlah = new PiecewiseStringMatcher("blah blah");
Moxie.expect(xmlStreamWriter).anyTimes().on().writeCharacters(Moxie.argThat(addsUpToBlahBlah));
xmlStreamWriter.writeCharacters("blah ");
xmlStreamWriter.writeCharacters("bl");
// In a normal test we'd assert true here.
// Here we assert false to verify that the behavior we're looking for has NOT occurred.
Assert.assertFalse(addsUpToBlahBlah.hasMatchedEntirely());
}
}
I am having a problem with EasyMock 2.5.2 and JUnit 4.8.2 (running through Eclipse). I have read all the similar posts here but have not found an answer. I have a class containing two tests which test the same method. I am using matchers.
Each test passes when run alone.
The first test always passes - this is true if I switch the order of the tests in the file.
Here is a simplified version of the test code:
private Xthing mockXthing;
private MainThing mainThing;
#Before
public void setUp() {
mockXthing = EasyMock.createMock(Xthing.class);
mainThing = new MainThing();
mainThing.setxThing(mockXthing);
}
#After
public void cleanUp() {
EasyMock.reset(mockXthing);
}
#Test
public void testTwo() {
String abc = "abc";
EasyMock.expect(mockXthing.doXthing((String) EasyMock.anyObject())).andReturn(abc);
EasyMock.replay(mockXthing);
String testResult = mainThing.testCallingXthing((Long) EasyMock.anyObject());
assertEquals("abc", testResult);
EasyMock.verify(mockXthing);
}
#Test
public void testOne() {
String xyz = "xyz";
EasyMock.expect(mockXthing.doXthing((String) EasyMock.anyObject())).andReturn(xyz);
EasyMock.replay(mockXthing);
String testResult = mainThing.testCallingXthing((Long) EasyMock.anyObject());
assertEquals("xyz", testResult);
EasyMock.verify(mockXthing);
}
The second (or last) test always fails with the following error:
java.lang.IllegalStateException: 1 matchers expected, 2 recorded
Any insight to this would be greatly appreciated.
Thanks,
Anne
I haven't looked meticulously closely yet, but this looks suspect:
String testResult = mainThing.testCallingXthing((Long) EasyMock.anyObject());
anyObject() is a matcher and you're calling it after the replay. It's not used to produce any object. It's used to instruct EasyMock to allow any object. EasyMock is detecting that extra matcher but it is not harmful until the second test. At that point, the number of matchers that EasyMock has recorded but hasn't yet used (2) doesn't line up with the number of parameters expected for the second doXthing call (1).
You should be passing in real parameters to testCallingXthing (or a mock that is in replay mode). Try passing in null directly, or a real value like 2.
for me this failure (in my case 2 matchers expected, 4 recorded.) meant "you are mixing easymock and mockito in the same unit test, so accidentally calling easymock's notNull() method for a mockito argument. Which causes the failure but only if the tests are run in a certain order.
Try:
String testResult = mainThing.testCallingXthing(eq(EasyMock.anyLong()));
There are more refined matchers than anyObject(). These allow you to make type-based assertions about collaborators.
From the EasyMock documentation:
eq(X value)
Matches if the actual value is equals the expected value. Available for all primitive types and for objects.
anyBoolean(), anyByte(), anyChar(), anyDouble(), anyFloat(), anyInt(), anyLong(), anyObject(), anyShort()
You should reset mock after each test method to get rid of this problem. Adding below code will solve this problem.
#After
public void after(){
EasyMock.reset(mockXthing)
}