Android Test Project JUnit Failure Trace Not Showing Expected + Actual - java

I'm writing some tests for an Android 2.3.3 project, using the Android JUnit Test runner, and I'm seeing some weird results in the failure traces on assertions. Here's a simple example:
import junit.framework.TestCase;
public class TU_Test extends TestCase {
public void testStuff() {
assertEquals("aft", "af");
}
}
The assertion obviously fails, and here's the trace copied from Eclipse:
junit.framework.ComparisonFailure: expected:<...t> but was:<...>
at com.redprairie.test.TU_Test.testStuff(TU_Test.java:33)
at java.lang.reflect.Method.invokeNative(Native Method)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154)
at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:529)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1448)
The displayed expected + actual are not very helpful... it seems to display the difference / missing characters between the two, but it would be much more helpful if I could see the full values of each. I'm normally (using JUnit 4 + not using the Android Test Runner) able to double click the failure trace in Eclipse and see a diff of the two results. Is there any way to achieve this using the Android Test Runner and it's JUnit 3 style tests? It's kind of a pain in the ass to always set breakpoints.
Thanks!

Have you tried catching ComparisonFailure? You can call getActual() and getExpected() on the exception object to find out the values.
Take a look at the following javadoc: http://kentbeck.github.com/junit/javadoc/4.10/org/junit/ComparisonFailure.html

Since I couldn't find a real solution to this, I ended up just wrapping the assertion and building the message myself like so:
private void _assertEquals(String expected, String actual) {
assertEquals("expected <" + expected + "> but was <" + actual + ">", expected, actual);
}

Related

Eclipse JUnit assertEquals(String, String) - disable square bracket diff in Failure Trace

JUnit assertEquals Changes String
In short: assertEquals("Hello World", "HelloWorld"); appears in the Failure Trace as expected:<Hello[ ]World> but was:<Hello[]World>
Great. Definitely didn't waste my time trying to figure out why the heck my parser was randomly throwing square brackets around.
How do I disable this?
I am using Eclipse 4.19.0 for Java, JUnit 4.10.
You cannot.
These brackets are added by junit.framework.ComparisonCompactor that is being used in org.junit.ComparisonFailure assertion error's getMessage method (this error is being thrown by assertEquals btw)
// ComparisonFailure class
public String getMessage() {
return new ComparisonCompactor(MAX_CONTEXT_LENGTH, fExpected, fActual).compact(super.getMessage());
}
in ComparisonCompactor these brackets are hardcoded and it seems that there is no configuration that can be provided, also ComparisonCompactor cannot be injected (the same for ComparisonFailure - you are not able to provide custom implementation for them)
public class ComparisonCompactor {
private static final String ELLIPSIS= "...";
private static final String DELTA_END= "]";
private static final String DELTA_START= "[";
// ...
private String compactString(String source) {
String result= DELTA_START + source.substring(fPrefix, source.length() - fSuffix + 1) + DELTA_END;
As far as I see even in Junit 4.13 it looks exactly the same so even bumping up dependency will not help here (however you could give a try to Junit5 that with usage of Assertions.assertEquals will produce expected: <Hello world> but was: <Helloworld> output - but obviously it won't be just bumping up dependency version)
By the way Eclipse has nothing to do with it, the same output you will get in other IDEs (or in console)
What I would suggest (however it's not an answer for your question) is to use assertion library like AssertJ that can give you more control but also make you assertions more fluent.
An example in AssertJ would look like
assertThat("Hello World").isEqualTo("HelloWorld");
and is producing
Expecting:
<"Hello World">
to be equal to:
<"HelloWorld">
but was not.

JUnit: Is it possible to output multi-line messages in any of the assert functions?

I tried to get a multi-line message when there's an assert. e.g. I have the following code:
errLog = "ERROR! Account Name not found in result \n Expected Result: " + acctName + "\n" + "Actual Result: " + output + "\n";
assertTrue(errLog, output.contains(accountType));
Where output is retrieved from the application during runtime and acctName is some data got passed in.
The result I got is:
ERROR! Account Name not found in result Expected Result: Name123 Actual Result: My name is Name456 type:junit.framework.AssertionFailedError
The result I expected is:
ERROR! Account Name not found in result
Expected Result: Name123
Actual Result: My name is Name456
type:junit.framework.AssertionFailedError
I've looked up everywhere online but it seemed like every example I read is only for output single line message. So, is it not possible to do multi-line messages using the existing assert functions? I know I can probably rewrite the assert function to accomodate what I need but as a newbie in JUnit, I guess it doesn't hurt to ask around first. Thanks in advance
I suspect this might come down to your IDE - I just ran the following in IntelliJ IDEA 12:
#Test
public void testABC() {
assertTrue("This\nis\na\ntest", false);
}
and got back:
java.lang.AssertionError: This
is
a
test
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.assertTrue(Assert.java:43)
The simplest (and more readble) way would be to write a function which extracts the account type from output, and use that in an assert:
assertEquals(acctName, extractAccountNameFromOutput(output));
which would give you a standard JUnit output if the test fails. When a test fails, you want the output to be immediately understandable, so it's probably better not to change the default output style provided by JUnit.

How make JUnit print assertion results

How can I get the results of my JUnit assertions to be printed [to standard output]?
I have some tests like this:
#Test
public void test01()
{
Position p = getPositionAt('a', 1);
assertNotNull("a1 exists", p);
assertNotNull("figure exists a1", p.getFigure());
p = getPositionAt('a', 2);
assertNotNull("exists a2", p);
assertNull("figure exists a2", p.getFigure());
p = getPositionAt('b', 1);
assertNotNull("exists b1", p);
assertNull("figure exists b1", p.getFigure());
}
This is the printed output format I am hoping to get:
a1 exists -success
figure exists a1 -success
exists a2 -success
figure exists a2 -succcess
exists b1 -succcess
figure exists b1 -failed
Is there way to do this using runners and suites? Or does there exist any assertSuccess(), assertFailed() methods?
First, you have two issues not one. When an assertion fails, an AssertionError exception is thrown. This prevents any assertion past this point from being checked. To address this you need to use an ErrorCollector.
Second, I do not believe there is any way built in to JUnit to do this. However, you could implement your own methods that wrap the assertions:
public static void assertNotNull(String description, Object object){
try{
Assert.assertNotNull(description, object);
System.out.println(description + " - passed");
}catch(AssertionError e){
System.out.println(description + " - failed");
throw e;
}
}
All the assertXXX methods have a form that allows for displaying a String on error:
assertNotNull("exists a2", p); // prints "exists a2" if p is null
There is no particular value in printing a message on success.
EDIT
Junit typically provides 2 forms of an assert. To follow the example above, you can test for a null value in 1 of 2 ways:
assertNotNull(p)
or
assertNotNull("my message on failure", p)
The framework will print the error messages with no other effort required by you (it's provided by the framework).
To test for exceptions you would use the following pattern:
try{
someCall();
catch(Exception e){
fail(): // exception shouldn't happen, use assertTrue(true) if it should
}
Again, there are versions of these methods for adding a message
Check the API
One last resort option is to pair each assert with a corresponding System.out.println, though obviously that is less than ideal. Still, it will solve the problem if all else fails.
Existing Answers/Comments here contain enough info to understand how to print something based on JUnit assertions - but they also explain how doing so is probably not what you actually want to do, and is probably missing the point of running unit tests in the first place.
You should be viewing the results of the tests themselves, instead of trying to print something while you don't understand how/where to view test results themselves.
Now then how/where to view results themselves depends on how you are running your tests - you need to understand how you are running your tests, and then research how to view test results according to how you are running them. Here are a few (but not limited to) examples:
Running tests in IntelliJ
Running tests in Eclipse
Running tests on command line
Running tests in Jenkins

WebDriver + TestNG - How to handle test results

I'm quite new to WebDriver and TestNG framework. I've started with a project that does a regression test of an e-commerce website. I'm done with the login and registration and so on. But there is something that I don't quite understand.
Example, I have this easy code that searches for a product.
driver.get(url + "/k/k.aspx");
driver.findElement(By.id("q")).clear();
driver.findElement(By.id("q")).sendKeys("xxxx"); //TODO: Make this dynamic
driver.findElement(By.cssSelector("input.submit")).click();
Now I want to check if xxxx is represented on the page. This can be done with
webdriver.findElement(By.cssSelector("BODY")).getText().matches("^[\\s\\S]*xxxxxx[\\s\\S]*$")
I store this in a Boolean and check if its true or false.
Now to the question, based on this Boolean value I want to say that the test result is success or fail. How can I do that? What triggers a testNG test to fail?
TestNG or any other testing tool decides success or failure of a test based on assertion.
Assert.assertEquals(actualVal, expectedVal);
So if actualVal and expectedVal are same then test will pass else it will fail.
Similarly you will find other assertion options if you using any IDE like Eclipse.
If you want to stop your test execution based on the verification of that text value, then you can use Asserts. However, if you want to log the outcome of the test as a failure and carry on, you should try using soft assertions, which log the verification as passed or failed and continue with the test. Latest Testng comes equipped to handle this - info at Cedric's blog
write this code where your if condition fails
throw new RuntimeException("XXXX not found: ");
u can use throw exception, and each method which will cal this meth should also throw Excetion after method name or you can use try catch. sample:
protected Boolean AssertIsCorrectURL(String exedctedURL) throws Exception {
String errMsg = String.format("Actual URL page: '%s'. Expected URL page: '%s'",
this.driver.getCurrentUrl(), exedctedURL);
throw new Exception(errMsg);
}
You can do this.
boolean result = webdriver.findElement(By.cssSelector("BODY")).getText().matches("^[\s\S]xxxxxx[\s\S]$")
Assert.assertTrue(result);

Additional logging JBehave

The scenario is this:
We are using JBehave and Selenium for system, integration and end to end testing.
I am checking the results of a calculation on a page with in excess of 20 values to validate.
Using Junit Assert the entire test will fail on the first instance of one of the values being incorrect. What I wanted to do was that if an assertion failure is met then the test continues to execute so that I can then collate all of the values that are incorrect in one test run rather than multiple test runs.
To do this I capture the assertions and write out to a log file anything that fails the validation. This has left me with a couple of issues:
1) The log file where I write out the assertions failures do not contain the name of the JBehave Story or Scenario that was being run when the exception occurred.
2) The JBehave Story or Scenario is listed as having 'Passed' and I want it to be listed as 'Failed'.
Is there any way that I can either log the name of the Story and Scenario out to the additional log file OR get the additional logging written to the JBehave log file?
How can I get the Story / Scenario marked as failed?
In the JBehave configuration I have:
configuredEmbedder()
.embedderControls()
.doIgnoreFailureInStories(true)
.doIgnoreFailureInView(false)
.doVerboseFailures(true)
.useStoryTimeoutInSecs(appSet.getMaxRunningTime());
and
.useStoryReporterBuilder(
new StoryReporterBuilder()
.withDefaultFormats()
.withViewResources(viewResources)
.withFormats(Format.HTML, Format.CONSOLE)
.withFailureTrace(true)
.withFailureTraceCompression(true)
.withRelativeDirectory("jbehave/" + appSet.getApplication())
Yes, you can create your own StoryReporter:
public class MyStoryReporter implements org.jbehave.core.reporters.StoryReporter{
private Log log = ...
#Override
public void successful(String step) {
log.info(">>successStep:" + step);
}
#Override
public void failed(String step, Throwable cause) {
log.error(">>error:" + step + ", reason:" + cause);
}
...
}
and register it like this:
.useStoryReporterBuilder(
new StoryReporterBuilder()
.withReporters(new MyStoryReporter())
..

Categories