We Keep Coding

TestNG reporting issues due to multi-threading

I have a testNG framework integrated with testrails. Because of the restrictions around the testrail api, the framework was designed to collate test result in bulk and upload them all at once when the test run is complete.
To do so, I've created a BaseTest class that provides a variable id that each test method can set itself to match a corresponding test case in test rails. Once the test method assigns this variable, we pack it in to the result object:
public abstract class BaseTest {
protected static final ThreadLocal<Integer> testrailIds = new ThreadLocal();
protected int testRailCaseId = -1;
//TODO: set down a clear and strong process for handling environment/domain. Once that is in place, report it
// loudly so it is clear where the tests ran
protected String baseUrl = "";
protected static final Logger LOG = Logger.getLogger(BaseTest.class);
/**
* records the case ID stored at the individual test level inside the result set for the
* test so that we can access it later when reporting our results
* #param result the result context object
*/
#AfterMethod(alwaysRun = true)
public void afterMethod(ITestResult result) {
if(testRailCaseId == -1) {
LOG.warn("NO CASE ID HAS BEEN SET FOR THIS TEST");
}
result.setAttribute("case_id", testrailIds.get());
}
Once all tests have executed we build a request object in the afterSuite method and pipe both the test cases and test results to testrail.
for(ISuiteResult suiteResult: suite.getResults().values()) {
ctx = suiteResult.getTestContext();
for (ITestResult result : ctx.getPassedTests().getAllResults()) {
cases.add((int) result.getAttribute("case_id"));
JSONObject resultJson = new JSONObject();
resultJson.put("case_id", result.getAttribute("case_id"));
resultJson.put("status_id", 1);
payload.add(resultJson);
}
for (ITestResult result : ctx.getFailedTests().getAllResults()) {
cases.add((int) result.getAttribute("case_id"));
JSONObject resultJson = new JSONObject();
resultJson.put("case_id", result.getAttribute("case_id"));
resultJson.put("status_id", 5);
payload.add(resultJson);
}
}
// get a clean instance of the api
TestRailApi tr = new TestRailApi();
//now dump that arraylist into a json param and add it to the test run
tr.updateRun(runId, cases);
//once the test run has been created, clean up again and build the results request
tr = new TestRailApi();
tr.addResultsForCases(runId, payload);
the testRailCaseId is set at the beginning of each test method with a simple assignment
this.testRailCaseId = 491;
or
testrailIds.set(489);
This worked fine until we started using multi-threading. Now, the value of testRaidCaseId is being overwritten by parallel tests, resulting in smaller result sets than expected.
I've been attempting to manage the threads through a ThreadLocal (as seen in the code above), but have been unsuccessful so far -- values I try to set in the before method or in the tests are coming up empty in the after methods.
The test methods themselves are fine, my only struggle is with shared content being passed into them from the parent.
Anyone have any guidance for how to manage my variables across the baseTest through the test methods to ensure my various ids don't clobber each other?
Sample test case:
#Test
#Parameters({ "domain", "username", "password" })
public void logInAuthEmptyToken(#Optional("http://REDACTED.com") String domain, String username, String password) {
this.testRailCaseId = 385;
Map<String, String> loginInfo = BaseAuthTests.login(domain, username, password);
AuthServiceApi auth = new AuthServiceApi(domain);
auth.addAuthTokens("", loginInfo.get("arrival_key"), loginInfo.get("profile_key"));
auth.executeLogin(400);
}

How to write junit for this code?

How do I call populateMapWithFormattedDates method in JUnit and how to write JUnit populateMapWithFormattedDates for this method. I dont know how to write JUnit for nested methods so kindly help.
protected Map<String, String> populateDispatch(final RequestDispatchData requestDispatchData)
{
final Map<String, String> map = getDispatchFieldMapper().populateMapper(requestDispatchData);
populateMapWithFormattedDates(requestDispatchData, map);
}
private void populateMapWithFormattedDates(final RequestDispatchData requestDispatchData, final Map<String, String> map)
{
String dateFormatted = map.get("ticket_date");
Date date = null;
try
{
date = new SimpleDateFormat("MM/dd/yy").parse(dateFormatted);
}
catch (ParseException parseException)
{
customLogger.logMessage(diagnosticMethodSignature, DiagnosticType.EXCEPTION,
"Exception in parsing start date of ticket " + parseException);
}
map.put("startDateDDMMYY", DateEnum.DDMMYY.getFormattor().format(date));
map.put("startDateDDMMMYY", DateEnum.DDMMMYY.getFormattor().format(date));
map.put("startDateDMY", DateEnum.DMY.getFormattor().format(date));
map.put("startDateYYMMDD", DateEnum.YYMMDD.getFormattor().format(date));
}

Simple: you don't test private methods directly.
Instead, you focus on the "public contract" of those methods that get invoked "from the outside". In your case, that would be:
Map<String, String> populateDispatch(...
Thus you want write tests like:
#Test
public void populateDispatchForValidDate() {
RequestDispatchData request = ...
Map<String, String> actualOutput = underTest.populateDispatch(request);
assertThat(actualOutput.size(), is(5));
}
The above is just meant as an example. What it does:
create a "request" object. This could be a mock; or a real object - depends on what exactly your various methods are doing with this object. And how easy it is to create a "real" RequestDispatchData object with "test data"
it invokes that method under test
it asserts one/several properties of the result coming back
Looking at your production code, that code is doing way too many things within that single method. You might want to read about "clean code" and improve that code. That probably lead to the creation of some helper classes which would be easier to test then.

There is nothing such as a nested method in Java. It's a nested function call is what it is. Plus, yea, you cannot call the private functions of a class through its object so testing them individually by calling them is not possible.
You can although have a public or protected function doing the call somewhat like a getter.

I believe your code is some what like,
protected Map<String, String> populateDispatch(final RequestDispatchData requestDispatchData)
{
final Map<String, String> map = getDispatchFieldMapper().populateMapper(requestDispatchData);
return populateMapWithFormattedDates(requestDispatchData, map);
}
note that you have missed the return statement, and update the map on certain condition from ,
private void populateMapWithFormattedDates(final RequestDispatchData requestDispatchData, final Map<String, String> map)
{
// Map manipulation here
}
So if you have minimum dependency on the getDispatchFieldMapper().populateMapper(), then you can directly invoke populateDispatch() from your test code, else you may have to find a way to inject a custom implementation of DispatchFieldMapper to prepare the map for testing your target method.
Injection of DispatchFieldMapper can be via overriding the getDispatchFieldMapper() or use a setDispatchFieldMapper() on your class.
While preparing your custom DispatchFieldMapper, make sure the populateMapper() returns a map with all data required for your testing.

It is not good idea to call non accessible method while testing directly form the test class.
Second thing : Non accessible method is always called form some accessible method or scope otherwise that code is dead code just remove that.
Because method is privet, so if it is in use then it called somewhere from code of current class. in your code it called form populateDispatch, so actual way to write test case for populateMapWithFormattedDates method is cover all the scenarios for populateDispatch method and populateDispatch is also used form sub class of the current class call it form there.
But you can call private method in junit like:
Deencapsulation.invoke(<object of class in called method is exist>, "populateMapWithFormattedDates", <object of RequestDispatchData class>, <object of Map<String, String> class>);
Again it is a way to call private method but you should not use this...

You should decouple the populateMapWithFormattedDates method like this:
// I created an utility class but it's a suggestion.
// I'm using an util class because you don't use requestDispatchData for
// anything. But if you do, maybe it's a good idea to implement this code
// on RequestDispatchData class
class DispatchMapUtils {
// Note that I took of the requestDispatchData
public static Map<String, String> populateMapWithFormattedDates(final Map<String, String> map) throws ParseException {
// Your code without try-catch.
// Throw the exception to the caller of this method
// and try-catch there to use the customLogger
}
}
With this code, your test would be something like this:
#Test
public void shouldFormatTicketDateInVariousFormat() {
Map<String, String> map;
// Instantiate and put some initial datas
map = new ...
map.put('ticket_date') = ..
// Call the method!
DispatchMapUtils.populateMapWithFormattedDates(map);
// Do the assertions!
Assert.assertTrue(map.get("startDateDDMMYY").equals(...));
}
#Test
public void shouldThrowExceptionWhenTicketDateIsInvalid() {
// More testing code
}

How to mock a local variable with PowerMockito?

I want to mock "source" when the method "ProductAdapterService.adapt" is called by other class.
How to deal that? I really tried a lot of ways. Please help me. I am a new guy.Thanks a lot!
public class ProductAdapterService {
private final SearchParameter parameter;
private List<Festival> festivals;
public ProductAdapterService(SearchParameter parameter) {
this.parameter = parameter;
}
public SingleProduct adapt(SearchHit hit, boolean bidding) {
//I want to mock "source", I don't want to use "hit.getSource()"
Map<String, Object> source = hit.getSource();
SingleProduct sp = new SingleProduct();
sp.setId(TypeConverter.toInt(source.get(FieldName.PRODUCT_ID)));
sp.setName(TypeConverter.toString(source.get(FieldName.NAME)));
sp.setPrice(this.price(source.get(FieldName.PRICE), source.get(FieldName.PRICE_MAP), source.get(FieldName.FIRST_START_CITIES)));
sp.setLevel(TypeConverter.toInt(source.get(FieldName.PRODUCT_LEVEL)));
sp.setDepartureCityId(this.departureCity(source.get(FieldName.DEPARTURE_CITY_ID), source.get(FieldName.FIRST_START_CITIES)));
sp.setSaleMode(TypeConverter.toString(source.get(FieldName.SALE_MODE)));
sp.setBrandName(this.providerBrandName(source.get(FieldName.PROVIDER_BRAND)));
sp.setSaleCount(TypeConverter.toInt(source.get(FieldName.MONTHLY_ORDER)));
sp.setCommentCount(TypeConverter.toInt(source.get(FieldName.COMMENT_COUNT)));
sp.setCommentScore(TypeConverter.toFloat(source.get(FieldName.COMMENT_SCORE)));
sp.setBuType(BuType.GT);
sp.setType(this.productType(source.get(FieldName.SEARCH_TAB_TYPE_SHOW), sp.getSaleMode()));
sp.setSaleout(this.saleout(source.get(FieldName.NON_SALEOUT_CITIES), sp.getDepartureCityId()));
if (!sp.isSaleout()) {
sp.setFestival(this.festival(source.get(FieldName.FESTIVAL_IDS)));
}
System.out.println("sp.getName(): " + sp.getName());
return sp;
}}
And below is my test code:
public class TabSearcherTest0 {
#Test
public void test() {
SearchParameter parameter = SearchParameter.create();
Ghost.begin();
parameter.getFiltered().setTab(TabType.ALL);
parameter.getPoi().setKeyword("Spa");
parameter.getClient().setTrace(TraceMode.MAIN);
Map<String, Object> mapMock = new HashMap<String, Object>();
mapMock.put("productgroupid", "12877");
mapMock.put("productid", "5539739");
mapMock.put("firststartcitys", "[1, 2]");
mapMock.put("nonsaleoutcities", "[1, 2]");
mapMock.put("productdiamondlevel", "4");
mapMock.put("commentcount", "0");
mapMock.put("price", "0.0");
mapMock.put("name", "TestName");
mapMock.put("searchtabtypeshow", "1");
mapMock.put("comment", "0.0");
mapMock.put("salemode", "S");
mapMock.put("providerbrandid", "999999");
mapMock.put("departurecityid", "2");
// how to inject the map?
// ???
SearchModel model = SearchContext.createContext(parameter).search();
Ghost.end();
System.out.println(model);
}}

You are getting "mocking" the wrong way. You only used it when you can not use the real class implementation; but you need to control how some object reacts to methods calls to it.
Your method to test looks like:
public SingleProduct adapt(SearchHit hit, boolean bidding) {
//I want to mock "source", I don't want to use "hit.getSource()"
Map<String, Object> source = hit.getSource();
Wrong: you want to make sure that hit.getSource() is used. Because your production code is using is; and you write your unit tests to text that code. So you want that your production code does its "normal" thing.
So, the very simply solution here is:
#Test
public void testAdapt() {
SearchHit mockedHit = mock(SearchHit.class);
Map<String, Object> resonseForGetSource = new HashMap<>();
resonseForGetSource.put("productgroupid", "12877");
...
doReturn(resonseForGetSource).when(mockedHit.getSource());
ProductAdapterService underTest = ...
underTest.adapt(mockedHit, true);
... probably some asserts
or something alike (don't nail me on the doReturn/when details here)
What you can see here: your production code needs that map to do its job; so you just make sure that such a map object shows up in your production code.
And in case it would be possible to use a real SearchHit object (that you could configure with such a map); then using that would even be better than mocking that object.
You absolutely try to minimize your usage of mocking. You only use it to gain control over objects that are used during a certain test.
And beyond that: you are not clear about the scope of your unit testing. In order to test that one method, you dont need no ghosts. The unit test code you are showing simply doesn't make sense in the context of the class you are showing us here! Thus: you better step back and carefully look into "which units do I have" and "how to unit test exactly that unit X". You don't text "X" by testing "Y"!

Why doAnswer() stubs are not reset?

I have 2 tests which call the same code but check different things.
The first one uses doAnswer() to stub a method call and put parameters checker.
The second one doesn't need it.
But when I run my tests, the second test fails because fails the checker that was injected in the first test. And if I run only the second test it passes.
So, should I reset doAnswer() stub with empty checker im my setUp() method? Why so?
This is a checker that I use in the first test:
final Map<String, String> expectedParams = new HashMap<>();
expectedParams.put("param1", "A");
doAnswer(new Answer() {
public Object answer(InvocationOnMock invocation) {
Object[] args = invocation.getArguments();
String name = (String) args[0];
Assert.assertEquals("event1", name);
Map<String, String> params = (Map<String, String>) args[1];
Assert.assertEquals(expectedParams, params);
return null;
}}).when(mMyMockedObject).myTestedMethod(anyString(), anyMap());
and this is the beginning
#Mock
MyMockedClass mMyMockedObject;
#Before
public void setUp() throws Exception {
System.out.println("setUp()");
MockitoAnnotations.initMocks(this);
}
In the second test I never call doAnswer() but my checker is called I don't know why :(
Adding the whole test.
#RunWith(LocalRobolectricTestRunner.class)
#Config(manifest = Config.NONE)
public class MyTest {
private static final String MAIN = "main";
#Mock
MyMockedClass mMyMockedObject;
#Before
public void setUp() throws Exception {
System.out.println("setUp()");
MockitoAnnotations.initMocks(this);
// this will connect MAIN with mMyMockedObject
SomClassICantShareNda.someMethodNda(MAIN, mMyMockedObject);
}
Hey, this is where I understand that this is my fault, not Mockito.
This is where I add new mMyMockedObject but forget to erase old instance of mMyMockedObject (inside SomClassICantShareNda, in static collection). So, after each test new mMyMockedObject will be created and added to collection.

Sorry, that was my fault.
The code duplicated many mocked object, cached them in a static collection and didn't reset it in setUp()

How to test ehcache using mockito?

Im helper method use ehcache, to reduce queries to Db. Now want to implement JUnit+Mockito test to ensure that ehcache works properly. Have such variant of test:
#Autowired
private DBService service;
#Autowired
private DiscountHelper discountHelper;
#Autowired
private CacheManager cacheManager;
#Before
public void setUp() throws Exception {
assertNotNull(cacheManager);
}
#Test
public void testGetDiscountWithCache() throws RuntimeException,
InterruptedException {
String id1 = "id1";
String id2 = "id2";
String id3 = "id3";
List<String> discountsId = new ArrayList<String>();
discountsId.add(id1);
discountsId.add(id2);
discountsId.add(id3);
List<Map<String, Object>> attrList = new ArrayList<Map<String, Object>>();
attrList.add(new Discount().getAttributes());
attrList.add(new Discount().getAttributes());
attrList.add(new Discount().getAttributes());
Cache cache = cacheManager.getCache(DiscountHelper.CACHE_NAME);
assertNotNull(cache);
assertEquals(0, cache.getSize());
// First run with empty cache
when(service.getAllItems(eq(Discount.TABLE_NAME))).thenReturn(attrList);
List<Discount> actualResult = discountHelper
.getAllDiscountsUsingCache();
assertNotNull(actualResult);
assertEquals(attrList.size(), actualResult.size());
verify(service).getAllItems(eq(Discount.TABLE_NAME));
cache = cacheManager.getCache(DiscountHelper.CACHE_NAME);
// In cache should be 1 record
assertNotNull(cache);
assertEquals(1, cache.getSize());
}
And test method is:
#Cacheable(cacheName = CACHE_NAME, refreshInterval = 1000 * 900, decoratedCacheType = DecoratedCacheType.REFRESHING_SELF_POPULATING_CACHE)
public List<Discount> getAllDiscountsUsingCache() throws RuntimeException,
InterruptedException {
List<Map<String, Object>> result = dbService
.getAllItems(Discount.TABLE_NAME);
List<Discount> discountList = new ArrayList<Discount>();
for (Map<String, Object> entry : result) {
discountList.add(new Discount(entry));
}
return discountList;
}
And this perfectly works. In test i`m sure that after invocation of method I get something in cache. As you can see I also verify that was called method getAllItems in db service. That is good for first time invocation. Next I add second invocation of discountHelper.getAllDiscountsUsingCache() in the same test like this:
when(service.getAllItems(eq(Discount.TABLE_NAME))).thenReturn(attrList);
actualResult = discountHelper
.getAllDiscountsUsingCache();
assertNotNull(actualResult);
assertEquals(attrList.size(), actualResult.size());
verify(service, times(0)).getAllItems(eq(Discount.TABLE_NAME));
So I just want to check that on second invocation there will not be calls to DB service via method getAllItems by this verify: verify(service, times(0)).getAllItems(eq(Discount.TABLE_NAME)); and result will be obtain from cache.
But it doesn't work, I still get invocation of DB method. I find this tutorial http://blog.goyello.com/2010/07/29/quick-start-with-ehcache-annotations-for-spring/ and try variant with delegate object for ehcache testing, but it still invoke db methods on test. What`s wrong?
By the way in production ehcache works good as I see in logs of tomcat, so this is a problem of test. Any suggestions how to fix it?

In the second invocation you use the same mock object (service) created (as I assume from a question's tag) by Springockito during the Spring context initialization. The mock remembers getAllItems() call from the first invocation. You can:
reset the mock before the second invocation using reset(service),
or
check after the second invocation) if there is still one (not two) getAllItems() invocation.