We Keep Coding

TestNG reporting issues due to multi-threading

I have a testNG framework integrated with testrails. Because of the restrictions around the testrail api, the framework was designed to collate test result in bulk and upload them all at once when the test run is complete.
To do so, I've created a BaseTest class that provides a variable id that each test method can set itself to match a corresponding test case in test rails. Once the test method assigns this variable, we pack it in to the result object:
public abstract class BaseTest {
protected static final ThreadLocal<Integer> testrailIds = new ThreadLocal();
protected int testRailCaseId = -1;
//TODO: set down a clear and strong process for handling environment/domain. Once that is in place, report it
// loudly so it is clear where the tests ran
protected String baseUrl = "";
protected static final Logger LOG = Logger.getLogger(BaseTest.class);
/**
* records the case ID stored at the individual test level inside the result set for the
* test so that we can access it later when reporting our results
* #param result the result context object
*/
#AfterMethod(alwaysRun = true)
public void afterMethod(ITestResult result) {
if(testRailCaseId == -1) {
LOG.warn("NO CASE ID HAS BEEN SET FOR THIS TEST");
}
result.setAttribute("case_id", testrailIds.get());
}
Once all tests have executed we build a request object in the afterSuite method and pipe both the test cases and test results to testrail.
for(ISuiteResult suiteResult: suite.getResults().values()) {
ctx = suiteResult.getTestContext();
for (ITestResult result : ctx.getPassedTests().getAllResults()) {
cases.add((int) result.getAttribute("case_id"));
JSONObject resultJson = new JSONObject();
resultJson.put("case_id", result.getAttribute("case_id"));
resultJson.put("status_id", 1);
payload.add(resultJson);
}
for (ITestResult result : ctx.getFailedTests().getAllResults()) {
cases.add((int) result.getAttribute("case_id"));
JSONObject resultJson = new JSONObject();
resultJson.put("case_id", result.getAttribute("case_id"));
resultJson.put("status_id", 5);
payload.add(resultJson);
}
}
// get a clean instance of the api
TestRailApi tr = new TestRailApi();
//now dump that arraylist into a json param and add it to the test run
tr.updateRun(runId, cases);
//once the test run has been created, clean up again and build the results request
tr = new TestRailApi();
tr.addResultsForCases(runId, payload);
the testRailCaseId is set at the beginning of each test method with a simple assignment
this.testRailCaseId = 491;
or
testrailIds.set(489);
This worked fine until we started using multi-threading. Now, the value of testRaidCaseId is being overwritten by parallel tests, resulting in smaller result sets than expected.
I've been attempting to manage the threads through a ThreadLocal (as seen in the code above), but have been unsuccessful so far -- values I try to set in the before method or in the tests are coming up empty in the after methods.
The test methods themselves are fine, my only struggle is with shared content being passed into them from the parent.
Anyone have any guidance for how to manage my variables across the baseTest through the test methods to ensure my various ids don't clobber each other?
Sample test case:
#Test
#Parameters({ "domain", "username", "password" })
public void logInAuthEmptyToken(#Optional("http://REDACTED.com") String domain, String username, String password) {
this.testRailCaseId = 385;
Map<String, String> loginInfo = BaseAuthTests.login(domain, username, password);
AuthServiceApi auth = new AuthServiceApi(domain);
auth.addAuthTokens("", loginInfo.get("arrival_key"), loginInfo.get("profile_key"));
auth.executeLogin(400);
}

Sonarqube gives me issues of deleted codes and can't filter issues

I'm trying to get Issues from a fileName on sonarqube with this code
public class SonarqubeServiceImpl implements SonarqubeService {
private final String SONAR_URL = "http://localhost:9000/";
private final String PROJECT_KEY = "refactor2";
private final String SRC_FOLDER = "src/main/java/com/uca/refactor2/activities";
private String EXECUTE_SONAR;
private String GET_ISSUES_URL = SONAR_URL + "api/issues/search?q=";
private static String POM_PATH = "pom.xml";
private RestTemplate restTemplate = new RestTemplate();
private InvocationRequest request;
private Invoker invoker;
#PostConstruct
private void init() {
System.out.println("PostConstruct");
buildSonarCommand();
configureMavenWithSonar();
}
#Override
public void runAnalysis() throws MavenInvocationException {
// Assuming everything is set up (done in init)
invoker.execute(request);
}
#Override
public IssuesResponse getIssuesFromFileName(String fileName) {
String URL = GET_ISSUES_URL + fileName;
return restTemplate.getForObject(URL, IssuesResponse.class);
}
private void configureMavenWithSonar() {
request = new DefaultInvocationRequest();
request.setPomFile(new File(POM_PATH));
request.setGoals(Collections.singletonList(EXECUTE_SONAR));
invoker = new DefaultInvoker();
// Set maven home in case env variables are not set
// (and using own installation)
invoker.setMavenHome(new File("apache-maven-3.5.2"));
}
private void buildSonarCommand() {
StringBuilder builder = new StringBuilder();
builder.append("sonar:sonar ");
builder.append(String.format("-Dsonar.host.url=%s ", SONAR_URL));
builder.append(String.format("-Dsonar.projectKey=%s ", PROJECT_KEY));
builder.append(String.format("-Dsonar.sources=%s ", SRC_FOLDER));
EXECUTE_SONAR = builder.toString();
}
}
I'm having two problem with this code. First one is that this is retrieving issues from filenames of source folder that were deleted long time ago and I don't understand why, because if I enter to sonarqube local web api on localhost/9000 that code is not there
and second and most important, I'm trying to get Issues from a filename but sonarqube is giving me issues from all the project(even projects when I had more than one), including deleted codes from the first problem I said above.
I'm fetching issues with this URL
http://localhost:9000/api/issues/search?q=" + fileName;
I'm using SonarQube version: 6.7.1
this is my first time with sonarqube so maybe I'm missing something

Where did you see that q is a valid parameter? It's not in the list I'm looking at, which is probably why you're getting random results.
You need to be using componentKeys instead, altho the right-hand side is going to be more than just the file name; instead it will be something like projectKey:path/to/file so you'll need to work out the details of that.
Regarding the issues you're getting back on deleted files, closed issues are cleaned out of the database 30 days (default value) later. Since your query is currently pulling a random set of issues, unconstrained by status, that explains seeing these "ghost" issues.

Kie Workbench remove existing facts from session

I am new to Kie Workbench. I am using Java Rest calls to fire rules in kie workbench. Please find the code below:
public class RuleEngineConnector {
#Value("${brms.execution.server.url}")
private String SERVER_URL;
#Value("${brms.execution.server.username}")
private String USER;
#Value("${brms.execution.server.password}")
private String PASSWORD;
#Value("${brms.containerId}")
private String CONTAINER_ID;
private static final MarshallingFormat FORMAT = MarshallingFormat.JAXB;
public String getAdapter(AdapterRuleDO adapterRule) {
KieServicesConfiguration cfg = KieServicesFactory.newRestConfiguration(SERVER_URL, USER, PASSWORD);
cfg.setMarshallingFormat(FORMAT);
Set<Class<?>> extraClasses = new HashSet<Class<?>>();
extraClasses.add(AdapterRuleDO.class);
cfg.addJaxbClasses(extraClasses);
KieServicesClient kieServicesClient = KieServicesFactory.newKieServicesClient(cfg);
ServiceResponse<ExecutionResults> response = getRulesResponse(adapterRule, kieServicesClient);
List<AdapterRuleDO> listOfObjects = (List<AdapterRuleDO>) response.getResult().getValue("get-adapter");//to be changed
return listOfObjects.get(0).getAdapterName();
}
private ServiceResponse<ExecutionResults> getRulesResponse(AdapterRuleDO adapterRule, KieServicesClient kieServicesClient) {
List<Command<?>> commands = new ArrayList<Command<?>>();
KieCommands commandsFactory = KieServices.Factory.get().getCommands();
commands.add(commandsFactory.newInsert(adapterRule, "adapterRule"));
commands.add(commandsFactory.newFireAllRules());
commands.add(commandsFactory.newGetObjects("get-adapter"));
BatchExecutionCommand batchExecution = commandsFactory.newBatchExecution(commands);
RuleServicesClient ruleServicesClient = kieServicesClient.getServicesClient(RuleServicesClient.class);
ServiceResponse<ExecutionResults> response = ruleServicesClient.executeCommandsWithResults(CONTAINER_ID, batchExecution);
return response;
}
}
I am getting the rules fired correctly and the values are getting properly updated in AdapterRuleDO class after the rule is fired. One problem is when I again call this method to execute rules for the second time, I receive two AdapterRuleDO object and for each subsequent calls I get one additional object. It seems that the objects in the session are stored and not getting cleared for each call. How can I achieve that for every call I will get only one AdapterRuleDO object in return?
Please note I have only one such decision table where this fact has been used.

After searching different blogs and the user forums got a solution which worked fine.The above problem can be resolved by the following steps:
1) Use the "adapterRule" to get the result instead of "get-adapter".
2) In KIE Workbench, search for deployment descriptor and make this following change:
<runtime-strategy>PER_REQUEST</runtime-strategy>
By default, runtime strategy is SINGLETON.
Hope this makes sense and help somebody out.

If you are interested in stateless evaluation try to configure your session as stateless. This will create a new session for each request.
You should be able to do this in the kie-workbench.
Hope it helps,

In place of below line:
BatchExecutionCommand batchExecution = commandsFactory.newBatchExecution(commands);
Use this line:
BatchExecutionCommand batchExecution = commandsFactory.newBatchExecution(commands,Ksession_name);

Issue when executing asynchronous tasks using ExecutorService

I had asked a question earlier
regarding ExecutorService and Apache Velocity initialization. To give a quick recap -- I have a Java EE frontend which accepts user requests and then for each of these requests, uses ExecutorService(SingleThreadedExecutor set as a daemon) to kick off a lengthy workflow.This workflow is contained in a library and works well and as expected when run in a standalone mode through eclipse. When called from the website(servlet) I observed that the workflows were consistently getting hung at the point where the Velocity Engine was being initialized (Velocity.init() or ve.init()). Hence my aforementioned question.
When none of the answers/suggestions worked i inferred that this was something to do with the way Velocity started up and decided to shift to FreeMarker. Now i see that the workflow is getting hung in the exact same place for the FreeMarker implementation as well. This 'place' is the mail-building part which evaluates a template against a coupel of passed data objects and returns the mail string.The class that calls the Freemark'ing class and the FreeMark class are as follows --
public class mailBuilder {
private static final Logger log = Logger.getLogger( mailBuilder.class );
static String a;
static String b;
public mailBuilder(CustomDataStructure input)
{
a = input.getA();
b = input.getB();
}
public static String returnMailstring() throws Exception
{
log.info("Gathering elements to construct email.");
String mailText=null;
Map context = new HashMap();
context.put("a",a);
context.put("b",b);
log.info("Calling Freemarker");
mailText=FreeMarkIT.ReturnReportString(context);
log.info("Freeemarker returned string");
return mailText;
}
}
FreeMarkIT class is as follows --
public class FreeMarkIT {
private static final Logger log = Logger.getLogger( FreeMarkIT.class );
private static Configuration config;
private static Template template;
public static String ReturnReportString(Map model) throws IOException, TemplateException
{
StringWriter sw = new StringWriter();
try
{
log.info("Going to get the template");
config= new Configuration();
log.info("Now really");
template=config.getTemplate("src/resource/email_template.vm");
log.info("Done initializing template");
template.process(model, sw);
sw.flush();
}
catch(Exception e)
{
System.out.println(e.getMessage());
}
return sw.getBuffer().toString();
}
}
Now, from my logging it looks like the worker thread hangs at the line
config=new Configuration()
Again, this works as expected in a standalone mode when run from eclipse but however hangs when called from the servlet using ExecutorService.
Im beginning to think/realize that this may have nothing to do with either of Velocity or FreeMarker and have something to do with ExecutorService.
Any advice or suggestion would be of immense help.
Thanks

your code is not thread-safe since you are sharing config and template across all thread instances (and continuously re-setting them). the easiest way to make it thread safe would be to make config and template local variables in the method instead of static members. as #JBNizet pointed out in the comments, you have a similar problem in mailBuilder with a and b. you might want to first check out some tutorials on object oriented programming fundamentals and then circle back to this issue (hint, in general you should avoid static member variables except for constants).

how to invoke schedular every day

what i do is ,when i run first time a servlet (which is invoked from jsp) that while put an entry of that service,daily in conf file.i want to run a scheduler which will invoke program(servlet- which runs and send mail) for that service daily 10 .
below is the code i use to execute a task.but problem is when i stop the server ,the scheduler stops and nothing happens
public class Schedule
{
public static final String CONF_PATH = "../webapps/selen/WEB-INF/scheduling.properties";
public static Properties schProps = null;
public static FileInputStream sis = null;
public static long period;
public static Timer timer = new Timer();
public static String servicename = null;
public static String keyValues = null;
public static String reValues[] = null;
public static String schedulingValue = null;
public static String service_url = null;
public static String browserlist = null;
public static String testType = null;
public static String mailCheacked = null;
public static String toaddr = null;
public static HttpServletRequest request = null;
public static HttpServletResponse response = null;
public static String serversURL = null;
public static String contextPath = null;
public static Date delay = null;
public void scheduleLoad(String serviceValue) throws Exception
{
try
{
schProps = new Properties();
sis = new FileInputStream(CONF_PATH);
schProps.load(sis);
servicename = SServlet.serviceName;
keyValues = schProps.getProperty(serviceValue);
reValues = keyValues.split(",");
String request = reValues[0];
String response = reValues[1];
schedulingValue = reValues[2];
service_url = reValues[3];
browserlist = reValues[4];
testType = reValues[5];
mailCheacked = reValues[6];
toaddr = reValues[7];
serversURL = reValues[8];
contextPath = reValues[9];
if(reValues[2].equals("Daily"))
{
Calendar cal =Calendar.getInstance();
cal.set(Calendar.HOUR,10);
cal.set(Calendar.MINUTE,20);
cal.set(Calendar.SECOND,0);
delay = cal.getTime();
period = 1000 * 60 * 60 * 24;
schedule();
}
else if(reValues[2].equals("Stop"))
{
stop();
}
}
catch(NullPointerException npe)
{
System.out.println("null point exception ");
}
finally
{
if(sis !=null)
{
sis.close();
}
}
}
public static void schedule()
{
MyTimerTask mt = new MyTimerTask(request,response,servicename,service_url,browserlist,mailCheacked,testType,schedulingValue,toaddr,serversURL,contextPath);
timer.schedule(mt,delay,period);
}
public static void stop()
{
timer.cancel();
}
}
class MyTimerTask extends TimerTask
{
public HttpServletRequest request;
public HttpServletResponse response;
public String servicename;
public String service_url;
public String browserlist;
public String mailCheacked;
public String testType;
public String schedulingValue;
public String toaddr;
public String serversURL;
public String contextPath;
public MyTimerTask(HttpServletRequest request,HttpServletResponse response, String servicename,String service_url,String browserlist,String mailCheacked,String testType,String schedulingValue,String toaddr,String serversURL, String contextPath)
{
this.request = request;
this.response = response;
this.servicename = servicename;
this.service_url = service_url;
this.browserlist = browserlist;
this.mailCheacked = mailCheacked;
this.testType = testType;
this.schedulingValue = schedulingValue;
this.toaddr = toaddr;
this.serversURL = serversURL;
this.contextPath = contextPath;
}
public void run()
{
SServlet sservlet = new SServlet();
sservlet.sServerloading(request,response,servicename,service_url,browserlist,mailCheacked,testType,schedulingValue,toaddr,false,1,serversURL,contextPath);
}
}

The JDK Timer runs in the JVM, not in the operating system. It's not CRON or Windows scheduler. So when you stop your server (Tomcat? JBoss? Glassfish?), you are effectivly stopping the JVM that the Timer lives in so of course it won't run any more. If you want a timer (scheduler) that runs independently of your server, you will have to start it in it's own JVM, either as a standalone java program using the java command or inside another server instance.
On a side note, if you're open to some critique, a small review of your code:
Avoid mixing static and non-static contexts if possible. Your Schedule class instance method scheduleLoad() makes heavy use of static member variables for statefull storage. Variables are either only used in the execution of a method (in which case they should be declared inside that method) or they are used to describe the state of an object (in which case they should be private instance members of the class) or they are global constants or immutable global variables (in which case they should be declared static final). Exceptions to these exist, but are less common.
Avoid declaring member variables public if they are not also final. Adhere to the JavaBean pattern, use getters and setters. If a variable is, in reality, a constant then it should be public static final.
Avoid using classes or parameters out of scope. For instance, your MyTimerTask uses HttpServletRequest and HttpServletResponse as member variables and method parameters. This makes no sense as MyTimerTask is not used in the scope of a servlet request (and will subsequently always be null, right?). Or, if that is indeed the case, if you are explicitly setting the static members of the Schedule in some servlet and then invoking scheduleLoad(), see my first point about improper use of static context. Your code would not be thread-safe and concurrent invocation of whichever servlet that uses the Schedule would produce unpredictable behaviour.
UPDATE:
It's hard to know where to start as I'm not sure what your level of expertise is in Java. If you are unfamiliar with how to execute stand-alone java applications, I would suggest having a go at some tutorials. Oracle has a bunch at http://download.oracle.com/javase/tutorial/. http://download.oracle.com/javase/tutorial/getStarted/index.html is a good place to start as it walks you through a very basic "hello world" type application with a main method and how to execute it using the java command, as well as some common mistakes and problems.
Once you've figured all that out, take a few minutes to figure out what your application should do, which resources it will require and if it needs to call any "external" systems. You mentioned that it should "execute a servlet to send mail". Does that mean that it has to call a specific servlet or is it just the sending mail that is really what you are after. In that case, maybe you can just move all the mail sending logic to your standalone program? If not, you will have to call the servlet using a http request (like a browser would). There are a number of existing frameworks for doing things like that. Apache HttpClient is a very popular one.

If you stop program it does not work. It is not a bug. It is a feature. BTW if you shutdown your computer nothing happens too :).
But If you questing is how to make my scheduled task more robust, e.g. how to make task to continue working when server stops and then starts again you have to persist somewhere the state of you scheduler, i.e. in your case the last time of task execution. You can implement this yourself: create special file and store the data there. You can use cross platform pure java Preferences API to do this: the data will be stored in file system in Unix and Registry in windows. you can save state in DB too.
But you can use other products that have already implemented this functionality. The most popular and well-known is Quartz.
But Quartz still need some java process to be up and running. If you want to be able to run your tasks even if no java process is running use platform dependent tools: cron tab for Unix and scheduler API for windows (it is accessible via VBScript, JScript, command line).
Unix has cron