We Keep Coding

Can we execute only failed test cases in Jenkins?

If we build and execute selenium (TestNG) suite in Jenkins and if some tests fails; after fixing is there any way to execute only those failed test cases in Jenkins?

Yes, from the docs:
Every time tests fail in a suite, TestNG creates a file called testng-failed.xml in the output directory. This XML file contains the necessary information to rerun only these methods that failed, allowing you to quickly reproduce the failures without having to run the entirety of your tests.
If I understand your use case, you are looking to save the list of failed tests, make changes to the code and re-execute that list, is that correct ? In that case you can store that testng-failed.xml file and use it for the next execution in Jenkins, possibly adding a checkbox to the job that lets you choose wether to use this test suite or the default one.

Please try below to run specific tests (example: testcases you have fixed) with mnetioning below include tag in testNG.xml
<classes>
<class name="test.IndividualMethodsTest">
<methods>
<include name="testMethod" />
</methods>
</class>
</classes>

From my QA and automation perspective, it's always better to run the entire suite from jenkins, if you want to check why they fail you can do it locally. the other option is to parameterize the xml but it is a lot of work, it could be by maven arguments and writing in the xml

Why in Jenkins?, you can build one Testng Retry Listener where it keeps polling the entire test execution and re-executes only Failed testcases.
Refer this below.
public class RetryFailedTestCases implements IRetryAnalyzer {
private int retryCnt = 0;
// Mentioned maxRetryCnt (Maximiun Retry Count) as per your requirement. Here I
// took 3, If any failed testcases then it runs two times
private int maxRetryCnt = 1;
// This method will be called everytime a test fails. It will return TRUE if a
// test fails and need to be retried, else it returns FALSE
public boolean retry(ITestResult result) {
if (retryCnt < maxRetryCnt) {
System.out.println("Retrying " + result.getName() + " again and the count is " + (retryCnt + 1));
retryCnt++;
return true;
}
return false;
}
}

Webdriver test pause unexpectedly

I have a Selenium WebDriver based testcase, which pauses during execution. It should upload thousands of files to a website. When it chooses the file to upload it sometimes doesn't click ok, but waits for manual interaction. In most cases it is working perfectly.
I use StringSelection to copy and paste file source to input field.
StringSelection cp = new StringSelection(fileSource);
Toolkit.getDefaultToolkit().getSystemClipboard().setContents(cp, null);

I think your test could be running to quickly? If this is the case, then you could potentially use WebDriverWait?? WebDriverWait could be used to wait for the 'OK' element to be visible prior to clicking and therefore proceeding.
I might be wrong, but I can't really tell what the issue is without the rest of the code.
Personally, I use the following method which I can then call
public void waitForElementToBeVisible(String cssSelector) throws Throwable {
try {
WebDriverWait wait = new WebDriverWait(driver, 30);
wait.until(ExpectedConditions.or(
ExpectedConditions.visibilityOfElementLocated(By.cssSelector(cssSelector))
));
} catch (Exception e) {
System.out.println("Timeout exceeded");
closeDriver();
}
}

For behavior "When it chooses the file to upload it sometimes doesn't click ok, but waits for manual interaction. In most cases it is working perfectly." I prefer use failed retry count. Every step with click should be wrapped up on the test and if test result=failed - retry test some times(3 or 5). JUnit have good mechanizm for that:
#RunWith(Runner.class)
public abstract class AbstractTest extends LibTest {
#Rule
public JUnitRetry retry = new JUnitRetry(3);
}
public class Test extends AbstractTest
#Test
public void testCp(String fileSource){
StringSelection cp = new StringSelection(fileSource);
Toolkit.getDefaultToolkit().getSystemClipboard().setContents(cp, null);
}
}

Below code working fine for the similar scenario in our environment.
StringSelection cp = new StringSelection(fileSource);
Toolkit.getDefaultToolkit().getSystemClipboard().setContents(cp, null);
Robot robot=new Robot();
robot.keyPress(KeyEvent.VK_ENTER);
robot.keyRelease(KeyEvent.VK_ENTER);
We can also use AutoIT to perform this type actions.
Please find AutoIT code to handle this case.
Download AutoIT ,write this code in AutoIT name it as 'Loadfromdisk' and compile. .exe will be generated, please place exe somwhere in your local drive(ex: E:\Loadfromdisk.exe)
AutoItSetOption("WinTitleMatchMode","2") ;
$title=WinGetTitle("[CLASS:DirectUIHWND; INSTANCE:2]")
WinActivate($title)
WinWaitActive($title)
If WinExists($title) Then
WinFlash($title,"", 4, 500) ;Just to Flash the window
EndIf
Sleep(1000)
ControlSetText($title, "", "Edit1", fileSource)
Sleep(1000)
ControlClick($title,"","Button1")
Exit
Load from disk Selenium Java code, this will load the file placed at 'filesource' path mentioned in AutoIT code into web application
String strAutoIT = "E:\\Loadfromdisk.exe";
Thread.sleep(3000);
String[] astrArg = null;
astrArg=new String[]{strAutoIT};
Runtime.getRuntime().exec(astrArg);
Please see whether this helps to run your testcase.

Selenium - TestNG - Running groups using Code

I'm making a GUI for use of TestNG, currently I have a dropdown box which uses then allows you to press a button which I want to run a group of tests
#Test(groups = {"Group1"})
public void Test()
//Test Data
#Test(groups = {"Group1"})
public void Test2()
//Test Data
#Test(groups = {"Group2"})
public void Test3()
//Test Data
I'm currently running this code via the run configurations and calling the group to run there.
Is there any way I can do this via a button press so the user can press run Group1 and then run Group2 in an executable program?

You can create a synthetic testng.xml in the Java code using the group inputs. But this might get messy with large size testng.xml.
You can also call testng from the command line (Have to figure out this from your GUI code) which has many parameters available like include groups, exclude groups etc etc.
java org.testng.TestNG testng1.xml [testng2.xml testng3.xml ...] -groups "grp1,grp2"
Refer to [http://testng.org/doc/documentation-main.html#running-testng}

You can make use of BeanShell feature that TestNG provides for doing this. You can define a JVM argument that reads the value set using the drop down selected option and then run it.
You can read more about using Beanshell and TestNG from this blog post that I created.

TestNG parallel test/method using dataprovoder

I have a testNG method just like this:
#Test(dataProvider="takeMyProvider")
public void myTest(String param1, String param2){
System.out.println(param1 + " " + param2);
}
My dataprovider returns 10 elements. My method will be executed 10 times in one thread. How is it possible to parallel this? For example
I want to have 5 methods in parallel. The webdriver should open 5 browsers at the same time. After these 5 tests in parallel the other 5 test should be executed
or
the webdriver should open 10 browsers and do all 10 elements parallel
Does anybody have an idea?

You can define the parallelism via a suite file in TestNG. Example following runs methods in parallel with 10 threads:
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="MySuiteNameHere" parallel="methods" thread-count="10">
<test name="Selenium Tests">
<classes>
<class name="foo.bar.FooTest"/>
</classes>
</test>
</suite>
You also need to note that your data provider can is thread safe to allow it to not force the method to run sequentially.
// data providers force single threaded by default
#DataProvider(name = "takeMyProvider", parallel = true)
Be careful, though. TestNG does not create new instances of the class object when running with parallel methods. That means that if you save values on the test class object you can run into threading issues.
Also note, if you set the thread count to 5, it does not wait for the first 5 to all be finished and then start up the next 5. It basically puts all the test methods into a queue and then starts up x threads. Each thread then simply polls the next element from the queue when it is available.

TestNG's #Test annotation already has what you want... To some degree:
// Execute 10 times with a pool of 5 threads
#Test(invocationCount = 10, threadPoolSize = 5)
What this won't do is fit your first scenario exactly, that is, run the first 5, wait for them to finish, run the other 5.

many thx for your feedback and useful tipps.
My tests ran - maybe - in any parallel way but only in one browser instance.
Lets jump in in detail:
My dataprovider returns an object[][]
#Dataprovider(name = "takeMyProvider", parallel = true)
public object[][] myProvider(){
return new object[][]{{"1", "name1"}, {"2", "name2"} {"3", "name3"}}
}
This test method is executed three times
#Test(dataProvider="takeMyProvider")
public void myTest(String param1, String param2){
System.out.println(param1 + " " + param2);
}
but just in one browser instance. Thats not what I want.
I want testNG to start 3 chrome instances and doing the 3 tests in parallel.
Btw I am running the tests on a selenium grid. Maybe with 100 nodes.
It would be perfect when 100 nodes doing this test in parallel. Or even 1.000, depends on the dataprovider.
Does anybody have an idea?
Best regards

Make use of a java object in Perl

Scenario :
there is a java class as given below :
public class TestSelenium {
public void googleTest() throws Exception {
WebDriver driver = new InternetExplorerDriver();
driver.get("http://www.google.com/webhp?complete=1&hl=en");
}
}
There is a perl program which makes use of Inline::Java module to call the googleTest of java class written above. Perl program looks like.
use warnings;
use Selenium::Remote::Driver;
use Inline Java => 'STUDY',
CLASSPATH => 'C:\selenium\selenium-java-2.37.0\selenium-2.37.0\libs\selenium-java-2.37.0.jar;C:\selenium\SeleniumTestPoc\bin\MyJar.jar;C:\selenium\selenium-java-2.37.0\selenium-2.37.0\libs\selenium-server-standalone-2.37.0.jar',
STUDY => ['TestSelenium'];
$test= TestSelenium->new;
$test->googleTest;
Now the above Perl code will open IExplorer and go to google.com page. In my Perl program further to $test->googleTest; I want to make use of same browser that was opened by java(WebDriver driver = new InternetExplorerDriver();) and perform a search for text "Cheese".
Question is, can the object of WebDriver class("driver" in this case) be further used in my Perl program so that I can use same browser and perform different UI operations on it in Perl?

I'm assuming that if you bring the WebDriver variable outside of the googleTest() method (making it available for all methods in the class) then you can write other methods in TestSelenium that reference the same WebDriver which you can then call in perl.
Modify TestSelenium to something like this:
public class TestSelenium {
private WebDriver driver = new InternetExplorerDriver();
public void googleTest() throws Exception {
driver.get("http://www.google.com/webhp?complete=1&hl=en");
}
public void searchForCheese() throws Exception {
// Your actions here can still reference the driver
}
}
And then after you call $test= TestSelenium->new;
Performing $test->googleTest; and $test->searchForCheese; should theoretically reference the same driver and in turn the same browser window.
EDIT :
If you want to pass the driver object back to perl try something like this.
Change the googleTest() java method to return the driver once it's done with the method (I will be using your original method but both should work):
public WebDriver googleTest() throws Exception {
WebDriver driver = new InternetExplorerDriver();
driver.get("http://www.google.com/webhp?complete=1&hl=en");
return driver;
}
And then I think you can hook onto the driver handle in perl via $mydriver= $test->goggleTest;
I assume you'll then be able to access the WebDriver hooks via $mydriver