Background Story
I'm working at a software firm developing a test automation framework to replace our old spaghetti tangled system.
Since our system requires a login for almost everything we do, I decided it would be best to use #BeforeMethod, #DataProvider, and #Factory to setup my tests. However, I've run into some issues.
Sample Test Case
Lets say the software system is a baseball team roster. We want to test to make sure a user can search for a team member by name.
(Note: I'm aware that BeforeMethods don't run in any given order -- assume that's been taken care of for now.)
#BeforeMethod
public void setupSelenium() {
// login with username & password
// acknowledge announcements
// navigate to search page
}
#Test(dataProvider="players")
public void testSearch(String playerName, String searchTerm) {
// search for "searchTerm"
// browse through results
// pass if we find playerName
// fail (Didn't find the player)
}
This test case assumes the following:
The user has already logged on (in a BeforeMethod, most likely)
The user has already navigated to the search page (trivial, before method)
The parameters to the test are associated with the aforementioned login
The Problems
So lets try and figure out how to handle the parameters for the test case.
Idea #1
This method allows us to associate dataproviders with usernames, and lets us use multiple users for any specific test case!
#Test(dataProvider="players")
public void testSearch(String user, String pass, String name, String search) {
// login with user/pass
// acknowledge announcements
// navigate to search page
// ...
}
...but there's lots of repetition, as we have to make EVERY function accept two extra parameters. Not to mention, we're also testing the acknowledge announcements feature, which we don't actually want to test.
Idea #2
So lets use the factory to initialize things properly!
class BaseTestCase {
public BaseTestCase(String user, String password, Object[][] data);
}
class SomeTest {
#Factory
public void ...
}
With this, we end up having to write one factory per test case... Although, it does let us have multiple users per test-case.
Conclusion
I'm about fresh out of ideas. There was another idea I had where I was loading data from an XML file, and then calling the methods from a program... but its getting silly.
Any ideas?
First, it seems like you are trying to do too much in each test case. For instance, if you are searching for something, why should you need to test navigation as part of searching?
Second, it seems like your requirements are unclear. Can you create a test where you send it a single search term and get back a single result? Seems like this should be your first test.
Third, why can't an authorized session connection/object be a requirement for a test? From the looks of your code, it looks like you are making some kind of call via HTTP. Even if you aren't, it looks like you must be using some kind of broker to send a message to your app, given that you are having to pass along user name and pass on each request... Why not just automate that whole thing into an "authorized broker" object that gives you a way to send your request along a pre-authorized connection?
Then just pass the authorized broker into your tests to handle your message passing. Or better yet, set up your authorized broker in your pre-test (BeforeMethods?) functions, and make it available as a class member variable.
I'm still not sure I understand your problem...
Are you not satisfied with your second attempt, i.e. having to write a #Factory per test case? Then why not put the #Factory annotation on the base class constructor? This way, all your subclasses need to do is call super and you're done.
What are you trying to achieve exactly?
Cederic,
I should have made an account from day one, I can't actually edit my old post anymore, or reply to comments (at this point). Let me try and expand things a little.
Here's an example of what I've come up with so far. I guess I answered my own question... this works, but its a little nasty.
My question is now, I have all my tests in one factory. That's no good for several reasons. One, I have to add them all by hand (or put them in reflectively). Two, I have to run everything essentially as one suite. Any ideas?
package example2;
import java.lang.reflect.Method;
import java.util.HashMap;
import org.testng.annotations.*;
public class WebTestBase {
protected String host, username, password;
protected HashMap<String, Object[][]> dataSet;
public WebTestBase(String host, String username, String password, HashMap<String, Object[][]> dataSet) {
this.host = host;
this.username = username;
this.password = password;
this.dataSet = dataSet;
}
#DataProvider(name="dataSet")
public Object[][] dataSet(Method m) {
return dataSet.get(m.getName());
}
#BeforeMethod
public void login() {
System.out.println("Logging in to " + host + " with " + username + ":" + password);
}
#AfterMethod
public void logout() {
System.out.println("Logging out!");
}
}
package example2;
import java.util.HashMap;
import org.testng.annotations.Factory;
public class WebTestFactory {
#Factory
public Object[] factory() {
HashMap<String, Object[][]> dataSetOne = new HashMap<String, Object[][]>();
dataSetOne.put("searchRoster", new Object[][] {
{"mcguire", "McGuire, Mark"},
{"ruth", "Ruth, Babe"}
});
HashMap<String, Object[][]> dataSetTwo = new HashMap<String, Object[][]>();
dataSetTwo.put("addPlayer", new Object[][] {
{"Sammy Sosa", 0.273}
});
Object[] tests = new Object[] {
new SearchTest("localhost", "user", "pass", dataSetOne),
new AddTest("localhost", "user", "pass", dataSetTwo)
};
return tests;
}
}
package example2;
import java.util.HashMap;
import org.testng.annotations.Test;
public class SearchTest extends WebTestBase {
public SearchTest(String host, String username, String password,
HashMap<String, Object[][]> dataSet) {
super(host, username, password, dataSet);
}
#Test(dataProvider="dataSet")
public void searchRoster(String searchTerm, String playerName) {
System.out.println("Searching for " + searchTerm);
System.out.println("I found " + playerName + " in the search results!");
}
}
package example2;
import java.util.HashMap;
import org.testng.annotations.Test;
public class AddTest extends WebTestBase {
public AddTest(String host, String username, String password,
HashMap<String, Object[][]> dataSet) {
super(host, username, password, dataSet);
}
#Test(dataProvider="dataSet")
public void addPlayer(String playerName, double battingAvg) {
System.out.println("Adding " + playerName + " with avg " + battingAvg);
}
}
Related
I have implemented parallel testing using Selenium and TestNG for a web application, however, my webapp got a restriction where only one session can be handled at the time (it is not supporting multi-sessions, when you login with the same user it will disconnect your older session), the solution of which I have thought to solve it is to create a unique user for each test (using API), so I have implemented it but when I am running the tests in parallel (using testng.xml file with thread-count="2") I am creating the same user twice! with the same credentials, I want to be able to create a unique user where it will create one unique user for each run.
This is my code:
public class BaseApiTest extends BaseTest {
protected String token;
protected String CREATED_ADMIN_TEST_USER;
protected String CREATED_ADMIN_TEST_PASSWORD;
private static final AtomicReference<String> ACCESS_TOKEN = new AtomicReference<>();
#BeforeClass(alwaysRun=true)
public void baseApiSetup() throws InterruptedException, ApiException {
generateToken();
createAdminUser();
}
private void generateToken() {
........
..........
...........
token = "Bearer " + ACCESS_TOKEN.get();
context.setAttribute("api-key", token);
context.setAttribute("HOST", HOST);
T_Logger.info("Host url address is: =[{}]", HOST);
T_Logger.info("new api key token =[{}]", token);
}
private void createAdminUser() throws ApiException, InterruptedException {
UsersAPIUnitTest usersAPITU = new UsersAPIUnitTest(context);
usersAPITU.createUser();
CREATED_ADMIN_TEST_USER = UsersAPIUnitTest.getEmail();
CREATED_ADMIN_TEST_PASSWORD = UsersAPIUnitTest.getPassword();
}
}
and this is used by the login page with the newly created user:
protected void adminSignIn() {
loginPage.login(CREATED_ADMIN_TEST_USER, CREATED_ADMIN_TEST_PASSWORD, true);
writeToLoggerSignIn(CREATED_ADMIN_TEST_USER);
}
and then I am starting to run my tests.
Expected: each test class will contain its own unique user
Actual: all the users that are being created are the same user with the same credentials
========================EDIT===============================
This is how I create new user:
public String createUserCms(String name, String email, String phone, String password) throws ApiException {
NewUserPayload body = new NewUserPayload();
body.setStatus(true);
body.setName(name);
body.setEmail(email);
body.setPhone(phone);
body.setPassword(password);
body.setPasswordConfirmation(password);
printBody(body);
return usersApi.createUser(token, body);
}
I have been working on a generic solution for this problem so that all automation teams can use it, still work in progress. I'm sure you would have explored other options, but creating users every time you run a test is not a good idea, rather you have to create the users once and use them otherwise you end up adding more users into your system (if you are cleaning up the users #afterMethod that's fine)
But for now, in your case, you have to make sure that you pass a unique name/email when creating the user and return that username/email address.
If you can share your createUser method snippet, we can help you more.
I want to Run Test in a same class with two credentials, and both are in same TestRunner class, I want to declare in #Beforemethod class so that no need to write same code in Every #Test.
If I understood, You want to run same test twice, but with different credentials.
If this is the case use dataProvider:
#DataProvider(name="login")
public Object[][] getData() {
return new Object[][] {
{"test2#test.com", "test",false},
{"test#test.com", "abcabc",true}
};
}
and here is method how to call it.
#Test(dataProvider="login")
public void testLogin(String usernameEmail, String password,boolean flag) throws InterruptedException {
if(flag){
Assert.assertTrue(!errorMessage.isDisplayed());
}else{
Assert.assertTrue(errorMessage.isDisplayed());
}
}
Hope I understood and it would help You,
Updated answer with pseudo code of a sort, so You could create method without annotation just simple method inside test class or in different one, depending on Your logic, and provide parameter to that method within test methods here is example:
Don't know how You data is set in #BeforeMethod, because there is no code example, but here is something:
#Test
public void testLogin_1(){
login(email, password)
}
#Test
public void testLogin_2(){
login(email2, password2)
}
private void login(String email, String password){
inputEmail(email);
inputPassword(password);
clickSubmit();
Assert.assertEquals();
// do some asserts so if You want to assert some error cases.
}
Hope this helps,
I need to unit test a method, and I would like mock the behavior so that I can test the necessary part of the code in the method.
For this I would like access the object returned by a private method inside the method I am trying to test. I created a sample code to give a basic idea of what I am trying to achieve.
Main.class
Class Main {
public String getUserName(String userId) {
User user = null;
user = getUser(userId);
if(user.getName().equals("Stack")) {
throw new CustomException("StackOverflow");
}
return user.getName();
}
private User getUser(String userId) {
// find the user details in database
String name = ""; // Get from db
String address = ""; // Get from db
return new User(name, address);
}
}
Test Class
#Test (expected = CustomException.class)
public void getUserName_UserId_ThrowsException() {
Main main = new Main();
// I need to access the user object returned by getUser(userId)
// and spy it, so that when user.getName() is called it returns Stack
main.getUserName("124");
}
There are only two ways to access private:
using reflection
extend the scope
maybe waiting for Java 9 to use new scope mechanisms?
I would change the scope modifier from private to package scope. Using reflection is not stable for refactoring. It doesn't matter if you use helpers like PowerMock. They only reduce the boiler-plate code around reflection.
But the most important point is you should NOT test too deep in whitbox tests. This can make the test setup explode. Try to slice your code into smaller pieces.
The only information the method "getUserName" needs from the User-object is the name. It will validate the name and either throw an exception or return it. So it should not be necessary to introduce a User-object in the test.
So my suggestion is you should extract the code retreiving the name from the User-object into a separate method and make this method package scope. Now there is no need to mock a User-Object just the Main-Object. But the method has its minimal information available to work properly.
class Main {
public String getUserName(String userId) {
String username = getUserNameFromInternal(userId);
if (userName.equals("Stack")) {
throw new CustomException("StackOverflow");
}
return user.getName();
}
String getUserNameFromInternal(String userId) {
User user = getUser(userId);
return user.getName();
}
...
}
The test:
#Test (expected = CustomException.class)
public void getUserName_UserId_ThrowsException() {
Main main = Mockito.mock(new Main());
Mockito.when(main.getUserNameInternal("124")).thenReturn("Stack");
main.getUserName("124");
}
Your problem that call to new within your private method.
And the answer is not to turn to PowerMock; or to change the visibility of that method.
The reasonable answer is to "extract" that dependency on "something that gives me a User object" into its own class; and provide an instance of that class to your "Main" class. Because then you are able to simply mock that "factory" object; and have it do whatever you want it to do.
Meaning: your current code is simply hard-to-test. Instead of working around the problems that are caused by this, you invest time in learning how to write easy-to-test code; for example by watching these videos as a starting point.
Given your latest comment: when you are dealing with legacy code, then you are really looking towards using PowerMockito. The key part to understand: you don't "mock" that private method; you rather look into mocking the call to new User() instead; as outlined here.
You can use a PowerMock's mockPrivate but I don't recommend it.
If you has such a problem it usually mean that your design is bad.
Why not making the method protected?
I'm developing desktop software with JavaFX and Java Spark which is basically a barebones framework for developing web apps, but I'm trying to use it strictly to put/get sensitive methods and variables on a server so that users can't access them. REST seems to be the correct approach but I'm struggling to understand 2 interrelated REST concepts. The Spark docs are light but I've integrated the few good tutorials into a working demo below but I've hit a wall. I'll briefly explain:
With the help of Postman I've been able to put a few records onto the server by using a path of http://localhost:4567/secrets and Body of:
{
"title" : "demofield1",
"content" : "12345"
}
Each record contains a title as an identifier (demofield1) and content as the sensitive data that should remain hidden from users at all times (12345). It's pretty trivial to put these strings onto a server and then get them by using title as a parameter, shown below. The demo code simply has a Model class for creating and returning a record (secret), a JSON conversion method, and a get and put Spark method. Secrets are stored locally in a HashMap for now, I'm assuming a real app would simply swap in a server DB.
The get method works as expected, returning the correct JSON record and storing the content as a String with this line: String secretString = model.getCertainSecret(title).getContent();
With that said...
Questions (partial answers fully appreciated too):
secretString above now holds a confidential value (12345) which is obtained using a supposedly secure REST method. But couldn't a user simply reverse-engineer my source code and write System.out.println(secretString) and have that 12345 revealed? I don't understand how a simple string is protected after retrieving it from the server, despite not being explicitly shown. The code seems correct yet the value is easily obtainable. What am I missing?
How do you put entire java methods on a server? A lot of code I need to protect isn't just strings but methods containing Tasks, Platform.runLater()->, and needs to interact with other desktop software. For example, one of my methods uses JACOB to identify when a certain area of a third-party software is clicked. I can't even fathom what a get/put would look like in that context.
My assumption was that a server-side DB would store all content from my put requests, but I don't understand how it stores and returns a method? Should I be reading about servlets or SaaS or something? I'm focused on desktop users.
Code:
import com.fasterxml.jackson.core.JsonParseException;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import lombok.Data;
import org.apache.log4j.BasicConfigurator;
import java.io.IOException;
import java.io.StringWriter;
import java.util.HashMap;
import java.util.Map;
import static spark.Spark.get;
import static spark.Spark.put;
public class ServerDemo
{
private static final int HTTP_BAD_REQUEST = 400;
#Data
static class NewSecretPayload {
private String title;
private String content;
public boolean isValid() {
return title != null && !title.isEmpty();
}
}
public static class Model {
private int nextId = 1;
private Map<String, Secret> secrets = new HashMap<>();
#Data
class Secret {
private int id;
private String title;
private String content;
}
public int createSecret(String title, String content){
int id = nextId++;
Secret secret = new Secret();
secret.setId(id);
secret.setTitle(title);
secret.setContent(content);
secrets.put(title, secret);
return id;
}
public Secret getCertainSecret(String titleToUse){
if(null != secrets.get(titleToUse)){
return secrets.get(titleToUse);
}else{
return null;
}
}
}
public static String dataToJson(Object data) {
try {
ObjectMapper mapper = new ObjectMapper();
mapper.enable(SerializationFeature.INDENT_OUTPUT);
StringWriter sw = new StringWriter();
mapper.writeValue(sw, data);
return sw.toString();
} catch (IOException e) {
throw new RuntimeException("IOException from a StringWriter?");
}
}
public static void main(String[] args) {
Model model = new Model();
BasicConfigurator.configure();
put("/secrets", (request, response) -> {
try {
ObjectMapper mapper = new ObjectMapper();
NewSecretPayload creation = mapper.readValue(request.body(), NewSecretPayload.class);
if (!creation.isValid()) {
response.status(HTTP_BAD_REQUEST);
return "";
}
int id = model.createSecret(creation.getTitle(), creation.getContent());
response.status(200);
response.type("application/json");
return id;
} catch (JsonParseException jpe) {
response.status(HTTP_BAD_REQUEST);
return "";
}
});
get("/secrets/:title", (req, res) -> {
String title = req.params(":title");
if (model.getCertainSecret(title) != null) {
res.status(200);
res.type("application/json");
String secretString = model.getCertainSecret(title).getContent();
return dataToJson(model.getCertainSecret(title));
}
res.status(400);
return new ResponseError("No user with title "+title+" was found", title);
});
}
}
Lets dig down to your first problem "Keeping string secret" :--
Restrict : The simplest way is not to provide the data to malicious user.
Masking : Mask the data you are providing to end user. You will have the original data mapped to masked data. You will provide masked data to end user. Here the end user can never get the original data as it is a one way process. When end user sends masked-data you can always retrieve the original data from it.
Encrypting : If the end user needs to see the data you can encrypt it and send it. You can make a sanity check of your code before decrypting the data. The sanity check can give you idea if the code is ever modified. If code fails the sanity check you can always exit the process.
I want to make a class that can interact with a database, which has the following desired functionality:
It has a method to return all fields from the database, later can be changed such that it can also limit with it returns.
It has a method to insert a specific instance of this class.
It has a method to update a specific instance of this class.
I will show the code in a moment after further explanation.
Now I want to extract an interface, or rather an abstract class I think might be more appriopiate, to be sure that all classes/datafields follow the same 'interface', and to be able to use them as a supertype in Lists etc.
The data class, in this case Account.java, should represent a table in a database which stores {Username, Password}, I am omitting an explicite unique identifier for now, still not sure if I will make an additional ID field or use the uniqueness of the Username field.
It would be best if the abstract class itself would handle all the MySQL interaction 'mess'.
Account.java so far:
package testthing;
import java.util.Map;
/**
*
* #author Frank
*/
public class Account {
private final static String ALL_QUERY = "SELECT * FROM accounts";
private final static String INSERT_QUERY = "INSERT INTO accounts (username, password) VALUES(?, ?)";
private final static String UPDATE_QUERY = "UPDATE accounts SET password=? WHERE username=?";
private String username;
private String password;
public Account(final String username, final String password) {
this.username = username;
this.password= password;
}
public String getUsername() {
return username;
}
public void setUsername(final String username) {
this.username = username;
}
public String getPassword() {
return password;
}
public void setPassword(final String password) {
this.password = password;
}
public static Map<String, Account> getAll() {
//return a map using the ALL_QUERY string
}
public void insert() {
//insert this using INSERT_QUERY
}
public void update() {
//update this using UPDATE_QUERY
}
}
I know that I haven't been that clear I'm afraid, but I hope this is enough to help me get going.
Basically I want to always be able to use the followings methods when working with a TableObject, which Account would be a subset of:
Account.getAll();
new Account("test", "test").insert();
currentAccount.setPassword("newPassword"); currentAccount.update();
All the nasty SQL stuff should be hidden inside the proposed abstract class. The only thing you cannot escape in a TableObject class is the definition of the SQL queries.
Regards.
EDIT: In current example Account.getAll() returns a Map<String, Account>, but in reality the first generic argument should be the type of the key in the database. So if you would use an unique ID then it would need to return a Map<Integer, Account>. I hope this change makes it in time for people to read it.
Is it not more logical, to have the connection code and all the "nasty" stuff in the superclass, but simply have a more general method in the superclass, that is used by it's sub classes. For example:
public void executeUpdate(String query)
{
// Code to execute update.
}
public Map<String, Data> getData(String query)
{
// Code to get data.
return data;
}
This way, these methods are more general. It means that you can implement several classes that simply pass query data, rather than constantly having to update a superclass every single time you want to add new functionality.
Obviously I've just assumed a type Data here, but that might be something to look into. The aim here is to decouple your classes as much as possible. This means you can add as many new classes as you want, and they can use their supertype without hinderence.
This also means things like
Account.getAll();
is a little less complicated, because if you have getAll in your superclass, where is the reference that you want to get all accounts? If the code is actually in the Account class, you can customize the query, and send it off to the getData method to be executed.