We Keep Coding

Java Properties Class

using java 8, tomcat 8
Hi, i am loading a file using properties, but i have a check before loading which returns the same properties object if its already been loaded (not null). which is a normal case scenario but i want to know if there is any way that if any change occur in target file, and some trigger should be called and refreshes all the properties objects. here is my code.
public static String loadConnectionFile(String keyname) {
String message = "";
getMessageFromConnectionFile();
if (propertiesForConnection.containsKey(keyname))
message = propertiesForConnection.getProperty(keyname);
return message;
}
public static synchronized void getMessageFromConnectionFile() {
if (propertiesForConnection == null) {
FileInputStream fileInput = null;
try {
File file = new File(Constants.GET_CONNECTION_FILE_PATH);
fileInput = new FileInputStream(file);
Reader reader = new InputStreamReader(fileInput, "UTF-8");
propertiesForConnection = new Properties();
propertiesForConnection.load(reader);
} catch (Exception e) {
Utilities.printErrorLog(Utilities.convertStackTraceToString(e), logger);
} finally {
try {
fileInput.close();
} catch (Exception e) {
Utilities.printErrorLog(Utilities.convertStackTraceToString(e), logger);
}
}
}
}
the loadConnectionFile method executes first and calls getMessageFromConnectionFile which has check implemented for "null", now if we remove that check it will definitely load updated file every time but it will slower the performance. i want an alternate way.
hope i explained my question.
thanks in advance.

Java has a file watcher service. It is an API. You can "listen" for changes in files and directories. So you can listen for changes to your properties file, or the directory in which your properties file is located. The Java Tutorials on Oracle's OTN Web site has a section on the watcher service.
Good Luck,
Avi.

Testing multiple environments with different data - best practice

I need to be able to run the same tests on different environments (max 3) but with different data for each one.
I have a test method:
#Test (groups = "core", description = "Login: Valid log in")
public void validLogin() {
User user = UserData.user_2();
loginPage.logOn(user);
}
In the UserData class I have:
public static User user_2() {
return new User().newUser("user2", "password");
}
"user2" does not exist on all environments. I may not be able to change the data that is available on all of the environments to match the test data.
The tests will be executed either using Maven and TestNg so I can send in the parameter for the execution environment.
My initial thought is to use:
public static User user_2() {
switch(env) {
case "env1": return new User().newUser("user2", "password"); break;
case "env2": return new User().newUser("user2Z", "password"); break;
case "env3": return new User().newUser("user2X", "password"); break;
}
I have a limited number of data classes and methods (<100) but several thousand tests.
What is the best way of setting up and handling the data required for testing against the different environments?

When it comes to different users ,
You always wish that all my test cases should remain as it is and with minimal change.
So this is what i follow.
I create a file lets say username.properties file in eclipse.
username=xyz#gmail.com
password=passswd1
You can create multiple users here with name i.e.
rohan=rohan#gmail.com
rohan's password: rohan
Now we need to call this file in our class.
See below example.
Main test
SignInPage.SendkeysMethodForSignInPAgeForPropertyFile(driver, By.cssSelector("input[id='Email']") , "username" );
SignInPage.SendkeysMethodForSignInPAgeForPropertyFile(driver, By.cssSelector("input[id='Passwd'][type='password']"), "password");
So here username will be taken from properties file.
This will go to SendkeysMethodForSignInPAgeForPropertyFile which is:
public class SignInPage {
public void SendkeysMethodForSignInPAgeForPropertyFile(WebDriver driver, By by, String Text) {
WebUtils.SendkeysForPropertyFile(driver,by, Text);
}
}
Which will go to SendkeysForPropertyFile method as:
public static void SendkeysForPropertyFile(WebDriver driver, By by, String Text) {
ReadFileData File = new ReadFileData();
Properties Values = File.ReadFile();
WebElement Element = driver.findElement(by);
Element.clear();
if (Text == "username"){
Element.sendKeys(Values.getProperty("username"));
}
else {
Element.sendKeys(Values.getProperty("password"));
}
Which will read from ReadFileData() class which is:
public class ReadFileData {
public Properties ReadFile() {
File file = new File("D:\\Selenium\\Gmail_Web_UI\\Loginproperty.properties");
FileInputStream fileInput = null;
try {
fileInput = new FileInputStream(file);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
Properties prop = new Properties();
//load properties file
try {
prop.load(fileInput);
} catch (IOException e) {
e.printStackTrace();
}
return prop;
}
This helps to keep our username and password safe in a single file.
Reply to me for further query. Happy Learning :-)

multi-threaded website parser and out of memory problems

Hi.
I'm coding a website parser that is aimed to be fast and thus multi-threaded.
The external libraries I'm using are: apache HTTP client, Jsoup (for HTML parsing) and GPars (for message-driven threads).
Now I'll show some concept that I'm trying to implement
static StaticDispatchActor<String> httpActor;
public static void main(String[] args) throws ClientProtocolException, IOException {
int numThreads = 25;
try{
numThreads = Integer.parseInt(args[0]);
} catch (Exception e){
System.out.println("Number of threads defaulted to "+numThreads);
}
final int numberOfThreads = numThreads;
final ExecutorService threadpool = Executors.newCachedThreadPool();
final Async async = Async.newInstance().use(threadpool);
AtomicInteger jobCount = new AtomicInteger(0);
//.....
// This is a parser itself which parses usernames out of every page.
Actor jsoupUser = new StaticDispatchActor<String>(){ // actor to parse users
HashSet<String> users = new HashSet<>(); // found users
public void onMessage(String html){ // takes html -> adds parsed users 2 set
users.addAll(Jsoup.parse(html)
.select("a[href*=/profile/]").stream() // select links
.map(e -> e.text()) // extract usernames
.filter(s -> s.length() > 0) // empty lines -> out
.collect(Collectors.toSet()));
System.out.print("Users: "+users.size()+", Jobs: "+jobCount.get()+"\r");
}
}.start();
// This actor shall extract new links to parse out of every page
Actor jsoupLinker = new StaticDispatchActor<String>(){ // link extractor
HashSet<String> usedLinks = new HashSet<>(); // already found links
public synchronized void removeBack(String url){
#Override
public void onMessage(String html) {
Set<String> links = Jsoup.parse(html).select("a[href]").stream().parallel()
.map(e -> e.attr("href").replace("#comments", "")// here also some replacements...
)
.filter(s -> (!usedLinks.contains(s) && /* other filters */ )
.collect(Collectors.toSet());
links.forEach(url -> httpActor.send(url)); // send to process new URLs
}
}.start(); // start actor
// this actor is the processor of new links and where the error comes in:
httpActor = new StaticDispatchActor<String>(){ // process responses async
public void onMessage(String url) {
try{
while(jobCount.get()>numberOfThreads); // wait for running threads to be less than wanted value; without this number of running jobs goes out of any control
async.execute(Request.Get(defaultWebSiteUrl+url), new FutureCallback<Content>(){ #Override // do request and process async
public void completed(Content c) {
jobCount.decrementAndGet();
try{
String s = c.asString();
jsoupUser.send(s);
jsoupLinker.send(s);
} catch (OutOfMemoryError e1){
System.out.println("out of my memory, "); // This is the thrown error the question is about - [1]
}
}
#Override public void failed(Exception e) {
jobCount.decrementAndGet();
try {
throw e;
} catch (ConnectException e4){ // if the request is timed out resend it
httpActor.send(url);
System.out.println("resent\r\n");
} catch (HttpResponseException e0){
} catch (Exception e1) { // for all other exceptions
e1.printStackTrace();
}
}
#Override public void cancelled() {
jobCount.decrementAndGet(); // never done actually
}
});
jobCount.incrementAndGet();
} catch (IllegalArgumentException e3){
System.out.println("some illigal shit");
}
}
};
httpActor.start();
Now the problem is, although I limited a number of running jobs, my code somehow goes out of memory (search for [1] in the code to see where).
Maybe you have any idea on how to resolve it. Or there is some showcase for similar task, because I fill very wrong about the whole application design and maybe I should change it at all?
Thank you.

So, using BiziClop's tip, I was able to figure out the mistake.
If some is interested, I, as you can see above, was sending HTML code received from server as a string to 2 different actors and than, inside these actors, parsing them. That was the cause for all out-of-memory errors, since these HTML pages are pretty big especially given how many of them are waiting to be processed in the message queue.
The solution I used is just parse the document and select needed elements and transfer their list to matching actors for further processing.
Document doc = Jsoup.parse(c.asString());
jsoupUser.send(doc.select("a[href*=/profile/]"));
jsoupLinker.send(doc.select("a[href]"));
Still, if any one has anything to say about how to improve the algorithm I'll really appreciate it.

Orient DB - Export database with Specified datatables

Im working with orient db where i have to export specified data tables using java. Here is the code im working with:
ODatabaseDocumentTx db = new ODatabaseDocumentTx("remote:localhost/sampleDB").open("admin", "admin");
try {
OCommandOutputListener listener = new OCommandOutputListener() {
#Override
public void onMessage(String iText) {
// System.out.print(iText);
}
};
Set<String> abcd = new HashSet<String>();
abcd.add("sample_demo1_OnlineShopping");
System.out.println(abcd);
ODatabaseExport export = new ODatabaseExport(db, "DataCont/Data.gz", listener);
export.setIncludeInfo(false);
export.setIncludeClusterDefinitions(false);
export.setIncludeSchema(false);
export.setIncludeIndexDefinitions(false);
export.setIncludeManualIndexes(false);
export.setIncludeClasses(abcd);
// export.exportRecords();
export.exportDatabase();
export.close();
} catch (IOException e) {
e.printStackTrace();
} finally {
db.close();
}
The problem is, its not including any classes. Output file was like this:
{"records":[]}
But when i tries without the "setIncludeClasses" it prints all the classes available in the database. What would the issue for this problem. Thanks in advance.

Change this line to:
abcd.add("sample_demo1_OnlineShopping".toUpperCase());
I haven't found it in the documentation, but includeClasses is expecting classes in UpperCase (same for excludeClasses).
See the source code.

PayPal REST API java sdk - custom config file

Good afternoon all!
I use PayPal REST API java sdk and I want to have different configurations for different environments of my application. Here is how I'm trying to do so:
private static boolean IS_PRODUCTION = false;
private static String PAYPAL_ACCESS_TOKEN;
private static void initPayPal() {
InputStream is = null;
try {
is = ApplicationConfig.class.getResourceAsStream(
IS_PRODUCTION? "/my_paypal_sdk_config.properties" : "/my_paypal_sdk_config_test.properties");
PayPalResource.initConfig(is);
String clientID = ConfigManager.getInstance().getConfigurationMap().get("clientID");
String clientSecret = ConfigManager.getInstance().getConfigurationMap().get("clientSecret");
PAYPAL_ACCESS_TOKEN = new OAuthTokenCredential(clientID, clientSecret).getAccessToken();
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
IOUtils.closeQuietly(is);
}
}
and while trying to get the clientID I have
java.io.IOException: Resource 'sdk_config.properties' could not be found
Strange behavior - I thought I've just configured the sdk to use my own properties file.
Please advice how could I set up those settings properly!

So here is the solution I found:
Create an empty sdk_config.properties file in default location
Load your own properties:
private static void initPayPal() {
InputStream is = null;
try {
is = ApplicationConfig.class.getResourceAsStream(
IS_PRODUCTION ? "/my_paypal_sdk_config.properties" : "/my_paypal_sdk_config_test.properties");
Properties props = new Properties();
props.load(is);
PayPalResource.initConfig(props);
ConfigManager.getInstance().load(props);
String clientID = ConfigManager.getInstance().getConfigurationMap().get("clientID");
String clientSecret = ConfigManager.getInstance().getConfigurationMap().get("clientSecret");
PAYPAL_ACCESS_TOKEN = new OAuthTokenCredential(clientID, clientSecret).getAccessToken();
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
IOUtils.closeQuietly(is);
}
}

We have made some good improvements to the PayPal Java SDK on integration steps. We are removing the need for sdk_config.properties file as they do not work as well, specially for multi-configuration settings.
Now, all you do is create an APIContext instance with clientId, clientSecret, and mode. You pass that context object for any API operation from there on.
Here is how the code would look like for different configurations:
APIContext defaultContext = new APIContext(clientId1, clientSecret1, "sandbox");
APIContext sandboxContext = new APIContext(clientId2, clientSecret2, "sandbox");
APIContext someOtherContext = new APIContext(clientId3, clientSecret3, "live");
APIContext liveContext = new APIContext(clientId, clientSecret, "live");
// Now pass any of the above context in these calls, and it would use those configurations.
Payment payment = new Payment();
// Fill in all the details.
payment.create(defaultContext);
// Replace that defaultContext with any of those other contexts.
Here is the wiki page explaining that: https://github.com/paypal/PayPal-Java-SDK/wiki/Making-First-Call

I had the same error with SDK 0.11 version. I use my own properties file, but code still looked for "sdk_config.properties". I put it into root in my CLASSPATH, but still got the same error. Then I made obvious and horrible solution: put empty "sdk_config.properties" into "rest-api-sdk-0.11.0.jar" library. This street magic solved my problem.