We Keep Coding

(Too) complex configuration management (Java properties)

I'm working at a company having a (too) complex configuration management process:
In each module there is an application.properties file. There are properties for the developers like: database.host = localhost
Properties which change in other environments are maintained in an application.properties file in an override-properties folder (for each module) like: database.host=#dbhost#
There is a default-deployment.properties file with default values for other environments like: database.HOST=noValueConfigured.DB_HOST
A postconfigure.properties file with DATABASE_ HOST=#configure.DB_HOST#
Those files are only needed if a property value depends on the environments (is different for development, testing, live).
Finally there is a Excel document with a sheet for every environment and a row like: configure.DB_HOST - a comment ... - 127.0.0.1 (just as example). The Excel is responsible for generating the correct property files for the rpm packages.
This process is not only complex but also error prone.
How could it be simplified/improved?
The approach should be compatbiel with Spring DI.

I would start with a master configuration file and generate the properties files to start with.
Ultimately you could have a set of proprties files which can be deployed in all environments e.g.
database.host = localhost
database.host.prod = proddb1
database.host.uat = uatdb1
i.e. use the environment/host/region/service at the end as a search path. This has the advantage that you can see the variations between environments.
You can implement this collect like this
public class SearchProperties extends Properties {
private final List<String> searchList;
public SearchProperties(List<String> searchList) {
this.searchList = searchList;
}
#Override
public String getProperty(String key) {
for (String s : searchList) {
String property = super.getProperty(key + "." + s);
if (property != null)
return property;
}
return super.getProperty(key);
}
You might construct this like
Properties prop = new SearchProperties(Arrays.asList(serverName, environment));
This way, if there is a match for that server, it will override, the environment which will overidden the default.
In Java 8 you can do
public String getProperty(String key) {
return searchList.stream()
.map(s -> key + "." + s)
.map(super::getProperty)
.filter(s -> s != null)
.findFirst()
.orElseGet(()-> super.getProperty(key));
}

There should be only one file, even if it has a lot of properties. Also, there should be only one property for each functionality, like database.host, not database.host and database_host, or anything similar.
You need to create hierarchy for such and for every property in order to know which one will be user. For example, if there is some head global value for database.host, it should be used for that property. If not, check next level in hierarchy, like specific environments (like production value). If such does not exist, check next level, like local or test level. And for bottom level, have a default value. In such way, you have two dimension of consuming properties and as such, decreases chances for error dramatically.
In one company I used to work, we had automated deployer which would handle such level setup, we would just set variable on its web site for level we wanted and it would go from top to bottom and set them. We never had problems with such setup and we would have more then 50 variables in app.properties file.

If not to take into consideration all the redesign methods mentioned in the previous comments, you can wrap all the complexity into Tomtit task manager which is good with these types if tasks.
Just create properties files templates and populate them using environments

Dynamically change JDBC config in Jmeter

I need to make sure,whether table has data or not ,based on the environment,say for example i have two two database,one is development another one is production like given below.
Production
host1
dbuser1
dbpassword1
dburl
Tablename : studentinfo
Development
dbuser2
dbpassword2
dburl2
Tablename : studentinfo
FYI : studentinfo has same structure in both environment .
In Jmeter ,User Defined Variable( UDV), I have configured two set of database information. Using BeanShell Processor I have trying to change the database connectivity information, Is there any way to change the Db Config dynamically ?
Given below is my Jmeter UDV
env : prod
prod_db_url: dburl
prod_db_user:usr
prod_db_password:password
dev_db_url: dburl
dev_db_user:usr
dev_db_password:password
In My Beanshell Preprocessor
String env=vars.get("env");
if(env.equlas("prod")){
// Load the Prod db into vars
} else if (env.equals("dev")){
// Load the Dev db into vars
}
Here ,I am setting the values in vars, and trying to get the information from DB Configuration variables. but i am not able get values the in DB config.
Can anyone explain ? what went wrong or what is the approach to get connection?

JDBC Connection Configuration is initialized after User Defined Variables but before any PreProcessor, that's why you don't see values changes.
Consider using JMeter Properties instead of JMeter Variables, i.e. change ${dburl} in JDBC Connection Configuration to ${__P(dburl,)}. Do the same for credentials variables.
Depending on how you run your test you can set properties value:
Via -J command-line argument like:
jmeter -Jdburl=jdbc://somedb:port/etc -Juser=foo -Jpassword=bar
Put it into user.properties file (located in JMeter's "bin" folder) like
dburl=jdbc://somedb:port/etc
user=foo
password=bar
References:
__P() function user manual entry
Apache JMeter Properties Customization Guide

You do not need a Beanshell PreProcessor for this.
For the below UDV,
env : prod
prod_db_url: dburl
prod_db_user:usr
prod_db_password:password
dev_db_url: dburl
dev_db_user:usr
dev_db_password:password
Just by changing the value of env, you can access all the variables values by using
${__V(${env}_db_url)} // return prod or dev db url depends on the value of env.
Another nice solution:
Can you have the same variables and store them in 2 different property files?
prod.proeprties
db_url=dburl
db_user=usr
db_password=password
dev.properties
db_url=dburl
db_user=usr
db_password=password
You can use the JMeter - Property File Reader.
Property reader file path would be /path/to/${env}.properties
Access all the variables using ${__P(db_url)}, ${__P(db_user)}

Getting instance name of a WebSphere app Server

My Web service will run on a Jboss App Server or a Websphere app Server. For both of them I need to know the instance name, for Jboss I can use System.getProperty("jboss.server.name"), but what can I use for WebSphere? I can't use WebSphere-specific methods, I need to call System.properties
Thanks

An alternative, at least for WebSphere, is to look it up in the JNDI tree.
This is what I use:
InitialContext ic = new javax.naming.InitialContext();
String serverName = ic.lookup("servername").toString();
This way I don't have to configure anything as WebSphere binds that information for me.
Cell and node name can also be retrieved using "thisNode/cell/cellname" and "thisNode/nodename". Something useful in clusters.

I agree with specifying server name as an environment variable (Manglu's touch is also fine). Just to make the discussion complete, here is how you get get instance name via runtime (this API is deprecated in recent versions but still in use);
import com.ibm.websphere.runtime.ServerName;
System.out.println("Display name: " + ServerName.getDisplayName());
System.out.println("Full name: " + ServerName.getFullName());
Sample output would be like
Display name: server1
Full name: was7host01Node01Cell\was7host01Node01\server1

To keep it platform neutral you can set a variable as a JVM argument for the Websphere server (one for each node if its clustered). For Websphere 7, you will find the following in the Admin Console ...
Servers > Server Types > Websphere application servers > [your cluster node] >
> Java and Process Management > Process Definition > Java Virtual Machine >
> Generic JVM arguments
and add a variable like this ...
-DServerName=serverNodeA
You can then access the value in your code as ...
String serverName = System.getproperty("ServerName");
This technique can be used with all application servers so long as you have access to add arguments to the JVM. I'm sure there must be Websphere specific API to query the node name, but then you're typing your code to the server which makes it difficult to unit test and is not portable. I prefer this approach.

kurtcebe solution works well. For those using maven, you won't be able to get the jar easily into your project. Because of this, use Class.forname...
try {
Class<?> c = Class.forName("com.ibm.websphere.runtime.ServerName");
LOGGER.debug("Class found" + c);
Method m = c.getMethod("getFullName", new Class<?>[0]);
LOGGER.debug("Method found" + m);
Object o = m.invoke(DeliveryServiceUtils.class, new Object[0]);
LOGGER.debug("Method invoked, response is " + o);
processName = o.toString();
}
catch (Exception ex) {
processName = "unknown - " + ex.getClass().getName() + ": " + ex.getMessage();
}

The approach suggested by Brad is good but I would do it subtle differently.
In the custom property of the Server JVM, I would add a property Server-Name and specify its value as WAS_SERVER_NAME.
here is the bread crumb for this:
Servers -> -> Process Definition -> Java Virtual Machine > Custom properties
Add a new one wiht say ServerName with value ${WAS_SERVER_NAME}
Every WAS Server instance has this WebSphere Variable set to the name of the Server instance. You don't need to worry about making typos (or similar errors here) and this approach works for all WAS Server.
You can create a Server with such values set as a template and when you create a new server, these are always present for you.
HTH

Read a Environment Variable in Java with Websphere

I've a little problem with Websphere application server 7.0 (WAS7) and the reading of Environment Varaibles.
With TomCat, I've defined a variable as
<Environment name="myVar" type="java.lang.String" value="myVarOnServeur"
and I read it with a lookup on the initialContext :
Context ctx = new InitialContext();
String myVar = (String) ctx.lookup( "java:comp/env/myVar" );
and it works!
But with Websphere, I define a environment variable on the GUI but I can't read it in my java code. I've a NamingException.
(source: fullahead.org)
How can I do to fix my problem?

to define inside web.xml
<env-entry>
<env-entry-name>varName</env-entry-name>
<env-entry-value>56</env-entry-value>
<env-entry-type>java.lang.String</env-entry-type>
</env-entry>
to see with java
Context envEntryContext = (Context) new InitialContext().lookup("java:comp/env");
String mydata = (String)envEntryContext.lookup("varName");

You are looking at the wrong place.
You should add the variable in Environment->Naming->Name space bindings->New.
If you choose Binding type String, "Binding identifier" and "Name in namespace..." myVar, you can get variable's value with:
Context ctx = new InitialContext();
String myVar = (String) ctx.lookup( "cell/persistent/myVar" );

On WAS follow the above setting where name is your key and value is your property value. in my example i used Name : Test Value : This is the test value. After setting this values restart your application server. on your Java code call System.getProperty("TEST") where test is the name for your property and the value will show

You can put something like the following in your web.xml file, which should be in your application's WEB-INF directory:
<env-entry>
<env-entry-name>myVar</env-entry-name>
<env-entry-type>java.lang.String</env-entry-type>
<env-entry-value>myVarOnServeur</env-entry-value>
</env-entry>
By the way this is a standard syntax and should work across all the application servers. I'm using it with WebSphere, JBoss and WebLogic. It can be queried exactly as you do in your example.

If what you want is to define and manage your own variables, have a look at Environment->Naming->Name space bindings. You can bind jndi names to String constants there. see String binding settings

You should be able to resolve these via WebSphere's AdminOperations MBean:
//sample code from WAS 7 Infocenter
private String expandVariable(String s) throws
javax.management.JMException {
com.ibm.websphere.management.AdminService as =
com.ibm.websphere.management.AdminServiceFactory.getAdminService();
String server = as.getProcessName();
String mBeanName = "*:*,type=AdminOperations,process=" + server;
java.util.Set result = as.queryNames(
new javax.management.ObjectName(mBeanName) , null);
return (String) as.invoke((javax.management.ObjectName)
result.iterator().next(),
"expandVariable",
new Object[]{"${"+s+"}"},
new String[]{"java.lang.String"});
}
See Creating, editing and deleting WebSphere variables.

Websphere 7.0 - 8.5
Set Variable
Admin Console ---> Websphere Application servers -----> Your_sever_name ---> Java and process management ---> Process definition -->Java Virtual Machine --> Custom properties
Get Value in Java
System.getProperty("Your_Variable")

I would just like to elaborate on creating a variable in WebSphere that can be used by a Java app, to hopefully help others, as I had to do a bit of additional research to figure this out.
Let's say you want to create a variable in WebSphere named ENV which contains a value of dev (or int, or prod, or any other value).
In the left panel of the WebSphere admin console, select Servers >
Server Types > WebSphere application servers.
Select the application server that contains the app.
Expand Java and Process Management and select process definition.
Select Java Virtual Machines.
Select Custom properties.
Select New.
Create the name and value of the variable and select OK.
Select Save.
Restart the application server for this change to take effect.
In this example, a variable named ENV with a vaule of "dev" was created.
Next, the Java app will need to be configured to use the ENV variable in WebSphere. In the below markup, the Java app has a class named "Environment". This class creates a variable named env. System.getProperty("ENV") is the magic that gets the variable from WebSphere. It is noteworthy that this Java code should also work with other application servers, such as JBoss or Tomcat, so you don't need to customize the Java code to a particular platform.
While definitely not required, I also am returning env. I am just doing this for demonstration, so that we can get the variable in a JSP page, so that we can see the variables with our own eyes in a JSP page, for validation that this works as expected.
package com.example.main;
public class Environment {
public String env;
public Environment() {
env = System.getProperty("ENV");
}
public String getEnvironment(){
return env;
}
}
Inside of the tags of a JSP page, I add the following markup to get the env variable from the Environment class, which in turn gets the ENV variable from WebSphere.
<%#page import="com.sample.main.Environment"%>
<%
Environment foo = new Environment();
String env = foo.getEnvironment();
out.print("Environment : " + env;
%>
Now, once the app has been deployed to WebSphere, the environment should be displayed, which is how I know that I was able to successfully get the variable from the application server.

The thread is kind of old but just wanted to provide some info. This is with WebSphere 8.5.5
I tried getting WebSphere variables defined in the console via [Environment > WebSphere variables] using
System.getProperty("Variable");
It did not give the variable to me. I looked around a bit on the web and came across the following:
https://www.setgetweb.com/p/WAS855/ae/was2873.html
The following function listed there returns the variables
private static String expandVariable(String s) throws
javax.management.JMException
{
com.ibm.websphere.management.AdminService as = com.ibm.websphere.management.AdminServiceFactory.getAdminService();
String server = as.getProcessName();
java.util.Set result = as.queryNames(new javax.management.ObjectName("*:*,"
+ "type=AdminOperations,process=" + server), null);
return (String)as.invoke((javax.management.ObjectName) result.iterator().next(),"expandVariable",
new Object[] {"${"+s+"}"}, new String[] {"java.lang.String"});
}
Then call
expandVariable("Variable");

I don't see anything there that says that those entries can be read via ctx.lookup( "java:comp/env/..." );

Can a JVM retrieve a list of agents that have been loaded into it via the attach api?

Is it possible to get a list of agents loaded into the current JVM by the Java 1.6 attach api? If so how?
Agents loaded at launch can be determined via RuntimeMXBean but I can't see a way to get a handle on ones added after launch.

(This question is similar to How to find list of java agents attached with a running JVM?. For the sake of completeness, I will add this answer to both questions.)
Checking agents that have been added using the Attach API:
If you are interested in the agents that have been added to an application at run time using the Attach API, you can use the DiagnosticCommandMBean.
This bean offers a diagnostic command called vmDynlib, a parameterless method that returns a String that list all dynamically loaded libraries.
Here is a snippet that prints all dynamic libraries loaded by the application's VM:
ObjectName diagnosticsCommandName = new ObjectName("com.sun.management:type=DiagnosticCommand");
String operationName = "vmDynlibs";
String result = (String) ManagementFactory.getPlatformMBeanServer().invoke(diagnosticsCommandName, operationName, null, null);
System.out.println(result);
This results in an output similar to this one:
Dynamic libraries:
0x00007ff7b8600000 - 0x00007ff7b8637000 C:\Program Files\Java\jdk1.8.0_181\bin\java.exe
0x00007ffdfeb00000 - 0x00007ffdfecf0000 C:\WINDOWS\SYSTEM32\ntdll.dll
0x00007ffdfe300000 - 0x00007ffdfe3b2000 C:\WINDOWS\System32\KERNEL32.DLL
0x00007ffdfbb30000 - 0x00007ffdfbdd3000 C:\WINDOWS\System32\KERNELBASE.dll
0x00007ffdfe950000 - 0x00007ffdfe9f3000 C:\WINDOWS\System32\ADVAPI32.dll
...
You can then check this text if it contains a certain .so or .dll file.
The same inspection can be performed non-programatically.
For this, you can use the jconsole tool.
Connect to a VM, switch to the tab MBeans, select com.sun.management, select DiagnosticCommand, select Operations, select vmDynlibs, and invoke it.
In the image, you can see one of my test agents attached to the application.
The agent was attached using the Attach API, thus this agent would not be visible by checking the application's command line arguments (i.e., no -agentpath=... would be seen on the arguments) but is only visible as dynamically loaded library.
Checking agents that have been added via command-line:
To have the complete reference, I will also post how to detect agents that have been added via the command line.
You can check them by using the RuntimeMXBean.
This bean offers the method getInputArguments, which returns a list of all VM arguments.
You can iterate over the list and check it for the arguments agentpath, agentlib and javaagent, similar to the following code snippet:
RuntimeMXBean runtimeMXBean = ManagementFactory.getRuntimeMXBean();
List<String> jvmArgs = runtimeMXBean.getInputArguments();
System.out.println("JVM arguments:");
for (String arg : jvmArgs) {
if (arg.startsWith("-agentpath") || arg.startsWith("-agentlib") || arg.startsWith("-javaagent")) {
System.out.print("***** ");
}
System.out.print(arg);
if (arg.startsWith("-agentpath") || arg.startsWith("-agentlib") || arg.startsWith("-javaagent")) {
System.out.println(" *****");
} else {
System.out.println();
}
}

No, I don't think there is a portable way to find out about the agents. What are you trying to accomplish? Maybe there is another approach...