We Keep Coding

Read Data from File in a Java Class Library Without Main Method

I have a set of Java console applications which act as my system's backend Services and which I package each into their own Jars. I also have written a class library (e.g. CoreLibrary) which consists of a set of classes and no main() execution method of it's own. This CoreLibrary is a dependency for each of these Services so I bundle The same CoreLibrary Jar into each of these Services Jars.
A number of classes in my CoreLibrary use some constant value(s) (configurations) for which I had defined a class like:
public class Config {
public static final String serverIP = "localhost:9092";
}
Now I want to shift this configuration into a config.json File like:
{
"serverIP" : "localhost:9092"
}
and access it through something like:
public class Config {
public static final String serverIP = getConfigValue();
private String getConfigValue()
{
try {
Reader reader = Files.newBufferedReader(pathToConfig);
CoreConfig item = gson.fromJson(reader, CoreConfig.class);
reader.close();
return item.getServerIP();
} catch (IOException e) {
e.printStackTrace();
return "localhost:9091";
}
}
}
Assuming that having a config file outside my application's Jar would allow me to change the file contents (values) each time before I run my Services Jars instead of going through the process of generating a Jar for each service every time I change this value. I also wouldn't want to load the value from the file from each Service but instead have the value loaded upon startup.
Is there a proper way to achieved this in Java (loading a value from a file from a class library on its own) ? and would it even be considered a valid code practice since item.getServerIP() may possibly produce NullPointerException if the file or json object couldn't be read ?
Note: I'm not using Spring or any such framework. It's plain Java 11 code and uses gson library.

Starting Instrumentation Agent after VM Startup

I was hoping for someone to explain this item since I might be getting this wrong:
I was reading about Java Agent Instrumentation which says that the agent can start after VM startup. So if I want to dynamically replace some class (without brining down the app) is this what I am going to go for using agent-main? Or do I need to do something more here?
I know people might ask "Are you talking about JRebel" - not really because I want to do something simple and JRebel is an overkill.
instrument docs - Java docs for Instrumentation
I understand all the instrumentation overrides, but I am slightly confused how I can hook this agent with -agent argument after the app has started.

First your agent class needs to specify an agentmain method like:
public class MyAgent {
public static void agentmain(final String args, final Instrumentation inst) {
try {
System.out.println("Agent loaded.");
} catch (Exception e) {
// Catch and handle every exception as they would
// otherwise be ignored in an agentmain method
e.printStackTrace();
}
}
}
Compile it and pack it inside a jar-file for example. If you choose the jar-variant then it must specify the Agent-Class key in its manifest-file (MANIFEST.MF). It points to the class implementing the agentmain method. It could look like:
Manifest-Version: 1.0
Agent-Class: package1.package2.MyAgent
If it is located inside those packages, as an example.
After that you can load the agent via the VirtualMachine#loadAgent method (documentation). Note that the mechanism used by those classes are part of the Attach library of Java. They decided, as most users don't need it, to not directly add it to the systems path but you can just add it. It is located at
pathToYourJDKInstallation\jre\bin\attach.dll
And it needs to be somewhere where the system property java.library.path is pointing at. You could for example just copy it to your .../Windows/System32 folder or adjust the property or stuff like that.
As an example, if you want to inject an agent-jar inside another currently running jar, you could use a method like this:
public void injectJarIntoJar(final String processIdOfTargetJar,
final String pathToAgentJar, final String[] argumentsToPass) {
try {
final VirtualMachine vm = VirtualMachine.attach(processIdOfTargetJar);
vm.loadAgent(pathToAgentJar, argumentsToPass.toString());
vm.detach();
} catch (AttachNotSupportedException | AgentLoadException
| AgentInitializationException | IOException e) {
System.err.println("Unable to inject jar into target jar.");
}
}
With the same technique you can inject dll-libraries (if they implement the corresponding agent-methods via the native agent interface) into jars.
Actually, if that helps you, I have written some small library for that kind of stuff some time ago. See Mem-Eater-Bug, the corresponding class is Injector.java and the whole project has a small Wiki.
It has an example showing how to use that technique to manipulate a SpaceInvaders game written as Java application.

So apparently you want to reload classes at runtime. Such that your project can react to changes of the code without restarting.
To achieve this you need to prepare your project and write a very clean architecture, it involves using interfaces, factory-patterns, proxy-patterns and a routine that checks for updates and then destroys and rebuilds all current objects.
Unfortunately this might not be an easy task, but it is doable, depending on the size of your project and the amount of code that should react dynamically to changes.
I found this article helpful, let me explain how it works. You can easily load a class with ClassLoader.loadClass(...) and you can also use that to reload a class, very easy. However at the time you have compiled your code classes are some kind of hardwired already. So your old code will continue to create instances of the old classes although you have reloaded the class.
This is the reason why we need some kind of architecture that allows exchanging the old class with the new class. Also it is pretty obvious that current instances of the old class can not automatically be transferred to the new version as everything could have changed. So you will also need a custom method that collects and rebuilds those instances.
The approach described in the article uses an Interface instead of an actual class in the first place. This allows to easily exchange the class behind that interface without breaking the code that uses the interface.
Then you need a factory where you ask for instances of that Interface. The factory can now check if the underlying class-file has changed, if so it reloads it and obtains a reference to the new class version. It can now always create an instance of the interface which uses the up-to-date class.
The factory, by that, is also able to collect all created instances in order to exchange them later, if the code base has changed. But the factory should reference them using WeakReference (documentation), else you have a big memory leak because the Garbage Collector would not be able to delete instances because the factory holds references to them.
Okay, now we are able to always obtain up-to-date implementations of an Interface. But how can we easily exchange existing instances. The answer is by using a proxy-pattern (explanation).
It is simple, you have a proxy class which is the actual object you are working with. It has all the methods of the Interface and upon calling methods it simply forwards to the real class.
Your factory, as it has a list of all current instances using WeakReference, can now iterate the list of proxies and exchange their real class with a new up-to-date version of the object.
Existing proxies that are used all around your project will now automatically use the new real version as the proxy itself has not changed, only its internal reference to the real target has changed.
Now some sample code to give you a rough idea.
The interface for the objects you want to monitor:
public interface IExample {
void example();
}
The real class, which you want to rebuild:
public class RealExample implements IExample {
#Override
public void example() {
System.out.println("Hi there.");
}
}
The proxy class that you will actually use:
public class ProxyExample implements IExample {
private IExample mTarget;
public ProxyExample(final IExample target) {
this.mTarget = target;
}
#Override
public void example() {
// Forward to the real implementation
this.mRealExample.example();
}
public void exchangeTarget(final IExample target) {
this.mTarget = target;
}
}
The factory you will mainly use:
public class ExampleFactory {
private static final String CLASS_NAME_TO_MONITOR = "somePackage.RealExample";
private final List<WeakReference<ProxyExample>> mInstances;
private final URLClassLoader mClassLoader;
public ExampleFactory() {
mInstances = new LinkedList<>();
// Classloader that will always load the up-to-date version of the class to monitor
mClassLoader = new URLClassLoader(new URL[] {getClassPath()}) {
public Class loadClass(final String name) {
if (CLASS_NAME_TO_MONITOR.equals(name)) {
return findClass(name);
}
return super.loadClass(name);
}
};
}
private IExample createRealInstance() {
return (IExample) this.mClassLoader.loadClass(CLASS_NAME_TO_MONITOR).newInstance();
}
public IExample createInstance() {
// Create an up-to-date instance
final IExample instance = createRealInstance();
// Create a proxy around it
final ProxyExample proxy = new ProxyExample(instance);
// Add the proxy to the monitor
this.mInstances.add(proxy);
return proxy;
}
public void updateAllInstances() {
// Iterate the proxies and update their references
// Use a ListIterator to easily remove instances that have been cleared
final ListIterator<WeakReference<ProxyExample>> instanceIter =
this.mInstances.listIterator();
while (instanceIter.hasNext()) {
final WeakReference<ProxyExample> reference = instanceIter.next();
final ProxyExample proxy = reference.get();
// Remove the instance if it was already cleared,
// for example by the garbage collector
if (proxy == null) {
instanceIter.remove();
continue;
}
// Create an up-to-date instance for exchange
final IExample instance = createRealInstance();
// Update the target of the proxy instance
proxy.exchangeTarget(instance);
}
}
}
And finally how to use it:
public static void main(final String[] args) {
final ExampleFactory factory = new ExampleFactory();
// Get some instances using the factory
final IExample example1 = factory.createInstance();
final IExample example2 = factory.createInstance();
// Prints "Hi there."
example1.example();
// Update all instances
factory.updateAllInstances();
// Prints whatever the class now contains
example1.example();
}

Attaching an agent at runtime requires use of the attach API which is contained in the tools.jar until Java 8 and is contained in its own module starting from Java 9. The location of the tools.jar and the name of its classes is system-dependent (OS, version, vendor) and as of Java 9 it does not exist at all but must be resolved via its module.
If you are looking for an easy way to access this functionality, try out Byte Buddy which has a subproject byte-buddy-agent for this. Create a Java agent as you are used to it but add anAgent-Main entry where you would put your Pre-Main in the manifest. Also, name the entry method agentmain, not premain.
Using byte-buddy-agent, you can write a program:
class AgentLoader {
public static void main(String[] args) {
String processId = ...
File agentJar = ...
ByteBuddyAgent.attach(processId, agentJar);
}
}
and you are done.

how to register a java class if the static initializer isn't called till the class is referenced

I've an interface implemented by classes that perform a file processing, say searching or whatever.
public interface FileProcessorInterface {
public void processFile(String fileName);
}
Then i have a different implementation for each file type:
public class TxtProcessor implements FileProcessorInterface {
#Override public void processFile(String fileName) { //do the work }
}
Thus i have the Utilizer of the processor, that has a method that allows for registering each class, something like this:
class Utilizer {
Map <String, Class> registered = new HashMap<>();
public void registerClass(String fileExt, Class clazz) {
registered.put(fileExt, clazz);
}
public void processFile(String fileName) {
//1) get the registered class from registered map (omitted because easy and not relevant)
//2) create an instance of the class using reflection (omitted because easy and not relevant)
FileProcessorInterface p = ....
p.processFile(fileName);
}
So far it's ok.
Now, i'm providing many implementations of my interface.
And i am tempted to provide each implementation class with a static initializer that register itself in the Utilizer, in the case of my previous TxtProcessor it would be:
class TxtProcessor implements FileProcessorInterface {
//previous code
static {
Utilizer.registerClass("txt", TxtProcessor.class);
}
}
The problem is that this static method will never be called because in the "statically reachable" code of the application there is no reference to my TxtProcessor class, since it is instantiated via reflection. So the jvm does not call the static initializer.
Say that i have two parts: the "generic code" that is the Utilizer and on the other side the implementations; it has to be thought as something provided dinamically and so it is not known by the Utilizer part.
Infact the idea was exactly that each class would register itself leaving the Utilizer untouched.
It is hard for me conceiving a solution that does not put some form of 'knowledge' of the implementations on the Utilizer side (and that stays simple), just because of the problem of the static initializer not called. How to overcome this?

Using reflections seems to be the best fit here. It's like geared to do this.
All you need is a small static block in Utilizer as
static {
Reflections reflections = new Reflections(
new ConfigurationBuilder()
.setUrls(ClasspathHelper.forPackage("path.to.all.processors.pkg"))
.setScanners(new SubTypesScanner())
);
reflections.getSubTypesOf(path.to.all.processors.pkg.FileProcessor.class);
}
If you don't want a third-part dependency, just add a FileProcessors.properties file to your classpath
txt=path.to.all.processors.pkg.TxtProcessor
doc=path.to.all.processors.pkg.DocProcessor
pdf=path.to.all.processors.pkg.PdfProcessor
and then register all the listed classes from Utilizer as
static {
Properties processors = new Properties();
try {
processors.load(Utilizer.class
.getResourceAsStream("FileProcessors.properties"));
} catch (IOException e) {
e.printStackTrace();
}
for (String ext : processors.stringPropertyNames()) {
Utilizer.registerClass(ext, Class.forName(processors.getProperty(ext));
}
}
This no longer requires a static block in every FileProcessor now.

You can look at Reflections library. It allow you to find all the classes which implement an interface, have an annotation or extend a class.

You Could...
Use the same concept as JDBC does for loading it's drivers. This would require you to use Class#forName to initialize the class when the program is first loaded. While this does mean that the implementation is still dynamic from the point of view of your utility class, it is specified at run time by your application...
This gives you control over which implementation you might want to use
You Could...
Use the same concept as something like java.awt.Toolkit uses when it initializes it's instance.
It basically looks up the resource (in this case a System property) and then loads the class dynamically using Class.
Personally, I normally look for a named resource (usually a properties file) and load a key from it.
Something like getClass().getResource("/some/gloabl/configFile");, which every implementation would need to provide.
Then, if available, read the properties file and find the key I'm after.
If more then one implementation is linked in though, there is no guarantee which one will be loaded.

Quick and dirty: You can statically initialize your Utilizer in main() with correct association.
Better solution: externalize in a resource file association like
txt=path.to.package.TxProcessor
load it in Utilizer and load FileProcessorInterface implementors with Class.forName()

you can force the static init by Class.forName(fqn, true, classLoader) or the short form Class.forName(fqn)

You could have a registry file (for example, some XML file), that would contain the list of all classes you support :
<item type="txt">somepackage.TxtProcessor</item>
<item type="gif">somepackage.GIFProcessor</item>
...
Your Utilizer would load this file into its registry.

properties at application level

I am currently having a properties file and I am loading this file in each class where there is a need to get the properties
static PropertiesConfiguration config = null;
config = new PropertiesConfiguration("Interface.properties");
This is working fine. But I know this is not the efficient way to load properties file multiple times.
Can anyone help me how to create the properties file at application level and create a java file (say config.java) + calling the method which returns the property.

One way to solve this is to utilize the dependency injection. You can have a singleton bean that holds the property instance and then you inject this bean as a dependency to each of your class that needs to access those properties.
One way to implement dependency injection to use the Spring framework. For instance, you can achieve loading the property file using Spring's PropertyPlaceHolderConfigurer. Then this bean becomes your reference bean for rest of the application. A tutorial on this can be found here.
Another choice for you is to load the property file in the constructor of the main entry class or the main method of your application and then pass the object to the classes that needs it. However, this will make your application more tightly coupled and maintenance would be harder in the future.
Another option is to create a singleton class that loads the properties and have a method that returns the values as needed.

I used this approach to set the properties at application level.
Define a properties file (say configure.properties).
Create a java class Config:
public class Config {
private static Config instance;
private PropertiesConfiguration configure;
private Config() throws ConfigurationException {
configure = new PropertiesConfiguration("configure.properties");
}
public static Config getInstance() {
if (null == instance) {
try {
instance = new Config();
} catch (ConfigurationException ex) {
throw new RuntimeException(ex);
}
}
return instance;
}
public PropertiesConfiguration getConfigure() {
return configure;
}
public void setConfig(PropertiesConfiguration configure) {
this.configure = configure;
}
}
This java class load the properties file at the startup and calls the getInstance method to get the value of the property. To get the value of the property anywhere else in the application import Config and
Config.getInstance().getConfig().getString("property.given.in.properties");
Sorry if variable names doesn't make much sense.
Thanks all for your input.

I just load my properties file in the System Properties:
public void loadStreamToSysProperties(InputStream in) throws IOException
{
Properties p = System.getProperties();
p.load(in);
}
Then I can get them out of System.getProperty where ever I am in the code.
In this example the InputStream is a Stream I created from the file name/path.

How can I load java class from database?

With the following source code:
package util.abc;
public class Test{
public String out(){
return "Hello World!";
}
}
I can use:
Class c = Class.forName("util.abc.Test");
to instantiate this Class, but I must put this source file(Test.java) in the classpath /util/abc/
I want to dynamically load this class from the database (store the source code as string or binary)
Is this possible ?

Assuming you already compiled the class, you can create a DatabaseClassLoader, which loads the class from a database.
public class DatabaseClassLoader extends ClassLoader {
public DatabaseClassLoader(ClassLoader parent, ... /* connection to database */ ) {
super(parent);
// store the connection
}
public Class findClass(String name) {
byte[] data = loadDataFromDatabase(name);
return defineClass(name, data, 0, data.length);
}
private byte[] loadDataFromDatabase(String name) {
// this is your job.
}
}
If the database only contains the source code, you'll have to compile it first - look into the Java compiler API for how to do this without any files.
Pay attention, the class loaded this way will stay alive as long as the classloader is alive, so you'll need a new class loader to be able to reload the class in case of changes.
Also, if you want to interact with the class by other ways than reflection, you would better let it implement some interface (which itself is in your class path), and let the application class loader be the parent of your database class loader.
Ah, and how to load:
Class<?> c = Class.forName("util.abc.Test", myClassLoader);
or directly
Class<?> c = myClassLoader.loadClass("util.abc.Test");
Here is a method which creates objects of your interface (of any interface, in fact):
public <X> X getImplementingObject(Class<X> interfaceClass, String className)
throws ClassNotFoundException, IllegalAccessException, InstantiationException
{
ClassLoader loader = new DatabaseClassLoader(interfaceClass.getClassLoader(), ...);
Class<?> cl = loader.loadClass(className);
Class<? extends X> c = cl.asSubclass(interfaceClass);
return c.newInstance();
}
(It needs the class to have a no-argument constructor which does not throw any exceptions, of course (if it does, you'll get this exception thrown, too).)
This creates a new ClassLoader for every such class, so they can only cooperate with each other by means of the interface (or reflection).
For on-the-fly compiling you should look at the Java Compiler API, as mentioned in the answer from dcn. But I think it would be better to do the compiling on the side which puts the classes into the database than the side who pulls them out.

If you want to store the source code in you DB, you can use the Java 6 Compiler API to compile it at runtime. See here for an example.
In order to load classes at runtime, you can either use an URLClassLoader if you can specify the location of the bytecode with a URL, or use ClassLoader.defineClass and supply the bytecode as an array of bytes.
Either way you should pay attention, that in order to use your dynamically loaded class, it should implement an interface that is known at compile time.