After a WebLogic app has been running fine for a few weeks I suddenly get an Exception:
<Oct 25, 2014 9:31:11 PM EDT> <Error> <HTTP> <BEA-101020>
<[ServletContext#60724164[app:whatever3000 module:whatever3000.war path:
spec-version:2.5]] Servlet failed with Exception
java.lang.ExceptionInInitializerError
after which the application is completely defunct with NoClassDefFoundError until the app server is restarted.
The full stack trace shows the origin of the issue is a ConcurrentModificationException in a static initializer.
Specifically the equivalent / minimized code is as follows:
package a;
import b;
public class Whatever {
void doIt()
{
Password p = new Password();
}
}
package b;
public final class Password implements Serializable
{
private static final int PARAM1 = CommonStuff.someStaticMethod();
...
}
import java.util.Properties;
public class CommonStuff
{
private static Properties prp = new Properties();
static {
CommonStuff.load();
}
public static void load()
{
prp.putAll(System.getProperties()); <---FAIL
This is the origin of the Exception:
java.util.ConcurrentModificationException
at java.util.Hashtable$Enumerator.next(Hashtable.java:1017)
at java.util.Hashtable.putAll(Hashtable.java:469)
at b.CommonStuff.load(CommonStuff.java:55)
at b.CommonStuff.<clinit>(CommonStuff.java:77)
at b.Password.<clinit>(Password.java:44)
at a.doIt(Whatever.java:99)
So it seems at some point during the runtime of the application, WebLogic decided to reload classes from package b but when the static block runs it finds that the Properties object has been modified.
I don't know if it's being called concurrently or if it's being called multiple times. It could be that the Properties object is the original instance created when the app loaded fresh the first time and the reloading of CommonStuff class is trying to call putAll() again.
Would it help if I do:
private static Properties prp = null;
static {
CommonStuff.prp = new Properties();
CommonStuff.load();
}
I cannot just try things blindly because it's in a production app at a huge company. So I'm trying to understand where I'm going wrong and how to property initialize these variables while classes are being reloaded in the middle of the night.
Any ideas?
Could this be a WebLogic ClassLoader issue?
Classes/instances cannot access some class's members before this class is initialized. Because of this no one can access newly created prp before static constructor returns. Moving prp initializer inside on static {} block makes no difference. Also "old" prp in the "old" CommonStuff and "new" prp are not connected in any way (because "old" and "new" CommonStuff are different classes for JVM). This all make possibility of concurrent modifications of prp look pretty weird.
I believe the reason is in another place. Notice the first line of the stack trace: exception is thrown by Enumerator of Hashtable. Here is the code of the putAll method (as in JDK 8, probably didn't change for many years):
for (Map.Entry<? extends K, ? extends V> e : t.entrySet())
put(e.getKey(), e.getValue());
Here is the Enumerator that throws an exception - and it is not prp, it is argument's Enumerator.
So the exception is related not to prp but to Map returned by System.getProperties(). The reason is that iterating over system properties map is not thread safe. It seems that another thread is modifying it at the same time.
You need to initialize prp differently. I think clone() is the simplest way.
Related
As I installed Sonarlint plugin in my eclipse IDE, i got the above (Heading) Major Issue in Sonar Report.
The Code is:
public class Demo
{
public static final Map<String, String> CARD_TYPES;
static
{
CARD_TYPES = new HashMap<String, String>()
{
{ //Move the contents of this initializer to a standard constructor or to field* initializers
put("visa", "001");
put("diner", "002");
}
};
}
//..code goes here
}
The Query is: what exactly should be done in above Static Block, to
Resolve the above issue ?
Your code example seems to be altered for this post? Doesn't compile because CARD_TYPES is not declared anywhere. Also there is a * in front of the inline comment which shouldn't be there.
Regardless, in your SonarLint report you can select a specific issue.
which then allows you to see the details of the Rule that it is reporting this issue.
These descriptions also contain compliant and not compliant code examples to help you understand the issue. In your case a compliant solution would be.
Basically: initialize the map directly at the declaration. Then use the static context of the class to access and fill it, rather than the context of the map.
For performance reasons, I have a class that stores a Map whose key is a Class<?> and its value is function of that class's fields. The map is populated during code execution according to the type of the calling object. The above is a generalization/simplification
public class Cache {
private static final Map<Class<?>, String> fieldsList = ...;
//Synchronization omitted for brevity
public String getHqlFor(Class<?> entity){
if (!fieldsList.containsKey(entity))
fieldsList.put(entity,createHql(entity));
return fieldsList.get(entity);
}
}
During development, thanks to the help of Jrebel, I often make modifications to classes by changing entire properties or just their names. I can continue development just fine. However, if I already put a value into the cache it will be stale forever.
What I am asking here is if it is possible to intercept the event that a class in the classpath has changed. Very broad... But my specific problem is very simple: since I have such a need only during development, I just want to wipe that cache in case any class in my classpath changes.
How can I accomplish this? I don't need to do anything special than intercepting the event and simply wiping the cache
JRebel has a plugin API that you can use to trigger code on class reloads. The tutorial complete with example application and plugin available here: https://manuals.zeroturnaround.com/jrebel/advanced/custom.html
The JRebel plugin is a self-contained jar built against the JRebel SDK, which is attached to the running application via the JVM argument -Drebel.plugins=/path/to/my-plugin.jar. The JRebel agent attached to the application will load and start plugins from this argument.
If the application is not started with the JRebel agent, the plugin is simply not loaded.
In your example you want to register a ClassEventListener that will clear the Cache.fieldsList map. As it is a private field, you need to access it via reflection or add a get/clear method via a ClassBytecodeProcessor
public class MyPlugin implements Plugin {
void preinit() {
ReloaderFactory.getInstance().addClassReloadListener(new ClassEventListenerAdapter(0) {
#Override
public void onClassEvent(int eventType, Class<?> klass) throws Exception {
Cache.clear();
}
});
}
// ... other methods ...
}
And to clear the map
public class CacheCBP extends JavassistClassBytecodeProcessor {
public void process(ClassPool cp, ClassLoader cl, CtClass ctClass) {
ctClass.addMethod(CtMethod.make("public static void clear() { fieldsList.clear(); }", ctClass));
}
}
However a better option is to only clear/recalculate the single class entry on class reload if possible. The example didn't display whether the info computed from one class depended on superclass infos, but if this is true, the JRebel SDK has methods to register a reload listener on the class hierarchy as well.
There is an existing class ClassValue which already does the job for you:
public class Cache {
private final ClassValue<String> backend = new ClassValue<String>() {
#Override
protected String computeValue(Class<?> entity) {
return createHql(entity);
}
};
public String getHqlFor(Class<?> entity){
return backend.get(entity);
}
}
When you call get, it will call computeValue if this is the first call for this specific Class argument or return the already existing value otherwise. It does already care thread safety and for allowing classes to get garbage collected. You don’t need to know when class unloading actually happens.
I was hoping for someone to explain this item since I might be getting this wrong:
I was reading about Java Agent Instrumentation which says that the agent can start after VM startup. So if I want to dynamically replace some class (without brining down the app) is this what I am going to go for using agent-main? Or do I need to do something more here?
I know people might ask "Are you talking about JRebel" - not really because I want to do something simple and JRebel is an overkill.
instrument docs - Java docs for Instrumentation
I understand all the instrumentation overrides, but I am slightly confused how I can hook this agent with -agent argument after the app has started.
First your agent class needs to specify an agentmain method like:
public class MyAgent {
public static void agentmain(final String args, final Instrumentation inst) {
try {
System.out.println("Agent loaded.");
} catch (Exception e) {
// Catch and handle every exception as they would
// otherwise be ignored in an agentmain method
e.printStackTrace();
}
}
}
Compile it and pack it inside a jar-file for example. If you choose the jar-variant then it must specify the Agent-Class key in its manifest-file (MANIFEST.MF). It points to the class implementing the agentmain method. It could look like:
Manifest-Version: 1.0
Agent-Class: package1.package2.MyAgent
If it is located inside those packages, as an example.
After that you can load the agent via the VirtualMachine#loadAgent method (documentation). Note that the mechanism used by those classes are part of the Attach library of Java. They decided, as most users don't need it, to not directly add it to the systems path but you can just add it. It is located at
pathToYourJDKInstallation\jre\bin\attach.dll
And it needs to be somewhere where the system property java.library.path is pointing at. You could for example just copy it to your .../Windows/System32 folder or adjust the property or stuff like that.
As an example, if you want to inject an agent-jar inside another currently running jar, you could use a method like this:
public void injectJarIntoJar(final String processIdOfTargetJar,
final String pathToAgentJar, final String[] argumentsToPass) {
try {
final VirtualMachine vm = VirtualMachine.attach(processIdOfTargetJar);
vm.loadAgent(pathToAgentJar, argumentsToPass.toString());
vm.detach();
} catch (AttachNotSupportedException | AgentLoadException
| AgentInitializationException | IOException e) {
System.err.println("Unable to inject jar into target jar.");
}
}
With the same technique you can inject dll-libraries (if they implement the corresponding agent-methods via the native agent interface) into jars.
Actually, if that helps you, I have written some small library for that kind of stuff some time ago. See Mem-Eater-Bug, the corresponding class is Injector.java and the whole project has a small Wiki.
It has an example showing how to use that technique to manipulate a SpaceInvaders game written as Java application.
So apparently you want to reload classes at runtime. Such that your project can react to changes of the code without restarting.
To achieve this you need to prepare your project and write a very clean architecture, it involves using interfaces, factory-patterns, proxy-patterns and a routine that checks for updates and then destroys and rebuilds all current objects.
Unfortunately this might not be an easy task, but it is doable, depending on the size of your project and the amount of code that should react dynamically to changes.
I found this article helpful, let me explain how it works. You can easily load a class with ClassLoader.loadClass(...) and you can also use that to reload a class, very easy. However at the time you have compiled your code classes are some kind of hardwired already. So your old code will continue to create instances of the old classes although you have reloaded the class.
This is the reason why we need some kind of architecture that allows exchanging the old class with the new class. Also it is pretty obvious that current instances of the old class can not automatically be transferred to the new version as everything could have changed. So you will also need a custom method that collects and rebuilds those instances.
The approach described in the article uses an Interface instead of an actual class in the first place. This allows to easily exchange the class behind that interface without breaking the code that uses the interface.
Then you need a factory where you ask for instances of that Interface. The factory can now check if the underlying class-file has changed, if so it reloads it and obtains a reference to the new class version. It can now always create an instance of the interface which uses the up-to-date class.
The factory, by that, is also able to collect all created instances in order to exchange them later, if the code base has changed. But the factory should reference them using WeakReference (documentation), else you have a big memory leak because the Garbage Collector would not be able to delete instances because the factory holds references to them.
Okay, now we are able to always obtain up-to-date implementations of an Interface. But how can we easily exchange existing instances. The answer is by using a proxy-pattern (explanation).
It is simple, you have a proxy class which is the actual object you are working with. It has all the methods of the Interface and upon calling methods it simply forwards to the real class.
Your factory, as it has a list of all current instances using WeakReference, can now iterate the list of proxies and exchange their real class with a new up-to-date version of the object.
Existing proxies that are used all around your project will now automatically use the new real version as the proxy itself has not changed, only its internal reference to the real target has changed.
Now some sample code to give you a rough idea.
The interface for the objects you want to monitor:
public interface IExample {
void example();
}
The real class, which you want to rebuild:
public class RealExample implements IExample {
#Override
public void example() {
System.out.println("Hi there.");
}
}
The proxy class that you will actually use:
public class ProxyExample implements IExample {
private IExample mTarget;
public ProxyExample(final IExample target) {
this.mTarget = target;
}
#Override
public void example() {
// Forward to the real implementation
this.mRealExample.example();
}
public void exchangeTarget(final IExample target) {
this.mTarget = target;
}
}
The factory you will mainly use:
public class ExampleFactory {
private static final String CLASS_NAME_TO_MONITOR = "somePackage.RealExample";
private final List<WeakReference<ProxyExample>> mInstances;
private final URLClassLoader mClassLoader;
public ExampleFactory() {
mInstances = new LinkedList<>();
// Classloader that will always load the up-to-date version of the class to monitor
mClassLoader = new URLClassLoader(new URL[] {getClassPath()}) {
public Class loadClass(final String name) {
if (CLASS_NAME_TO_MONITOR.equals(name)) {
return findClass(name);
}
return super.loadClass(name);
}
};
}
private IExample createRealInstance() {
return (IExample) this.mClassLoader.loadClass(CLASS_NAME_TO_MONITOR).newInstance();
}
public IExample createInstance() {
// Create an up-to-date instance
final IExample instance = createRealInstance();
// Create a proxy around it
final ProxyExample proxy = new ProxyExample(instance);
// Add the proxy to the monitor
this.mInstances.add(proxy);
return proxy;
}
public void updateAllInstances() {
// Iterate the proxies and update their references
// Use a ListIterator to easily remove instances that have been cleared
final ListIterator<WeakReference<ProxyExample>> instanceIter =
this.mInstances.listIterator();
while (instanceIter.hasNext()) {
final WeakReference<ProxyExample> reference = instanceIter.next();
final ProxyExample proxy = reference.get();
// Remove the instance if it was already cleared,
// for example by the garbage collector
if (proxy == null) {
instanceIter.remove();
continue;
}
// Create an up-to-date instance for exchange
final IExample instance = createRealInstance();
// Update the target of the proxy instance
proxy.exchangeTarget(instance);
}
}
}
And finally how to use it:
public static void main(final String[] args) {
final ExampleFactory factory = new ExampleFactory();
// Get some instances using the factory
final IExample example1 = factory.createInstance();
final IExample example2 = factory.createInstance();
// Prints "Hi there."
example1.example();
// Update all instances
factory.updateAllInstances();
// Prints whatever the class now contains
example1.example();
}
Attaching an agent at runtime requires use of the attach API which is contained in the tools.jar until Java 8 and is contained in its own module starting from Java 9. The location of the tools.jar and the name of its classes is system-dependent (OS, version, vendor) and as of Java 9 it does not exist at all but must be resolved via its module.
If you are looking for an easy way to access this functionality, try out Byte Buddy which has a subproject byte-buddy-agent for this. Create a Java agent as you are used to it but add anAgent-Main entry where you would put your Pre-Main in the manifest. Also, name the entry method agentmain, not premain.
Using byte-buddy-agent, you can write a program:
class AgentLoader {
public static void main(String[] args) {
String processId = ...
File agentJar = ...
ByteBuddyAgent.attach(processId, agentJar);
}
}
and you are done.
I know Java loads Classes in first Access (creating new Instance, calling static method or static field), but in this simple example I try to execute a jar file that uses some classes which there aren't in my ClassPath at run time. I expect (because of loading classes in first access) print my messages in static block and main method before an exception occurred. but I got "Exception in thread "main" java.lang.NoClassDefFoundError: com/example/DateAbstract" and nothing printed.
This occurred when I used an abstract class or interface in main class which that classes or interfaces are in another jar file.
public class Driver {
static { System.out.println("I am first.[static block]"); }
public static void main(String[] args) {
System.out.println("I am first.[ main method]");
DateAbstract date = new CustomDate();
System.out.println(date.sayDate());
}
in my another jar :
public class CustomDate extends DateAbstract {
#Override
public String sayDate() {
return new Date().toString();
}
public abstract class DateAbstract {
public abstract String sayDate();
}
when I use this code for add my classes to classpath at runtime. nothing changed. I got execption before execute static block.
public class Driver {
static {
System.out.println("I am first.[static block]");
try {
URL url = new File("lib/DateApi.jar").toURI().toURL();
URLClassLoader urlClassLoader = (URLClassLoader) URLClassLoader.getSystemClassLoader();
Method method = URLClassLoader.class.getDeclaredMethod("addURL", URL.class);
method.setAccessible(true);
method.invoke(urlClassLoader,url);
} catch (Exception e) {
e.printStackTrace();
}
}
public static void main(String[] args) {
System.out.println("I am first.[ main method]");
DateAbstract date = new CustomDate();
System.out.println(date.sayDate());
}
}
Questions :
why is this happening and how to solve it ?
It’s not correct to say that in Java classes are loaded on their first access. You are confusing this with the initialization of a class, which implies executing the Java code of static initializer blocks and field initializers. The loading and verification might happen at an earlier time; the specification provides some freedom to the JVMs in this regard.
The key point here is that your main method instantiates an object of type CustomDate, stores it into a variable of the compile-time type DateAbstract and then tries to invoke sayDate() on that variable. This combination of instantiating CustomDate and invoking DateAbstract.sayDate() on it requires the verification of its correctness, i.e. whether CustomDate is a subtype DateAbstract. So the loading of these two classes will already happen at verification time.
You can easily check that this is the cause. If you change the type of the local variable date to CustomDate, the instantiated type and the receiver type of the method invocation are the same, so the correctness can be proven without loading the type, so it will be indeed deferred to the actual attempt to instantiate CustomDate, hence the messages will be printed.
Still, the loading time is an implementation-specific detail. A different JVM could load the referenced classes eagerly, even if they are not required for verification. The only safe way to ensure a deferred loading, is to use dynamic loading, e.g. Class.forName(String). Note that within the class detached this way, all types might be again referenced ordinarily. So if you do the dynamic loading once after the class path has been adjusted, there is not much impact on how you have to write the code nor its performance. Of course, having the code adjusting the class path and the code depending on it within the same class won’t work reliably.
Unsure about static variables.
import java.io.File;
public class Logger {
public static final File log = new File(File.listRoots()[0], "log.log");
public static void log (String message) {
/* ... */
}
}
Is the variable log pointing to the same memory throughout the life of the program? Basically is the log definition new File(File.listRoots()[0], "log.log") calculated multiple times or just one, and when?
Thanks in advance.
It is invoked once per classloader. Which, normally, means once.
A static variable is initialized as soon as the class declaring it is loaded by the classloader, and stays there until the classloader is destroyed, which in most cases means - at the end of the program execution / application lifecycle.
Just once. AFAIK, when class is loaded.