Rewritten from scratch # Friday, 25 May, about 16:00 GMT
(Code is cleaner now, bug can be reproduced and the question is more clear)
Original problem: I'm writing a server app that's required to accept files from clients over the net and process them with certain classes, which are loaded from locally stored .jar-files via URLClassLoader. Almost everything works correctly, but those jar-files are hot-swapped (without restarting the server app) from time to time to apply hotfixes, and if we're unlucky enough to update .jar-file at the same time class from it is being loaded, ClassFormatError is thrown, with remarks about "truncated class" or "excess bytes at the end". That's to be expected, but the whole application becomes unstable and starts to behave weird after that - those ClassFormatError exceptions keep happening when we try to load the class again from the same jar that was updated, even though we use new instance of URLClassLoader and it happens in different app thread.
The app is running and compiled on Debian Squeeze 6.0.3/Java 1.4.2, migration is not within my power.
Here's a simple code that mimics app behavior and roughly describes the problem:
1) Classes for main app and per-client threads:
package BugTest;
public class BugTest
{
//This is a stub of "client" class, which is created upon every connection in real app
public static class clientThread extends Thread
{
private JarLoader j = null;
public void run()
{
try
{
j = new JarLoader("1.jar","SamplePlugin.MyMyPlugin","SampleFileName");
j.start();
}
catch(Exception e)
{
e.printStackTrace();
}
}
}
//Main server thread; for test purposes we'll simply spawn new clients twice a second.
public static void main(String[] args)
{
BugTest bugTest = new BugTest();
long counter = 0;
while(counter < 500)
{
clientThread My = null;
try
{
System.out.print(counter+") "); counter++;
My = new clientThread();
My.start();
Thread.currentThread().sleep(500);
}
catch(Exception e)
{
e.printStackTrace();
}
}
}
}
2) JarLoader - a wrapper for loading classes from .jar, extends Thread. Here we load a class which implements a certain interface a:
package BugTest;
import JarPlugin.IJarPlugin;
import java.io.File;
import java.io.FileNotFoundException;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
public class JarLoader extends Thread
{
private String jarDirectory = "jar/";
private IJarPlugin Jar;
private String incomingFile = null;
public JarLoader(String JarFile, String JarClass, String File)
throws FileNotFoundException, MalformedURLException, ClassNotFoundException, InstantiationException, IllegalAccessException
{
File myjarfile = new File(jarDirectory);
myjarfile=new File(myjarfile,JarFile);
if (!myjarfile.exists())
throw new FileNotFoundException("Jar File Not Found!");
URLClassLoader ucl = new URLClassLoader(new URL[]{myjarfile.toURL()});
Class JarLoadedClass =ucl.loadClass(JarClass);
// ^^ The aforementioned ClassFormatError happens at that line ^^
Jar = (IJarPlugin) JarLoadedClass.newInstance();
this.setDaemon(false);
incomingFile = File
}
public void run()
{
Jar.SetLogFile("log-plug.txt");
Jar.StartPlugin("123",incomingFile);
}
}
3) IJarPlugin - a simple interface for pluggable .jars:
package JarPlugin;
public interface IJarPlugin
{
public void StartPlugin(String Id, String File);
public void SetLogFile(String LogFile);
}
4) the actual plugin(s):
package SamplePlugin;
import JarPlugin.IJarPlugin;
public class MyMyPlugin implements IJarPlugin
{
public void SetLogFile(String File)
{
System.out.print("This is the first plugin: ");
}
public void StartPlugin(String Id, String File)
{
System.out.println("SUCCESS!!! Id: "+Id+",File: "+File);
}
}
To reproduce the bug, we need to compile a few different .jars using same class name, whose only difference is number in "This is the Nth plugin: ". Then start the main application, and then rapidly replace the loaded plugin file named "1.jar" with other .jars, and back, mimicing the hotswap. Again, ClassFormatError is to be expected at some point, but it keeps happening even when the jar is completely copied (and is NOT corrupt in any way), effectively killing any client threads which try to load that file; the only way to get out from this cycle is to replace the plugin with another one. Seems really weird.
The actual cause:
It all became sort of clear once I simplified my code even more and got rid of clientThread class, simply instancing and starting the JarLoader inside the while loop in main. When ClassFormatError was thrown, it not just printed the stack trace out, but actually crashed the whole JVM (exit with code 1). The reason is not as obvious as it seems now (it wasn't for me, at least): ClassFormatError extends Error, not Exception. Hence it passes through catch(Exception E) and the JVM exits because of uncaught exception/error, BUT since I spawned thread which caused error from another spawned (client) thread, only that thread crashed. I guess it's because of the way Linux handles Java threads, but I'm not sure.
The (makeshift) solution:
Once uncaught error cause became clear, I tried to catch it inside the "clientThread". It sort of worked (I removed the stacktrace printout and printed my own message), but the main problem was still present: the ClassFormatError, even though caught properly, kept happening until I replace or remove the .jar in question. So I took a wild guess that some sort of caching might be a culprit, and forced URLClassLoader reference invalidation and Garbage Collection by adding this to clientThread try block:
catch(Error e)
{
System.out.println("Aw, an error happened.");
j=null;
System.gc();
}
Surprisingly, it seems to work! Now error only happens once, and then file class just loads normally, as it should. But since I just made an assumption, but not understood a real cause, I'm still worried - it works now, but there's no guarantee that it will work later, inside a much more complicated code.
So, could anyone with deeper understanding of Java enlighten me on what's the real cause, or at least try to give a direction? Maybe it's some known bug, or even expected behavior, but it's already way too complicated for me to understand on my own - I'm still a novice. And can I really rely on forcing GC?
Related
I am new to programming and trying to insert the mp3 file on Mac, but I have errors with these codes. I have been looking for solutions for a long time but I was not able to find the right answers. I would like to know what I did wrong.
import javazoom.jl.player.Player;
import java.io.BufferedInputStream;
import java.io.FileInputStream;
public class Music {
public static void main(String [] args) {
String filename = "src_music_typing.mp3";
MusicPlayer music = new MusicPlayer(filename);
music.play();
}
}
class MusicPlayer {
private final String mp3File;
private Player jlPlayer;
public MusicPlayer(String mp3File) {
this.mp3File = mp3File;
}
public void play() {
try {
FileInputStream fis = new FileInputStream(mp3File);
BufferedInputStream bis = new BufferedInputStream(fis);
jlPlayer = new Player(bis);
} catch (Exception e) {
System.out.println("problem file is " + mp3File);
System.out.println(e.getMessage());
}
new Thread() {
public void run() {
try {
jlPlayer.play();
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
}.start();
}
public void close() {
if(jlPlayer != null) jlPlayer.close();
}
}
Problem:
problem file is src_music_typing.mp3
src_music_typing.mp3 (No such file or directory)
Cannot invoke "javazoom.jl.player.Player.play()" because "this.this$0.jlPlayer" is null
The error is telling you simply that src_music_typing.mp3 does not exist; evidently you aren't running this in the directory you think you're running it in. Trivial solution: Make that path string (String filename = "src_...") an absolute path instead.
NB: It's a cavalcade of problems, here. Your code is bad and it leads to inefficient error messages. Inefficient enough to confuse you, for example.
You should never catch an exception just to log it and then blindly continue; I know a ton of code snippets do this, but that part of them is just bad. You don't want to do that - dealing with an error by blindly continuing on is, obviously, a really silly thing to do!
The right way to deal with exceptions that you don't explicitly know how to handle is instead to just throw them on. your play method should be declared as throws IOException, as this is inherent to your API design, this is fine (it's inherent because your music player class as a property that represents a file name, and anything file related is expected to throw IOExceptions, hence, fine - not leaking an abstraction).
Then the whole try/catch bit can just go away, yay! Your code is better and shorter and easier to understand, win win win!
Because you didn't do that, and you just run blindly on, you get a second error that is complaining about attempting to invoke play() on a null pointer. This error is meaningless, in that it's merely a symptom, not the cause. The cause is the first error message. This is one of a few key reasons why 'keep blindly going' is a really bad idea - it means you get a ton of meaningless, confusing errors after the actual problem, resulting in a ton of error output, most of which is just hiding the actual problem.
If you can't throw them on, a distant second best solution is to put this in your catch blocks: throw new RuntimeException("uncaught", e);. This preserves all error information (type, message, stack trace, causal chain - all of it), and still ensures code does not blindly continue when your method is an unknown (to you) state. If you have an IDE that inserts catch blocks for you, update its template.
NB: main can and usually should be declared as static void main(String[] args) throws Exception {.
https://github.com/IshayKom/RCProject
Classes As Shown in the eclipse package manager
(I don't really deal with github so I don't know how to upload classes correctly)
I got an error running my project.
I use VMWare to work on my project but I don't think that this specific error requires the use of multiple PCs or VMs.
It basically should receive ClientInformation from the InitiHandler class and start the process of matching two clients together. (Security isn't required in this project at the moment)
The steps to recreate this issue as follows: Enabling the "server" with the required information. After that go on "controlled client" mode, write the required information, and attempt to send info to the server.
I tried searching for a solution and looking at what mistake I did this time but I just can't get my mind into it. If anyone can spot my mistake it'll super helpful.
The following is the class which the error happened in:
package Server;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.net.ServerSocket;
import java.net.Socket;
public class ServerInitializer extends Thread {
private int port;
private ClientInformation[] ci = null;
private ClientInformation c = null;
private boolean run = true;
ServerSocket serversocket;
ObjectInputStream ois;
Socket client;
public ServerInitializer(int port, int clientlimit) {
this.port = port;
ci = new ClientInformation[clientlimit];
start();
}
#Override
public void run() {
try
{
serversocket = new ServerSocket(port);
while(run)
{
client = serversocket.accept();
System.out.println("New Client Has Been Connected, ip:" + client.getInetAddress().getHostAddress());
ois = new ObjectInputStream(client.getInputStream());
c = (ClientInformation) ois.readObject();
new ServerThread(ci,c, client, run);
}
serversocket.close();
}
catch (IOException | ClassNotFoundException e) {
e.printStackTrace();
}
}
public void terminate()
{
this.run = false;
}
public boolean getrun()
{
return run;
}
}
The error itself:
"Exception in thread "Thread-0" java.lang.ClassCastException: class Startup.ClientInformation cannot be cast to class Server.ClientInformation (Startup.ClientInformation and Server.ClientInformation are in unnamed module of loader 'app')
at Server.ServerInitializer.run(ServerInitializer.java:35)"
If it's too much of a pain to see my mistake based on what I currently wrote let me know what to do to make it easier for you to spot it.
Another Question: How can I use the terminate function to basically disable any while loops that are happening in other classes/threads? or what I wrote is good enough? I couldn't test this because of my error. (This can apply to the server itself in my project or the client-side)
You have a class named ClientConfiguration, which has package Server; on the top. You have a completely different, totally unrelated class which by pure coincidence is also named ClientConfiguration, but has package Startup; at the top, which is why I say it is unrelated. Because that's what that means. You are then sending an instance of one of these, and upon receiving it, you assume it is the other. It isn't, hence, CCEx.
If you intended to have 2 different classes with the same name, stop confusing them; if this sounds difficult (I think it'd have some issues with this, too!), then rename one.
If you never intended to have 2 separate classes, then fix that problem. Possibly you already did and simply replaced the package statement (and moved the source file to another folder) for the only ClientConfiguration you ever had, but then one of the two (client or server) is running old code.
I want to individually log every unique error I have, as searching though a dozen log files each +10k lines in length is time wasting and tedious.
I catch all exceptions I possibly can, but oftentimes other threads or libraries will shoot off their own errors without any way to process them myself.
Is there any workaround for this?
(E.G. an event for when printStackTrace() is called.)
Is there any workaround for this?
(E.G. an event for when printStackTrace() is called.)
Remap System.err to intercept throwables. If you look at the source code for Throwable.printStackTrace() you'll see that it indirectly calls System.err.println(this);
For example:
import java.io.PrintStream;
public class SpyPrintStream extends PrintStream {
public static void main(String[] args) {
System.setErr(new SpyPrintStream(System.err));
System.setOut(new SpyPrintStream(System.out));
new Exception().printStackTrace();
}
public SpyPrintStream(PrintStream src) {
super(src);
}
#Override
public void println(Object x) {
if (x instanceof Throwable) {
super.println("Our spies detected "+ x.getClass().getName());
}
super.println(x);
}
}
Keep in mind there is all kinds of issues with using this code and it is not going to work in cases where printStackTrace is called with stream that is not standard stream.
You could always do a deep dive into java.lang.instrument if you really want to trap all exceptions.
I catch all exceptions I possibly can, but oftentimes other threads or libraries will shoot off their own errors without any way to process them myself.
Most libraries either throw exceptions back to the caller or use a logging framework. Capture the exception or configure the logging framework.
I want to individually log every unique error I have, as searching though a dozen log files each +10k lines in length is time wasting and tedious.
Logging frameworks include options to deal with this. DuplicateMessageFilter is an example.
Food for thought:
public class DemoClass {
private Map<String, Exception> myExceptions = new HashMap<>();
public void demoMethod() {
try {
// throwing an exception for illustration
throw new IOException("some message");
} catch (IOException e) {
myExceptions.putIfAbsent(e.getLocalizedMessage(), e);
// actually handle the exception
...
}
}
public void finished() {
for (Exception e : myExceptions.values()) {
e.printStackTrace();
}
}
}
You could store any exception you haven't seen yet. If your specific scenario allows for a better way to ensure you only save an exception only once you should prefer that over mapping by Exception.getLocalizedMessage()
The overall concept of my program involves loading a plugin from .class files, running it, shutting it down, manually updating the .class file, and finally turning it back on. Currently it
Loads the .class file via URLClassLoader and begins execution of the main method.
The main method spawns another thread (VIA ScheduledThreadPoolExecutor) which queries a service every regularly.
Spawns a new thread to handle cleanup:
Calls .shutdown() on ScheduledThreadPoolExecutor, and all threads die.
Calls .close() on URLClassLoader and sets URLClassLoader variable to null.
The cleanup thread then sleeps to allow manual replacement of .class files.
Cleanup thread then starts process over again, loading new .class files and running them.
Everything during these steps works, and the new .class files work as expected.
The issue I'm running into begins when the plugin is started again. Each time it runs through the restart process, it spawns one extra instance of the plugin.
1st run: 1 running plugin
2nd run: 2 running plugins
3rd run: 3 running plugins
and so on
What I find strange is that it isn't spawning double the amount of plugins on each start, it's only spawning one additional. The extra piece of code must only be executed one additional time, rather than by each previous thread. Could each subsequent second call to startup(), which creates a new URLClassLoader (the old one is closed and nulled out), also start up all of the past URLClassLoaders somehow? Tried running this in debug mode, and it isn't tracking every active thread. I need to maintain the previous URLClassLoader, without it any reference to existing objects running in the background are removed.
Hard to give SSCCE given the complexity of the program.
public class PluginHandler
{
private static URLClassLoader cl = null;
private static String = "somedir";
public void restart()
{
(new Thread() {
public void run() {
if (cl != null) {
cl.invokeClass(pckg, "main", "-shutdown");
cl.close();
cl = null;
}
try {
Thread.sleep(15000);
} (catch InterruptedException e) {
System.out.println("Interrupted");
}
cl = URLClassLoader cl = new URLClassLoader(new URL[] { new File(path).toURI().toURL() } ));
cl.invokeClass(pckg, "main", "-startup");
}).start();
}
public URLClassLoader invokeClass(String pckgName, String mthdName, String[] args)
throws Exception
{
Class<?> loadedClass = cl.loadClass(pckgName);
Method m = loadedClass.getMethod(mthdName, args.getClass());
m.invoke(null, new Object[] { args });
return urlcl;
}
}
public class PluginMain
{
public static void main(String[] args) {
if (args[0].equals("-startup") {
new PluginController.run();
}
else if (args[0].equals("-shutdown") {
PluginController.shutdown();
}
}
}
public class PluginController implements Runnable
{
static ScheduledThreadPoolExecutor st;
static ScheduledFuture<?> sf;
public void run() {
st = new Scheduled ThreadPoolExecutor(1);
sf = st.scheduleWithFixedDelay(new Plugin(), 0, 10, Time_Unit.SECONDS);
sf.wait();
System.out.println("Returns from run()"); //prints after shutdown is run.
}
public static void shutdown() {
sf.cancel();
st.shutdown();
}
}
public class Plugin implements Runnable
{
public void run() {
//Queries some service
}
}
Edit: All plugins are running the same lines of code at the same time. I mentioned those sleeps, which I suspect would throw off the different threads from being completely synchronized.
Mark W's suggestion lead me down a rabbit hole of using jmap and other process analysis programs to find out what exactly was going on. There's a ton of useful utilities out there. With the combination of VisualVM and Eclipse Memory Analyzer (MAT) I was able to figure out that I wasn't closing the FileHandler for my log. So many hours down the drain!
A program that I've developed is crashing the JVM occasionally due to this bug: http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8029516. Unfortunately the bug has not been resolved by Oracle and the bug report says that there are no known workarounds.
I've tried to modify the example code from the bug report by calling .register(sWatchService, eventKinds) in the KeyWatcher thread instead, by adding all pending register request to a list that I loop through in the KeyWatcher thread but it's still crashing. I'm guessing this just had the same effect as synchronizing on sWatchService (like the submitter of the bug report tried).
Can you think of any way to get around this?
From comments:
It appears that we have an issue with I/O cancellation when there is a pending ReadDirectoryChangesW outstanding.
The statement and example code indicate that the bug is triggered when:
There is a pending event that has not been consumed (it may or may not be visible to WatchService.poll() or WatchService.take())
WatchKey.cancel() is called on the key
This is a nasty bug with no universal workaround. The approach depends on the specifics of your application. Consider pooling watches to a single place so you don't need to call WatchKey.cancel(). If at one point the pool becomes too large, close the entire WatchService and start over. Something similar to.
public class FileWatcerService {
static Kind<?>[] allEvents = new Kind<?>[] {
StandardWatchEventKinds.ENTRY_CREATE,
StandardWatchEventKinds.ENTRY_DELETE,
StandardWatchEventKinds.ENTRY_MODIFY
};
WatchService ws;
// Keep track of paths and registered listeners
Map<String, List<FileChangeListener>> listeners = new ConcurrentHashMap<String, List<FileChangeListener>>();
Map<WatchKey, String> keys = new ConcurrentHashMap<WatchKey, String>();
boolean toStop = false;
public interface FileChangeListener {
void onChange();
}
public void addFileChangeListener(String path, FileChangeListener l) {
if(!listeners.containsKey(path)) {
listeners.put(path, new ArrayList<FileChangeListener>());
keys.put(Paths.get(path).register(ws, allEvents), path);
}
listeners.get(path).add(l);
}
public void removeFileChangeListener(String path, FileChangeListener l) {
if(listeners.containsKey(path))
listeners.get(path).remove(l);
}
public void start() {
ws = FileSystems.getDefault().newWatchService();
new Thread(new Runnable() {
public void run() {
while(!toStop) {
WatchKey key = ws.take();
for(FileChangeListener l: listeners.get(keys.get(key)))
l.onChange();
}
}
}).start();
}
public void stop() {
toStop = true;
ws.close();
}
}
I've managed to create a workaround though it's somewhat ugly.
The bug is in JDK method WindowsWatchKey.invalidate() that releases native buffer while the subsequent calls may still access it. This one-liner fixes the problem by delaying buffer clean-up until GC.
Here is a compiled patch to JDK. In order to apply it add the following Java command-line flag:
-Xbootclasspath/p:jdk-8029516-patch.jar
If patching JDK is not an option in your case, there is still a workaround on the application level. It relies on the knowledge of Windows WatchService internal implementation.
public class JDK_8029516 {
private static final Field bufferField = getField("sun.nio.fs.WindowsWatchService$WindowsWatchKey", "buffer");
private static final Field cleanerField = getField("sun.nio.fs.NativeBuffer", "cleaner");
private static final Cleaner dummyCleaner = Cleaner.create(Thread.class, new Thread());
private static Field getField(String className, String fieldName) {
try {
Field f = Class.forName(className).getDeclaredField(fieldName);
f.setAccessible(true);
return f;
} catch (Exception e) {
throw new IllegalStateException(e);
}
}
public static void patch(WatchKey key) {
try {
cleanerField.set(bufferField.get(key), dummyCleaner);
} catch (IllegalAccessException e) {
throw new IllegalStateException(e);
}
}
}
Call JDK_8029516.patch(watchKey) right after the key is registred, and it will prevent watchKey.cancel() from releasing the native buffer prematurely.
You might not be able to work around the problem itself but you could deal with the error and handle it. I don't know your specific situation but I could imagine the biggest issue is the crash of the whole JVM. Putting all in a try block does not work because you cannot catch a JVM crash.
Not knowing more about your project makes it difficult to suggest a good/acceptable solution, but maybe this could be an option: Do all the file watching stuff in a separate JVM process. From your main process start a new JVM (e.g. using ProcessBuilder.start()). When the process terminates (i.e. the newly started JVM crashes), restart it. Obviously you need to be able to recover, i.e. you need to keep track of what files to watch and you need to keep this data in your main process too.
Now the biggest remaining part is to implement some communication between the main process and the file watching process. This could be done using standard input/output of the file watching process or using a Socket/ServerSocket or some other mechanism.