We Keep Coding

Replace a class within the Java class library with a custom version

The class BasicLabelUI in javax/swing/plaf/basic is affected by a confirmed bug.
In my application I need functionality provided by the fixed version (filed for v9).
Due to both legal and technical reasons, I'm still bound to the affected JDK version.
My approach was to create a package javax/swing/plaf/basic inside my project, containing the fixed version.
How can I force my project to favor my included version of the class over the defective class in the installed JDK?
This has to be somewhat portable as the fixed class also has to be working on customer side and the defective class in the JDK installation has to be disregarded. Therefore, I dont want to modify the JDK, but rather bypass this particular class.

As mentioned by the other answers, you could in theory of course unzip your JVM's rt.jar file and replace the file with a compatible bugfixed version.
Any classes of the Java Class library such as those of Swing are loaded by the bootstrap class loader which looks up its classes from this rt.jar. You can generally not prepend classes to this classpath without adding them to this file. There is a (non-standard) VM option
-Xbootclasspath/jarWithPatchedClass.jar:path
where you would prepend a jar file that includes the patched version, but this does not necessarily work on any Java virtual machine. Also, it is illegal to deploy an application that changes this hehavior! As it is stated in the official documentation:
Do not deploy applications that use this option to override a class in
rt.jar because this violates the Java Runtime Environment binary code
license.
If you however appended a class to the bootstrap class loader (what is possible without using non-standard APIs by using the instrumentation API), the runtime would still load the original class as the bootstrap class loader in this case searches the rt.jar first. It is therefore impossible to "shadow" the broken class without modifying this file.
Finally, it is always illegal to distribute a VM with a patched file, i.e. putting it into a production system for a customer. The license agreement states clearly that you need to
[...] distribute the [Java runtime] complete and unmodified and only bundled as part of your applets and applications
Changing the VM that you distribute is therefore not recommended as you might face legal consequences when this is ever uncovered.
Of course, you can in theory build your own version of the OpenJDK but you could not call the binary Java anymore when you distribute it and I assume that your customer would not allow for this by what you suggest in your answer. By experience, many secure environments compute hashes of binaries before execution what would prohibit both approaches of tweaking the executing VM.
The easiest solution for you would probably be the creation of a Java agent that you you add to your VM process on startup. In the end, this is very similar to adding a library as a class path dependency:
java -javaagent:bugFixAgent.jar -jar myApp.jar
A Java agent is capable of replacing a class's binary representation when the application is started and can therefore change the implementation of the buggy method.
In your case, an agent would look something like the following where you need to include the patched class file as a ressource:
public static class BugFixAgent {
public static void premain(String args, Instrumentation inst) {
inst.addClassFileTransformer(new ClassFileTransformer() {
#Override
public byte[] transform(ClassLoader loader,
String className,
Class<?> classBeingRedefined,
ProtectionDomain protectionDomain,
byte[] classfileBuffer) {
if (className.equals("javax/swing/plaf/basic/BasicLabelUI")) {
return patchedClassFile; // as found in the repository
// Consider removing the transformer for future class loading
} else {
return null; // skips instrumentation for other classes
}
}
});
}
}
The javadoc java.lang.instrumentation package offers a detail description of how to build and implement a Java agent. Using this approach, you can use the fixed version of the class in question without breaking the license agreement.
From experience, Java agents are a great way for fixing temporary bugs in third party libraries and in the Java Class Library without needing to deploy changes in your code or even being required to deploy a new version for a customer. As a matter of fact, this is a typical use case for using a Java agent.

How can I force my project to favor my included version of the class over the defective class in the installed JDK?
Simple answer - you can't. At least, not while strictly obeying the constraint that you should use the affected Java version.
Assuming that you can identify an appropriate version in the OpenJDK source repos, it would be possible to build your own flavor of the Java libraries with a bug patched. However, that won't be real Java. Certainly, it won't qualify as "the affected Java version" that you are constrained to use. (And besides, you are committing yourself to an endless cycle of reapplying your patch to each new patch release of the current version of Java ...)
It is also possible in theory to put a modified version of some Java standard library class into a JAR and prepend it to the JVM's bootstrap classpath using the -Xbootclasspath command line option. But that is tantamount to changing "the affected Java version" too.
Doing it by using a Java agent to use a patched version of the class is breaking the rules too. And it is more complicated. (If you are going to break your rules, do it the easy way ...)
If you and your customers do decide that tweaking the JVM is an acceptable solution, then doing it via the bootstrap classpath is probably the simplest and cleanest approach. And it is DEFINITELY legal1.
However, I'd recommend that you find a workaround for the bug until a version of Java 9 with your fix is released.
1 - Actually, even the build-from-modified-source approach is legal, because the Oracle Binary license does not apply to that. The Binary license is about distributing a modified version of an Oracle binary. The other possible issue is that you may be violating the terms for using the Java trademark(s) if you distribute a version that is incompatible with "true" Java, and call your distro "Java". The solution to that is ... don't call it "Java"!
However, don't just follow my advice. Ask a lawyer. Better yet, don't do it at all. It is unnecessarily complicated.

URLClassLoader.getResources("") (empty resource name) not giving roots of jars

Consider a URLClassLoader parameterized with a collection of URLs which is a mix of expanded directories and jar files. For example:
URL[] urls = new URL[] {
new URL("file:/D:/work/temp/jars/spring-security-core-3.2.0.RELEASE.jar"),
new URL("file:/D:/work/temp/jars/spring-security-config-3.2.0.RELEASE.jar"),
...
new URL("file:/D:/work/temp/domain/bin/"),
new URL("file:/D:/work/temp/web/bin/"),
...
}
URLClassLoader cl = new URLClassLoader(urls);
The classloader correctly handles getResources() requests for resources located somewhere inside a package like "org/my/package/conf.properties". By correctly handles I mean the classloader successfully finds all matches inside both directories and jars.
A special empty string name passed in getResources("") is supposed to yield the URLs for all available roots (in both the directories and the jars). However there is a known limitation in ClassLoaders which results in only returning roots that correspond to directories. All roots to jars are discarded.
Using classloader.getURLs[] instead of classloader.getResources("") will not work with me as I have a complex graph of interdependent URLClassLoaders, so the results are going to be completely different. Also my classloaders are to be consumed by a third party classpath scanning facilities that uses getResources("") calls in order to set up an internal search base. This way resources located in jars are simply not found.
I currently have a working fix where I extend from URLClassLoader and manually handle requests with an empty string by forcing roots for jars in addition to those for directories within the returned collection of URLs.
However my questions are:
What was the conceptual/technical reason for this limitation (where paths to jars are not returned)?
By fixing this manually, do I violate any important contract?
Is there any nice way to get the desired behavior?
Thanks for any thoughts on that!

What was the conceptual/technical reason for this limitation (where paths to jars are not returned)?
The behavior of ClassLoader.getResources("") is unspecified.
The implementation for loading resources from the file system in URLClassPath$Loader is solely based on URLs. It constructs a new file URL by adding the resource name to the base URL of the directory
and returns the URL when it points to an existing resource.
There's no special handling for an empty resource name.
Whether this is wanted behavior or not is undocumented.
The implementation for JAR files in URLClassPath$JarLoader works on an index over JAR files. To get the same behaviour for JAR files the implementation would require a special handling for empty resource names, i.e. it would need to check for an empty resource name first and return the file URL of the JAR file instead of searching within the index. The implementation does not have a special handling for empy resource names. Whether this is wanted behavior or not is again undocumented.
Since the API specification does not specify the behavior for
empty resource names both implementations are valid.
Some may argue that exposing roots is a security issues, especially when running in a sandbox. Others may argue that getResources() should return null for empty resources since there actually does not exist a resource with the name "".
In any case current behavior of URLClassLoader leads to unexpected behavior in Class.getResource(). When this method is called with an empty string for a class in the default package it returns the root directory of the class when the class was loaded from the file system. This violates the contract of the method. For details see for example this open Java bug: https://bugs.openjdk.java.net/browse/JDK-8202687.
By fixing this manually, do I violate any important contract?
As long as you only override the findResource() method
of your ClassLoader, call the super method and then add the additional
URLs of your JAR files you shouldn't violate any contract.
But be aware that there are already implementations out there, that have a special handling for URLClassLoaders. For example
Spring's PathMatchingResourcePatternResolver has a special handling (here) for class loaders that are instances of URLClassLoader, which adds additional URLs for JARs.
Is there any nice way to get the desired behavior?
There is no nice way to get the desired behavior since every solution would be based on unspecified behavior that may theoretically change with every new JRE version.
With the introduction of multi-release JAR files in Java 9 the behavior already changed:
For a multi-release JAR file with Java 8 classes and Java 9 classes
ClassLoader.getResource("") returns now an URL for the JAR file when it is executed within a JRE version > 8. With JRE 8 it still returns no URL for the same JAR file. With it, the returned URLs for an empty resource string depend now even on the JRE version, resp. the type of JAR file.
There exist workarounds to get also the URLs for JAR files. PathMachintResourcePatternResolver for example loads JAR file names from the java.class.path system property (in case of the system class loader) and loads additional URLs by calling URLClassLoader.getURLs() (in case of a URLClassLoader). But again, these are only workarounds based on unspecified behavior.
Ideally searches on the classpath are only performed in the context of a java package. Frameworks like Spring (boot) perform searches on the classpath only in the context of a java package. This avoids to rely on unspecified behavior of class loaders and also avoids to search in JAR files of irrelevant third-party libraries. So, whenever possible I recommend to search on the classpath in context of a java package instead of searching resources by using an empty resource name.

Compile-time define in java [duplicate]

When I used to write libraries in C/C++ I got into the habit of having a method to return the compile date/time. This was always a compiled into the library so would differentiate builds of the library. I got this by returning a #define in the code:
C++:
#ifdef _BuildDateTime_
char* SomeClass::getBuildDateTime() {
return _BuildDateTime_;
}
#else
char* SomeClass::getBuildDateTime() {
return "Undefined";
}
#endif
Then on the compile I had a '-D_BuildDateTime_=Date' in the build script.
Is there any way to achieve this or similar in Java without needing to remember to edit any files manually or distributing any seperate files.
One suggestion I got from a co-worker was to get the ant file to create a file on the classpath and to package that into the JAR and have it read by the method.
Something like (assuming the file created was called 'DateTime.dat'):
// I know Exceptions and proper open/closing
// of the file are not done. This is just
// to explain the point!
String getBuildDateTime() {
return new BufferedReader(getClass()
.getResourceAsStream("DateTime.dat")).readLine();
}
To my mind that's a hack and could be circumvented/broken by someone having a similarly named file outside the JAR, but on the classpath.
Anyway, my question is whether there is any way to inject a constant into a class at compile time
EDIT
The reason I consider using an externally generated file in the JAR a hack is because this is) a library and will be embedded in client apps. These client apps may define their own classloaders meaning I can't rely on the standard JVM class loading rules.
My personal preference would be to go with using the date from the JAR file as suggested by serg10.

I would favour the standards based approach. Put your version information (along with other useful publisher stuff such as build number, subversion revision number, author, company details, etc) in the jar's Manifest File.
This is a well documented and understood Java specification. Strong tool support exists for creating manifest files (a core Ant task for example, or the maven jar plugin). These can help with setting some of the attributes automatically - I have maven configured to put the jar's maven version number, Subversion revision and timestamp into the manifest for me at build time.
You can read the contents of the manifest at runtime with standard java api calls - something like:
import java.util.jar.*;
...
JarFile myJar = new JarFile("nameOfJar.jar"); // various constructors available
Manifest manifest = myJar.getManifest();
Map<String,Attributes> manifestContents = manifest.getAttributes();
To me, that feels like a more Java standard approach, so will probably prove more easy for subsequent code maintainers to follow.

I remember seeing something similar in an open source project:
class Version... {
public static String tstamp() {
return "#BUILDTIME#";
}
}
in a template file. With Ant's filtering copy you can give this macro a value:
<copy src="templatefile" dst="Version.java" filtering="true">
<filter token="BUILDTIME" value="${build.tstamp}" />
</copy>
use this to create a Version.java source file in your build process, before the compilation step.

AFAIK there is not a way to do this with javac. This can easily be done with Ant -- I would create a first class object called BuildTimestamp.java and generate that file at compile time via an Ant target.
Here's an Ant type that will be helpful.

Unless you want to run your Java source through a C/C++ Preprocessor (which is a BIG NO-NO), use the jar method. There are other ways to get the correct resources out of a jar to make sure someone didn't put a duplicate resource on the classpath. You could also consider using the Jar manifest for this. My project does exactly what you're trying to do (with build dates, revisions, author, etc) using the manifest.
You'll want to use this:
Enumeration<URL> resources = Thread.currentThread().getContextClassLoader().getResources("META-INF/MANIFEST.MF");
This will get you ALL of the manifests on the classpath. You can figure out which jar they can from by parsing the URL.

Personally I'd go for a separate properties file in your jar that you'd load at runtime... The classloader has a defined order for searching for files - I can't remember how it works exactly off hand, but I don't think another file with the same name somewhere on the classpath would be likely to cause issues.
But another way you could do it would be to use Ant to copy your .java files into a different directory before compiling them, filtering in String constants as appropriate. You could use something like:
public String getBuildDateTime() {
return "#BUILD_DATE_TIME#";
}
and write a filter in your Ant file to replace that with a build property.

Perhaps a more Java-style way of indicating your library's version would be to add a version number to the JAR's manifest, as described in the manifest documentation.

One suggestion I got from a co-worker
was to get the ant file to create a
file on the classpath and to package
that into the JAR and have it read by
the method. ... To my mind that's a
hack and could be circumvented/broken
by someone having a similarly named
file outside the JAR, but on the
classpath.
I'm not sure that getting Ant to generate a file is a terribly egregious hack, if it's a hack at all. Why not generate a properties file and use java.util.Properties to handle it?

Best way to override MATLAB's default static javaclasspath

MATLAB is configured to search its static java class path before searching the user-modifiable dynamic path. Unfortunately, the static path contains quite a number of very old public libraries, so if you are trying to use a new version you may end up loading the wrong implementation and get errors.
For instance, the static path contains an old copy of the google-collections.jar, which has long been supplanted by Google's guava library and which has some of the same class names (e.g. com.google.common.base.Objects). As a result, if you invoke a Guava method that uses a newer method of one of such a class, you will end up getting surprising NoSuchMethodErrors because the google-collections jar is found first.
As of R2012b, MATLAB lets you specify additional jars to add to the static path by putting a javaclasspath.txt file in your preferences folder, but that adds jars to the end of the path, and doesn't let you override jars that are built into MATLAB.
So what is the best way around this?

I got an official response from Mathworks:
As of MATLAB R2013a (also in R2012b), classes can be added to the front of the static Java class path by including the following line in javaclasspath.txt:
<before>
Any directory that is after this line in javaclasspath.txt will be added to the front of the static Java class path. This is an undocumented use of javaclasspath.txt as of R2013a.
But overall in MATLAB, the ability to add classes to the front of the static Java classpath is not available through javaclasspath.txt in MATLAB 8.0 (R2012b).
MATLAB searches for classpath.txt in the following order:
In the startup directory. As of MATLAB 8.0 (R2012b) a warning will be shown if the file is found there and it will be ignored.
In the first directory on the MATLABPATH environment variable. (This environment variable is used in the bin/matlab shell script on Linux and in general is not used by the end-user).
In the toolbox/local directory.
Although the MATLABPATH environment variable of point 2 is normally not used by end-users we can use it in a workaround to allow reading a custom classpath.txt outside of the toolbox/local directory.
On Windows:
You will need to create the MATLABPATH environment variable. The first directory on it should be your directory with the custom classpath.txt AND you will also need to add the toolbox\local directory as second option. So from a cmd prompt you could do:
set MATLABPATH=c:\Users\user\Documents\myMATLABClasspath;c:\Program Files\MATLAB\R2012b
\toolbox\local
matlab.exe

One hack that appears to work is to add the jar to the top of the classpath.txt file that can be found in your MATLAB installations toolbox/local folder. Unfortunately, this is automatically generated and may get rewritten at some unspecified time, such as when you install new toolboxes, so this approach would require you to have some way to notice when this happens and reapply the hack.

If you're distributing a jar that's intended to be used with matlab, it may be better to use proguard as described at http://code.google.com/p/guava-libraries/wiki/UsingProGuardWithGuava.
If you specify that all of your classes and their (public) fields and methods are to be preserved and include guava as a program jar (not a library), then it will rename all of guava's methods and update your compiled bytecode to reference the new names.
It seems a bit hackish, but depending on the audience, it may be significantly easier than teaching your users about static vs. dynamic classpath, and it won't break any matlab code that depends on the old behavior.

Instead of obfuscating the package as suggested by #user2443532, I have found it easier to "shade" the conflicting package instead of obfuscating it - unless you actually need obfuscation. One easy way to do this is to build your package using Maven and use the maven-shade-plugin. Internal calls are modified automatically, so you don't need to modify any of the Java code.
Direct calls from Matlab will need to be modified - for example, calls to com.opensource.Class become shaded.com.opensource.Class.
For more info on shading, see What is the maven-shade-plugin used for, and why would you want to relocate Java packages?

Speed up application start by adding own application classes to classes.jsa

To speed up the startup time of the JVM, the Sun developers decided it is a good idea to precompile the standard runtime classes for a platform during installation of the JVM. These precompiled classes can be found e.g. at:
$JAVA_HOME\jre\bin\client\classes.jsa
My company currently develops a Java standalone application which brings its own JRE, so it would be a fantastic option to speed up our application start time by adding our own application classes to this jsa file, too.
I don't believe the JSA file was created by magic, so: How is it created? And how can I trick the JVM into incorporating my own classes?
EDIT: I already found out the following:
The classes.jsa is created by the command
java -Xshare:dump
The list of classes to incorporate in the dump can be found in $JAVA_HOME/jre/lib/classlist.
I even managed to add my own classes here (and to add them into the rt.jar for java to find them), and to generate my own checksum below the classlist file.
The final problem is: Only classes in the packages java, com.sun, and org.w3c seem to be recognized, if I leave the same classes in their original packages, they won't be loaded. I searched the whole OpenJDK source for pointer about this, but it seems to have something to do with protection domains. If someone is interested enough in this topic and knowledgeable enough, please add some pointers for me to investigaete further.

As of Java 8u40 (and Embedded Java 8u51), Java now supports Application Class Data Sharing (AppCDS) (ie your own classes in the shared archive). On our embedded java, we've found a startup improvement of >40%! Pretty awesome for almost no work on our part...
https://blogs.oracle.com/thejavatutorials/entry/jdk_8u40_released

You were almost there, you only need a couple steps to make it work. To add your own classes to the clients.js you need the following steps:
The qualified name your classes (you have it)
The classpath of these classes (you have it)
Know how to recalculate the checksum (you have it)
Dump the new file, providing the classpath of the classes you are now precompiling with the Java classes.
Run the program, providing the same classpath that you used to dump the new classes.jsa
To provide the classpath where are the classes you are adding to the classlist, use the -Xbootclasspath/a command. It will append the directories/JARs when JVM is searching the places where the boot classes are. The default space for the classes.jsa is quite small, if you need to improve it you can use the -XX:SharedReadWriteSize and -XX:SharedReadOnlySize commands. Your dump command you look similar to this:
java -Xshare:dump -Xbootclasspath/a:C:/myfiles/directoryA/;C:/myfiles/directoryB/;C:/myJars/myJar.jar;
The last step is just run the java application normally, rememebering of turn on the share mode. You also need to add the Xbootclasspath excatly as you added on the dump. It will look similar to this:
java myapp.java -Xshare:on -Xbootclasspath/a:C:/myfiles/directoryA/;C:/myfiles/directoryB/;C:/myJars/myJar.jar;
Now every class that you put on the classlist is being shared with other instances running in the same JVM.

Interesting idea. As I read it though, it's used for sharing data across VMs and for speeding up classloading, not compiling. I'm not sure how much of a boost you would get, but it might be worth a try if you have a big lag at startup already (though the VM already tries to mitigate that).
As for trying it yourself, it appears this file is normally created when the Sun VM is installed, but you can also control it. Some details are in this older Sun Java 5 Class Data Sharing document (which you may have already seen?). Some Sun Java 6 docs also mention it a few times, but don't add much to the documentation. It seems it was originally an IBM VM feature. And, to continue the link dump, it's explained a bit in this article.
I don't personally know much about it, so I don't know how you might control it. You can regenerate it, but I don't think it's intended for you to put custom stuff into. Also, even if you can "trick" it, that would probably violate a Sun/Oracle license of some sort (you can't mess with rt.jar and redistribute, for instance). And, all that said, I doubt you would see a serious improvement in startup time unless you have thousands or tens of thousands of classes in your app?
(And this isn't really an answer, I know, but it was too big to fit in a comment, and I found the question interesting so I investigated a bit and put links here in case anyone finds the same info useful.)

It took a little figuring out but I have 4 Java8 VMs (version 1.8.0_162) running using shared classes. The following script was used to set up and test sharing and with a little modification could be used elsewhere:
#!/bin/bash
# Libraries to load
LIBS1="./lib/protobuf-java-2.6.1.jar:\
./lib/jetty-server-9.2.18.v20160721.jar:./lib/jetty-util-9.2.18.v20160721.jar:./lib/servlet-api-3.1.jar:./lib/jetty-http-9.2.18.v20160721.jar:./lib/jetty-io-9.2.18.v20160721.jar:\
./lib/derby.jar:\
./lib/json-simple-1.1.1.jar:"
LIBS2=":./lib/GTFS.jar"
# Uncomment these lines for the first phase where you are determining the classes to archive. During this phase aim to get as many classes loaded as possible
# which means loading a schedule and retrieving the stop list and next vehicle information
#
#APPCDS="-Xshare:off -XX:+UnlockCommercialFeatures -XX:+UseAppCDS -XX:DumpLoadedClassList=../GtfsAppCds.lst"
#java -Xmx512m $APPCDS -Dderby.system.home=database -classpath $LIBS1$LIBS2 com.transitrtd.GtfsOperatorManager
# Uncomment these lines when the class list is created and run to create the shared archive. Classes marked as unverifiable will need to be removed from the
# archived class list in GtfsAppCds.lst and the lines below run again. LIBS2 above contains jars which are left out of the archive. These are jars which change
# frequently and would therefore cause the archive to be frequently rebuilt.
#
#APPCDS="-Xshare:dump -XX:+UnlockCommercialFeatures -XX:+UseAppCDS -XX:SharedClassListFile=../GtfsAppCds.lst -XX:SharedArchiveFile=../GtfsAppCds.jsa"
#java -Xmx512m $APPCDS -classpath $LIBS1
# Uncomment these lines when wishing to verify the application is using the shared archive.
#
#APPCDS="-Xshare:on -XX:+UnlockCommercialFeatures -XX:+UseAppCDS -XX:SharedArchiveFile=../GtfsAppCds.jsa -verbose:class"
#java -Xmx512m $APPCDS -Dderby.system.home=database -classpath $LIBS1$LIBS2 com.transitrtd.GtfsOperatorManager
Note that the shared archive file (i.e.the jsa file) is architecture dependent and will need to be built on each target platform type.
Also if a jar uses sealed packages a security exception is thrown, see
https://docs.oracle.com/javase/tutorial/deployment/jar/sealman.html
for information on sealed packages. This was the case above with derby.jar but the problem could be solved by unpacking the jar file, replacing Sealed:true with Sealed:false in the manifest and repacking it.
jars built with older versions of java cannot be used in a shared archive, in the case above the derby version needed to be upgraded from 10.10 to 10.14 to benefit.