We Keep Coding

Files with the .SCL.lombok extension

When the lombok jar file is opened in Intellij, all files other than the annotations end with .SCL.lombok (e.g. HandleAccessors.SCL.lombok). I was just wondering what the reason for this was and how it's handled.

The reason for it
Lombok has a public API - the stuff you're supposed to interact with. That'd be, for example, the #lombok.Getter annotation. Those are just class files in that jar, the aim is simply: add that jar to your classpath and your IDE autocomplete dialogs and the like will automatically start suggesting these, as per design.
But, lombok also has lots of classes that just 'make it tick', these aren't meant for public consumption. Things like lombok.eclipse.HandleGetter, which is the implementation for handling the #Getter annotation inside the eclipse agent. There is no point or purpose to referring to this class anywhere, in any project - it's an internal lombok thing. If we just stuck that jar file into the jar, and you typed Handle and hit your IDE's autocomplete shortcut key, you'd still get the suggestion.
Similarly, we ship a few dependencies straight into lombok.jar - it's a 'shaded jar' (a jar with all deps included), though we don't have many, keeping lombok.jar a nice small size. Still, ASM (a bytecode manipulation library) is in it, and that is fairly popular.
The standard shading solution offered by most shading tools is to prefix something to the name. ASM's org.objectweb.asm.AnnotationVisitor class would become org.projectlombok.shading.org.objectweb.asm.AnnotationVisitor. Point is, your IDE doesn't know that, and if you ALSO use asm in your project (where you also use lombok), and you want AnnotationVisitor thus you type AnnV and hit cmd+space or whatnot, your IDE suggests both. That's ugly and we'd like to avoid this.
Hence, we built our own shader, and it works by not having class files in the first place. This way, IDEs and any other automated tool doesn't even know either our ASM classes, or our implementation details, even exists. The only files that such tools (such as your IDE) sees are the types you're meant to see: lombok.Builder, lombok.extern.slf4j.Slf4j, lombok.experimental.UtilityClass, etcetera.
How does it work
Java's classloader architecture is abstracted: You can make your own. The primitives offered by a class loader is simply this: "Convert this byte array containing bytecode (i.e. the contents of a class file) into a Class<?> definition", and the primitives that you're supposed to implement when you write your own classloader is twofold:
Here is a resource key, such as "/com/foo/load.png". Please provide me an InputStream with this data.
Here is a fully qualified class name, such as "com.foo.MyApp". Please provide me with a Class<?> instance representing it.
Out of the box, java ships with a default classloader. This default classloader answers these questions by checking your CLASSPATH - which can be provided in various ways (via the jar manifest's Class-Path entry, or via the -cp argument to the JVM executable, or the CLASSPATH environment variable), and scanning each entry on the classpath for the resource requested, capable of reading the file system as well as opening jar files.
But that's just a classloader. One implementation of the general principle that's baked into java. You can write your own. You can write a classloader that generates resources on the fly, or that loads them from a network.
Or, as lombok does, that loads them by opening its own jar and looking for .SCL.lombok files.
Thus, lombok works like this: When you launch it, the 'entrypoint' (the class containing public static void main - or in lombok's case, for javac mode it's the annotation processor entrypoint and for eclipse it's agentmain), we 'hide' it from you using some fancy trickery: agentmain does not need to be in a public class (it can't be .SCL.lombok files - our classloader isn't available yet, we need to bootstrap that up first!). annotation processors do have to be in a public class, but, it's a public class inside a package private class, thus, just about every IDE knows it's 'invisible' and won't show it, but javac's annotation runner accepts it.
From there, we register a classloader that is capable of loading classes by way of reading in an .SCL.lombok file, and this lets us hide everything else we want to hide.
I want to develop lombok and this is getting in the way!
No need; just clone our repo, run ant eclipse or ant intellij, and off you go. There is no way to extend lombok without first forking it; we'd like lombok to be able to be extensible without it, but that would be far more complicated than simply not doing the .SCL.lombok thing. Eclipse runs on top of equinox, a runtime modularization system, and making that work properly requires all sorts of stuff that would make 'just toss some extra handlers on the classpath' not a feasible route to extending lombok in the first place.

How to read the classes from another JRT?

From Java 11, how can I read the content of another runtime image?
In order to list the content of a Java runtime image, JEP 220 suggests the following solution:
A built-in NIO FileSystem provider for the jrt URL scheme ensures that development tools can enumerate and read the class and resource files in a run-time image by loading the FileSystem named by the URL jrt:/, as follows:
FileSystem fs = FileSystems.getFileSystem(URI.create("jrt:/"));
byte[] jlo = Files.readAllBytes(fs.getPath("modules", "java.base",
"java/lang/Object.class"));
This snippet works and will allow me to read the content of java/lang/Object.class in the runtime image of the Java installation that is executing the code.
How can I get it to read the content of java/lang/Object.class in another Java installation, given its java home?
I have read this SO answer which explains how to read a Java runtime image's content from a Java 8 runtime. Unfortunately, this won't work with newer Java runtimes, since, I believe, the filesystem for jrt:/ will always point to the current runtime image.

You may still use jrt:/ scheme as described in this answer, you just need to provide an alternative java.home path in the environment argument when creating a FileSystem object:
public static void listModules(String javaHome) throws IOException {
FileSystem fs = FileSystems.newFileSystem(
URI.create("jrt:/"),
Collections.singletonMap("java.home", javaHome));
try (Stream<Path> stream = Files.list(fs.getPath("/modules"))) {
stream.forEach(System.out::println);
}
}
Or, to read a single resource:
public static byte[] readResource(String javaHome, String module, String path) throws IOException {
FileSystem fs = FileSystems.newFileSystem(
URI.create("jrt:/"),
Collections.singletonMap("java.home", javaHome));
return Files.readAllBytes(fs.getPath("modules", module, path));
}

I think what you want is impossible. To wit:
Up to JDK8, you can rely on e.g. Paths.get(pathToJdk8Home, "jre", "lib", "rt.jar") to exist, which you can then turn into a URL (you're looking for jar:file:/that/path), and you can then toss that URL at FileSystems.newFileSystem), see this documentation for more.
But from JDK9 and up, the core java API is loaded in jmod files, and jmod files have an unspecified format by design - right now jmods are just zips, but unlike jars you explicitly get no guarantees that they will remain zip formatted, and there is no jmod URL scheme and no JmodFileSystemProvider. It is, in effect, impossible to read a jmod file in a way that is future compatible. Unfortunately the OpenJDK project has been on a murderous spree turning a ton of useful things, such as 'read a jmod', into implementation details. Bit user-hostile - just be aware of that, and I'm trying to do some expectation management: Stuff like this is way, way harder, and a huge maintenance burden (as you're forced to dip into workarounds, hacks, and going beyond spec thus needing to check it still works for every point release). See also this SO answer.
The jrt scheme can only load data from jmods that are actually 'loaded' into the VM's mod base, which I gather is explicitly not what you want (in fact, I'm pretty sure you cannot load e.g. the JDK11 core jmods into a JDK14, as it already loaded its jmods, and you'd get a split package violation). The jrt:// URL scheme, per its spec, isn't base file system related. You specify a module name (or nothing, and you get all loaded modules as one file system). There is no place for you to list a JDK installation path or jmod file, so that can't help you either.
Thus, you have only two options:
Accept that what you want cannot be done.
Accept that you're going to have to write hackery (as in, go beyond things that specifications guarantee you), and you accept the large maintenance burden that comes with the territory.
The hackery would involve:
Detect targeted JDK version or go on a hunting spree within the provided JDK installation directory (using e.g. Files.walk) to find a file named rt.jar. If it's there, load it up as ZipFileSystem and carry on. Modules 'do not exist', just turn any desired class into a path by replacing dots with slashes and appending .class (note that you'll need the binary name; e.g. package com.foo; class Outer { class Inner {}} means you want the name of Inner to be com.foo.Outer$Inner, so that you turn that into /com/foo/Outer$Inner.class).
For JDK9 and up, hunt for a file at JDK_HOME/jmods/java.base.jmod, and throw that at ZipFileSystem. A given class is in subdir classes. So, you're looking for e.g. the entry classes/java/lang/Object.class within the zip (that jmod is the zip). However, festoon this code with comments stating that this is a total hack and there is zero guarantee that this will work in the future. I can tell you, however, that JDK16, at least, still has zip-based jmod files.
Alternatively, given that you have a JDK installation path, you can use ProcessBuilder to exec List.of("JDK_HOME/bin/jmod" /* or jmod.exe, you'll have to check which one to call! */, "extract", "JDK_HOME/jmods/java.base.jmod"), but note that this will extract all of those files into the current working directory (you can set the cwd for the invoked process to be some dir you just created for the purpose of being filled with the files inside). Quite a big bazooka if all you wanted was the one file. (You can also use the --dir option instead). The advantage is that this will still work even if hypothetically JDK17 is using some different format; presumably JDK17 will still have both bin/jmod as well as jmods/java.base.jmod, and the bin/jmod of JDK17 should be able to unpack the jmod files in your JDK17 installation. Even if you are running all this from e.g. JDK16 which wouldn't be able to read them.

Compile-time define in java [duplicate]

When I used to write libraries in C/C++ I got into the habit of having a method to return the compile date/time. This was always a compiled into the library so would differentiate builds of the library. I got this by returning a #define in the code:
C++:
#ifdef _BuildDateTime_
char* SomeClass::getBuildDateTime() {
return _BuildDateTime_;
}
#else
char* SomeClass::getBuildDateTime() {
return "Undefined";
}
#endif
Then on the compile I had a '-D_BuildDateTime_=Date' in the build script.
Is there any way to achieve this or similar in Java without needing to remember to edit any files manually or distributing any seperate files.
One suggestion I got from a co-worker was to get the ant file to create a file on the classpath and to package that into the JAR and have it read by the method.
Something like (assuming the file created was called 'DateTime.dat'):
// I know Exceptions and proper open/closing
// of the file are not done. This is just
// to explain the point!
String getBuildDateTime() {
return new BufferedReader(getClass()
.getResourceAsStream("DateTime.dat")).readLine();
}
To my mind that's a hack and could be circumvented/broken by someone having a similarly named file outside the JAR, but on the classpath.
Anyway, my question is whether there is any way to inject a constant into a class at compile time
EDIT
The reason I consider using an externally generated file in the JAR a hack is because this is) a library and will be embedded in client apps. These client apps may define their own classloaders meaning I can't rely on the standard JVM class loading rules.
My personal preference would be to go with using the date from the JAR file as suggested by serg10.

I would favour the standards based approach. Put your version information (along with other useful publisher stuff such as build number, subversion revision number, author, company details, etc) in the jar's Manifest File.
This is a well documented and understood Java specification. Strong tool support exists for creating manifest files (a core Ant task for example, or the maven jar plugin). These can help with setting some of the attributes automatically - I have maven configured to put the jar's maven version number, Subversion revision and timestamp into the manifest for me at build time.
You can read the contents of the manifest at runtime with standard java api calls - something like:
import java.util.jar.*;
...
JarFile myJar = new JarFile("nameOfJar.jar"); // various constructors available
Manifest manifest = myJar.getManifest();
Map<String,Attributes> manifestContents = manifest.getAttributes();
To me, that feels like a more Java standard approach, so will probably prove more easy for subsequent code maintainers to follow.

I remember seeing something similar in an open source project:
class Version... {
public static String tstamp() {
return "#BUILDTIME#";
}
}
in a template file. With Ant's filtering copy you can give this macro a value:
<copy src="templatefile" dst="Version.java" filtering="true">
<filter token="BUILDTIME" value="${build.tstamp}" />
</copy>
use this to create a Version.java source file in your build process, before the compilation step.

AFAIK there is not a way to do this with javac. This can easily be done with Ant -- I would create a first class object called BuildTimestamp.java and generate that file at compile time via an Ant target.
Here's an Ant type that will be helpful.

Unless you want to run your Java source through a C/C++ Preprocessor (which is a BIG NO-NO), use the jar method. There are other ways to get the correct resources out of a jar to make sure someone didn't put a duplicate resource on the classpath. You could also consider using the Jar manifest for this. My project does exactly what you're trying to do (with build dates, revisions, author, etc) using the manifest.
You'll want to use this:
Enumeration<URL> resources = Thread.currentThread().getContextClassLoader().getResources("META-INF/MANIFEST.MF");
This will get you ALL of the manifests on the classpath. You can figure out which jar they can from by parsing the URL.

Personally I'd go for a separate properties file in your jar that you'd load at runtime... The classloader has a defined order for searching for files - I can't remember how it works exactly off hand, but I don't think another file with the same name somewhere on the classpath would be likely to cause issues.
But another way you could do it would be to use Ant to copy your .java files into a different directory before compiling them, filtering in String constants as appropriate. You could use something like:
public String getBuildDateTime() {
return "#BUILD_DATE_TIME#";
}
and write a filter in your Ant file to replace that with a build property.

Perhaps a more Java-style way of indicating your library's version would be to add a version number to the JAR's manifest, as described in the manifest documentation.

One suggestion I got from a co-worker
was to get the ant file to create a
file on the classpath and to package
that into the JAR and have it read by
the method. ... To my mind that's a
hack and could be circumvented/broken
by someone having a similarly named
file outside the JAR, but on the
classpath.
I'm not sure that getting Ant to generate a file is a terribly egregious hack, if it's a hack at all. Why not generate a properties file and use java.util.Properties to handle it?

Adding a proxy to a networking app without changing code possible?

I want to use a Java application, which is not written by me and also not maintained anymore, meaning I can't just go to the author and ask for a new feature. What options do I have to add proxy capabalities. Is it possible to add some shell arguments? Is it manageable work to add the binary code for the proxy directly into the .class file? Could I write my own class, import the stuff from the .class files and overwrite the network functionality?

This might work for you:
java -Dhttp.proxyHost=IP_address -Dhttp.proxyPort=8080 -jar App.jar

It is possible to replace a class with your own implementation by creating a class with the same name/package and placing it earlier in the classpath so it takes precedence. However you cannot break binary compatibility if this class is used by other parts of the code.
There are also instrumentation APIs available to modify classes as they are loaded. See the Instrumentation API.

If you are creating your own source file that references the compiled .class, then in your file try doing this:
java.util.Properties systemProperties = System.getProperties();
systemProperties.setProperty("http.proxyHost", PROXY_HOST);
systemProperties.setProperty("http.proxyPort", PROXY_PORT);
I think that should work.

how to avoid "duplicate class" in Java

Suppose I have have a java project myProject and am using an external library jar (someJar.jar), which has a class com.somepackage.Class1.class.
Now I find an updated version of Class1.java which fixes a bug in the original jar.
I include the new Class1.java in my source code under package com.somepackage
When I build the project (e.g., using Netbeans), there is a dist\myProject.jar which contains the classcom.somepackage.Class1.class and a dist\lib\someJar.jar which also contains a class with the same name.
When I run the file (e.g, using java -jar dist\myProject.jar), the new version of Class1.class is used (as I want).
How does Java decide which class file to run in case of such duplicates? Is there any way I can specify precedence ?
Is there any 'right' way to avoid such clashes?
In Proguard, when I try to compress my code, I get a duplicate class error. How do I eliminate this?

Java decides which one to use based on the order of the classpath. List yours first and you'll be fine.
The "right" way would be to fix the orignal source, but sometimes that's not always an option.
I haven't used ProGuard, but I have re-jarred libaries before that had duplicate classes. The solution in my case was to tell Ant to ignore duplicate classes. I would assume ProGuard would have that support too.

Can you not create an updated jar file which contains the bug fix? It's going to make things a lot simpler if you don't have two versions of the same fully-qualified class around.

1) Updated Jar is a better solution.
2) Use a different class name. Is there a reason, why you want to use the same class name and same packing? I don't think there is a reason.
3) create a wrapper/ proxy class, that encapsulate all the calls to the jar and you can decide to call this new class that fixes the bug ( provided it has a different name and packaging)