We Keep Coding

Can ToolProvider.getSystemJavaCompiler() access runtime generated in memory source files when using JPMS modules?

I have developed an application that generates Java source at runtime from an external source. The generated source code implements an interface residing in say org.foo and also calls other classes residing in org.foo. The application compiled the generated Java source to a specific package, say org.foo.generated. The compiler provided by ToolProvider.getSystemJavaCompiler() found the source files using a ForwardingJavaFileManager, compiled the source files and the application was able to run them.
Now I have split up my application in JPMS modules and the compiler is complaining it cannot find the org.foo package where the interface implemented by the generated source code lives. I call the compiler from within the same module (org.foo) as where the generated code lives. The error message looks like this:
package org.foo does not exist
I changed my ForwardingFileManager by adding some methods to lookup modules, as described here. It seems like the compiler, residing in the java.compiler module cannot access the org.foo module. So, I added
--add-reads java.compiler=org.foo
to the command line, but that did not change anything. Any clues? Does somebody have any positive experiences combining runtime generated source files and JPMS with Janino instead of the ToolProvider compiler?
Below is a minimal reproducable example as requested in the comments. The source files can also be retrieved from github:
https://github.com/evg/demo-jpms-compiler.git
I get the following output without the module-info.java:
All files compiled without errors: true
0 diagnostics reported by compiler task
I get the following output with the module-info.java:
All files compiled without errors: false
3 diagnostics reported by compiler task
DIAGNOSTIC /org/foo/generated/MyGreeter.java:5: error: cannot find symbol
public class MyGreeter implements IGreeter {
^
symbol: class IGreeter
DIAGNOSTIC /org/foo/generated/MyGreeter.java:3: error: package org.foo does not exist
import org.foo.*;
^
DIAGNOSTIC /org/foo/generated/MyGreeter.java:6: error: method does not override or implement a method from a supertype
#Override
^
The command lines Eclipse uses.
Without module-info.java:
C:\Program Files\Java\jdk-11.0.2\bin\javaw.exe -Dfile.encoding=Cp1252 -classpath "H:\git\demo-jpms-compiler\demo-jpms-compiler\target\classes" org.foo.Main
With module-info.java:
C:\Program Files\Java\jdk-11.0.2\bin\javaw.exe -Dfile.encoding=Cp1252 -p "H:\git\demo-jpms-compiler\demo-jpms-compiler\target\classes" -m org.foo/org.foo.Main
The files used:
module-info.java
module org.foo {
requires java.compiler;
}
Main.java
package org.foo;
import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import javax.tools.DiagnosticCollector;
import javax.tools.JavaCompiler;
import javax.tools.JavaFileObject;
import javax.tools.ToolProvider;
public class Main {
public static void main(String[] args) {
new Main().run();
}
private void run() {
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
Iterable<? extends JavaFileObject> compilationUnits = List.of(new JavaSourceFromString("org.foo.generated.MyGreeter", getJavaSource()));
JavaCompiler.CompilationTask task = compiler.getTask(null, null, diagnosticCollector , compilerOptions, null, compilationUnits);
boolean allFilesCompiledWithoutErrors = task.call();
log("All files compiled without errors: " + allFilesCompiledWithoutErrors);
log(diagnosticCollector.getDiagnostics().size() + " diagnostics reported by compiler task");
diagnosticCollector.getDiagnostics().stream().forEach(diagnostic->{
log("DIAGNOSTIC " + diagnostic);
});
}
private void log(String text) {
System.out.println(text);
}
private String getJavaSource() {
String[] lines = new String[] {
"package org.foo.generated;",
"",
"import org.foo.*;",
"",
"public class MyGreeter implements IGreeter {",
" #Override",
" public String getGreeting() {",
" return \"Hello, generated world\";",
" }",
"",
"}"
};
return Stream.of(lines).collect(Collectors.joining(System.lineSeparator()));
}
private DiagnosticCollector<JavaFileObject> diagnosticCollector = new DiagnosticCollector<>();
private List<String> compilerOptions = Arrays.asList("-Xlint:deprecation");
}
IGreeter.java
package org.foo;
public interface IGreeter {
public String getGreeting();
}
JavaSourceFromString.java
package org.foo;
import java.net.URI;
import javax.tools.SimpleJavaFileObject;
/**
* A file object used to represent source coming from a string.
*/
public class JavaSourceFromString extends SimpleJavaFileObject {
/**
* The source code of this "file".
*/
final String code;
/**
* Constructs a new JavaSourceFromString.
*
* #param name
* the name of the compilation unit represented by this file object
* #param code
* the source code for the compilation unit represented by this file
* object
*/
public JavaSourceFromString(String name, String code) {
super(URI.create("string:///" + name.replace('.', '/') + Kind.SOURCE.extension), Kind.SOURCE);
this.code = code;
}
#Override
public CharSequence getCharContent(boolean ignoreEncodingErrors) {
return code;
}
}

Generating a new java class by writing java program in another java class [duplicate]

This question already has answers here:
How do I create a file and write to it?
(35 answers)
Closed 3 years ago.
I want to create a java program that generates another java class in the same project. For example in the class Dragon.java, i want to write java code that creates another java class called fire.java. I do not want to use any GUI from eclipse, just pure code that generates another class from the execution of written programming in java.
I have tried making objects of a non existent class in hopes of the program automatically producing a class with that name.
Again, it doesn't have to be just a java class, is there a way to make other forms of files also? for example fol.flow, or of different names.

Creating a new Java file is easy. You can use any FileWriter technique. But what need to be taken care of is that new Java file is valid java file and can be compiled to class file.
This link has working example of doing the same.
import java.io.*;
import java.util.*;
import java.lang.reflect.*;
public class MakeTodayClass {
Date today = new Date();
String todayMillis = Long.toString(today.getTime());
String todayClass = "z_" + todayMillis;
String todaySource = todayClass + ".java";
public static void main (String args[]){
MakeTodayClass mtc = new MakeTodayClass();
mtc.createIt();
if (mtc.compileIt()) {
System.out.println("Running " + mtc.todayClass + ":\n\n");
mtc.runIt();
}
else
System.out.println(mtc.todaySource + " is bad.");
}
public void createIt() {
try {
FileWriter aWriter = new FileWriter(todaySource, true);
aWriter.write("public class "+ todayClass + "{");
aWriter.write(" public void doit() {");
aWriter.write(" System.out.println(\""+todayMillis+"\");");
aWriter.write(" }}\n");
aWriter.flush();
aWriter.close();
}
catch(Exception e){
e.printStackTrace();
}
}
public boolean compileIt() {
String [] source = { new String(todaySource)};
ByteArrayOutputStream baos= new ByteArrayOutputStream();
new sun.tools.javac.Main(baos,source[0]).compile(source);
// if using JDK >= 1.3 then use
// public static int com.sun.tools.javac.Main.compile(source);
return (baos.toString().indexOf("error")==-1);
}
public void runIt() {
try {
Class params[] = {};
Object paramsObj[] = {};
Class thisClass = Class.forName(todayClass);
Object iClass = thisClass.newInstance();
Method thisMethod = thisClass.getDeclaredMethod("doit", params);
thisMethod.invoke(iClass, paramsObj);
}
catch (Exception e) {
e.printStackTrace();
}
}
}

At first I thought you wanted code generation, but you simply want to write to files or create them?
The simplest code to create file and write to it:
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.nio.file.StandardOpenOption;
public class Testing {
public static void main(String[] args) throws IOException {
Files.writeString(Paths.get("D://output.txt"), "some text to write", StandardOpenOption.CREATE);
}
}
It uses only java standard classes, you don't need any libraries or anything external. Just make sure to write to the valid path, where you have access.
If you want to generate files with java code, you can just do it with the method above, but creating the String with code content is really hard, there are libraries for it and they are not easy to use for beginners. For example javapoet. I personally used javaparser, it has a lot of other possibilities besides generating code.

Compiling and using Java from within another .jar file [duplicate]

This question already has answers here:
How to provide an interface to JavaCompiler when compiling a source file dynamically?
(3 answers)
Closed 5 years ago.
The community reviewed whether to reopen this question 4 months ago and left it closed:
Original close reason(s) were not resolved
(This question is similar to many questions I have seen but most are not specific enough for what I am doing)
Background:
The purpose of my program is to make it easy for people who use my program to make custom "plugins" so to speak, then compile and load them into the program for use (vs having an incomplete, slow parser implemented in my program). My program allows users to input code into a predefined class extending a compiled class packaged with my program. They input the code into text panes then my program copies the code into the methods being overridden. It then saves this as a .java file (nearly) ready for the compiler. The program runs javac (java compiler) with the saved .java file as its input.
My question is, how do I get it so that the client can (using my compiled program) save this java file (which extends my InterfaceExample) anywhere on their computer, have my program compile it (without saying "cannot find symbol: InterfaceExample") then load it and call the doSomething() method?
I keep seeing Q&A's using reflection or ClassLoader and one that almost described how to compile it, but none are detailed enough for me/I do not understand them completely.

Take a look at JavaCompiler
The following is based on the example given in the JavaDocs
This will save a File in the testcompile directory (based on the package name requirements) and the compile the File to a Java class...
package inlinecompiler;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.Writer;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import javax.tools.Diagnostic;
import javax.tools.DiagnosticCollector;
import javax.tools.JavaCompiler;
import javax.tools.JavaFileObject;
import javax.tools.StandardJavaFileManager;
import javax.tools.ToolProvider;
public class InlineCompiler {
public static void main(String[] args) {
StringBuilder sb = new StringBuilder(64);
sb.append("package testcompile;\n");
sb.append("public class HelloWorld implements inlinecompiler.InlineCompiler.DoStuff {\n");
sb.append(" public void doStuff() {\n");
sb.append(" System.out.println(\"Hello world\");\n");
sb.append(" }\n");
sb.append("}\n");
File helloWorldJava = new File("testcompile/HelloWorld.java");
if (helloWorldJava.getParentFile().exists() || helloWorldJava.getParentFile().mkdirs()) {
try {
Writer writer = null;
try {
writer = new FileWriter(helloWorldJava);
writer.write(sb.toString());
writer.flush();
} finally {
try {
writer.close();
} catch (Exception e) {
}
}
/** Compilation Requirements *********************************************************************************************/
DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<JavaFileObject>();
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
StandardJavaFileManager fileManager = compiler.getStandardFileManager(diagnostics, null, null);
// This sets up the class path that the compiler will use.
// I've added the .jar file that contains the DoStuff interface within in it...
List<String> optionList = new ArrayList<String>();
optionList.add("-classpath");
optionList.add(System.getProperty("java.class.path") + File.pathSeparator + "dist/InlineCompiler.jar");
Iterable<? extends JavaFileObject> compilationUnit
= fileManager.getJavaFileObjectsFromFiles(Arrays.asList(helloWorldJava));
JavaCompiler.CompilationTask task = compiler.getTask(
null,
fileManager,
diagnostics,
optionList,
null,
compilationUnit);
/********************************************************************************************* Compilation Requirements **/
if (task.call()) {
/** Load and execute *************************************************************************************************/
System.out.println("Yipe");
// Create a new custom class loader, pointing to the directory that contains the compiled
// classes, this should point to the top of the package structure!
URLClassLoader classLoader = new URLClassLoader(new URL[]{new File("./").toURI().toURL()});
// Load the class from the classloader by name....
Class<?> loadedClass = classLoader.loadClass("testcompile.HelloWorld");
// Create a new instance...
Object obj = loadedClass.newInstance();
// Santity check
if (obj instanceof DoStuff) {
// Cast to the DoStuff interface
DoStuff stuffToDo = (DoStuff)obj;
// Run it baby
stuffToDo.doStuff();
}
/************************************************************************************************* Load and execute **/
} else {
for (Diagnostic<? extends JavaFileObject> diagnostic : diagnostics.getDiagnostics()) {
System.out.format("Error on line %d in %s%n",
diagnostic.getLineNumber(),
diagnostic.getSource().toUri());
}
}
fileManager.close();
} catch (IOException | ClassNotFoundException | InstantiationException | IllegalAccessException exp) {
exp.printStackTrace();
}
}
}
public static interface DoStuff {
public void doStuff();
}
}
Now updated to include suppling a classpath for the compiler and loading and execution of the compiled class!

I suggest using the Java Runtime Compiler library. You can give it a String in memory and it will compile and load the class into the current class loader (or one of your choice) and return the Class loaded. Nested classes are also loaded. Note: this works entirely in memory by default.
e.g.
// dynamically you can call
String className = "mypackage.MyClass";
String javaCode = "package mypackage;\n" +
"public class MyClass implements Runnable {\n" +
" public void run() {\n" +
" System.out.println(\"Hello World\");\n" +
" }\n" +
"}\n";
Class aClass = CompilerUtils.CACHED_COMPILER.loadFromJava(className, javaCode);
Runnable runner = (Runnable) aClass.newInstance();
runner.run();

Conevert string to java code [duplicate]

This question already has answers here:
How do I programmatically compile and instantiate a Java class?
(3 answers)
Closed 7 years ago.
Is there a way for a running Java program to compile Java source code (passed as a string)?
Class newClass = Compiler.compile ("class ABC { void xyz {etc. etc. } }");
Ideally, any classes referenced by the passed-in source code would be resolved by the program's class loader.
Does something like this exist?

Sure. Have a look at the JavaCompiler class and the other classes in the javax.tools package.
They've been around since Java 1.6.
Here is some example code.
(As pointed out by #Sergey Tachenov in the comments, it needs JDK to be installed as the necessary tools.jar file comes with JDK but not JRE.)

What you need is a class that extends JavaFileObject
import java.net.URI;
import javax.tools.SimpleJavaFileObject;
public class JavaSourceFromString extends SimpleJavaFileObject {
final String code;
public JavaSourceFromString( String name, String code) {
super( URI.create("string:///" + name.replace('.','/')
+ Kind.SOURCE.extension),Kind.SOURCE);
this.code = code;
}
#Override
public CharSequence getCharContent(boolean ignoreEncodingErrors) {
return code;
}
}
Which can be used as follows:
JavaCompiler jc = ToolProvider.getSystemJavaCompiler();
if( jc == null) throw new Exception( "Compiler unavailable");
String code = "public class CustomProcessor { /*custom stuff*/ }";
JavaSourceFromString jsfs = new JavaSourceFromString( "CustomProcessor", code);
Iterable<? extends JavaFileObject> fileObjects = Arrays.asList( jsfs);
List<String> options = new ArrayList<String>();
options.add("-d");
options.add( compilationPath);
options.add( "-classpath");
URLClassLoader urlClassLoader =
(URLClassLoader)Thread.currentThread().getContextClassLoader();
StringBuilder sb = new StringBuilder();
for (URL url : urlClassLoader.getURLs()) {
sb.append(url.getFile()).append(File.pathSeparator);
}
sb.append( compilationPath);
options.add(sb.toString());
StringWriter output = new StringWriter();
boolean success = jc.getTask( output, null, null, options, null, fileObjects).call();
if( success) {
logger.info( LOG_PREFIX + "Class has been successfully compiled");
} else {
throw new Exception( "Compilation failed :" + output);
}

Depends on what you want to do.
If you just want to run some code you could use BeanShell. It's not a java compiled class, but is very usefull to make something flexible

You could try my essence jcf library which does this. When running in debug you can have the source written to a file so you can step into the code. Otherwise, it does everything in memory. It wraps the JavaCompiler in tools.jar
It takes a String, compiles and loads it into the current class loader and returns the Class. It handles nested/inner classes.
http://vanillajava.blogspot.com/2010/11/more-uses-for-dynamic-code-in-java.html
Note: I haven't got this working in OSGi. ;)

Javassist can generate and load at runtime classes and methods from Strings of source code.
It is also possible to dump in the file system the generated class if you need to.
Currently there are minor limitations in the code you can pass in those strings, for example it cannot include generics, enumerations, or autoboxing and inboxing of primitives.
More information here:
http://www.csg.ci.i.u-tokyo.ac.jp/~chiba/javassist/

Loading a class from an external jar

This is my first java program, so please excuse me if its too naive.
I have a 3rd party jar. I want to instantiate a class in the jar and be able to use its methods. Some details about the class in the jar:
Class File: rediff.inecom.catalog.product.CSVAPI
Constructor: CSVAPI()
Method: UpdateCSVAPI(key, csvpath)
Return: String
I have written the following program:
import java.io.File;
import java.net.URL;
import java.net.URLClassLoader;
import java.io.IOException;
class MyLoaderClass{
public void myLoaderFunction(){
File file = new File("vendorcatalogapi.jar");
try {
URL url = file.toURI().toURL();
URL[] urls = new URL[]{url};
ClassLoader cl = new URLClassLoader(urls);
Class cls = cl.loadClass("rediff.inecom.catalog.product.CSVAPI");
Object cls_object = cls.newInstance();
System.out.println(cls_object);
String output = cls_object.UpdateCSVAPI(12345,"myfile.csv");
System.out.println(output);
System.out.println("try");
}
catch (Exception e) {
System.out.println("catch");
e.printStackTrace();
}
}
public static void main(String args[]){
new MyLoaderClass().myLoaderFunction();
}
}
I am trying to compile it using:
javac -cp vendorcatalogapi.jar temp.java
But I am getting the following error:
temp.java:17: error: cannot find symbol
String output = cls_object.UpdateCSVAPI(12345,"myfile.csv");
^
symbol: method UpdateCSVAPI(int,String)
location: variable cls_object of type Object
1 error
Looks like the object is not correctly initialized. Please can someone help me with the correct way of doing it

If this is your first java program, then loading the class dynamically is probably overkill. Just use it normally and let the default class loader load it:
import java.io.File;
import java.net.URL;
import java.net.URLClassLoader;
import java.io.IOException;
import rediff.inecom.catalog.product.CSVAPI;
class MyFirstClass{
public void myFunction() {
CSVAPI cvsapi = new CSVAPI();
System.out.println(cvsapi);
String output = cvsapi.UpdateCSVAPI(12345,"myfile.csv");
System.out.println(output);
System.out.println("Success!");
}
public static void main(String args[]){
new MyFirstClass().myFunction();
}
}
Compile (note that the source code file name must match the class name):
javac -cp vendorcatalogapi.jar MyFirstClass.java
Run:
java -cp .:vendorcatalogapi.jar MyFirstClass (on Unix based)
java -cp .;vendorcatalogapi.jar MyFirstClass (on Windows)

You have to let the compiler know that cls_object is an instance of CSVAPI. If you don't, you can only use the object methods (toString, equals, etc.).
To do this, you can do the following:
rediff.inecom.catalog.product.CSVAPI cls_object = (rediff.inecom.catalog.product.CSVAPI) cls.newInstance();
Please, note that you need to have CSVAPI in your classpath!

Object class doesnt know the methods of rediff.inecom.catalog.product.CSVAPI class.
Class cls = cl.loadClass("rediff.inecom.catalog.product.CSVAPI");
Object cls_object = cls.newInstance();
So, explicit casting is required
rediff.inecom.catalog.product.CSVAPI object =
(rediff.inecom.catalog.product.CSVAPI) cls.newInstance();
will do the job.