We Keep Coding

Can a native image generated by GraalVM replace IKVM generated DLLs?

I have some Java-app and a customer with some UWP-app implemented in C#, distributed through the Windows Store etc., who wants to use some pieces of my app. Those pieces are pretty OS-independent, only parsing of some special binary file formats, applying some business logic configured using YAML files and stuff. No network, GUI, only some accesses to files etc.
We currently use IKVM to make the code of interest available to C# but ran into different problems already. Some were supporting .NET Core, some had to do with the native toolchain in Release etc. While right now things seem to work after applying some workarounds, I'm looking for alternatives to IKVM already a bit.
The only thing I currently use of IKVM is simply creating a DLL of my code using ikvmc, which can then be referenced in the UWP-project. The compiler is summarized like the following:
The ikvmc tool converts Java bytecode to .NET dll's and exe's.
That's where the support to create native Windows images of GraalVM came into my mind. Others seem to already build native binaries for Windows and according to the docs, GraalVM is able to create shared libs using "--shared". From my understanding, IKVM implements a JVM in .NET and maps things as needed and possible. That sounds pretty much like what "Substrate VM" does in case of a native image, doesn't it?
This executable includes the application, the libraries, the JDK and
does not run on the Java VM, but includes necessary components like
memory management and thread scheduling from a different virtual
machine, called “Substrate VM”. Substrate VM is the name for the
runtime components (like the deoptimizer, garbage collector, thread
scheduling etc.).
https://www.graalvm.org/docs/reference-manual/native-image/
So, is there any chance that a native image in form of a DLL can replace the DLL created by ikvmc currently? Did anyone try that already and has any experiences? Did anyone try already to create a native DLL and consume that in some other native Windows app? From my understanding UWP "only" applies additional restrictions which one might be able to work around again. Or is this approach totally impossible for some reasons?
Thanks all for your input!

I'm not very familiar with the IKVM project, so this answer is mostly about the generic question:
Can you create a native DLL/shared library and consume that in some other native Windows app?
It should be possible. You can compile Java code into a shared library. The entry points are marked with the #CEntrypoint annotation.
You can then use the generated shared library and the header files to consume your library from a native application.
This way for example GraalVM distributions use the GraalVM JIT compiler by default:
The GraalVM JIT is written in Java
Compiled as a shared library with the native-image
Used in Hotspot.
Here's a page describing how to consume those from Java through the JNI: https://www.graalvm.org/reference-manual/native-image/ImplementingNativeMethodsInJavaWithSVM/
which could be very similar to how would you use a shared library from a C# application.

GraalVM native images are not very flexible, unlike IKVM.NET images. Unless you like writing wrappers and playing with P/Invoke, you should stick to IKVM.NET.
NOTE: I am behind an IKVM.NET fork

Is it possible to compile a class on the fly in Java without using external libraries?

It seems to me that the Java Compiler API allows to compile at runtime a class, writing its output (the .class file) to the file system. However, in-memory compilation is not supported. Is this correct ? or is possible to use this API to compile a class in memory (from a String) and instantiating such class afterwards ?
(I know I can compile the class to the file system and load it afterwards with a custom class loader, but I am wondering if I can compile it in memory, without passing by the file system).
Is there another alternative mechanism to do such in memory compilation using the J2SE only?
BeanShell (I library that can do what I want) mentions in its web page that it may be included in the J2SE at "some point in the future", however, the status of its JSR is "Dormant" (whatever does it mean).
Update:
Ideally, I would like to know if this can be done with the J2SE only (or if there are any expected enhancements to J2SE that will allow me to do this in, for example, Java 8). However, tips about how to do that requirying the JDK to be installed are also appreciated (thanks Evgeniy).

It is possible if you have JDK, java complier is in tools.jar which comes with JDK only. See http://docs.oracle.com/javase/7/docs/api/javax/tools/package-summary.html

Java Compiler API .. in-memory compilation is not supported. Is this correct?
No. The STBC uses the JavaCompiler to do exactly that.
..and instantiating such class afterwards?
The STBC does not go as far as trying to load/run the class, but I believe it should be possible. I imagine it might require a custom 'in memory' class loader though.
..the JavaCompiler API can do that independently if the JDK is installed or not?
From the page..
System Requirements
STBC will run on any computer with a version 1.6+ Java Plug-In* JDK (AKA SDK).
* The API that STBC uses is merely a public interface to the compiler in the tools.jar that is distributed only with JDKs (though the 'public JRE' of the JDK also seems to acquire a tools.jar). ..

Java Applet allowed for Javac or other jar access?

I am involved in a project that will need to run via web and have access to java's compiler tools and/or javacc api. My team is thinking of using a java applet to make it web based. I'm wondering if there are certain limitations on what an applet can and cannot do in this case. I would assume that since access to the compiler would be done on the server, not the client's machine, that this wouldn't be a problem. Does an applet allow us to separate the two as described?

An applet (and even a JavaFX applet) can work in this situation if the applet is signed. There are numerous subtle pitfalls with applets, so I would advise prototyping before committing to that technology. Follow the JavaFX deployment guide to see how to deploy a JavaFX based applet.
I had thought that to compile Java, you needed to have the full Java Development Kit installed (which would be tricky to ensure in an applet deployment situation). But it seems that the compile API is included in the javax.tools API included with the standard Java Runtime Environment. So this likely means that you could develop your solution, including client based deployment and compilation of Java code, without requiring the user install the full Java Development Kit.
You may alternately wish to consider a client/server solution where the compilation can be performed on the server. An example of such an approach (with a Java WebStart based solution) is the TopCoder Algorithm Competition Application. Here is a jnlp file (http://apps.topcoder.com/wiki/display/tc/The+Algorithm+Competition+Arena) to run this application. I suggest you register an ID at TopCoder using the application and try out writing and compiling some code using it. The TopCoder implementation uses plain Swing as it was written before JavaFX existed, but you could equally use JavaFX for your implementation if you preferred.
If you additionally need an editor (with syntax aware text styling) for the code you will be compiling, you could use something like this CodeMirror based editor embedded in JavaFX. The CodeMirror based solution wraps the editor in html based WebView control. For JavaFX 8 you will probably be able to make use of the new TextFlow control for a syntax highlighting text editor, but that API is not part of a supported public release yet.
Update
I got this work using the strategy outlined in this answer.
The image is an html page allowing access as an applet or a webstart application to the client code editor. The top area of the image is the code editing area which is based on a WebEngine with an embedded syntax highlighting CodeMirror JavaScript editor that supports Java editing. The bottom area of the image is the output of compiling the code in the editor locally on the client machine and subsequently running it. The output constists of any compilation errors, any program output to sysout, as well as any runtime exceptions printed to syserr. The tricky parts of the solution were:
Working out how to capture sysout and syserr and redirect them to a JavaFX control.
Finding the Java compiler.
The default Oracle Java Runtime Environment Provider merely provides a generic interface to a Java Compiler implementation, but it provides no java compiler implementation itself - that implementation is only included in the tools.jar included with the jdk. So when I packaged my applet, I included the tools.jar in the packaging for the applet. I had some difficulty getting the service provider interface to get me an instance of the javac compiler, so in the end, I just instantiated it using the following line:
JavaCompiler compiler = new com.sun.tools.javac.api.JavacTool();
The above is somewhat brittle as sun may change their private com.sun classes at any time - but at least it worked in this instance.
Another thing to be aware of is that if you ship a tools.jar with a javac compiler which is earlier than the runtime environment that you have available for your system, then you might get some warnings such as below:
warning: C:\Program Files\Java\jre8\lib\rt.jar(java/lang/Object.class): major version 52 is newer than 51, the highest major version supported by this compiler.
It is recommended that the compiler be upgraded.
The above warning occurred because I shipped the applet with a java 7 tools.jar and ran the applet with a java 8 runtime (note that the applet worked fine regardless of those warnings).
Update
I put the code for this solution in a github repository (project name conception). The updated solution uses the Eclipse Compiler for Java rather than the Oracle Java Compiler. Mostly because, for the Eclipse Compiler, it is a separate jar (only 1.8meg rather than the 14meg tool jar of the oracle distribution) and the licensing is a bit clearer. Because the Java compiler interface is pluggable, the Oracle compiler can still be used if tools.jar is placed on the classpath.

Yeah applets can access them and can be a good choice. But it has very limited/ dull look and feell. Go for JavaFx in this you can define your own StyleSheet so it will give you a very good look and feel and yeah definitely it will separate the two layers too.
JavaFx Oracle Documentation

Dynamically load and run Java code from within Java application

I am thinking about writing a Java application where the user can write some modules in Java and add them to a library in this application. The modules will be some calculations that use data from the main app. Maybe a bit like VBA in Office.
I would appreciate if someone would give me some hints where to start as I couldn't find something useful on the net.
Thanks in advance!

You can try to develop a module framework from scratch, if you wish.
However, if you are developing anything serious you should consider using OSGi.

Theoretically it is possible. You can allow user to write java classes, then you can compile the class using java compiler, generate .class files. you can load them using your custom class loader (probably URLClassLoader or its subclass, etc.
BUT
It is very serious application. Actually it is a kind of IDE. So, of you really want this check out a possibility to create Eclipse based application, i.e. implement several eclipse plugins.
Other approach may be to allow user wring code in one of popular scripting languages. For example groovy that has java-like syntax but can be run without compilation and does not require creating classes etc. Javascript is an option too. Javascript interpreter is a part of JDK, so you even do not need external dependencies.

you might be better off incorporating some form of jvm scripting language. Something like groovy, jruby or jython. Those don't need actual compilation and can be stored in your system as source. Plus they can be quite nice to write code in too. Groovy has the advantage of being a superset of java. (For the most part) you could just degrade to java code, and run it in the groovy interpreter.

Using Java Compiler API without requiring install of JDK

Hello All
I am writing some software that will allow users to create their own Java classes for a specific use in my software package. Obviously, my software will need to be able to invoke a Java compiler to compile the user-generated classes for use in my program. However, I do not want to require users to download and install the entire JDK just so they can have access to the javac Java compiler. I understand that in Jave 6 there is a new Java Compiler API but even then, users with just the JRE and not the JDK will get a null object when they try to instantiate the Java compiler tool.
So, what is the best way to enable my program to compile Java classes while requiring the end user to just have the JRE installed on their machines? If this is not possible, what is the minimal set of libraries/jar files I would need to install on the user machine?
I suppose another possibility might be to use JWS (javaws) to launch the app over the web. In this case, is it possible for my software to not require the JDK (mainly tools.jar I think)? Would I somehow have to bundle tools.jar with my software?

You can use standalone Java compiler from Eclipse. It doesn't need JDK installed, and it's single JAR file you can use in your project. This is the compiler that is used inside Eclipse itself, but it can be integrated into other programs too. You can find current latest stable version at http://download.eclipse.org/eclipse/downloads/drops/R-3.6.2-201102101200/index.php, look for Look for JDT Core Batch Compiler. Documentation is available at http://help.eclipse.org/helios/index.jsp?topic=/org.eclipse.jdt.doc.isv/guide/jdt_api_compile.htm
You can also use Janino, which can compile also smaller units than full Java classes: blocks, expressions. It's not full compiler though, it does not support all features of Java language that your users use (generics, annotations, enums, ...)

To use the java compiler, you need to include tools.jar into your application (e.g. it has to be reachable by the classloader who wants to load the compiler - most easily by the System class loader).

Maybe http://asm.ow2.org/ this will be usefull? To generate bytecode on fly?

It sounds like a better solution would be to use some scripting language that can run within the JVM without having to be compiled.
Possible the Java Scripting API would be of use to you. I'm sure there are other options as well.