Calling clojure from java

Calling clojure from java - java

Most of the top google hits for "calling clojure from java" are outdated and recommend using clojure.lang.RT to compile the source code. Could you help with a clear explanation of how to call Clojure from Java assuming you have already built a jar from the Clojure project and included it in the classpath?

Update: Since this answer was posted, some of the tools available have changed. After the original answer, there is an update including information on how to build the example with current tools.
It isn't quite as simple as compiling to a jar and calling the internal methods. There do seem to be a few tricks to make it all work though. Here's an example of a simple Clojure file that can be compiled to a jar:
(ns com.domain.tiny
(:gen-class
:name com.domain.tiny
:methods [#^{:static true} [binomial [int int] double]]))
(defn binomial
"Calculate the binomial coefficient."
[n k]
(let [a (inc n)]
(loop [b 1
c 1]
(if (> b k)
c
(recur (inc b) (* (/ (- a b) b) c))))))
(defn -binomial
"A Java-callable wrapper around the 'binomial' function."
[n k]
(binomial n k))
(defn -main []
(println (str "(binomial 5 3): " (binomial 5 3)))
(println (str "(binomial 10042 111): " (binomial 10042 111)))
)
If you run it, you should see something like:
(binomial 5 3): 10
(binomial 10042 111): 49068389575068144946633777...
And here's a Java program that calls the -binomial function in the tiny.jar.
import com.domain.tiny;
public class Main {
public static void main(String[] args) {
System.out.println("(binomial 5 3): " + tiny.binomial(5, 3));
System.out.println("(binomial 10042, 111): " + tiny.binomial(10042, 111));
}
}
It's output is:
(binomial 5 3): 10.0
(binomial 10042, 111): 4.9068389575068143E263
The first piece of magic is using the :methods keyword in the gen-class statement. That seems to be required to let you access the Clojure function something like static methods in Java.
The second thing is to create a wrapper function that can be called by Java. Notice that the second version of -binomial has a dash in front of it.
And of course the Clojure jar itself must be on the class path. This example used the Clojure-1.1.0 jar.
Update: This answer has been re-tested using the following tools:
Clojure 1.5.1
Leiningen 2.1.3
JDK 1.7.0 Update 25
The Clojure Part
First create a project and associated directory structure using Leiningen:
C:\projects>lein new com.domain.tiny
Now, change to the project directory.
C:\projects>cd com.domain.tiny
In the project directory, open the project.clj file and edit it such that the contents are as shown below.
(defproject com.domain.tiny "0.1.0-SNAPSHOT"
:description "An example of stand alone Clojure-Java interop"
:url "http://clarkonium.net/2013/06/java-clojure-interop-an-update/"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}
:dependencies [[org.clojure/clojure "1.5.1"]]
:aot :all
:main com.domain.tiny)
Now, make sure all of the dependencies (Clojure) are available.
C:\projects\com.domain.tiny>lein deps
You may see a message about downloading the Clojure jar at this point.
Now edit the Clojure file C:\projects\com.domain.tiny\src\com\domain\tiny.clj such that it contains the Clojure program shown in the original answer. (This file was created when Leiningen created the project.)
Much of the magic here is in the namespace declaration. The :gen-class tells the system to create a class named com.domain.tiny with a single static method called binomial, a function taking two integer arguments and returning a double. There are two similarly named functions binomial, a traditional Clojure function, and -binomial and wrapper accessible from Java. Note the hyphen in the function name -binomial. The default prefix is a hyphen, but it can be changed to something else if desired. The -main function just makes a couple of calls to the binomial function to assure that we are getting the correct results. To do that, compile the class and run the program.
C:\projects\com.domain.tiny>lein run
You should see output shown in the original answer.
Now package it up in a jar and put it someplace convenient. Copy the Clojure jar there too.
C:\projects\com.domain.tiny>lein jar
Created C:\projects\com.domain.tiny\target\com.domain.tiny-0.1.0-SNAPSHOT.jar
C:\projects\com.domain.tiny>mkdir \target\lib
C:\projects\com.domain.tiny>copy target\com.domain.tiny-0.1.0-SNAPSHOT.jar target\lib\
1 file(s) copied.
C:\projects\com.domain.tiny>copy "C:<path to clojure jar>\clojure-1.5.1.jar" target\lib\
1 file(s) copied.
The Java Part
Leiningen has a built-in task, lein-javac, that should be able to help with the Java compilation. Unfortunately, it seems to be broken in version 2.1.3. It can't find the installed JDK and it can't find the Maven repository. The paths to both have embedded spaces on my system. I assume that is the problem. Any Java IDE could handle the compilation and packaging too. But for this post, we're going old school and doing it at the command line.
First create the file Main.java with the contents shown in the original answer.
To compile java part
javac -g -cp target\com.domain.tiny-0.1.0-SNAPSHOT.jar -d target\src\com\domain\Main.java
Now create a file with some meta-information to add to the jar we want to build. In Manifest.txt, add the following text
Class-Path: lib\com.domain.tiny-0.1.0-SNAPSHOT.jar lib\clojure-1.5.1.jar
Main-Class: Main
Now package it all up into one big jar file, including our Clojure program and the Clojure jar.
C:\projects\com.domain.tiny\target>jar cfm Interop.jar Manifest.txt Main.class lib\com.domain.tiny-0.1.0-SNAPSHOT.jar lib\clojure-1.5.1.jar
To run the program:
C:\projects\com.domain.tiny\target>java -jar Interop.jar
(binomial 5 3): 10.0
(binomial 10042, 111): 4.9068389575068143E263
The output is essentially identical to that produced by Clojure alone, but the result has been converted to a Java double.
As mentioned, a Java IDE will probably take care of the messy compilation arguments and the packaging.

As of Clojure 1.6.0, there is a new preferred way to load and invoke Clojure functions. This method is now preferred to calling RT directly (and supersedes many of the other answers here). The javadoc is here - the main entry point is clojure.java.api.Clojure.
To lookup and call a Clojure function:
IFn plus = Clojure.var("clojure.core", "+");
plus.invoke(1, 2);
Functions in clojure.core are automatically loaded. Other namespaces can be loaded via require:
IFn require = Clojure.var("clojure.core", "require");
require.invoke(Clojure.read("clojure.set"));
IFns can be passed to higher order functions, e.g. the example below passes plus to read:
IFn map = Clojure.var("clojure.core", "map");
IFn inc = Clojure.var("clojure.core", "inc");
map.invoke(inc, Clojure.read("[1 2 3]"));
Most IFns in Clojure refer to functions. A few, however, refer to non-function data values. To access these, use deref instead of fn:
IFn printLength = Clojure.var("clojure.core", "*print-length*");
IFn deref = Clojure.var("clojure.core", "deref");
deref.invoke(printLength);
Sometimes (if using some other part of the Clojure runtime), you may need to ensure that the Clojure runtime is properly initialized - calling a method on the Clojure class is sufficient for this purpose. If you do not need to call a method on Clojure, then simply causing the class to load is sufficient (in the past there has been a similar recommendation to load the RT class; this is now preferred):
Class.forName("clojure.java.api.Clojure")

EDIT This answer was written in 2010, and worked at that time. See Alex Miller's answer for more modern solution.
What kind of code are calling from Java? If you have class generated with gen-class, then simply call it. If you want to call function from script, then look to following example.
If you want to evaluate code from string, inside Java, then you can use following code:
import clojure.lang.RT;
import clojure.lang.Var;
import clojure.lang.Compiler;
import java.io.StringReader;
public class Foo {
public static void main(String[] args) throws Exception {
// Load the Clojure script -- as a side effect this initializes the runtime.
String str = "(ns user) (defn foo [a b] (str a \" \" b))";
//RT.loadResourceScript("foo.clj");
Compiler.load(new StringReader(str));
// Get a reference to the foo function.
Var foo = RT.var("user", "foo");
// Call it!
Object result = foo.invoke("Hi", "there");
System.out.println(result);
}
}

EDIT: I wrote this answer almost three years ago. In Clojure 1.6 there is a proper API exactly for the purpose of calling Clojure from Java. Please Alex Miller's answer for up to date information.
Original answer from 2011:
As I see it, the simplest way (if you don't generate a class with AOT compilation) is to use clojure.lang.RT to access functions in clojure. With it you can mimic what you would have done in Clojure (no need to compile things in special ways):
;; Example usage of the "bar-fn" function from the "foo.ns" namespace from Clojure
(require 'foo.ns)
(foo.ns/bar-fn 1 2 3)
And in Java:
// Example usage of the "bar-fn" function from the "foo.ns" namespace from Java
import clojure.lang.RT;
import clojure.lang.Symbol;
...
RT.var("clojure.core", "require").invoke(Symbol.intern("foo.ns"));
RT.var("foo.ns", "bar-fn").invoke(1, 2, 3);
It is a bit more verbose in Java, but I hope it's clear that the pieces of code are equivalent.
This should work as long as Clojure and the source files (or compiled files) of your Clojure code is on the classpath.

I agree with clartaq's answer, but I felt that beginners could also use:
step-by-step information on how to actually get this running
information that's current for Clojure 1.3 and recent versions of leiningen.
a Clojure jar that also includes a main function, so it can be run standalone or linked as a library.
So I covered all that in this blog post.
The Clojure code looks like this:
(ns ThingOne.core
(:gen-class
:methods [#^{:static true} [foo [int] void]]))
(defn -foo [i] (println "Hello from Clojure. My input was " i))
(defn -main [] (println "Hello from Clojure -main." ))
The leiningen 1.7.1 project setup looks like this:
(defproject ThingOne "1.0.0-SNAPSHOT"
:description "Hello, Clojure"
:dependencies [[org.clojure/clojure "1.3.0"]]
:aot [ThingOne.core]
:main ThingOne.core)
The Java code looks like this:
import ThingOne.*;
class HelloJava {
public static void main(String[] args) {
System.out.println("Hello from Java!");
core.foo (12345);
}
}
Or you can also get all the code from this project on github.

This works with Clojure 1.5.0:
public class CljTest {
public static Object evalClj(String a) {
return clojure.lang.Compiler.load(new java.io.StringReader(a));
}
public static void main(String[] args) {
new clojure.lang.RT(); // needed since 1.5.0
System.out.println(evalClj("(+ 1 2)"));
}
}

If the use case is to include a JAR built with Clojure in a Java application, I have found having a separate namespace for the interface between the two worlds to be beneficial:
(ns example-app.interop
(:require [example-app.core :as core])
;; This example covers two-way communication: the Clojure library
;; relies on the wrapping Java app for some functionality (through
;; an interface that the Clojure library provides and the Java app
;; implements) and the Java app calls the Clojure library to perform
;; work. The latter case is covered by a class provided by the Clojure lib.
;;
;; This namespace should be AOT compiled.
;; The interface that the java app can implement
(gen-interface
:name com.example.WeatherForecast
:methods [[getTemperature [] Double]])
;; The class that the java app instantiates
(gen-class
:name com.example.HighTemperatureMailer
:state state
:init init
;; Dependency injection - take an instance of the previously defined
;; interface as a constructor argument
:constructors {[com.example.WeatherForecast] []}
:methods [[sendMails [] void]])
(defn -init [weather-forecast]
[[] {:weather-forecast weather-forecast}])
;; The actual work is done in the core namespace
(defn -sendMails
[this]
(core/send-mails (.state this)))
The core namespace can use the injected instance to accomplish its tasks:
(ns example-app.core)
(defn send-mails
[{:keys [weather-forecast]}]
(let [temp (.getTemperature weather-forecast)] ...))
For testing purposes, the interface can be stubbed:
(example-app.core/send-mails
(reify com.example.WeatherForecast (getTemperature [this] ...)))

Other technique that works also with other languages on top of JVM is to declare an interface for functions you want to call and then use 'proxy' function to create instance that implemennts them.

You can also use AOT compilation to create class files representing your clojure code. Read the documentation about compilation, gen-class and friends in the Clojure API docs for the details about how to do this, but in essence you will create a class that calls clojure functions for each method invocation.
Another alternative is to use the new defprotocol and deftype functionality, which will also require AOT compilation but provide better performance. I don't know the details of how to do this yet, but a question on the mailing list would probably do the trick.

Related

Unit test Java class that loads native library

I'm running unit tests in Android Studio. I have a Java class that loads a native library with the following code
static
{
System.loadLibrary("mylibrary");
}
But when I test this class inside my src/test directory I get
java.lang.UnsatisfiedLinkError: no mylibrary in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1864)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
How can I make it find the path of native .so libraries which is located at src/main/libs in order to unit test without errors?
Note: inside src/main/libs directory I have 3 more subdirectories: armeabi, mips and x86. Each one of those contains the proper .so file. I'm using the Non experimental version for building NDK libs.
I don't wanna use other 3rd party testing libraries as all my other "pure" java classes can be unit tested fine. But if that's not possible then I'm open to alternatives.
Here is my test code which throws the error
#Test
public void testNativeClass() throws Exception
{
MyNativeJavaClass test = new MyNativeJavaClass("lalalal")
List<String> results = test.getResultsFromNativeMethodAndPutThemInArrayList();
assertEquals("There should be only three result", 3, results.size());
}

The only solution I found that works without hacks is to use JUnit through instrumentation testing (androidTest directory).
My class can now be tested fine but with help of the android device or emulator.

If the library is required for your test, use an AndroidTest (under src/androidTest/...) rather than a junit test. This will allow you to load and use the native library like you do elsewhere in your code.
If the library is not required for your test, simply wrap the system load in a try/catch. This will allow the JNI class to still work in junit tests (under src/test/...) and it is a safe workaround, given that it is unlikely to mask the error (something else will certainly fail, if the native lib is actually needed). From there, you can use something like mockito to stub out any method calls that still hit the JNI library.
For example in Kotlin:
companion object {
init {
try {
System.loadLibrary("mylibrary")
} catch (e: UnsatisfiedLinkError) {
// log the error or track it in analytics
}
}
}

I am not sure whether this solves your problem or not but so far nobody has mentioned about strategy pattern for dealing with classes preloading library during their creation.
Let's see the example:
We want to implement Fibonacci solver class. Assuming that we provided implementation in the native code and managed to generate the native library, we can implement the following:
public interface Fibonacci {
long calculate(int steps);
}
Firstly, we provide our native implementation:
public final class FibonacciNative implements Fibonacci {
static {
System.loadLibrary("myfibonacci");
}
public native long calculate(int steps);
}
Secondly, we provide Java implementation for Fibonacci solver:
public final class FibonacciJava implements Fibonacci {
#Override
public long calculate(int steps) {
if(steps > 1) {
return calculate(steps-2) + calculate(steps-1);
}
return steps;
}
}
Thirdly, we wrap the solvers with parental class choosing its own implementation during its instantiation:
public class FibonnaciSolver implements Fibonacci {
private static final Fibonacci STRATEGY;
static {
Fibonacci implementation;
try {
implementation = new FibonnaciNative();
} catch(Throwable e) {
implementation = new FibonnaciJava();
}
STRATEGY = implementation;
}
#Override
public long calculate(int steps) {
return STRATEGY.calculate(steps);
}
}
Thus, the problem with finding path to the library using strategy. This case, however, does not resolve the problem if the native library is really necessary to be included during the test. It does not neither solve the problem if the native library is a third-party library.
Basically, this gets around the native library load problem by mocking out the native code for java code.
Hope this helps somehow:)

There is a way to configure library path of Gradle-run VM for local unit tests, and I'm going to describe it below, but spoiler: in my expericence, #ThanosFisherman is right: local unit tests for stuff that uses the Android NDK seem to be a fools errand right now.
So, for anyone else looking for a way to load shared (i.e. .so) libraries into unit tests with gradle, here's the somewhat lengthy abstract:
The goal is to set the shared library lookup path for the JVM running the unit tests.
Althoug many people suggest putting the lib path into java.library.path, I found that it doesn't work, at least not on my linux machine. (also, same results in this CodeRanch thread)
What does work though is setting the LD_LIBRARY_PATH os environment variable (or PATH is the closest synonym in Windows)
Using Gradle:
// module-level build.gradle
apply plugin: 'com.android.library' // or application
android {
...
testOptions {
unitTests {
all {
// This is where we have access to the properties of gradle's Test class,
// look it up if you want to customize more test parameters
// next we take our cmake output dir for whatever architecture
// you can also put some 3rd party libs here, or override
// the implicitly linked stuff (libc, libm and others)
def libpath = '' + projectDir + '/build/intermediates/cmake/debug/obj/x86_64/'
+':/home/developer/my-project/some-sdk/lib'
environment 'LD_LIBRARY_PATH', libpath
}
}
}
}
With that, you can run, e.g. ./gradlew :mymodule:testDebugUnitTest and the native libs will be looked for in the paths that you specified.
Using Android Studio JUnit plugin
For the Android Studio's JUnit plugin, you can specify the VM options and the environment variables in the test configuration's settings, so just run a JUnit test (right-clicking on a test method or whatever) and then edit the Run Configuration:
Although it sounds like "mission accomplished", I found that when using libc.so, libm.so and others from my os /usr/lib gives me version errors (probably because my own library is compiled by cmake with the android ndk toolkit against it's own platform libs). And using the platform libs from the ndk packages brought down the JVM wih a SIGSEGV error (due to incompatibility of the ndk platform libs with the host os environment)
Update As #AlexCohn incisively pointed out in the comments, one has to build against the host environment libs for this to work; even though your machine most likely is x86_64, the x86_64 binaries built against NDK environment will not do.
There may be something I overlooked, obviously, and I'll appreciate any feedback, but for now I'm dropping the whole idea in favor of instrumented tests.

Just make sure, the directory containing the library is contained in the java.library.path system property.
From the test you could set it before you load the library:
System.setProperty("java.library.path", "... path to the library .../libs/x86");
You can specify the path hard coded, but this will make the project less portable to other environments. So I suggest you build it up programmatically.

The .so files are to be placed under
src/main/jniLibs
Not under src/main/libs
(Tested with Android Studio 1.2.2)
For reference check the page - http://ph0b.com/android-studio-gradle-and-ndk-integration/, though some portions might be outdated.

This is very, very tricky. Setting java.library.path does not work, but trying to understand someone else’s Mac OSX approach I eventually found a working solution.
Legal release: all code examples directly copied into this post are available under CC0 but it would be appeciated to credit my employer ⮡ tarent, the LLCTO project at Deutsche Telekom, and the author mirabilos.
CAVEATS first:
with this, you’re testing a version of the native code compiled against your system libraries (usually glibc on GNU/Linux, and on BSD, Mac OSX and Windows it’s even trickier) so adding some instrumented tests should be done anyway, use the unittests only for faster testing of things that actually can be tested on the host OS
I’ve only tested this with a GNU/Linux host (and am, in fact, excluding these native tests on all other host OSes, see below)
it should work under unixoid OSes with GNU/BSD-style shared libraries as-is
with small adaptions from the “someone else’s” article linked above, it might probably work on Mac OSX
Windows… no, just no. Use WSL, which is basically Linux anyway and makes things much easier, and so much closer to Android which is also basically Linux just not GNU
IDE integration needs manual steps at each developer’s machine (but these are easily documented, see (much) below)
Prerequisites
You’ll need to make sure that all build dependencies of your native code are also installed in the host system. This includes cmake (because we sadly cannot reuse the NDK cmake) and a host C compiler. Note that these introduce further differences in the build: you’re testing something that has been built with the host C compiler (often GCC, not clang like in Android) against the host C library and other libraries by the host clang. Do consider this when writing your tests. I had to move one of the tests to instrumented because it was impossible to test under glibc.
For filesystem layout, we assume the following:
~/MYPRJ/build.gradle is the top-level build file (generated by IntelliJ / Android Studio)
~/MYPRJ/app/build.gradle is where the Android code in question is built (generated by IntelliJ / Android Studio)
~/MYPRJ/app/src/main/native/CMakeLists.txt is where the native code is situated
This means build.gradle (for the app) has something like this already, by the point where you begin wondering about whether your project can be unittested:
externalNativeBuild {
cmake {
path "src/main/native/CMakeLists.txt"
return void // WTF‽
}
}
Make sure your code builds on the host
Doing this ought to be easy at first glance:
$ rm -rf /tmp/build
$ mkdir /tmp/build
$ cd /tmp/build
$ cmake ~/MYPRJ/app/src/main/native/
$ make
(Make sure you give cmake the path to the directory the main CMakeLists.txt file is in, but not to that file itself!)
This will fail for everything nōntrivial, of course. Most people would use Android logging. (It will also fail because it cannot find <jni.h>, and because GNU libc requires an extra _GNU_SOURCE definition to access some prototypes, etc…)
So I wrote a header to include instead of <android/log.h> which abstracts the logging away…
#ifndef MYPRJ_ALOG_H
#define MYPRJ_ALOG_H
#ifndef MYPRJ_ALOG_TAG
#define MYPRJ_ALOG_TAG "MYPRJ-JNI"
#endif
#if defined(MYPRJ_ALOG_TYPE) && (MYPRJ_ALOG_TYPE == 1)
#include <android/log.h>
#define ecnlog_err(msg, ...) __android_log_print(ANDROID_LOG_ERROR, \
MYPRJ_ALOG_TAG, msg, ##__VA_ARGS__)
#define ecnlog_warn(msg, ...) __android_log_print(ANDROID_LOG_WARN, \
MYPRJ_ALOG_TAG, msg, ##__VA_ARGS__)
#define ecnlog_info(msg, ...) __android_log_print(ANDROID_LOG_INFO, \
MYPRJ_ALOG_TAG, msg, ##__VA_ARGS__)
#elif defined(MYPRJ_ALOG_TYPE) && (MYPRJ_ALOG_TYPE == 2)
#include <stdio.h>
#define ecnlog_err(msg, ...) fprintf(stderr, \
"E: [" MYPRJ_ALOG_TAG "] " msg "\n", ##__VA_ARGS__)
#define ecnlog_warn(msg, ...) fprintf(stderr, \
"W: [" MYPRJ_ALOG_TAG "] " msg "\n", ##__VA_ARGS__)
#define ecnlog_info(msg, ...) fprintf(stderr, \
"I: [" MYPRJ_ALOG_TAG "] " msg "\n", ##__VA_ARGS__)
#else
# error What logging system to use?
#endif
#endif
… and updated my CMakeLists.txt to indicate whether building for NDK (must be default) or native:
cmake_minimum_required(VERSION 3.10)
project(myprj-native)
option(UNDER_NDK "Build under the Android NDK" ON)
add_compile_options(-fvisibility=hidden)
add_compile_options(-Wall -Wextra -Wformat)
add_library(myprj-native SHARED
alog.h
myprj-jni.c
)
if (UNDER_NDK)
add_definitions(-DECNBITS_ALOG_TYPE=1)
find_library(log-lib log)
target_link_libraries(myprj-native ${log-lib})
else (UNDER_NDK)
add_definitions(-DECNBITS_ALOG_TYPE=2)
include(FindJNI)
include_directories(${JNI_INCLUDE_DIRS})
add_definitions(-D_GNU_SOURCE)
endif (UNDER_NDK)
Note this also already includes the fix for <jni.h> (FindJNI) and the extra definitions.
Now let’s try to build it again:
$ rm -rf /tmp/build
$ mkdir /tmp/build
$ cd /tmp/build
$ cmake -DUNDER_NDK=OFF ~/MYPRJ/app/src/main/native/
$ make
In my case, this was sufficient. If you’re still not there, fix this first before proceeding. If you cannot fix this, give up on buildhost-local unit tests for your JNI code and move the respective tests to instrumented.
Let Gradle build the host-native code
Add the following to the app build.gradle:
def dirForNativeNoNDK = project.layout.buildDirectory.get().dir("native-noNDK")
def srcForNativeNoNDK = project.layout.projectDirectory.dir("src/main/native").asFile
task createNativeNoNDK() {
def dstdir = dirForNativeNoNDK.asFile
if (!dstdir.exists()) dstdir.mkdirs()
}
task buildCMakeNativeNoNDK(type: Exec) {
dependsOn createNativeNoNDK
workingDir dirForNativeNoNDK
commandLine "/usr/bin/env", "cmake", "-DUNDER_NDK=OFF", srcForNativeNoNDK.absolutePath
}
task buildGMakeNativeNoNDK(type: Exec) {
dependsOn buildCMakeNativeNoNDK
workingDir dirForNativeNoNDK
commandLine "/usr/bin/env", "make"
}
project.afterEvaluate {
if (org.gradle.internal.os.OperatingSystem.current().isLinux()) {
testDebugUnitTest {
dependsOn buildGMakeNativeNoNDK
systemProperty "java.library.path", dirForNativeNoNDK.asFile.absolutePath + ":" + System.getProperty("java.library.path")
}
testReleaseUnitTest {
dependsOn buildGMakeNativeNoNDK
systemProperty "java.library.path", dirForNativeNoNDK.asFile.absolutePath + ":" + System.getProperty("java.library.path")
}
}
}
This defines a few new tasks to compile the buildhost-native version of the shared library, and hooks this up if the host OS is “Linux”. (This syntax will also work for other unixoid OSes — BSD, Mac OSX — but not for Windows. But we can probably test this under Linux only anyway. WSL counts as Linux.) It also sets up the JVM library path so that ../gradlew test will let the JVM pick up the library from its path.
Loose ends
There’s a few loose ends you might have noticed here:
In the last paragraph of the previous section, I mentioned that ../gradlew test will pick up the library. Testing from the IDE will not work yet; this involves manual setup.
I mentioned that the relevant unit tests must be skipped if the buildhost OS is not “Linux”; we have yet to do that. Unfortunately, JUnit 4 lacks such facilities, but switching the unit tests to JUnit 5 “Jupiter” will allow us to do that. (We’re not switching the instrumented tests, though; that’d be more invasive.)
You’ll probably not yet have noticed, but the logging output from the native code will not show up thanks to Gradle’s default settings which we’ll need to change.
So, let’s do that. First, edit your app build.gradle file again. There will be a dependencies { block. We’ll need to fill that with suitable dependencies for either JUnit:
dependencies {
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.7.0'
testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.7.0'
//noinspection GradleDependency
androidTestImplementation 'com.android.support.test:runner:1.0.1'
//noinspection GradleDependency
androidTestImplementation 'com.android.support.test.espresso:espresso-core:3.0.1'
//noinspection GradleDependency
androidTestImplementation 'junit:junit:4.12'
}
You’ll also have a line apply plugin: 'com.android.application' (or perhaps apply plugin: 'com.android.library') at the top. Directly below that line, insert this one:
apply plugin: 'de.mannodermaus.android-junit5'
Also, make sure that, under android { defaultConfig { the testInstrumentationRunner is still "android.support.test.runner.AndroidJUnitRunner" (the default as generated by IntelliJ / Android Studio).
Next, edit the top-level ~/MYPRJ/build.gradle file. You’ll already have a buildscript { dependencies { and will have to add a line to that section to make the JUnit5 plugin available in the first place:
//noinspection GradleDependency
classpath 'de.mannodermaus.gradle.plugins:android-junit5:1.5.2.0'
Then, add a new section under allprojects {:
tasks.withType(Test) {
testLogging {
outputs.upToDateWhen { false }
showStandardStreams = true
exceptionFormat = 'full'
}
systemProperty 'java.util.logging.config.file', file('src/test/resources/logging.properties').getAbsolutePath()
}
This ensures that…
tests are never skipped because Gradle thinks them up-to-date
logging output and exceptions are shown in full
if you have a ~/MYPRJ/app/src/test/resources/logging.properties it will set up java.util.logging with this (recommended)
Now see to your test, something like ~/MYPRJ/app/src/test/java/org/example/packagename/JNITest.java. First, you should add a “test” that can always run (I use one that merely tests whether my JNI class can be loaded), and ensure it displays some information first:
// or Lombok #Log
private static final java.util.logging.Logger LOGGER = java.util.logging.Logger.getLogger(JNITest.class.getName());
#Test
public void testClassBoots() {
LOGGER.info("running on " + System.getProperty("os.name"));
if (!LINUX.isCurrentOs()) {
LOGGER.warning("skipping JNI tests");
}
// for copy/paste into IntelliJ run options
LOGGER.info("VM options: -Djava.library.path=" +
System.getProperty("java.library.path"));
LOGGER.info("testing Java™ part of JNI class…");
[…]
}
Then, annotate the actual JNI tests that need to be skipped on other OSes:
#Test
#EnabledOnOs(LINUX)
public void testJNIBoots() {
LOGGER.info("testing JNI part of JNI class…");
final long tid;
try {
tid = JNI.n_gettid();
} catch (Throwable t) {
LOGGER.log(Level.SEVERE, "it failed", t);
Assertions.fail("JNI does not work");
return;
}
LOGGER.info("it also works: " + tid);
assertNotEquals(0, tid, "but is 0");
}
For comparison, instrumented tests (unittests that run on the Android device or emulator) — e.g. ~/MYPRJ/app/src/androidTest/java/org/example/packagename/JNIInstrumentedTest.java — look like this:
#RunWith(AndroidJUnit4.class)
public class JNIInstrumentedTest {
#Test
public void testJNIBoots() {
Log.i("ECN-Bits-JNITest", "testing JNI part of JNI class…");
final long tid;
try {
tid = JNI.n_gettid();
} catch (Throwable t) {
Log.e("ECN-Bits-JNITest", "it failed", t);
fail("JNI does not work");
return;
}
Log.i("ECN-Bits-JNITest", "it also works: " + tid);
assertNotEquals("but is 0", 0, tid);
}
}
See Testable.java if you need an assertThrows for instrumented tests (JUnit 5 already comes with one), by the way. (Note that this does not fall under the CC0 grant above but comes under a permissive licence.)
Now, you can run both tests, unittests and (if an Android emulator is started or device commected) instrumented tests:
../gradlew test connectedAndroidTest
Do so. Note the output of the VM options: logger call from the buildhost-native unit tests; in fact, copy it to the clipboard. You’ll now need it to set up testing in the IDE.
In the Project view (left-side tree), right-click either on your JNITest class or the entire src/test/java/ directory. Click on Run 'JNITest' (or Run 'Tests in 'java''), it will fail with an UnsatisfiedLinkError as in the original post.
Now click on the arrow in the test drop-down below the menu bar, then select Save JNITest configuration, then do it again and select Edit configurations… and select your configuration. Append the entire pasted thing to VM options: so the field will now look like -ea -Djava.library.path=/home/USER/MYPRJ/app/build/native-noNDK:/usr/java/packages/lib:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib (of course, the actual value will differ) and click OK. Re-run the test, and it will succeed.
Unfortunately, you’ll have to do this for every native test class once and for the entire directory, so all possible ways of invocation will be covered. You’ll also have to do this manually, by clicking around, for every IDE instance, and these values depend on the path the code was checked out into. I’ve not found a way to automate these (if you know of one, do tell).
Exception backtraces
If you’re throwing custom exceptions from your code, you’ll most likely wish to include file/lineno/function information. Use a constructor like MyprjNativeException(final String file, final int line, final String func, final String msg, … /* custom data */, final Throwable cause) and, after calling super(msg, cause) (possibly with a changed message), do this:
StackTraceElement[] currentStack = getStackTrace();
StackTraceElement[] newStack = new StackTraceElement[currentStack.length + 1];
System.arraycopy(currentStack, 0, newStack, 1, currentStack.length);
newStack[0] = new StackTraceElement("<native>", func, file, line);
setStackTrace(newStack);
Then, to throw an exception like this from native code:
#define throw(env,...) vthrow(__FILE__, __func__, env, __LINE__, __VA_ARGS__)
static void vthrow(const char *loc_file, const char *loc_func, JNIEnv *env,
int loc_line, /* custom args */ const char *msg, ...);
Use as follows:
if (func() != expected)
throw(env, /* custom args */ "foo");
Implementation (assuming you cache class and constructor method references) looks as follows (adjust for custom args):
static void vthrow(const char *loc_file, const char *loc_func, JNIEnv *env,
int loc_line, const char *fmt, ...)
{
jthrowable e;
va_list ap;
jstring jfile = NULL;
jint jline = loc_line;
jstring jfunc = NULL;
jstring jmsg = NULL;
jthrowable cause = NULL;
const char *msg;
char *msgbuf;
if ((*env)->PushLocalFrame(env, /* adjust for amount you need */ 5)) {
cause = (*env)->ExceptionOccurred(env);
(*env)->ExceptionClear(env);
(*env)->Throw(env, (*env)->NewObject(env, classreference, constructorreference,
jfile, jline, jfunc, jmsg, /* custom */…, cause));
return;
}
if ((cause = (*env)->ExceptionOccurred(env))) {
/* will be treated as cause */
(*env)->ExceptionClear(env);
}
va_start(ap, fmt);
if (vasprintf(&msgbuf, fmt, ap) == -1) {
msgbuf = NULL;
msg = fmt;
} else
msg = msgbuf;
va_end(ap);
jmsg = (*env)->NewStringUTF(env, msg);
free(msgbuf);
if (!jmsg)
goto onStringError;
if (!(jfunc = (*env)->NewStringUTF(env, loc_func)))
goto onStringError;
/* allocate NewStringUTF for any custom things you need */
/* exactly like the one for loc_func above */
/* increase PushLocalFrame argument for each */
jfile = (*env)->NewStringUTF(env, loc_file);
if (!jfile) {
onStringError:
(*env)->ExceptionClear(env);
}
e = (*env)->PopLocalFrame(env, (*env)->NewObject(env, classreference, constructorreference,
jfile, jline, jfunc, jmsg, /* custom */…, cause));
if (e)
(*env)->Throw(env, e);
}
Now using __FILE__ will put the full absolute path into the messages and backtraces. This is not very nice. There’s a compiler option to fix that, but NDK r21’s clang is much too old, so we need a workaround.
CMakeLists.txt:
if (NOT TOPLEV)
message(FATAL_ERROR "setting the top-level directory is mandatory")
endif (NOT TOPLEV)
[…]
if (UNDER_NDK)
[…]
execute_process(COMMAND ${CMAKE_CXX_COMPILER} --version OUTPUT_VARIABLE cxx_version_full)
string(REGEX REPLACE "^Android [^\n]* clang version ([0-9]+)\\.[0-9].*$" "\\1" cxx_version_major ${cxx_version_full})
if (${cxx_version_major} VERSION_GREATER_EQUAL 10)
add_definitions("-ffile-prefix-map=${TOPLEV}=«MyPrj»")
else (${cxx_version_major} VERSION_GREATER_EQUAL 10)
add_definitions(-DOLD_CLANG_SRCDIR_HACK="${TOPLEV}/")
endif (${cxx_version_major} VERSION_GREATER_EQUAL 10)
else (UNDER_NDK)
[…]
add_definitions("-ffile-prefix-map=${TOPLEV}=«MyPrj»")
endif (UNDER_NDK)
app build.gradle:
(straight after the apply plugin lines)
def dirToplev = project.layout.projectDirectory.asFile.absolutePath
(inside android { defaultConfig { add a new block)
externalNativeBuild {
cmake {
//noinspection GroovyAssignabilityCheck because Gradle and the IDE have different world views…
arguments "-DTOPLEV=" + dirToplev
}
return void // WTF‽
}
(later, where you call cmake)
commandLine "/usr/bin/env", "cmake", "-DTOPLEV=" + dirToplev, "-DUNDER_NDK=OFF", srcForNativeNoNDK.absolutePath
Then, replace the line jfile = (*env)->NewStringUTF(env, loc_file); with the following snippet:
#ifdef OLD_CLANG_SRCDIR_HACK
if (!strncmp(loc_file, OLD_CLANG_SRCDIR_HACK, sizeof(OLD_CLANG_SRCDIR_HACK) - 1) &&
asprintf(&msgbuf, "«ECN-Bits»/%s", loc_file + sizeof(OLD_CLANG_SRCDIR_HACK) - 1) != -1) {
msg = msgbuf;
} else {
msg = loc_file;
msgbuf = NULL;
}
#else
#define msg loc_file
#endif
jfile = (*env)->NewStringUTF(env, msg);
#ifdef OLD_CLANG_SRCDIR_HACK
free(msgbuf);
#else
#undef msg
#endif
Tieing it all together
This all is implemented in the ECN-Bits project. I’m posting a permalink because it’s currently on a nōn-default branch but expected to be merged (once the actual functionality is no longer WIP), so be sure to check master at some point in time as well (although this permalink is probably a better example as it has the testing down and there’s not as much “actual” code to get in the way). Note that these links do not fall under the CC0 grant from above; the files all have a permissive licence though (the files which don’t have it explicit (gradle/cmake files) have the same as the unittest class permalink), but enough of it was reposted in this article so that should not be a problem for you; these only serve to show an actually-compiling-and-testing example.
In this project, it’s not in app/ but as a separate library module.
top-level build.gradle
library build.gradle
instrumented tests
unittest class
unittest logging configuration
native CMakeLists.txt
native alog.h
native code including caching the references
Java™ code to which the JNI native code attaches including the Exception class

Try running your test code with java -XshowSettings:properties option and make sure your destination path for system libraries and in the output of this command, library path values are the same

Just to clarify, the System.loadlibrary() call was failing because the junit unit test uses host/system environment which was windows in my case. Hence the loadlibrary() call was trying to search for the .so files in standard shared libary folders. But this isn't what I was expecting to happen. Instead I wanted the libxxx.so files to be loaded from .aar file(contains android resources, jars, jni libs).
This can only happen by two ways:
Baking the libxxx.so into the APK manually: we ourselves place the libxxx.so files into jnilibs dir(which the system will search for when loading the shared lib) under src/java/ root of the apk by copying the required files.
adding the .aar as an external dependency to module level gradle build script (implementation/androidTestImplementation), by doing this we make this aar available for link editing used by the apk build process.
But in both cases the app runs in the android environment/vm and hence the System.loadlibrary() call will resolve to correct libxxx.so which would be part of the apk. Hence NO ISSUES.
However in case of unit tests, which are does not require instrument(ie, android device) and runs on the JVM running on the host system where the tests are running (ex: windows/linux/mac), The call to System.loadlibrary() resolves only the standard lib paths of host system for finding shared libs/exe and doesn't refer to android system environment. Hence the ISSUES.
Fixes:
unpack the libxxx.so into some temp dir and add this dir to the system's library search path (ex: java.library.path, %PATH% variable on windows etc). Now run the required unit tests which doesn't require the android environment but involves the native code testing using JNI if any. This should work!!
(Efficient method) Simply move these type of unit tests to androidTest(ie, Instrumentation tests) so that above explained loading and packing are intact and System.loadlibrary() can successfully find the libxxx.so when running inside the instrument(android device/os). This way you ensure appropriate lib type (x86, x86-64, arm-v7a, arm-v8a(AARCH64)) is invoked on target device and tests are run on specific target devices.

How do I set up classes in a Lein project?

I ran lein new app hm, then in hm/src/hm edited core.clj to be:
(ns hm.core
(:gen-class)
(:use [hm.hashmap]))
(defn -main []
(def j (new hm.hashmap))
(-add j "foo" "bar")
(println j))
and hashmap.clj to be:
(ns hm.hashmap
(:gen-class
:methods [[hashmap [] java.util.HashMap]
[add [String String]]]))
(defn -hashmap []
(def h (new java.util.HashMap))
h)
(defn -add [this key value]
(. this put key value)
this)
The goal is to make a wrapper around the HashMap so I can understand Clojure and how it ties with Java. I'm fairly new to Clojure. However, when I compile this, I get a lot of ClassNotFoundException in hashmap.clj. How can I make this work?

Note: This is a direct answer to your question. I don't recommend that you learn Clojure this way.
You need to compile your classes before you can run them. In your project.clj add this to the map:
:aot [hm.hashmap]
Then you need to run lein compile in order to compile the classes. You should see output saying the hm.hashmap class was compiled. After that run lein run to invoke the "main "function in hm.core.
I removed the :methods part of your gen-class because you're already defining them below, and that was causing the weird java.lang., error. You're going to run into other errors, but this should be enough to get you passed this issue.

Your code has some other issues, but the immediate problem here is that the signature of add is incomplete. Your add returns this, a hm.hashmap.
To fix, change the signature to return an Object, or, with additional edit, a java.util.HashMap. If you want this to work as otherwise written, you'll also need to extend rather than encapsulate.
(ns hm.hashmap
(:gen-class
:extends java.util.HashMap
:methods [[add [String String] java.util.HashMap]]))
Finally change -main in core.clj to call the method using .add instead of trying to access the private -add.
...
(.add j "foo" "bar")
...
Then
lein clean
lein compile hm.core hm.hashmap
lein run
should print
#<hashmap {foo=bar}>
Note that you cannot, as far as I know, specify returning an hm.hashmap in the signature due to the timing of the symbol resolution. See GC Issue 81: compile gen-class fail when class returns self.

access clojure via java classes

Hello I have a main method in a Java class and I would like to access and run my clojure functions from my java classes, is that possible right?
Help please

If you just want to call a function which you have defined in a Clojure script the following code might help you getting the job done:
test.clj:
(ns test)
(defn hello [name]
(println (str "Hi " name "!")))
TestRun.java:
import clojure.lang.RT;
public class TestRun {
public static void main(String[] args) throws Exception {
RT.loadResourceScript("test.clj");
// var(namespace, function name).invoke(parameters..)
RT.var("test", "hello").invoke("Daisy Duck");
}
}
Output:
Hi Daisy Duck!
Make sure you have the Clojure jar on your classpath

Do you have your Clojure code compiled and packaged in a jar? Do you have the jar in your classpath? If so, you should be able to use the classes in the jar just as if there were written in Java.

see the accepted answer to this question: Calling clojure from java
in short you add the mothods you want to expose to your namespace:
(ns com.domain.tiny
(:gen-class
:name com.domain.tiny
:methods [ [binomial [int int] double]]))
then write the functions. compile your class file with maven/leiningen
then call them from java:
System.out.println("(binomial 5 3): " + tiny.binomial(5, 3));
This is just an excerpt. take a look as the origional question.

Check the Java Scripting API for calling functions in script files:
http://download.oracle.com/javase/6/docs/technotes/guides/scripting/programmer_guide/index.html

Calling Haskell from Java with C in between

This probably sounds like a nightmare, but I'd really like to get this working. I am using this example for the most part: Calling C from Haskell and am trying to get this working on ubuntu.
I am running this in java:
package test;
public class JniTest {
public native int fib(int x);
}
this in c after creating the .h file with javah: (test_JniTest.c)
#include "test_JniTest.h"
#include "Safe_stub.h"
JNIEXPORT jint JNICALL Java_test_JniTest_fib(JNIEnv * e, jobject o, jint f)
{
return fibonacci_hs(f);
}
and then for reference in haskell (before stub): (Safe.hs)
module Safe where
import Foreign.C.Types
fibonacci :: Int -> Int
fibonacci n = fibs !! n
where fibs = 0 : 1 : zipWith (+) fibs (tail fibs)
fibonacci_hs :: CInt -> CInt
fibonacci_hs = fromIntegral . fibonacci . fromIntegral
foreign export ccall fibonacci_hs :: CInt -> CInt
and this is what i'm trying to compile it with:
ghc -c -O Safe.hs
followed by:
ghc -shared -o libTest.jnilib -optc-O test_JniTest.c
-I/usr/lib/jvm/java-6-sun-1.6.0.26/include -I/usr/lib/jvm/java-6-sun-1.6.0.26/include/linux
and I am getting this error:
/usr/bin/ld: test_JniTest.o: relocation R_X86_64_PC32 against
undefined symbol `fibonacci_hs' can not be used when making a shared
object; recompile with -fPIC /usr/bin/ld: final link failed: Bad value
collect2: ld returned 1 exit status
I am not a c expert by any means and have no idea what to do about this. I tried compiling various ways with -fPIC, but I kept on getting the same error. Any idea what I might be doing wrong?
Thanks!

Although I've pretty much answered this question here: Communication between Java and Haskell, since this issue is more about the error itself, I will be adding the details for that here. The issue stems from Haskell not supporting shared libraries very well, while Java requires them.
Buildings plugins as Haskell shared libs gives us this insight and workaround:
In principle you can use -shared without -dynamic in the link step. That would mean to statically link the rts all the base libraries into your new shared library. This would make a very big, but standalone shared library. However that would require all the static libraries to have been built with -fPIC so that the code is suitable to include into a shared library and we don't do that at the moment.
If we use ldd again to look at the libfoo.so that we've made we will notice that it is missing a dependency on the rts library. This is problem that we've yet to sort out, so for the moment we can just add the dependency ourselves:
$ ghc --make -dynamic -shared -fPIC Foo.hs -o libfoo.so \
-lHSrts-ghc6.11 -optl-Wl,-rpath,/opt/ghc/lib/ghc-6.11/
This is a workaround because it requires us to know the version of the rts library at build time.

If your goal is to actually get something done (as opposed to just playing around with JNI) I suggest tackling this as a garden variety RPC problem and utilizing one of the many framework/protocols for it:
Protocol Buffers from Google
Thrift from Facebook
Avro (well this is mostly a wire protocol)
From what you are trying to do, Thrift might be your best bet since it describes a full client/server RPC stack but I'm pretty sure any of them would pretty much work over a simple socket.

How do I integrate GEVA into Clojure

I am getting into Clojure and Grammatical Evolution at the same time and discovered GEVA, which is a GE tool written in Java. I have no background in Java. So I don't have to re-invent the wheel, how can I integrate GEVA into Clojure?
I can execute the default script from the CLI with:
java -jar GEVA.jar -main_class Main.Run
The GEVA source directory has the following subdirectories with packages named after the directory:
Algorithm com Exceptions FitnessEvaluation Fractal Individuals Main Mapper Operator Parameter UI Util
The Main subdirectory has the following files and tutorial subdir:
AbstractRun.java Experiment.java Run.java State.java Tutorials
The following classes are in the Run.java file:
public class Run extends AbstractRun
public Run()
public void experiment(String[] args)
public void setup(String[] args)
private void setSeed()
public static void main(String[] args)
From what I can tell, the static method 'main' in class Main.Run is what I want to execute. However when I try to follow the Java interop instructions, I am getting errors.
clgeva.test=> (.Main.Run/main)
java.lang.Exception: No such namespace: .Main.Run (NO_SOURCE_FILE:5)
clgeva.test=> (.Main.Run/main "")
java.lang.Exception: No such namespace: .Main.Run (NO_SOURCE_FILE:6)
I'm obviously not importing the namespace correctly to run the main function.
user=> (import Main)
java.lang.ClassNotFoundException: Main (NO_SOURCE_FILE:1)
user=> (import Main.Run)
Main.Run
user=> (main)
java.lang.Exception: Unable to resolve symbol: main in this context (NO_SOURCE_FILE:3)
user=> (.Main.Run/main)
java.lang.Exception: No such namespace: .Main.Run (NO_SOURCE_FILE:4)
What the heck am I doing wrong? I am sure I'm just missing something obvious, but I've been banging my head on it for a little while and need some direction. Thanks.

First, make sure you are loading the GEVA lib correctly - see http://clojure.org/libs
I'm guessing, from the java.lang.ClassNotFoundException error, that GEVA is not on your classpath.
Then you want to look at the Clojure documentation for Java interop: http://clojure.org/java_interop

The syntax for calling a static method with no arguments (I'll use an example that everyone has access to) is:
=> (java.lang.System/currentTimeMillis)
1284574337322
The above form is preferred, but the following is also possible, and can be useful in macros:
=> (. java.lang.System currentTimeMillis)
1284574477369
In this case, "java.lang." may be omitted, but I'm including it to match your case more closely.
=> (System/currentTimeMillis)
1284574617771
Adding a dot to the first form will not work, whether with or without a space:
user=> (.System/currentTimeMillis)
java.lang.Exception: No such namespace: .System (NO_SOURCE_FILE:44)
user=> (. System/currentTimeMillis)
java.lang.IllegalArgumentException: Malformed member expression, expecting (. target member ...) (NO_SOURCE_FILE:45)

You have the right idea, but you do have some syntax errors.
clgeva.test=> (.Main.Run/main)
Should have a space between the dot operator and the Main.Run/main reference. As it is, it's trying to find something called ".Main.Run" which obviously doesn't exist. Do it like so:
clgeva.test=> (. Main.Run/main)
Assuming you have your classpath set up correctly, it should work.
Importing the class using (import ...) and calling (main) won't work because main is a static method.

We Keep Coding

Java is a programming language and computing platform first released by Sun Microsystems in 1995.