I'm working on a Java program which has a C++ module. I'd like for my C++'s stdout/stderr to be sent to a slf4j/log4j logger.
Some possibilities:
Have my C++ module capture stdout/stderr in a string which is returned and sent to a logger. A downside is the C++ module might be expensive and this would be asynchronous.
Have my Java software create/manage a temp file to which the C++ module can write. Upon completion, my software would read and then delete the file and send the data to slf4j.
is there some way to get an ostream from log4j I could pass through my swig interface?
it looks like I could create a Director in swig to pass a string to Java from C++ to forward to the logger.
How have other people solved this problem?
There are three problems you need to solve to get C++ (called via SWIG generated wrappers) logging to log4j:
How we capture the C++ output (i.e. hook the iostream objects).
How we insert this hooking into the generated wrapper.
How we actually call log4j with the stuff we've captured.
In answering this question I wrote the following header file to demonstrate how my solution works:
#include <iostream>
void test1() {
std::cout << "OUT: " << "test1\n";
std::cerr << "ERR: " << "test1\n";
}
struct HelloWorld {
static void test2() {
std::cout << "OUT: " << "test2\n";
std::cerr << "ERR: " << "test2\n";
}
void test3() const {
std::cout << "OUT: " << "test3\n";
std::cerr << "ERR: " << "test3\n";
}
};
At the end of this I wanted to see both std::cout and std::cerr going to log4j in the correct order. I've answered that question before, in this instance to keep it simple and portable I started out using rdbuf() to swap the internal buffer used by std::cout and std::cerr to one that I'd actually created inside a std::stringstream, something like:
std::stringstream out; // Capture into this
// Save state so we can restore it
auto old_buf = std::cout.rdbuf();
// Swap buffer on cout
std::cout.rdbuf(out.rdbuf());
// Do the real call to C++ here
// ...
// Reset things
std::cout.rdbuf(old_buf);
// Walk through what we captured in out
Of course this won't capture output from libc functions (printf() etc.) or system calls (write() etc.) but it will get all your standard C++ output.
So that's problem #1 crossed off our list. For problem #2 SWIG has the %exception directive that matches what we want to do quite nicely, it gives us a chance to execute C++ code before and after a call into a wrapped function gets dispatched. In the above example all we need to do is use the special variable $action to get the substitution to happen where the comment "do the real call to C++ here".
For problem #3 we need to make some Java calls happen. I started out thinking that JNI wouldn't be too bad, perhaps a little verbose. Basically all we want to do is duplicate the following Java code (from the log4j docs):
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
public class HelloWorld {
private static final Logger logger = LogManager.getLogger("HelloWorld");
public static void main(String[] args) {
logger.info("Hello, World!");
}
}
but inside JNI rather than Java and passing the right string into the getLogger call.
So putting this all together into a SWIG interface we get:
%module test
%{
#include "test.hh"
#include <sstream>
#include <cassert>
static const char *defaultLogname="$module"; // Use if we're not in a class
%}
// Exception handling for all wrapped calls
%exception {
// Hook output into this:
std::stringstream out;
// Save old states
auto old_outbuf = std::cout.rdbuf();
auto old_errbuf = std::cerr.rdbuf();
// Do actual buffer switch
std::cout.rdbuf(out.rdbuf());
std::cerr.rdbuf(out.rdbuf());
try {
$action
}
catch (...) {
// TODO: use RAII instead of poor finally substitute?
std::cout.rdbuf(old_outbuf);
std::cerr.rdbuf(old_errbuf);
throw;
}
// Restore state
std::cout.rdbuf(old_outbuf);
std::cerr.rdbuf(old_errbuf);
// JNI calls to find mid and instance for Logger.error(String) for the right name
static const std::string class_name = "$parentclassname";
// prepare static call to org.apache.logging.log4j.LogManager.getLogger(String)
// start with class lookup:
jclass logmanagercls = JCALL1(FindClass, jenv, "org/apache/logging/log4j/LogManager");
assert(logmanagercls);
// find method ID for right overload of getLogger
jmethodID getloggermid = JCALL3(GetStaticMethodID, jenv, logmanagercls, "getLogger", "(Ljava/lang/String;)Lorg/apache/logging/log4j/Logger;");
assert(getloggermid);
// Prep name strign to pass into getLogger
jstring logname = JCALL1(NewStringUTF, jenv, (class_name.size() ? class_name.c_str(): defaultLogname));
// Actually get the Logger instance for us to use
jobject logger = JCALL3(CallStaticObjectMethod, jenv, logmanagercls, getloggermid, logname);
assert(logger);
// Lookup .error() method ID on logger, we need the jclass to start
jclass loggercls = JCALL1(GetObjectClass, jenv, logger);
assert(loggercls);
// and the method ID of the right overload
jmethodID errormid = JCALL3(GetMethodID, jenv, loggercls, "error", "(Ljava/lang/String;)V");
assert(errormid);
// Loop over all the lines we got from C++:
std::string msg;
while(std::getline(out, msg)) {
// Pass string into Java logger
jstring jmsg = JCALL1(NewStringUTF, jenv, msg.c_str());
JCALL3(CallVoidMethod, jenv, logger, errormid, jmsg);
}
}
// And of course actually wrap our test header
%include "test.hh"
I added some Java to prove this works:
public class run {
public static void main(String[] argv) {
System.loadLibrary("test");
test.test1();
HelloWorld.test2();
HelloWorld h1 = new HelloWorld();
h1.test3();
}
}
Compiled and ran with log4j 2.6 jars in current directory:
swig3.0 -c++ -java -Wall test.i
javac *.java
g++ -std=c++1y -Wall -Wextra -shared -o libtest.so test_wrap.cxx -I/usr/lib/jvm/default-java/include/ -I/usr/lib/jvm/default-java/include/linux -fPIC
LD_LIBRARY_PATH=. java -classpath log4j-api-2.6.jar:log4j-core-2.6.jar:. run
When runs gives:
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
22:31:08.383 [main] ERROR test - OUT: test1
22:31:08.384 [main] ERROR test - ERR: test1
22:31:08.386 [main] ERROR HelloWorld - OUT: test2
22:31:08.386 [main] ERROR HelloWorld - ERR: test2
22:31:08.386 [main] ERROR HelloWorld - OUT: test3
22:31:08.386 [main] ERROR HelloWorld - ERR: test3
Discussion points:
Is the JNI verbose? Definitely yes, but is that bad enough to go down another road? For me not (although the directors idea is viable it's not as simple as it sounds due to complexities with producing interfaces)
The timestamps on your log events will be "wrong" here because all the log information that gets captured will be pushed out after the underlying C++ function call returns, not during its execution
The cout/cerr messages get mixed and logged at the same level. I did this because of the previous point, if they weren't intermixed like this getting the relative ordering would be impossible without more work
If your C++ methods produce large quantities of output the buffer will grow rather large
If you've got a large C++ project I'd hope you had a better logging framework in use than just iostreams. If that's the case you're probably better off implementing its output interface(s) with an adaptor that passes directly to Java with timestamps, log level etc. info retained. (Or vice versa)
Even sticking with iostream the Boost iostreams library makes it easier1 to write something that would actually hook the output writes at the point they happen rather than the current hooking technique (but introduce bigger dependencies so it's a trade off). You could also do it without boost, but the basic_streambuf class is fairly unwieldy.
1 I can probably illustrate the boost version in this answer if you're interested.
Related
I like to control 'sooperlooper' from java, a sound looper. This uses the OSC protocol. First try was the Java OSC lib, but this doesn't do anything. Now I'm trying JNA to wrap liblo.so
The program I'm trying to replicate in JAVA is very simple (of course with different commands from "record"):
static lo_address addr;
int main(int argc, char *argv[])
{
addr = lo_address_new(<IP>, "9951");
lo_send(addr, "/sl/-1/down", "s", "record");
}
The C declaration of the failing method is (https://github.com/radarsat1/liblo/blob/master/lo/lo.h.in):
int lo_send(lo_address targ, const char *path, const char *type, ...);
If I understand correctly, the lo_address is some void pointer type declared elsewhere.
My library interface is:
public interface LibLo extends Library {
Pointer lo_address_new(String host, String port);
int lo_send(Pointer address, String command, String a, String... params);
}
And my caller code is this:
System.setProperty("jna.debug_load", "true");
System.setProperty("jna.debug_load.jna", "true");
LibLo libLo = Native.loadLibrary("liblo", LibLo.class);
Pointer pointer = libLo.lo_address_new(<IP>, "9951");
libLo.lo_send(pointer, "/sl/-1/down","s", "record");
It perfectly gets through the lo_address_new call. 'pointer' does have some value. I think my arguments are correct.
And I discovered even with incorrect arguments, it gets past the lo_address_new call.
My stdout is:
...
Found library 'liblo' at /usr/lib/x86_64-linux-gnu/liblo.so.7.3.0
Exception in thread "main" java.lang.UnsatisfiedLinkError: Error looking up function 'lo_send': /usr/lib/x86_64-linux-gnu/liblo.so.7.3.0: undefined symbol: lo_send
at com.sun.jna.Function.<init>(Function.java:245)
at com.sun.jna.NativeLibrary.getFunction(NativeLibrary.java:566)
at com.sun.jna.NativeLibrary.getFunction(NativeLibrary.java:542)
at com.sun.jna.NativeLibrary.getFunction(NativeLibrary.java:528)
at com.sun.jna.Library$Handler.invoke(Library.java:228)
at com.sun.proxy.$Proxy0.lo_send(Unknown Source)
at nl.ronaldteune.coverdownloader.Main.main(Main.java:30)
Other questions about UnsatisfiedLinkError point to more thorough problems. I can't find 'lo_send' when opening the lib with vim, I can find lo_send_message though - but it's more low level. However, I think my C program is using the same lib (compiled with -llo) and it doesn't have problems to run.
So... I'm stuck. Anyone know how I can debug further?
You can't map lo_send() to JNA (and it doesn't appear in your vim output) because it's a macro:
#define lo_send(targ, path, types...) \
lo_send_internal(targ, __FILE__, __LINE__, path, types, \
LO_MARKER_A, LO_MARKER_B)
You could, in theory, map lo_send_internal() but the source code comments for it say:
/* Don't call lo_send_internal directly, use lo_send, a macro wrapping this
* function with appropriate values for file and line */
Which makes sense as it needs to know the source code __FILE__ and __LINE__ number at compile time, and any hack you'd make in JNA for these values would have to assume you have the correct source code used to compile your binary .so. You'd also need the markers but they are at least constants, amusing words spelled out in hex code.
You could probably just toss in dummy values for the file and line to make your code work, but otherwise, your C wrapper function calling lo_send() looks like the best workaround in this case.
OK, I worked around it... With a wrapper lib:
#include <lo/lo.h>
static lo_address addr;
lo_address slGetAddress() {
return lo_address_new("IP", "9951");
}
void slSendFull(lo_address addr, const char *command) {
lo_send(addr, "/sl/-1/down", "s", command);
}
void slSendSimple(const char *command) {
addr = slGetAddress();
lo_send(addr, "/sl/-1/down", "s", command);
}
and this compile script:
#!/bin/bash
# sudo ln -s /usr/lib/arm-linux-gnueabihf/liblo.so.7 /usr/local/lib/liblo.so
# sudo ln -s /usr/lib/x86_64-linux-gnu/liblo.so.7 /usr/local/lib/liblo.so
gcc -c -Wall -O2 -fPIC lowrapper.c -llo
ld -shared lowrapper.o /usr/local/lib/liblo.so -o lowrapper.so
sudo cp lowrapper.so /usr/local/lib/
It's working now. Still interested in a real answer, but not in a hurry for that anymore :)
As you see here one can perform native C/C++ method calls from Java code, thanks to JVM’s JNI. But how about performing calls of Swift methods? Is this possible or going to be possible (can be implemented in a reasonable time frame) as Swift becomes more popular?
I want to access Apple’s native API that is only accessible when one writes apps in either Objective-C or Swift. As the JVM is just being ported to ARMv8 (64-bit), I could also imagine the JVM as an alternative runtime for iOS apps in the future. But this might be the future... The present is: The JVM runs on Mac OS X and one can write apps for Mac OS X in Swift that can access some APIs that might be not accessible to Java apps in the same way.
Well, roughly 5½ years later, it turned out, it wasn't the future... No JVMs on iOS.
But you can definitely do it, i.e., call Swift APIs from Java. That said, it's quite cumbersome, because AFAIK, you have to take a detour via C/C++.
Here's the deal:
As you know, you can use JNI to call C code.
From C you can call Swift.
The following relies heavily on this question.
Code
Your Java code is in helloworld/SwiftHelloWorld.java:
package helloworld;
public class SwiftHelloWorld {
static {
System.loadLibrary("SwiftHelloWorld");
}
public static native void printHelloWorldImpl();
public static void main(final String[] args) {
printHelloWorldImpl();
}
}
Now write the native C code (file helloworld_SwiftHelloWorld.c):
#include <jni.h>
#include <stdio.h>
#include "helloworld_SwiftHelloWorld.h"
#include "helloworld_SwiftHelloWorld_swift.h"
JNIEXPORT void JNICALL Java_helloworld_SwiftHelloWorld_printHelloWorldImpl
(JNIEnv *env, jclass clazz) {
int result = swiftHelloWorld(42);
printf("%s%i%s", "Hello World from JNI! ", result, "\n");
}
Which uses a header file named helloworld_SwiftHelloWorld_swift.h for our (yet to be written) Swift code:
int swiftHelloWorld(int);
Finally, our Swift code resides in SwiftCode.swift:
import Foundation
// force the function to have a name callable by the c code
#_silgen_name("swiftHelloWorld")
public func swiftHelloWorld(number: Int) -> Int {
print("Hello world from Swift: \(number)")
return 69
}
Building
To build all this, we first have to compile the Swift code to a dynamic library:
swiftc SwiftCode.swift -emit-library -o libSwiftCode.dylib -Xlinker -install_name -Xlinker libSwiftCode.dylib
We use the -Xlinker directives to ensure that the dylib's location is relative.
Before we can create the C dylib, we first have to generate the Java headers:
javac -h . helloworld/SwiftHelloWorld.java
Now that we have the Java headers and the Swift dylib, we can compile the C dylib, which links against the Swift dylib:
gcc -I"$JAVA_HOME/include" -I"$JAVA_HOME/include/darwin/" -o libSwiftHelloWorld.dylib -dynamiclib helloworld_SwiftHelloWorld.c libSwiftCode.dylib
Now that everything is in place, we must make sure that both dylibs are in the same directory and that that directory can be found by Java, i.e., you may need to set -Djava.library.path=<dir of your dylibs>.
Et voilà!
Swift called from Java!
I've got some code to talk to a hardware device on windows which is working in C++. The code does something pretty simple to react to a button push on the device and I have this compiled into a dll with an observer that is called when the button is pushed. I now need to interface this with a big Java program.
I was intending to use JNA but it only works with C and I cannot see how to do this with an Observer pattern in C. I've looked into using BridJ and SWIG (both of which cliam to work on C++ DLLs) to create an interface to the compiled dll (with the associated header file) but BridJ creates huge amounts of files (in JNAeratorStudio) and then stops with an error and I cannot see how to get started on Windows with SWIG (I'm using Visual Studio Express rather than full Visual Studio).
Does anyone know of a tutorial on integrating a C++ DLL with a Java Program - SWIG looks pretty promising but the tutorials are 'swampy'.
I've put some simple C code to talk to the DLL below:
#include <iostream>
#include <stdio.h>
#include "DeepFocusControlDll.h"
using namespace std;
using namespace DeepFocusControl;
class MyObserver : public DeepFocusControl::DeepFocusObserver{
void Event(){
printf("***Button Pushed***");
}
};
int main()
{
DeepFocusControl::AVA6Control* dfc = new DeepFocusControl::AVA6Control();
MyObserver* observer = new MyObserver();
dfc->AddObserver(observer);
bool connected = dfc->IsConnected();
printf("Connected %s\n", (connected)?"true":"false");
bool connectresult = dfc->Connect();
printf("Connecting %s\n", (connectresult)?"true":"false");
connected = dfc->IsConnected();
printf("Connected %s\n", (connected)?"true":"false");
dfc->SetHardwareAppLaunch(false);
char waitChar;
cin >> waitChar;
dfc->SetHardwareAppLaunch(true);
dfc->RemoveObserver(observer);
dfc->Disconnect();
printf("Disconnected\n");
cin >> waitChar;
}
If anyone knows a simpler way to use an observer pattern on this I can happily recode the C side too.
It sounds like you're looking for SWIG's directors feature. In its simplest form you can use directors with SWIG by giving an interface file like:
%module(directors=1) MyModule
%feature("director");
%{
#include "mydll.h"
%}
%include "mydll.h"
Given a header file "mydll.h":
class Observer {
public:
virtual void frobination() = 0;
virtual ~Observer() {}
};
inline void bar(Observer *o) {
o->frobination();
}
Then you can run SWIG:
swig -Wall -java -c++ mymodule.i
This will generate three Java classes: MyModule, MyModuleJNI and Observer. Of these MyModule will contain all the free functions from your header file, exposed as static member functions since Java has no such thing as free functions. You can safely ignore MyModuleJNI - it's glue generated by SWIG for connecting MyModule to the real C++ implementations. You'll need to compile mymodule_wrap.cxx for MyModuleJNI (and hence MyModule) to work correctly though and load the DLL using System.loadLibrary before you call any functions from them.
The Observer class directly corresponds to the Observer interface in mydll.h. You should derive from it in Java and override the frobinate function to give it your own implementation:
public class Test extends Observer {
#Override
public void frobination() {
System.out.println("go go gadget frobinator");
}
public static void main(String[] argv) {
System.loadLibrary("mymodule");
Test t = new Test();
MyModule.bar(t);
}
}
Which I can compile and run to do exactly what you'd hope.
If you want you can automate the call to System.loadLibrary by adding:
%pragma(java) jniclasscode=%{
static {
try {
System.loadLibrary("mymodule");
} catch (UnsatisfiedLinkError e) {
System.err.println("Native code library failed to load. \n" + e);
System.exit(1);
}
}
%}
to your SWIG interface file.
If your real header file is that simple it should be that simple to get the same results too. If it's more complicated you can instruct SWIG to special case some of its wrapping in various ways.
So, I'm trying to use swig to wrap a c++ library to make JNI calls to it. But, the _wrap.cxx file that swig is creating for me won't compile cleanly in g++. What am I doing wrong?
Here's a simple test case that will reproduce the error, along with a couple of other cases that don't error. My header file:
class MyClass {
};
class MyDirectored {
public:
virtual void Overridable (MyClass const clz);
virtual ~MyDirectored();
};
and here's my .i file:
%module("directors="1") swigtest
%{
#include "swig.h"
%}
%feature("director") MyDirectored
%include "swig.h"
I'm attempting to build via the following:
swig -c++ -package gen -java -outdir gen swig.i
g++ -c swig_wrap.cxx -o swig_wrap.o
And the g++ step yields the following error:
swig_wrap.cxx: In member function 'virtual void SwigDirector_MyDirectored::Overridable(MyClass)':
swig_wrap.cxx:436: error: invalid conversion from 'const MyClass*' to 'MyClass*'
Which appears to be a legitimate complaint - the resulting swig_wrap.cxx file looks like (lots of snippage)
void SwigDirector_MyDirectored::Overridable (MyClass const clz) {
//...
jlong jclz;
//...
*((MyClass **)&jclz) = &clz; //Error on this line
//...
}
I get the same error in both swig 2.0.4 and 1.3.40, Linux and Windows. Any suggestions? Any g++ trick I could use to ignore the constness error?
Random notes: I can't control the input header, so changing the function signature is a no-go. It doesn't appear to matter what type the input parameter is - class or struct. Making it a const reference instead of a const value parameter does "fix" the error by causing SWIG to explicitly cast away the constness (but again, I can't change the input header).
Thanks in advance!
I'm in the process of doing some Java<->.NET interop code using a native library between them, so far things are going fairly well.
However for some reason I cannot run any methods under JNIEnv.
System::String^ JNIStringToNet(JNIEnv * env, jstring js)
{
const char *buf = env->GetStringUTFChars(js, 0); // segfault
Now I can pass variables back and forth and do all other kinds of communication, so I'm figuring I haven't initialized this correctly or something.
I'm loading it like this:
this.Lib = (LibHandler)Native.loadLibrary(this.Name, LibHandler.class);
(I prefer Native.loadLibrary because it seems to allow me to do more with it easier, such as class sharing between multiple libraries and unhooking and rehooking it from the JVM on the fly).
Edit:
Seriously any method:
std::cout << "Getting version:" << std::endl;
std::cout << env->GetVersion() << std::endl;
Getting version:
(segfault)
Any ideas on wht JNIEnv would segfault for every method? This should be set up by the JVM, correct?
Edit 2:
This is a Java application that is calling a C++ library that will interface with a .NET library (so it's a CLR compiled C++ library, if that makes any difference), to limit any external factors I'm not even calling the .NET DLL but just converting strings that come in back out (or well... trying).
So for example from Java:
this.Lib = (LibHandler)Native.loadLibrary(this.Name, LibHandler.class);
this.Lib.myCPPMethod(); // Segmentation fault during this call, JVM crashes.
Curious if it was the CLR that was causing it: Disabled clr compilation and stripped everything out that was CLR related, still does it.
Edit 3:
Got it to dump:
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000000000000, pid=1596, tid=7248
#
# JRE version: 6.0_23-b05
# Java VM: Java HotSpot(TM) 64-Bit Server VM (19.0-b09 mixed mode windows-amd64 compressed oops)
# Problematic frame:
# C 0x0000000000000000
So yeah, looks like the JVM isn't giving me memory access for some reason.
Edit 4:
Actual call:
JNIEXPORT jstring JNICALL Query(JNIEnv * env, jobject jobj, jstring start, jstring end)
{
std::cout << "Getting version:" << std::endl;
jint j = env->GetVersion();
return jstring("test");
}
Edit 5:
Works with System.loadLibrary:
JNIEXPORT void JNICALL Java_LibTest_T(JNIEnv *env, jobject jobj)
{
std::cout << "Getting version:" << std::endl;
jint j = env->GetVersion();
std::cout << j << std::endl;
}
Output:
java -Djava.library.path="(dir)\lib\64" EntryPoint
Getting version:
65542
Ack! I mean some headway, but I can't unload libraries from the JVM that are loaded in System.loadLibrary can I?
I basically need to be able to unhook these libraries from the JVM and swap them out, on top of that they all need to "share" a single class and be able to be bound to the class at runtime... which is kinda why I went with Native.loadLibrary.
Currently I do this:
Load the DLL:
this.Lib = (LibHandler)Native.loadLibrary(this.Name, LibHandler.class);
Unhook it:
this.Lib = null;
Runtime.getRuntime().gc(); // Force the JVM to drop it immediately.
Class I load them all into:
public interface LibHandler extends Library{
void T();
}
Any way to work similarly with System.loadLibrary?
Edit 6:
Feel free to call me dumb, I'm using JNA, NOT JNI, which is completely different and a huge source of my problems.... is there a way to do this with JNI? Or can I get JNIEnv to register with JNA somehow? I'm guessing I can drop JNI from the C++ library and straight up use wstrings?
I'll get back with this tomorrow.
Well I feel bad.
Native.loadLibrary == JNA.
System.loadLibrary == JNI.
The purpose of JNA is to not require any real knowledge of the JVM environment, so you can run native libraries as is, so instead of jstring you can use char*...