We Keep Coding

How to map in JNA an array of structure inside a structure without setting the array size on the C++ declaration

I'm trying to map a structure which contains an array of structure in JNA.
The size of the embedded structure is not defined in advanced in the C++ declaration of the structure but in the java code.
My issue is that I get Exception in thread "main" java.lang.Error: Invalid memory access
The C++ header file is as follows :
typedef struct s_Param
{
char* key;
uint32_t key_value;
} Param;
typedef struct s_ParamList {
Param* init_param;
int param_list_size; // number of param items in the param List
} ParamList;
The C++ code is as follows :
int Init(
ParamList* i_initParamList,
logfunction i__callback,
const allocator* i__allocator,
void** o__content) {
...
if (i_initParamList != NULL){
printf("DLL PRINT init ---- i_initParamList = %p\n", i_initParamList);
printf("DLL PRINT init ---- i_initParamList->param_list_size = %i\n", i_initParamList->param_list_size);
if (i_initParamList->init_param == NULL){
printf("DLL PRINT init ---- i_initParamList->init_param must not be NULL\n");
returnedCode = 1;
}else{
for (int i = 0; i<i_initParamList->param_list_size;i++){
printf("DLL PRINT init ---- i_initParamList->init_param[i]->key = %s\n",i_initParamList->init_param[i].key);
printf("DLL PRINT init ---- i_initParamList->init_param[i]->key_value = %i\n",i_initParamList->init_param[i].key_value);
}
...
}
The java JNA code is as follows :
public interface MyLibrary extends Library {
#FieldOrder({ "key", "key_value" })
public static class Param extends Structure {
public static class ByReference extends Param implements Structure.ByReference {
}
public String key;
public int key_value;
public Param(){ // NOT sure that this adding constructor makes sense
super();
setAlignType(ALIGN_NONE);
}
}
#FieldOrder({ "init_param", "param_list_size" })
public static class ParamList extends Structure {
public static class ByReference extends ParamList implements Structure.ByReference {
}
public Param[] init_param;
public int param_list_size;
ParamList(){ // NOT sure that this adding constructor makes sense
super();
setAlignType(ALIGN_NONE);
}
}
public int Init(ParamList i_initParamList, logfunction i__callback, allocator i__allocator,
PointerByReference o__content);
...
}
The Sample.java code which calls the JNA library is as follows :
Note that I did not add the way the other parameters of Init function are managed as I do not have any issues with them.
int paramListSize = 4;
MyLibrary.Param[] params = new MyLibrary.Param[paramListSize];
for (int i = 0; i < paramListSize; i++) {
params[i] = new MyLibrary.Param();
}
params[0].key = "first";
params[0].key_value = 1;
params[1].key = "second";
params[1].key_value = 5;
params[2].key = "third";
params[2].key_value = 7;
params[3].key = "forth";
params[3].key_value = 9;
MyLibrary.ParamList paramList = new MyLibrary.ParamList.ByReference();
paramList.init_param = params;
paramList.param_list_size = paramListSize;
logger.debug("params = "+ params);
logger.debug("paramList = "+ paramList);
logger.debug("paramList.param_list_size = "+paramList.param_list_size);
int errInit = IFunctions.Init(paramList, logCallback, i__allocator, o__content);
The traces results coming from C++ and Java code are as follows :
10:41:28,303 DEBUG Sample:193 - params = [MyLibrary$Param;#1e67a849
10:41:28,312 DEBUG Sample:194 - paramList = MyLibrary$ParamList$ByReference(auto-allocated#0x1f3d49fe5b0 (52 bytes)) {
MyLibrary$Param init_param[4]#0x0=[MyLibrary$Param;#1e67a849
int param_list_size#0x30=0x0004
}
10:41:28,316 DEBUG Sample:195 - paramList.param_list_size = 4
Exception in thread "main" java.lang.Error: Invalid memory access
at com.sun.jna.Native.invokeInt(Native Method)
at com.sun.jna.Function.invoke(Function.java:426)
at com.sun.jna.Function.invoke(Function.java:361)
at com.sun.jna.MyLibrary$Handler.invoke(MyLibrary.java:265)
at com.sun.proxy.$Proxy3.init(Unknown Source)
at Sample.main(Sample.java:216)
DLL PRINT init ---- i_initParamList = 000001F3D49FE5B0
DLL PRINT _init ---- init_param address = 0000021B4ADAF3D0
DLL PRINT init ---- i_initParamList->param_list_size = 1
DLL PRINT init ---- i_initParamList->init_param[i]->key =
It works if I declare my Structure in C++ header as follows :
typedef struct s_ParamList {
Param init_param[4];
int param_list_size; // number of param items in the param List
} ParamList;
But I do not want to define the init_param array size in the C++ code as it must be defined on java code side.
Regarding added following code in the javacode :
ParamList(){ // NOT sure that this adding constructor makes sense
super();
setAlignType(ALIGN_NONE);
}
I'm not sure that I have to add this in both structures i.e in ParamList and in Param structure.
But anyway if I delete both of those constructor I have exactly the same issue.
I saw another way to manage my requirement (looking at toArray(size) method from chapter Structure of https://javadoc.io/doc/net.java.dev.jna/jna/latest/index.html for 5.8.0 JNA version.
Indeed I also followed the link How to fill an array of structures in JNA? but I get Invalid Memory access. I'll create a separate question regarding this second kind of implementation if I cannot have my Structure's array in a Structure.
And I had to split the Param array and the size of the array in 2 separate parameters of Init function instead of setting them into a unique Structure.
And as I already have many parameters in my Init function I would prefer to have only one structure parameter. And I guess that It should be possible in JNA.
Does any one have a clue?
Thanx for your help.

The problem is in your mapping of this structure
typedef struct s_ParamList {
Param* init_param;
int param_list_size; // number of param items in the param List
} ParamList;
The * indicates this is a pointer to memory elsewhere that you need to define.
Structures are treated as ByValue by default inside structures, and as ByRefrence by default in function arguments. So here you need to explicitly define the ByReference version of the structure (or use a plain Pointer, which is less type safe.)
So you'll get this as the main part of your structure.
#FieldOrder({ "init_param", "param_list_size" })
public static class ParamList extends Structure {
public Param.ByReference init_param;
public int param_list_size;
}
Next, you've indicated you want to define this array and allocate its memory yourself in Java. The important thing to remember here is that C treats arrays as contiguous memory, so you really only have two choices: allocate a large block yourself with Memory and set values at the offsets manually; or use Structure.toArray() designed for exactly this case: You start with an instantiated structure and then tell the toArray() method how many copies of it you need.
So your sample code would look like this:
int paramListSize = 4;
// Note the syntax for allocating a contiguous array
MyLibrary.Param.ByReference[] params =
(MyLibrary.Param.ByReference[]) new MyLibrary.Param.ByReference().toArray(paramListSize);
// set the values as you've alread done
params[0].key = "first";
params[0].key_value = 1;
// and so on...
// Now instantiate your structure and set its members
MyLibrary.ParamList paramList = new MyLibrary.ParamList();
// The first array member is the pointer to the start of the array
paramList.init_param = params[0];
paramList.param_list_size = paramListSize;
And here, you pass it to the native function. It is ByReference by default.
You can find another general example here

Can you consider following data structure?
typedef struct s_ParamList {
int param_list_size; // number of param items in the param List
Param init_param[0];
} ParamList;
If yes, please be accurate with memory allocation and array usage, since such approach is pretty dangerous and may easily cause data overrun

IntByReference returns 0 instead of the actual value

I have the following C code which I am calling from JNA. When I call it I am getting 0 as the return value while I'm getting the real value when I test it in C.
RFIDLIB_API uint32_t get(
ipj_key key /*[i]*/,
uint32_t bank_index /*[in]*/,
uint32_t value_index /*[out]*/,
uint32_t *value /*[out]*/)
{
return ipj_get(&iri_device,key,bank_index,value_index,value);
}
Below is how I have defined this in my JNA library.
public int get(ipj_key key, int bank_index, int value_index, IntByReference value);
Below is the ip_key structure
public class ipj_key extends Structure {
public int ipj_key;
#Override
protected List getFieldOrder() {
return Arrays.asList("ipj_key");
}
}
And below is how I am calling it in my main method.
public rfidlib rlib;
public static void main(String[] args) {
MainScannerTest mainClass = new MainScannerTest();
mainClass.rlib = (rfidlib) Native.loadLibrary("rfidlib", rfidlib.class);
ipj_key key = new ipj_key();
key.ipj_key = 0xD;
IntByReference value = new IntByReference();
mainClass.rlib.get(key, 0, 0, value);
System.out.println("Antenna power is : "+ value.getValue());
}
What am I doing wrong here? Why am I getting 0 as the return value? Please advice.

My C lib has a int* as an OUT parameter. I was also trying to use it with an IntByReference, without any success.
EDIT 2017-03-16: indeed it worked with IntByReference. There might me something wrong in your C code.
An alternative would be using a Pointer instead of IntByreference
(I made it work with IntByRef and also with Pointer in my case (64bit Win7, JNA 4.3.0 and Java 64 bits jdk 1.8.0_71))
With your code snippet, this would look like the following.
First change your JNA dll interface defintion (use Pointer instead of IntByReference):
public int get(ipj_key key, int bank_index, int value_index, /*change is here*/ Pointer value);
Then change of course the variable before call:
Pointer value = new Memory(Integer.size); // Here we assume that your java platform is 32 bits (Integer.size=32)
//this should match your uint32_t. No idea if there are consequences when mixing 32/64 bits java/OS/dll ...
Then, you can do the following:
System.out.println("Antenna power is : "+ value.getInt(0));
// The offset is set to 0, since there is no reason to read the int from another offset value.

JNI : java - C // Using Global variables

I would like to know whether JNI accepts the use of global variables declared in the .cpp (and of type created by me).
It would be for a speech recognition program. In fact, I would like to use C to declare all the settings of the decoder and Java to manage the recording thing on Android. I don't want to create a Decoder class on Java because it should be use both on Android and iOS (I would like to have 3 separate parts :
1) set up the decoder (only in C)
2) do the recording and manage the threads (only in Android-java)
3) decode the audio (in java but calling C functions that will use the settings of the decoder)
Example :
function.cpp :
struct decoder {
char *model;
};
In the function_2.ccp file :
decoder *ps;
void set_decoder(char *modelToUse){
ps->model = modelToUse;
return;
}
void process(int *buf ) {
process_raw(ps, buf); // modify buf accordingly to the parameters of ps
return;
}
char *getHyp (){
return ps->hyp;
}
void main(){
ps = set_decoder(var1);
jclass = android_class = (*env)->FindClass(env,"android/app/nameofactivity");
// find method_ID
jmethodID android_mid = (*env)->GetStaticMethodID(env, android_class, "java_method",String_signature);
// call method
jString hyp_java = (*env)->CallIntMethod(env, android_class, android_mid, int samplerate);
EDIT (add these lines) :
const char *hyp_tmp = (*env)->GetStringUTFChars(env, hyp_java, 0);
const char *hyp = hyp_tmp;
printf(hyp_java);
(*env)->ReleaseStringUTFChars(env, hyp_java, hyp_tmp);
//END of EDIT
}
In the java file (the methods process and getHyp are declared as native methods ; the read method is a method from Android):
file.java
String java_method(int samplerate){
int buf[samplerate];
recorder.read(buf);
this.process(buf);
return this.getHyp();
}
Is it possible to do that?
EDIT : In brief, I wonder whether I can declare a global variable "ps" in C and then call a java function that will call other C functions that will use the value of the declared variable "ps".

Converting String to Pointer for JNA

I'm trying to use JNA to query the effective permissions for a file in Windows. Eventually, I plan on using the GetEffectiveRightsFromAcl function, but to do so, I need to provide a pointer to a populated TRUSTEE structure. The JNA Platform (platform.jar) doesn't appear define this struct, so I'm trying to define it myself instead. Here's what I have so far:
public static class TRUSTEE extends Structure {
public TRUSTEE() {
super();
}
public TRUSTEE(Pointer p) {
super(p);
read();
}
public Pointer pMultipleTrustee;
public int MultipleTrusteeOperation;
public int TrusteeForm;
public int TrusteeType;
public Pointer ptstrName;
}
I'm trying to populate the structure like this:
private TRUSTEE createTrusteeForCurrentUser() {
TRUSTEE result = new TRUSTEE();
result.TrusteeForm = TRUSTEE_FORM.TRUSTEE_IS_NAME;
result.TrusteeType = TRUSTEE_TYPE.TRUSTEE_IS_USER;
String strName = "CURRENT_USER";
// How can I set result.ptstrName using strName?
}
This Google Groups thread recommends using String fields in structures when a char * is called for. However, I don't think this is appropriate in my situation, considering the ptstrName field is allowed to point to different types of things, depending on the value of TrusteeForm. So, I think I somehow need to convert from String to Pointer instead. I found the NativeString class in JNA, which would work, except it's a package-private class.
What's the recommended way to convert a Java String to a native format and obtain a Pointer to it? Am I even using the right data type for the TRUSTEE struct? I'm somewhat new to JNA, so please excuse me if I'm missing something obvious.
Update
I found a solution to my problem, but if anyone has a better solution I'd still like to hear it.

Assuming you want char * on the native side (you may need more memory allocated if the string contains non-ascii characters),
String myString = "CURRENT_USER";
Pointer m = new Memory(myString.length() + 1); // WARNING: assumes ascii-only string
m.setString(0, myString);
You can then use m wherever you need to reference the "native" string.
For wide strings (wchar_t *),
String myString = "CURRENT_USER";
Pointer m = new Memory(Native.WCHAR_SIZE * (myString.length() + 1));
m.setWideString(0, myString);

I solved the problem by copying the source code for package-private NativeString class and creating a public copy in my project. I had to make one minor alteration due to the use of a package-private method in the constructor.
Update: As #fragorl notes in the comments, the implementation of NativeString shown below is by now quite out-of-date.
Usage:
private static TRUSTEE createTrusteeForCurrentUser() {
TRUSTEE result = new TRUSTEE();
result.TrusteeForm = TRUSTEE_FORM.TRUSTEE_IS_NAME;
result.TrusteeType = TRUSTEE_TYPE.TRUSTEE_IS_USER;
result.ptstrName = new NativeString("CURRENT_USER",true).getPointer();
result.write();
return result;
}
NativeString.java:
/** Provides a temporary allocation of an immutable C string
* (<code>const char*</code> or <code>const wchar_t*</code>) for use when
* converting a Java String into a native memory function argument.
*
* #author Todd Fast, todd.fast#sun.com
* #author twall#users.sf.net
*/
public class NativeString implements CharSequence, Comparable {
private Pointer pointer;
private boolean wide;
/** Create a native string (NUL-terminated array of <code>char</code>).<p>
* If the system property <code>jna.encoding</code> is set, its value will
* be used to encode the native string. If not set or if the encoding
* is unavailable, the default platform encoding will be used.
*/
public NativeString(String string) {
this(string, false);
}
/** Create a native string as a NUL-terminated array of <code>wchar_t</code>
* (if <code>wide</code> is true) or <code>char</code>.<p>
* If the system property <code>jna.encoding</code> is set, its value will
* be used to encode the native <code>char</code>string.
* If not set or if the encoding is unavailable, the default platform
* encoding will be used.
*
* #param string value to write to native memory
* #param wide whether to store the String as <code>wchar_t</code>
*/
public NativeString(String string, boolean wide) {
if (string == null) {
throw new NullPointerException("String must not be null");
}
// Allocate the memory to hold the string. Note, we have to
// make this 1 element longer in order to accommodate the terminating
// NUL (which is generated in Pointer.setString()).
this.wide = wide;
if (wide) {
int len = (string.length() + 1 ) * Native.WCHAR_SIZE;
pointer = new Memory(len);
pointer.setString(0, string, true);
}
else {
byte[] data = Native.toByteArray(string);
pointer = new Memory(data.length + 1);
pointer.write(0, data, 0, data.length);
pointer.setByte(data.length, (byte)0);
}
}
public int hashCode() {
return toString().hashCode();
}
public boolean equals(Object other) {
if (other instanceof CharSequence) {
return compareTo(other) == 0;
}
return false;
}
public String toString() {
String s = wide ? "const wchar_t*" : "const char*";
s += "(" + pointer.getString(0, wide) + ")";
return s;
}
public Pointer getPointer() {
return pointer;
}
public char charAt(int index) {
return toString().charAt(index);
}
public int length() {
return toString().length();
}
public CharSequence subSequence(int start, int end) {
return CharBuffer.wrap(toString()).subSequence(start, end);
}
public int compareTo(Object other) {
if (other == null)
return 1;
return toString().compareTo(other.toString());
}
}

try using Pointer class in http://jna.java.net/javadoc/com/sun/jna/Pointer.html.

Javolution ByteBuffer question

I have the following implementation with Javolution:
public class RunScan extends Struct
{
public final Signed32 numOfClusters = new Signed32();
public final ClusterData[] clusters;
public final Signed32 numOfRecons = new Signed32();
public final ReconData[] recons ;
public RunScan (int numOfClusters, int numOfRecons)
{
this.numOfClusters.set(numOfClusters);
this.numOfRecons.set(numOfRecons);
clusters = array(new ClusterData[numOfClusters]);
recons = array(new ReconData[numOfRecons]);
}
}
public class ClusterData extends Struct
{
public final UTF8String scanType = new UTF8String(CommInterfaceFieldConstants.SCAN_TYPE_SIZE);
public final UTF8String patientId = new UTF8String(CommInterfaceFieldConstants.PATIENT_ID_SIZE);
.
.
.
}
public class ReconData extends Struct
{
public final UTF8String patientId = new UTF8String(CommInterfaceFieldConstants.PATIENT_ID_SIZE);
public final UTF8String scanSeriesId = new UTF8String(CommInterfaceFieldConstants.SCAN_SERIES_ID_SIZE);
.
.
.
}
In our communication class, before we put data onto socket, we need to get the bytes[] of the RunScan object but we get BufferUnderflowException in the line with "//<<<<<<<":
private byte[] getCmdBytes(Struct scCmd)
{
ByteBuffer cmdBuffer = scCmd.getByteBuffer();
int cmdSize = scCmd.size();
byte[] cmdBytes = new byte[cmdSize];
if (cmdBuffer.hasArray())
{
int offset = cmdBuffer.arrayOffset() + scCmd.getByteBufferPosition();
System.arraycopy(cmdBuffer.array(), offset, cmdBytes, 0, cmdSize);
}
else
{
String msg = "\n\ncmdBufferRemaining=" + cmdBuffer.remaining() + ", cmdBytesSize=" + cmdBytes.length + "\n\n";
System.out.println(msg);
cmdBuffer.position(scCmd.getByteBufferPosition());
cmdBuffer.get(cmdBytes); //<<<<<<<<<< underFlowException
}
return cmdBytes;
}
This method works in other cases. The exception happens because this line,
ByteBuffer cmdBuffer = scCmd.getByteBuffer();
only returns a 8 bytes (from the remaining() method) ByteBuffer of the RunScan object which are those two Signed32 fields, I think. But this line,
int cmdSize = scCmd.size();
returns a right length of the RunScan object which includes the size of those two arrays.
If I create those two array at the time I declare them (not "new" them in the constructor) with hard coded length, it works fine without any exception.
Anybody can help me figure out what's wrong with our implementation?

I ran into a similar situation with my code. Generally, with the current Struct object, you cannot have a variable length array defined in the same struct as the member that contains the number of elements in the array.
Try something like this:
public class RunScanHeader extends Struct
{
public final Signed32 numOfClusters = new Signed32();
public final Signed32 numOfRecons = new Signed32();
}
public class RunScanBody extends Struct
{
public final ClusterData[] clusters;
public final ReconData[] recons ;
public RunScan (int numOfClusters, int numOfRecons)
{
clusters = array(new ClusterData[numOfClusters]);
recons = array(new ReconData[numOfRecons]);
}
}
You'll then need a two phase approach to read and write, first read/write the header data, then read/write the body data.
Sorry I don't have more details at this time, if you can't solve this, let me know and I'll dig back through my code.

The initialization order is important has it defines the position of each field. Either your initialization is done when the field is declared (most common case). Or if you do it in the constructor you have to remember that the constructor is called after the member initialization. Here is an example with initialization done in the constructor:
public class RunScan extends Struct {
public final Signed32 numOfClusters;
public final ClusterData[] clusters;
public final Signed32 numOfRecons;
public final ReconData[] recons ;
public RunScan (int numOfClusters, int numOfRecons) {
// Initialization done in the constructor for all members
// Order is important, it should match the declarative order to ensure proper positioning.
this.numOfClusters = new Signed32();
this.clusters = array(new ClusterData[numOfClusters]);
this.numOfRecons = new Signed32();
this.recons = array(new ReconData[numOfRecons]);
// Only after all the members have been initialized the set method can be used.
this.numOfClusters.set(numOfClusters);
this.numOfRecons.set(numOfRecons);
}
}

get() will move the position of the ByteBuffer.
scCmd.getByteBuffer().slice().get(dest) might solve your issue with moving the position and unintended side effects.
scCmd.getByteBuffer().duplicate().get(dest) might also solve your issue if slice() produces the wrong picture of the origin buffer.
Additionally, it appears as though scCmd.getByteBuffer() creates a redundant reference and you are calling the source and child reference in the same method.
If scCmd.getByteBuffer() is already passing you a slice(), your redundant access to these methods is certainly going to do something other than what you planned.