My native code is
typedef struct driver_config {
unsigned int dllVersion;
unsigned int channelCount;
unsigned int reserved[10];
ChannelConfig channel[64];
} DriverConfig;
In Java my class looks like this
public class DriverConfig extends Structure {
public int dllVersion;
public int channelCount;
public int[] reserved= new int[10];
ChannelConfig[] channel = new ChannelConfig[64];
public DriverConfig() {
super();
init();
}
private void init() {
for (int i = 0; i < channel.length; i++) {
channel[i]= new ChannelConfig();
}
}
#Override
protected List<String> getFieldOrder() {
return Arrays.asList(new String[] { "dllVersion", "channelCount", "reserved" });
}
//toString()...
}
The method declaration is
int getDriverConfig(DriverConfig driverConfig);
I tried to access the method like this
DriverConfig driverConfig = new DriverConfig();
status = dll.INSTANCE.getDriverConfig(driverConfig);
System.out.println("DriverConfig Status: " + status);
System.out.println(driverConfig.toString());
If channel.length is replaced with less then 50 the array is initialized correctly but with channel.length it did not work. It even did not show any error just nothing.
Your getFieldOrder() array does not include the last element (channel) of your structure. I see in your comments that you attempted to do this but received an error because you have not declared it public. All elements of your structure must be listed in the FieldOrder and also declared public so they can be found with reflection.
Also, with JNA 5.x (which you should be using) the #FieldOrder annotation is preferred.
You haven't identified the mapping for ChannelConfig, but your question title and this API link matching your structure indicate that it is a nested structure array. Structure arrays must be allocated using contiguous memory, either by directly allocating the native memory (new Memory()) which requires knowing the structure size, or by using Structure.toArray(). Allocating in a loop as you have done will end up with memory for each new structure allocated at possibly/probably non-contiguous locations in native memory. Given that you state that it appears to work for some values, you might be getting lucky with contiguous allocations, but your behavior is certainly undefined.
Your structure mapping should therefore be:
#FieldOrder ({"dllVersion", "channelCount", "reserved", "channel"})
public class DriverConfig extends Structure {
public int dllVersion;
public int channelCount;
public int[] reserved= new int[10];
public ChannelConfig[] channel = (ChannelConfig[]) new ChannelConfig().toArray(64);
}
Related
I am using JNA and writing some code to enumerate through all of the modules available in a process. I successfully obtain the snapshot handle via CreateToolhelp32Snapshot, however on my first Module32First call, I am getting error code 87 for "Invalid Parameter".
Below is the relevant code that I am using:
Code in question:
private void getModule() {
Pointer hSnapshot = this.kernel32.CreateToolhelp32Snapshot(WinAPI.TH32CS_SNAPMODULE, this.processId);
if(hSnapshot != null) {
WinAPI.MODULEENTRY32 moduleEntry32 = new WinAPI.MODULEENTRY32();
moduleEntry32.write(); // Write the struct in memory (for dwSize to be registered)
Pointer moduleEntry32Pointer = moduleEntry32.getPointer();
if(this.kernel32.Module32First(hSnapshot, moduleEntry32Pointer)) {
// irrelevant stuff here
}
}
System.out.println(this.kernel32.GetLastError()); // Prints out "87"
this.kernel32.CloseHandle(hSnapshot);
}
Kernel32 Mapper Class:
Pointer CreateToolhelp32Snapshot(int dwFlags, int th32ProcessID);
boolean Module32First(Pointer hSnapshot, Pointer lpme);
boolean Module32Next(Pointer hSnapshot, Pointer lpme);
ModuleEntry32 Structure:
public static class MODULEENTRY32 extends Structure {
public int dwSize = this.size();
public int th32ModuleID;
public int th32ProcessID;
public int GlblcntUsage;
public int ProccntUsage;
public Pointer modBaseAddr;
public int modBaseSize;
public Pointer hModule;
public String szModule;
public String szExePath;
public MODULEENTRY32() {
super();
}
public MODULEENTRY32(Pointer p) {
super(p);
}
protected List<String> getFieldOrder() {
return Arrays.asList(new String[] {"dwSize", "th32ModuleID", "th32ProcessID", "GlblcntUsage", "ProccntUsage", "modBaseAddr", "modBaseSize", "hModule", "szModule", "szExePath"});
}
}
And finally, this is "WinAPI.TH32CS_SNAPMODULE":
public static final int TH32CS_SNAPMODULE = 0x00000008;
Note: The process I am enumerating on is already opened with OpenProcess and is valid. The processId is valid as well, and is obtained properly as well.
Any help would be appreciated.
Your mapping of a pointer as the second argument, while technically correct, requires you to do a lot more overhead with writing it. It is better to simply put the structure type as an argument. Structures are treated as .ByReference when used as function/method arguments, and handle all of the auto-read and write for you. So if you do this, you can omit your write() call:
boolean Module32First(Pointer hSnapshot, WinAPI.MODULEENTRY32 lpme);
boolean Module32Next(Pointer hSnapshot, WinAPI.MODULEENTRY32 lpme);
That said, the comment on your write call points to the root cause here: you need to set the dwSize value to the size of the structure. However, because of the order of initialization of Java Objects, the Structure does not have the information it needs to determine the size when initializing the dwSize variable, so this part is not giving you the correct size:
public int dwSize = this.size();
To solve this, just declare that variable and set the size later on in the constructor, e.g.,
public static class MODULEENTRY32 extends Structure {
public int dwSize;
// other declarations
public MODULEENTRY32() {
super();
dwSize = size();
}
}
In addition your mappings for szModule and szExePath are incorrect. You've only provided Pointers (a total of 16 bytes) in the structure's memory allocation, but these are fixed length character arrays. They should be defined:
public char[] szModule = new char[MAX_MODULE_NAME32 + 1];
public char[] szExePath = new char[MAX_PATH];
Note the char[] mapping is for the Unicode (_W) version of the mapping which is a reasonably safe assumption.
But rather than write yourself, you should use the mappings that already exists in the JNA user-contributed mappings in jna-platform. The MODULEENTRY32W type is already there (yes it uses DWORD so your version is simpler; but you can look at the code to see how they handled the size.) And the Module32FirstW and Module32NextW functions are also mapped. (The W suffix is for wide characters (Unicode) which all modern Windows systems use now.)
I'm trying to map a structure which contains an array of structure in JNA.
The size of the embedded structure is not defined in advanced in the C++ declaration of the structure but in the java code.
My issue is that I get Exception in thread "main" java.lang.Error: Invalid memory access
The C++ header file is as follows :
typedef struct s_Param
{
char* key;
uint32_t key_value;
} Param;
typedef struct s_ParamList {
Param* init_param;
int param_list_size; // number of param items in the param List
} ParamList;
The C++ code is as follows :
int Init(
ParamList* i_initParamList,
logfunction i__callback,
const allocator* i__allocator,
void** o__content) {
...
if (i_initParamList != NULL){
printf("DLL PRINT init ---- i_initParamList = %p\n", i_initParamList);
printf("DLL PRINT init ---- i_initParamList->param_list_size = %i\n", i_initParamList->param_list_size);
if (i_initParamList->init_param == NULL){
printf("DLL PRINT init ---- i_initParamList->init_param must not be NULL\n");
returnedCode = 1;
}else{
for (int i = 0; i<i_initParamList->param_list_size;i++){
printf("DLL PRINT init ---- i_initParamList->init_param[i]->key = %s\n",i_initParamList->init_param[i].key);
printf("DLL PRINT init ---- i_initParamList->init_param[i]->key_value = %i\n",i_initParamList->init_param[i].key_value);
}
...
}
The java JNA code is as follows :
public interface MyLibrary extends Library {
#FieldOrder({ "key", "key_value" })
public static class Param extends Structure {
public static class ByReference extends Param implements Structure.ByReference {
}
public String key;
public int key_value;
public Param(){ // NOT sure that this adding constructor makes sense
super();
setAlignType(ALIGN_NONE);
}
}
#FieldOrder({ "init_param", "param_list_size" })
public static class ParamList extends Structure {
public static class ByReference extends ParamList implements Structure.ByReference {
}
public Param[] init_param;
public int param_list_size;
ParamList(){ // NOT sure that this adding constructor makes sense
super();
setAlignType(ALIGN_NONE);
}
}
public int Init(ParamList i_initParamList, logfunction i__callback, allocator i__allocator,
PointerByReference o__content);
...
}
The Sample.java code which calls the JNA library is as follows :
Note that I did not add the way the other parameters of Init function are managed as I do not have any issues with them.
int paramListSize = 4;
MyLibrary.Param[] params = new MyLibrary.Param[paramListSize];
for (int i = 0; i < paramListSize; i++) {
params[i] = new MyLibrary.Param();
}
params[0].key = "first";
params[0].key_value = 1;
params[1].key = "second";
params[1].key_value = 5;
params[2].key = "third";
params[2].key_value = 7;
params[3].key = "forth";
params[3].key_value = 9;
MyLibrary.ParamList paramList = new MyLibrary.ParamList.ByReference();
paramList.init_param = params;
paramList.param_list_size = paramListSize;
logger.debug("params = "+ params);
logger.debug("paramList = "+ paramList);
logger.debug("paramList.param_list_size = "+paramList.param_list_size);
int errInit = IFunctions.Init(paramList, logCallback, i__allocator, o__content);
The traces results coming from C++ and Java code are as follows :
10:41:28,303 DEBUG Sample:193 - params = [MyLibrary$Param;#1e67a849
10:41:28,312 DEBUG Sample:194 - paramList = MyLibrary$ParamList$ByReference(auto-allocated#0x1f3d49fe5b0 (52 bytes)) {
MyLibrary$Param init_param[4]#0x0=[MyLibrary$Param;#1e67a849
int param_list_size#0x30=0x0004
}
10:41:28,316 DEBUG Sample:195 - paramList.param_list_size = 4
Exception in thread "main" java.lang.Error: Invalid memory access
at com.sun.jna.Native.invokeInt(Native Method)
at com.sun.jna.Function.invoke(Function.java:426)
at com.sun.jna.Function.invoke(Function.java:361)
at com.sun.jna.MyLibrary$Handler.invoke(MyLibrary.java:265)
at com.sun.proxy.$Proxy3.init(Unknown Source)
at Sample.main(Sample.java:216)
DLL PRINT init ---- i_initParamList = 000001F3D49FE5B0
DLL PRINT _init ---- init_param address = 0000021B4ADAF3D0
DLL PRINT init ---- i_initParamList->param_list_size = 1
DLL PRINT init ---- i_initParamList->init_param[i]->key =
It works if I declare my Structure in C++ header as follows :
typedef struct s_ParamList {
Param init_param[4];
int param_list_size; // number of param items in the param List
} ParamList;
But I do not want to define the init_param array size in the C++ code as it must be defined on java code side.
Regarding added following code in the javacode :
ParamList(){ // NOT sure that this adding constructor makes sense
super();
setAlignType(ALIGN_NONE);
}
I'm not sure that I have to add this in both structures i.e in ParamList and in Param structure.
But anyway if I delete both of those constructor I have exactly the same issue.
I saw another way to manage my requirement (looking at toArray(size) method from chapter Structure of https://javadoc.io/doc/net.java.dev.jna/jna/latest/index.html for 5.8.0 JNA version.
Indeed I also followed the link How to fill an array of structures in JNA? but I get Invalid Memory access. I'll create a separate question regarding this second kind of implementation if I cannot have my Structure's array in a Structure.
And I had to split the Param array and the size of the array in 2 separate parameters of Init function instead of setting them into a unique Structure.
And as I already have many parameters in my Init function I would prefer to have only one structure parameter. And I guess that It should be possible in JNA.
Does any one have a clue?
Thanx for your help.
The problem is in your mapping of this structure
typedef struct s_ParamList {
Param* init_param;
int param_list_size; // number of param items in the param List
} ParamList;
The * indicates this is a pointer to memory elsewhere that you need to define.
Structures are treated as ByValue by default inside structures, and as ByRefrence by default in function arguments. So here you need to explicitly define the ByReference version of the structure (or use a plain Pointer, which is less type safe.)
So you'll get this as the main part of your structure.
#FieldOrder({ "init_param", "param_list_size" })
public static class ParamList extends Structure {
public Param.ByReference init_param;
public int param_list_size;
}
Next, you've indicated you want to define this array and allocate its memory yourself in Java. The important thing to remember here is that C treats arrays as contiguous memory, so you really only have two choices: allocate a large block yourself with Memory and set values at the offsets manually; or use Structure.toArray() designed for exactly this case: You start with an instantiated structure and then tell the toArray() method how many copies of it you need.
So your sample code would look like this:
int paramListSize = 4;
// Note the syntax for allocating a contiguous array
MyLibrary.Param.ByReference[] params =
(MyLibrary.Param.ByReference[]) new MyLibrary.Param.ByReference().toArray(paramListSize);
// set the values as you've alread done
params[0].key = "first";
params[0].key_value = 1;
// and so on...
// Now instantiate your structure and set its members
MyLibrary.ParamList paramList = new MyLibrary.ParamList();
// The first array member is the pointer to the start of the array
paramList.init_param = params[0];
paramList.param_list_size = paramListSize;
And here, you pass it to the native function. It is ByReference by default.
You can find another general example here
Can you consider following data structure?
typedef struct s_ParamList {
int param_list_size; // number of param items in the param List
Param init_param[0];
} ParamList;
If yes, please be accurate with memory allocation and array usage, since such approach is pretty dangerous and may easily cause data overrun
I'm having some trouble in Java.
I have two classes, suprad and coupling.
Couplings are formed by an ArrayList of suprad's (the list is called supradscoupled), and two values(integer num and double q)
Every coupling instantiated is written in an ArrayList called Createdcouplings.
During the main program, I have to declare two couplings between two suprad's each (total of 4 suprad's at all). There is no issue when I declare the first coupling (formed by the two firsts suprad's).
But when I declare the second one trough adding the other two suprad's at the ToBeCoupledSupRads ArrayList, the older couplings are being cleared and then overwritten by the new coupling.
At the end, I get two couplings repeated formed only by the new one.
The MainWindow class:
public class MainWindow {
(...)
public static List<suprad> ToBeCoupledSupRads = new ArrayList<suprad>();
public static void main(){
(...)
int i = ToBeCoupledComboBox.getSelectedIndex();
suprad stemp = suprad.Createdsuprads.get(i);
(...)
// adding new suprad to this ArrayList makes the ArrayList Createdcouplings have its registered data overwritten!
ToBeCoupledSupRads.add(stemp);
(...)
}
}
The coupling class:
public class coupling {
public int num;
public static int total = 0;
public double q;
public static List<coupling> Createdcouplings = new ArrayList<coupling>();
public List<suprad> supradscoupled = new ArrayList<suprad>();
public coupling(List<suprad> nums, double q)
{
(...)
this.supradscoupled = aux;
total++;
this.num = total;
this.q=q;
// this is the ArrayList that is having its older data cleared and overwritten unwantedly
Createdcouplings.add(this);
}
}
I would appreciate some help, the Java project is being compiled with JavaSDK 1.8.0_221 with IntelliJ IDEA Community.
As far as I can see java.lang.instrument.Instrumentation.getSize returns the size without elements (as the size doesn't depend on the number of the list. It is 24 bytes). Looks strange. Is that correct that list items are not measured?
The size of objects in Java is fixed, because Java, like C++, is a statically-typed language. The size basically corresponds to whatever is written inside the class file. Since a class file layout is fixed, so is its size.
An ArrayList for example, looks roughly like this:
public class ArrayList implements List
{
Object[] elementData;
private int size;
}
The size of any ArrayList instance would be the size of elementData (a pointer to Object[]), size (an int), and overhead. So 24 sounds about right.
That is often referred to as a shallow size.
It sounds like what you want instead is the deep size, i.e. the size of the List + all of the objects it refers to + all the objects those objects refer to, etc.
That's not directly possible using instrumentation API, but with some hacky reflection + instrumentation you might be able to get it. The basic idea is to reflect the object and recursively call getObjectSize() on all referred objects, skipping circular references.
Something along the lines:
public class DeepSizeCounter {
private HashSet<Object> counted = new HashSet<>();
private long size = 0;
public long getSizeDeep(Object x) throws Exception {
counted.clear();
size = 0;
computeSizeDeep(x);
return size;
}
private void computeSizeDeep(Object x) throws Exception {
if (x != null && !counted.contains(x)) {
counted.add(x);
size += instrumentation.getObjectSize(x);
if (x.getClass().isArray()) {
if (!x.getClass().getComponentType().isPrimitive())
for (Object y : (Object[]) x)
computeSizeDeep(y);
} else {
for (Field f : x.getClass().getDeclaredFields()) {
if (!f.getType().isPrimitive() && (f.getModifiers() & Modifier.STATIC) == 0) {
f.setAccessible(true);
computeSizeDeep(f.get(x));
}
}
}
}
}
}
Use as:
long totalSize = new DeepSizeCounter().getSizeDeep(myList);
Beware that this approach is unreliable because objects can have backreferences to some global state which you can't distinguish automatically, so can end up counting way too many objects. Also if access control is on, the Field.setAccessible() hack won't work and you won't be able to count any non-public members.
I have the following implementation with Javolution:
public class RunScan extends Struct
{
public final Signed32 numOfClusters = new Signed32();
public final ClusterData[] clusters;
public final Signed32 numOfRecons = new Signed32();
public final ReconData[] recons ;
public RunScan (int numOfClusters, int numOfRecons)
{
this.numOfClusters.set(numOfClusters);
this.numOfRecons.set(numOfRecons);
clusters = array(new ClusterData[numOfClusters]);
recons = array(new ReconData[numOfRecons]);
}
}
public class ClusterData extends Struct
{
public final UTF8String scanType = new UTF8String(CommInterfaceFieldConstants.SCAN_TYPE_SIZE);
public final UTF8String patientId = new UTF8String(CommInterfaceFieldConstants.PATIENT_ID_SIZE);
.
.
.
}
public class ReconData extends Struct
{
public final UTF8String patientId = new UTF8String(CommInterfaceFieldConstants.PATIENT_ID_SIZE);
public final UTF8String scanSeriesId = new UTF8String(CommInterfaceFieldConstants.SCAN_SERIES_ID_SIZE);
.
.
.
}
In our communication class, before we put data onto socket, we need to get the bytes[] of the RunScan object but we get BufferUnderflowException in the line with "//<<<<<<<":
private byte[] getCmdBytes(Struct scCmd)
{
ByteBuffer cmdBuffer = scCmd.getByteBuffer();
int cmdSize = scCmd.size();
byte[] cmdBytes = new byte[cmdSize];
if (cmdBuffer.hasArray())
{
int offset = cmdBuffer.arrayOffset() + scCmd.getByteBufferPosition();
System.arraycopy(cmdBuffer.array(), offset, cmdBytes, 0, cmdSize);
}
else
{
String msg = "\n\ncmdBufferRemaining=" + cmdBuffer.remaining() + ", cmdBytesSize=" + cmdBytes.length + "\n\n";
System.out.println(msg);
cmdBuffer.position(scCmd.getByteBufferPosition());
cmdBuffer.get(cmdBytes); //<<<<<<<<<< underFlowException
}
return cmdBytes;
}
This method works in other cases. The exception happens because this line,
ByteBuffer cmdBuffer = scCmd.getByteBuffer();
only returns a 8 bytes (from the remaining() method) ByteBuffer of the RunScan object which are those two Signed32 fields, I think. But this line,
int cmdSize = scCmd.size();
returns a right length of the RunScan object which includes the size of those two arrays.
If I create those two array at the time I declare them (not "new" them in the constructor) with hard coded length, it works fine without any exception.
Anybody can help me figure out what's wrong with our implementation?
I ran into a similar situation with my code. Generally, with the current Struct object, you cannot have a variable length array defined in the same struct as the member that contains the number of elements in the array.
Try something like this:
public class RunScanHeader extends Struct
{
public final Signed32 numOfClusters = new Signed32();
public final Signed32 numOfRecons = new Signed32();
}
public class RunScanBody extends Struct
{
public final ClusterData[] clusters;
public final ReconData[] recons ;
public RunScan (int numOfClusters, int numOfRecons)
{
clusters = array(new ClusterData[numOfClusters]);
recons = array(new ReconData[numOfRecons]);
}
}
You'll then need a two phase approach to read and write, first read/write the header data, then read/write the body data.
Sorry I don't have more details at this time, if you can't solve this, let me know and I'll dig back through my code.
The initialization order is important has it defines the position of each field. Either your initialization is done when the field is declared (most common case). Or if you do it in the constructor you have to remember that the constructor is called after the member initialization. Here is an example with initialization done in the constructor:
public class RunScan extends Struct {
public final Signed32 numOfClusters;
public final ClusterData[] clusters;
public final Signed32 numOfRecons;
public final ReconData[] recons ;
public RunScan (int numOfClusters, int numOfRecons) {
// Initialization done in the constructor for all members
// Order is important, it should match the declarative order to ensure proper positioning.
this.numOfClusters = new Signed32();
this.clusters = array(new ClusterData[numOfClusters]);
this.numOfRecons = new Signed32();
this.recons = array(new ReconData[numOfRecons]);
// Only after all the members have been initialized the set method can be used.
this.numOfClusters.set(numOfClusters);
this.numOfRecons.set(numOfRecons);
}
}
get() will move the position of the ByteBuffer.
scCmd.getByteBuffer().slice().get(dest) might solve your issue with moving the position and unintended side effects.
scCmd.getByteBuffer().duplicate().get(dest) might also solve your issue if slice() produces the wrong picture of the origin buffer.
Additionally, it appears as though scCmd.getByteBuffer() creates a redundant reference and you are calling the source and child reference in the same method.
If scCmd.getByteBuffer() is already passing you a slice(), your redundant access to these methods is certainly going to do something other than what you planned.