We Keep Coding

JNA call Win32 API method IShellFolder::GetUIObjectOf

I have a problem with calling this method IShellFolder::GetUIObject I don't know how to create Pointer to Array of Pointers as the 3rd argument of this function. In the documentation the header of this function is:
HRESULT GetUIObjectOf(
HWND hwndOwner,
UINT cidl,
PCUITEMID_CHILD_ARRAY apidl,
REFIID riid,
UINT *rgfReserved,
void **ppv
);
This is my code:
String directory = "c:\\Users";
String file = "c:\\Users\\Duchon\\Downloads\\Baumüller Brno, s.r.o.PS43668.prpkg";
try {
PointerByReference psfDesktopPTR = new PointerByReference();
WinNT.HRESULT hResult = Shell32.INSTANCE.SHGetDesktopFolder(psfDesktopPTR);
if (COMUtils.SUCCEEDED(hResult)) {
IntByReference pcheaten = new IntByReference();
PointerByReference ppidl = new PointerByReference();
IntByReference pdwAttributes = new IntByReference();
MyIShellFolder psfDesktop = MyIShellFolder.Converter.PointerToIShellFolder(psfDesktopPTR);
hResult = psfDesktop.ParseDisplayName(null, null, new WString(file), pcheaten, ppidl, pdwAttributes);
PointerByReference iContextMenuPtr = new PointerByReference();
if (COMUtils.SUCCEEDED(hResult)) {
Pointer[] ppidls = new Pointer[1];
ppidls[0] = ppidl.getValue();
hResult = psfDesktop.GetUIObjectOf(null, 1, ppidl.getValue(), new Guid.REFIID(IContextMenu.IID_IContextMenu), new IntByReference(), iContextMenuPtr);
if (COMUtils.SUCCEEDED(hResult)) {
// QueryIContextMenu ...
}
}
}
}
catch (Exception e) {
e.printStackTrace(System.err);
}
But I get invalid memory access exception. I need a solution for array of files, not only for one. Thank you very much.

When you get an Invalid Memory Access error, it's a clue that you need to properly allocate Native memory. Your code above only declares a Java-side Pointer array.
Arrays in C use contiguous memory. This means you must allocate a single block of native memory large enough for the array; it is not enough to collect a bunch of individual allocations (which is what declaring a single Pointer variable in Java does).
You have two primary options for allocating this block of native memory:
Option 1. Use JNA's Memory class to explicitly allocate the size of memory you will need. If you allocating an array of Pointers, you will allocate like this: Memory m = new Memory(numberOfElements * Native.POINTER_SIZE); When you get the returned value into this memory you will use offsets to pull the appropriate pointer from the array, e.g., for the 0-indexed ith pointer, do Pointer p = m.getPointer(i * Native.POINTER_SIZE);
Option 2. Create a Structure of the appropriate size (in this case, containing a single element which is a Pointer) and use Structure.toArray() to allocate the Structure array. So you could define:
#FieldOrder ({"childId"})
class PCUITEMID_CHILD extends Structure {
public Pointer childId;
}
And then allocate the array
PCUITEM_CHILD[] pcuItemIdArray = new PCUITEMID_CHILD().toArray(numberOfElements);
Then you can pass this array variable, and access its result using traditional array syntax.
Pointer p = pcuItemIdArray[0].childId;

Java How to convert a java.nio.Buffer into a ByteBuffer

I'm facing a little problem I have two libraries one send me the output as java.nio.Buffer and the other receives the input as a java.nio.ByteBuffer how do I make the conversion?
Thanks
the Buffer is from javaCV from this piece of code:
private BytePointer[] image_ptr;
private Buffer[] image_buf;
// Determine required buffer size and allocate buffer
int size = avpicture_get_size(fmt, width, height);
image_ptr = new BytePointer[] { new BytePointer(av_malloc(size)).capacity(size) };
image_buf = new Buffer[] { image_ptr[0].asBuffer() };
// Assign appropriate parts of buffer to image planes in picture_rgb
// Note that picture_rgb is an AVFrame, but AVFrame is a superset of AVPicture
avpicture_fill(new AVPicture(picture_rgb), image_ptr[0], fmt, width, height);
picture_rgb.format(fmt);
picture_rgb.width(width);
picture_rgb.height(height);

First of all, since Buffer is an abstract class, and ByteBuffer is one of its subclasses, it's entirely possible that the output you're getting from the first library is in fact a ByteBuffer. If possible, check to see which implementation of Buffer the library is returning, because if it's actually returning a ByteBuffer you can just cast the output to ByteBuffer and be done.
If you don't know which implementation of Buffer the library returns, you'll have to resort to instanceof tests to determine what subclass it is, and copy the data from the returned Buffer to a new ByteBuffer after downcasting it to a subclass. This is because the Buffer interface doesn't actually provide any methods to read the data from the buffer; only the subclasses (ByteBuffer, ShortBuffer, LongBuffer, etc.) do. Fortunately, there are only 7 possible subclasses of Buffer, one for each primitive type.
Once you've determined which subclass of Buffer you have, you can copy the data to a ByteBuffer using the "asXXXBuffer()" method described in this answer, as #Tunaki pointed out.
The code would look something like this:
Buffer outputBuffer = library.getBuffer();
ByteBuffer byteBuffer;
if (outputBuffer instanceof ByteBuffer) {
byteBuffer = (ByteBuffer) outputBuffer;
} else if (outputBuffer instanceof CharBuffer) {
byteBuffer = ByteBuffer.allocate(outputBuffer.capacity());
byteBuffer.asCharBuffer().put((CharBuffer) outputBuffer);
} else if (outputBuffer instanceof ShortBuffer) {
byteBuffer = ByteBuffer.allocate(outputBuffer.capacity() * 2);
byteBuffer.asShortBuffer().put((ShortBuffer) outputBuffer);
} else if (outputBuffer instanceof IntBuffer) {
byteBuffer = ByteBuffer.allocate(outputBuffer.capacity() * 4);
byteBuffer.asIntBuffer().put((IntBuffer) outputBuffer);
} else if (outputBuffer instanceof LongBuffer) {
byteBuffer = ByteBuffer.allocate(outputBuffer.capacity() * 8);
byteBuffer.asLongBuffer().put((LongBuffer) outputBuffer);
} else if (outputBuffer instanceof FloatBuffer) {
byteBuffer = ByteBuffer.allocate(outputBuffer.capacity() * 4);
byteBuffer.asFloatBuffer().put((FloatBuffer) outputBuffer);
} else if (outputBuffer instanceof DoubleBuffer) {
byteBuffer = ByteBuffer.allocate(outputBuffer.capacity() * 8);
byteBuffer.asDoubleBuffer().put((DoubleBuffer) outputBuffer);
}
Note that the size of the ByteBuffer you allocate depends on which subclass of Buffer you're copying from, since different primitive types are stored using different numbers of bytes. For example, since an int is 4 bytes, if your library gives you an IntBuffer, you need to allocate a ByteBuffer with 4 times the capacity.

Error while passing array of structures conataining char array to native function using JNA

Here is my updated code after implementing suggestions. But still problems persist.
typedef struct S1{
char temp1[100];
char temp2[100];
}S1
...
int manipulateTemp(S1 s1Arr[] );
JNA interface looks like this
public interface Add extends Library
{
Add INSTANCE = (Add) Native.loadLibrary("add", Add.class);
public static class S1 extends Structure {
public byte[] temp1 = new byte[100];
public byte[] temp2 = new byte[100];
public static class ByReference extends S1 implements Structure.ByReference {
};
};
int manipulateTemp( S1[]);
}
//
public static byte[] toByteArray(char[] a ,Charset c){
CharBuffer cBuffer = CharBuffer.wrap(a);
ByteBuffer bBuffer = c.encode(cBuffer);
return bBuffer.array;
}
//in main method
Add lib = Add.INSTANCE;
Add.S1.ByReference s1Ref = new Add.S1.ByReference();
Add.S1[] s1Arr = (Add.S1[])s1Ref.toArray(10);
s1Ref.clear();
//initialize array
for(int i =0;i<s1Arr.lenth ;i++){
byte[] data = toByteArray("myString1".toCharArray,Charset.defaultCharSet
System.arrarycopy(data,0, s1Arr[i].temp1,0,data.length);
data = toByteArray("myString2".toCharArray,Charset.defaultCharSet
System.arrarycopy(data,0, s1Arr[i].temp2,0,data.length);
}
// calling native function
lib.manipulateTemp(s1Arr[]);
After execution
Exception in thread "main" java.lang.Error: Invalid memory access
at com.sun.jna.Function.invokeInt(Native Method)
at com.sun.jna.Function.invoke(Function.java:344)
at com.sun.jna.Function.invoke(Function.java:276)
at com.sun.jna.Library$Handler.invoke(Library.java:216)
at com.sun.proxy.$Proxy0.manipulateTemp((Unknown Source)
at LoanTest.newTestCalc.main(newTestCalc.java:288)
I even checked memory dump, structures are seems to be allocated stored correctly.Structure size is also correct = 200 bytes
Any clues about this error?

You need to copy values into the existing temp field, not overwrite it. When you overwrite it, you're actually changing its size, which JNA uses to determine the structure size. Following is how you should initialize your structure data:
class S1 extends Structure {
public byte[] temp = new byte[100];
...
}
S1 s = new S1();
S1[] array = (S1[])s.toArray(ARRAY_SIZE);
System.setProperty("jna.encoding", "utf-8"); // You probably want utf-8; utf-16 has 16-bit code units, so unless your native code is actually expecting a utf-16 encoding broken down into byte units, use utf-8
byte[] data = Native.toByteArray("myString"); // includes NUL terminator
System.arraycopy(data, 0, array[0].temp, 0, data.length);
// Repeat as needed for other members of the array
lib.manipulateTemp(array);
Note that the declarations manipulateTemp(S1 s) or manipulateTemp(S1[] array) will both work, although the latter is more accurate and conveys your intent explicitly.

excessive memory use by java

In a project of mine I constantly compress little blocks of data.
Now I find out that the jvm then grows to 6GB of ram (resident (RES) RAM, not shared or virtual or so) and then die because of out of memory.
It is as if the garbage collector never runs or so.
I've pulled out the relevant code and pasted it below. When I run it (java6, 32 bit linux) it grows to 1GB of ram.
Anyone got an idea how to reduce the memory usage?
import java.util.Random;
import java.util.zip.Deflater;
import java.util.zip.Inflater;
class test {
int blockSize = 4096;
Random r = new Random();
public test() throws Exception {
blockSize = 4096;
byte [] data = new byte[blockSize];
for(int index=0; index<blockSize; index++)
data[index] = (byte)r.nextInt();
for(long cnt=0; cnt<1000000; cnt++) {
byte [] result = compress(data);
if (result != null)
data[0] = result[0];
}
}
byte [] compress(byte [] in) {
assert in.length == blockSize;
Deflater compresser = new Deflater();
compresser.setInput(in);
compresser.finish();
byte [] out = new byte[in.length];
int outLen = compresser.deflate(out);
if (outLen < blockSize) {
byte [] finalOut = new byte[outLen];
System.arraycopy(out, 0, finalOut, 0, outLen);
return finalOut;
}
return null;
}
public static void main(String [] args) throws Exception {
new test();
}
}

Well, Folkert van Heusden solved his own problem, but to summarize:
Early in the compress(byte [] in)-method, we create a java.util.zip.Deflater.
We use the Deflater to do some stuff, and then we leave the compress()-method. We loose our reference to the deflater-variable. At this point, the Deflater is no longer in use, and is waiting to be killed by the garbage collector.
Deflater allocates both Java heap memory and C/C++/native heap memory. The native heap memory that are allocated by a Deflater, will be held until Deflater.finalize-method is called by the garbage collector. If the garbage collector doesn't run fast enough (there might be plenty free java heap memory), we can run out of C/C++ heap memory. If this happen, we will get "Out of memory"-errors.
The Oracle bug report JDK-4797189 is probably related. It contains a code snippet that illustrates and reproduces the problem:
public class Bug {
public static void main( String args[] ) {
while ( true ) {
/* If ANY of these two lines is not commented, the JVM
runs out of memory */
final Deflater deflater = new Deflater( 9, true );
final Inflater inflater = new Inflater( true );
}
}
}
The solution is to free the resources when you are finished by calling the Deflater.end()-method (or Inflater.end()).

Well, It seems to me that there is no memory leak in the code, so it actually seems the VM is not GC-ing byte arrays.
"Anyone got an idea how to reduce the memory usage?"
Well, I would try with
byte firstByteOfDataWhichIsCompressedAndThenUncompressed(byte [] in) { ... }
which specifically returns the first byte of the uncompressed array, rather than the whole array. I know, it's a horrible method name, and I hope you will find a better one.
The following code
for(long cnt=0; cnt<1000000; cnt++) {
byte [] result = compress(data);
if (result != null)
data[0] = result[0];
}
would become
for(long cnt=0; cnt<1000000; cnt++)
data[0] = firstByteOfDataWhichIsCompressedAndThenUncompressed(data);

Deep copy duplicate() of Java's ByteBuffer

java.nio.ByteBuffer#duplicate() returns a new byte buffer that shares the old buffer's content. Changes to the old buffer's content will be visible in the new buffer, and vice versa. What if I want a deep copy of the byte buffer?

I think the deep copy need not involve byte[]. Try the following:
public static ByteBuffer clone(ByteBuffer original) {
ByteBuffer clone = ByteBuffer.allocate(original.capacity());
original.rewind();//copy from the beginning
clone.put(original);
original.rewind();
clone.flip();
return clone;
}

As this question still comes up as one of the first hits to copying a ByteBuffer, I will offer my solution. This solution does not touch the original buffer, including any mark set, and will return a deep copy with the same capacity as the original.
public static ByteBuffer cloneByteBuffer(final ByteBuffer original) {
// Create clone with same capacity as original.
final ByteBuffer clone = (original.isDirect()) ?
ByteBuffer.allocateDirect(original.capacity()) :
ByteBuffer.allocate(original.capacity());
// Create a read-only copy of the original.
// This allows reading from the original without modifying it.
final ByteBuffer readOnlyCopy = original.asReadOnlyBuffer();
// Flip and read from the original.
readOnlyCopy.flip();
clone.put(readOnlyCopy);
return clone;
}
If one cares for the position, limit, or order to be set the same as the original, then that's an easy addition to the above:
clone.position(original.position());
clone.limit(original.limit());
clone.order(original.order());
return clone;

Based off of mingfai's solution:
This will give you an almost true deep copy. The only thing lost will be the mark. If orig is a HeapBuffer and the offset is not zero or the capacity is less than the backing array than the outlying data is not copied.
public static ByteBuffer deepCopy( ByteBuffer orig )
{
int pos = orig.position(), lim = orig.limit();
try
{
orig.position(0).limit(orig.capacity()); // set range to entire buffer
ByteBuffer toReturn = deepCopyVisible(orig); // deep copy range
toReturn.position(pos).limit(lim); // set range to original
return toReturn;
}
finally // do in finally in case something goes wrong we don't bork the orig
{
orig.position(pos).limit(lim); // restore original
}
}
public static ByteBuffer deepCopyVisible( ByteBuffer orig )
{
int pos = orig.position();
try
{
ByteBuffer toReturn;
// try to maintain implementation to keep performance
if( orig.isDirect() )
toReturn = ByteBuffer.allocateDirect(orig.remaining());
else
toReturn = ByteBuffer.allocate(orig.remaining());
toReturn.put(orig);
toReturn.order(orig.order());
return (ByteBuffer) toReturn.position(0);
}
finally
{
orig.position(pos);
}
}

One more simple solution
public ByteBuffer deepCopy(ByteBuffer source, ByteBuffer target) {
int sourceP = source.position();
int sourceL = source.limit();
if (null == target) {
target = ByteBuffer.allocate(source.remaining());
}
target.put(source);
target.flip();
source.position(sourceP);
source.limit(sourceL);
return target;
}

You'll need to iterate the entire buffer and copy by value into the new buffer.

I believe this should supply a full deep copy, including the mark, "out-of-bounds" data, etc...just in case you need the most complete sandbox-safe carbon copy of a ByteBuffer.
The only thing it doesn't copy is the read-only trait, which you can easily get by just calling this method and tagging on a ".asReadOnlyBuffer()"
public static ByteBuffer cloneByteBuffer(ByteBuffer original)
{
//Get position, limit, and mark
int pos = original.position();
int limit = original.limit();
int mark = -1;
try
{
original.reset();
mark = original.position();
}
catch (InvalidMarkException e)
{
//This happens when the original's mark is -1, so leave mark at default value of -1
}
//Create clone with matching capacity and byte order
ByteBuffer clone = (original.isDirect()) ? ByteBuffer.allocateDirect(original.capacity()) : ByteBuffer.allocate(original.capacity());
clone.order(original.order());
//Copy FULL buffer contents, including the "out-of-bounds" part
original.limit(original.capacity());
original.position(0);
clone.put(original);
//Set mark of both buffers to what it was originally
if (mark != -1)
{
original.position(mark);
original.mark();
clone.position(mark);
clone.mark();
}
//Set position and limit of both buffers to what they were originally
original.position(pos);
original.limit(limit);
clone.position(pos);
clone.limit(limit);
return clone;
}