Unable to set Java int array field from JNI - java

I'm developing an Android App and I'm receiving camera data from a lib in C++. I need to send this data from C++ to the Java code. For this, I'm using JNI. I'm able to set different fields in Java from the JNI and the C++ data (like the name or the type of the camera), but I'm unable to set the ID Field because it's an uint8_t array.
How can I do this?
I already tried several ways to do this but each time I've got an SIGSEGV error with an invalid address. For others fields I'm using
env->Set<Primitives>Field(jobject, jfieldID, value)
method but there are no methods like that for int array, are there?
So, I've tried to set this field by calling a method from my class and provide the int array as parameter but this function failed and returned the SIGSEGV error.
Then, I searched on the web and I tried to set the field through
env->GetObjectField(jobject, jfieldID)
and
env->SetIntArrayRegion(jintArray, start, end, myIntArray)
but here the first method returns always null.
JavaVM * mJVM; //My Java Virtual Machine
jobject mCameraObject, mThreadObject; //Previously initialize to call functions in the right thread
void onReceiveCameraList(void *ptr, uint32_t /*id*/, my::lib::Camera *arrayCamera, uint32_t nbCameras) {
JNIEnv *env;
mJVM->AttachCurrentThread(&env, nullptr);
if (env->ExceptionCheck())
return;
//Get Field, Method ID, Object and Class
jclass cameraClass = env->GetObjectClass(mCameraObject);
jfieldID camIDField = env->GetFieldID(cameraClass, "idCam", "[I");
jfieldID camNameField = env->GetFieldID(cameraClass, "label", "Ljava/lang/String;");
jfieldID camConnectedField = env->GetFieldID(cameraClass, "connected", "Z");
jfieldID camTypeField = env->GetFieldID(cameraClass, "typeProduit", "B");
jmethodID camReceptionMID = env->GetMethodID(env->GetObjectClass(mThreadObject), "onCamerasReception", "([Lcom/my/path/models/Camera;)V"); //Java function
jobjectArray cameraArray = env->NewObjectArray(nbCameras, cameraClass, mCameraObject); //Object return in the functions
//Put the cameras into the vector
std::vector<my::lib::Camera> vectorCameras;
if(!vectorCameras.empty())
vectorCameras.clear();
if ((arrayCamera != nullptr) && (nbCameras > 0)) {
for (uint32_t i = 0; i < nbCameras; ++i) {
vectorCameras.push_back(arrayCamera[i]);
}
}
//Set the my::lib::Camera field into Java::Camera
int c= 0;
for (auto & cam : vectorCameras)
{
jobject camera = env->AllocObject(cameraClass); //Object Camera to add in cameraArray object
// MY DATA TO SET ID FIELD ///
jint idArray[16];
for (int i = 0; i < 16 ; ++i) {
idArray[i] = cam.idCamera.data[i]; // uint8_t cam.idCamera.data[16]
}
///////// FIRST WAY /////////
jmethodID setIDCamMID = env->GetMethodID(env->GetObjectClass(camera), "setIDCam", "([I)V");
env->CallVoidMethod(camera, setIDCamMID, idArray);
///////// SECOND WAY /////////
jintArray jintArray1 = (jintArray)env->GetObjectField(camera, camIDField);
env->SetIntArrayRegion(jintArray1, 0, 16, idArray);
//Set<Primitives>Field : WORKING
env->SetObjectField(camera, camNameField, env->NewStringUTF((const char *) cam.labelCamera));
env->SetBooleanField(camera, camConnectedField, cam.isCameraConnected);
jbyte type;
if (cam.typeCamera == my::lib::TYPE_1 || cam.typeCamera == my::lib::TYPE_2 || cam.typeCamera == my::lib::TYPE_3) //type not known in JAVA
type = 0;
else
type = cam.typeCamera;
env->SetByteField(camera, camTypeField, type);
//Put camera object into cameraArray object
env->SetObjectArrayElement(cameraArray, c++, camera);
}//for
//Call Java method with cameraArray
env->CallVoidMethod(mThreadObject, camReceptionMID, dpCameraArray);
}//onreceiveCamera
Can someone tell me if I made a mistake or am using it the wrong way?
Is there any other way to set this data?

This yields a C++ array with elements of type jint:
// MY DATA TO SET ID FIELD ///
jint idArray[16];
for (int i = 0; i < 16 ; ++i) {
idArray[i] = cam.idCamera.data[i]; // uint8_t cam.idCamera.data[16]
}
It is important to understand that that is not a Java array. Therefore, this ...
///////// FIRST WAY /////////
jmethodID setIDCamMID = env->GetMethodID(env->GetObjectClass(camera), "setIDCam", "([I)V");
env->CallVoidMethod(camera, setIDCamMID, idArray);
... is incorrect. idArray is not the right type (and does not decay to a pointer to the right type) for the corresponding parameter to the Java method you are trying to invoke.
On the other hand, this ...
///////// SECOND WAY /////////
jintArray jintArray1 = (jintArray)env->GetObjectField(camera, camIDField);
env->SetIntArrayRegion(jintArray1, 0, 16, idArray);
... is ok, provided that the field already contains a reference to an int[] of length at least 16. Since it didn't work for you, I take it that the field's initial value does not satisfy that criterion.
If you need to create a new Java int[], then
use JNI's NewIntArray() function, which returns a jintArray with the length you specify. Then
use the appropriate JNI methods for setting the elements (see below), and finally
either assign the array directly to the target object's field with SetObjectField() or use the object's setter method to do so, as in your first attempt.
As for setting elements of the Java array, SetIntArrayRegion() will work fine for that (given an actual Java array of sufficient length), but that does require you to allocate a separate native array of primitives (your idArray) from which to copy the values. A slightly more efficient approach would be to use GetPrimitiveArrayCritical() to let Java provide the buffer -- possibly a direct pointer to the internal data -- and then ReleasePrimitiveArrayCritical() when you're done. Something like this:
// It is assumed here that the length of the array is sufficient, perhaps because
// we just created this (Java) array.
jint *idArray = (jint *) env->GetPrimitiveArrayCritical(jintArray1, NULL);
for (uint32_t i = 0; i < nbCameras; ++i) {
idArray[i] = cam.idCamera.data[i];
}
env->ReleasePrimitiveArrayCritical(jintArray1, idArray, 0);

For the first approach, you need to create a Java int[] first, fill it from idArray, and then call your method:
jintArray j_arr = env->NewIntArray(16);
env->SetIntArrayRegion(j_arr, 0, 16, idArray);
env->CallVoidMethod(camera, setIDCamMID, j_arr);
Your second approach does not work because you never called a constructor that filled the idCam field.
You can do that from JNI, however:
env->SetObjectField(camera, camIDField, j_arr);

Related

How to map in JNA an array of structure inside a structure without setting the array size on the C++ declaration

I'm trying to map a structure which contains an array of structure in JNA.
The size of the embedded structure is not defined in advanced in the C++ declaration of the structure but in the java code.
My issue is that I get Exception in thread "main" java.lang.Error: Invalid memory access
The C++ header file is as follows :
typedef struct s_Param
{
char* key;
uint32_t key_value;
} Param;
typedef struct s_ParamList {
Param* init_param;
int param_list_size; // number of param items in the param List
} ParamList;
The C++ code is as follows :
int Init(
ParamList* i_initParamList,
logfunction i__callback,
const allocator* i__allocator,
void** o__content) {
...
if (i_initParamList != NULL){
printf("DLL PRINT init ---- i_initParamList = %p\n", i_initParamList);
printf("DLL PRINT init ---- i_initParamList->param_list_size = %i\n", i_initParamList->param_list_size);
if (i_initParamList->init_param == NULL){
printf("DLL PRINT init ---- i_initParamList->init_param must not be NULL\n");
returnedCode = 1;
}else{
for (int i = 0; i<i_initParamList->param_list_size;i++){
printf("DLL PRINT init ---- i_initParamList->init_param[i]->key = %s\n",i_initParamList->init_param[i].key);
printf("DLL PRINT init ---- i_initParamList->init_param[i]->key_value = %i\n",i_initParamList->init_param[i].key_value);
}
...
}
The java JNA code is as follows :
public interface MyLibrary extends Library {
#FieldOrder({ "key", "key_value" })
public static class Param extends Structure {
public static class ByReference extends Param implements Structure.ByReference {
}
public String key;
public int key_value;
public Param(){ // NOT sure that this adding constructor makes sense
super();
setAlignType(ALIGN_NONE);
}
}
#FieldOrder({ "init_param", "param_list_size" })
public static class ParamList extends Structure {
public static class ByReference extends ParamList implements Structure.ByReference {
}
public Param[] init_param;
public int param_list_size;
ParamList(){ // NOT sure that this adding constructor makes sense
super();
setAlignType(ALIGN_NONE);
}
}
public int Init(ParamList i_initParamList, logfunction i__callback, allocator i__allocator,
PointerByReference o__content);
...
}
The Sample.java code which calls the JNA library is as follows :
Note that I did not add the way the other parameters of Init function are managed as I do not have any issues with them.
int paramListSize = 4;
MyLibrary.Param[] params = new MyLibrary.Param[paramListSize];
for (int i = 0; i < paramListSize; i++) {
params[i] = new MyLibrary.Param();
}
params[0].key = "first";
params[0].key_value = 1;
params[1].key = "second";
params[1].key_value = 5;
params[2].key = "third";
params[2].key_value = 7;
params[3].key = "forth";
params[3].key_value = 9;
MyLibrary.ParamList paramList = new MyLibrary.ParamList.ByReference();
paramList.init_param = params;
paramList.param_list_size = paramListSize;
logger.debug("params = "+ params);
logger.debug("paramList = "+ paramList);
logger.debug("paramList.param_list_size = "+paramList.param_list_size);
int errInit = IFunctions.Init(paramList, logCallback, i__allocator, o__content);
The traces results coming from C++ and Java code are as follows :
10:41:28,303 DEBUG Sample:193 - params = [MyLibrary$Param;#1e67a849
10:41:28,312 DEBUG Sample:194 - paramList = MyLibrary$ParamList$ByReference(auto-allocated#0x1f3d49fe5b0 (52 bytes)) {
MyLibrary$Param init_param[4]#0x0=[MyLibrary$Param;#1e67a849
int param_list_size#0x30=0x0004
}
10:41:28,316 DEBUG Sample:195 - paramList.param_list_size = 4
Exception in thread "main" java.lang.Error: Invalid memory access
at com.sun.jna.Native.invokeInt(Native Method)
at com.sun.jna.Function.invoke(Function.java:426)
at com.sun.jna.Function.invoke(Function.java:361)
at com.sun.jna.MyLibrary$Handler.invoke(MyLibrary.java:265)
at com.sun.proxy.$Proxy3.init(Unknown Source)
at Sample.main(Sample.java:216)
DLL PRINT init ---- i_initParamList = 000001F3D49FE5B0
DLL PRINT _init ---- init_param address = 0000021B4ADAF3D0
DLL PRINT init ---- i_initParamList->param_list_size = 1
DLL PRINT init ---- i_initParamList->init_param[i]->key =
It works if I declare my Structure in C++ header as follows :
typedef struct s_ParamList {
Param init_param[4];
int param_list_size; // number of param items in the param List
} ParamList;
But I do not want to define the init_param array size in the C++ code as it must be defined on java code side.
Regarding added following code in the javacode :
ParamList(){ // NOT sure that this adding constructor makes sense
super();
setAlignType(ALIGN_NONE);
}
I'm not sure that I have to add this in both structures i.e in ParamList and in Param structure.
But anyway if I delete both of those constructor I have exactly the same issue.
I saw another way to manage my requirement (looking at toArray(size) method from chapter Structure of https://javadoc.io/doc/net.java.dev.jna/jna/latest/index.html for 5.8.0 JNA version.
Indeed I also followed the link How to fill an array of structures in JNA? but I get Invalid Memory access. I'll create a separate question regarding this second kind of implementation if I cannot have my Structure's array in a Structure.
And I had to split the Param array and the size of the array in 2 separate parameters of Init function instead of setting them into a unique Structure.
And as I already have many parameters in my Init function I would prefer to have only one structure parameter. And I guess that It should be possible in JNA.
Does any one have a clue?
Thanx for your help.
The problem is in your mapping of this structure
typedef struct s_ParamList {
Param* init_param;
int param_list_size; // number of param items in the param List
} ParamList;
The * indicates this is a pointer to memory elsewhere that you need to define.
Structures are treated as ByValue by default inside structures, and as ByRefrence by default in function arguments. So here you need to explicitly define the ByReference version of the structure (or use a plain Pointer, which is less type safe.)
So you'll get this as the main part of your structure.
#FieldOrder({ "init_param", "param_list_size" })
public static class ParamList extends Structure {
public Param.ByReference init_param;
public int param_list_size;
}
Next, you've indicated you want to define this array and allocate its memory yourself in Java. The important thing to remember here is that C treats arrays as contiguous memory, so you really only have two choices: allocate a large block yourself with Memory and set values at the offsets manually; or use Structure.toArray() designed for exactly this case: You start with an instantiated structure and then tell the toArray() method how many copies of it you need.
So your sample code would look like this:
int paramListSize = 4;
// Note the syntax for allocating a contiguous array
MyLibrary.Param.ByReference[] params =
(MyLibrary.Param.ByReference[]) new MyLibrary.Param.ByReference().toArray(paramListSize);
// set the values as you've alread done
params[0].key = "first";
params[0].key_value = 1;
// and so on...
// Now instantiate your structure and set its members
MyLibrary.ParamList paramList = new MyLibrary.ParamList();
// The first array member is the pointer to the start of the array
paramList.init_param = params[0];
paramList.param_list_size = paramListSize;
And here, you pass it to the native function. It is ByReference by default.
You can find another general example here
Can you consider following data structure?
typedef struct s_ParamList {
int param_list_size; // number of param items in the param List
Param init_param[0];
} ParamList;
If yes, please be accurate with memory allocation and array usage, since such approach is pretty dangerous and may easily cause data overrun

Returning array from JNI to Java causing slow leak

I have a cache implemented in c++ using multimap and then I look up the cache and return the matching integer values as an array to the Java code.
The problem I am facing is, the Old generation progressively gets completely full and triggers continuous full GC from thereon. I found out, that, just returning a static array is causing the Old generation to fillup periodically and trigger a full GC. Eventually at a high load system will heap dump.
The code is like below
Java
for(JavaObject o: javaOjctes){
int[] values = cache.get(o.getkey1,o.getkey2,o.getkeyType[]);
//further processing
}
and on C++ side I have CacheImpl.cpp
JNIEXPORT jintArray JNICALL Java_com_test_cache_CacheService_get(JNIEnv *env, jobject thisObject, jstring key1,jstring key2,jobjectArray keyTypes){
const char *key1Value = (env)->GetStringUTFChars(key1,NULL);
env->ReleaseStringUTFChars(key1,key1Value);
//if(key1Value == NULL)
//return NULL;
const char *key2Value = (env)->GetStringUTFChars(key2,NULL);
//if(key2Value == NULL)
//return NULL;
string key2_str(key2value);
env->ReleaseStringUTFChars(key2,key2Value);
jsize length = env->GetArrayLength(keyTypes);
if(length == 0)
return NULL;
// I’m managing the C++ cache from Java side, so getting the
// reference of native cache from Java
jclass CacheService = env->GetObjectClass(thisObject);
Cache* cache= (Cache*) env->GetStaticLongField(CacheService,getCacheField(env,thisObject));
env->DeleteLocalRef(CacheService);
list<int> result;
for(int index=0;index<length;index++){
jstring keyType =(jstring) env->GetObjectArrayElement(keyTypes,index);
const char *key_type_str=env->GetStringUTFChars(keyType,NULL);
string key_type(key_type_str);
env->ReleaseStringUTFChars(keyType,key_type_str);
// Cache.cpp code is given below
list<int> intermediate=cache->get(key2_str+key_type,key1Value);
result.merge(intermediate);
intermediate=cache->get(key_type,key1Value);
result.merge(intermediate);
}
env->ReleaseStringUTFChars(key1,key1Value);
// I would return result list from below code
// For testing I commented all of the above code and
// tested with only below code. I could see Old gen
// Keep growing and GC spikes
jintArray Ids=env->NewIntArray(50);
if(Ids == NULL){
return NULL;
}
jint return_Array[50];
int j=0;
for(j=0; j<50 ;j++){
return_Array[j]=j;
}
env->SetIntArrayRegion(Ids,0,50,return_Array);
//env->DeleteLocalRef(Ids);
return Ids;
and The Cache.cpp code is like below
shared_ptr<multimap<string, SomeObject> > Cache::cacheMap;
list<int> Cache::get(string key,const char* key1Value){
SomeObject value;
list<int> IDs;
shared_ptr<multimap<string, SomeObject> > readMap=atomic_load(&cacheMap);
pair<mapIterator,mapIterator> result=readMap->equal_range(key);
for(auto iterator=result.first; iterator!=result.second; iterator++){
value=iterator->second;
if(!(value.checkID(key1Value))){
IDs.push_front(value.getID(key1Value));
}
}
return IDs;
}
In the code above I guess the Ids array
jintArray Ids=env->NewIntArray(50);
is not getting deleted so I tried to
//env->DeleteLocalRef(Ids);
This doesn't work as the empty array will be returned on Java side.
As described in the code comments, I commented out all the code except for returning a static array to Java. I could see that the Old Generation was getting filled up gradually triggering a Full GC, which I believe is a slow leak problem. I have a slow leak in this cache lookup flow. Am I missing something?
Thanks,
Raj

JNI : java - C // Using Global variables

I would like to know whether JNI accepts the use of global variables declared in the .cpp (and of type created by me).
It would be for a speech recognition program. In fact, I would like to use C to declare all the settings of the decoder and Java to manage the recording thing on Android. I don't want to create a Decoder class on Java because it should be use both on Android and iOS (I would like to have 3 separate parts :
1) set up the decoder (only in C)
2) do the recording and manage the threads (only in Android-java)
3) decode the audio (in java but calling C functions that will use the settings of the decoder)
Example :
function.cpp :
struct decoder {
char *model;
};
In the function_2.ccp file :
decoder *ps;
void set_decoder(char *modelToUse){
ps->model = modelToUse;
return;
}
void process(int *buf ) {
process_raw(ps, buf); // modify buf accordingly to the parameters of ps
return;
}
char *getHyp (){
return ps->hyp;
}
void main(){
ps = set_decoder(var1);
jclass = android_class = (*env)->FindClass(env,"android/app/nameofactivity");
// find method_ID
jmethodID android_mid = (*env)->GetStaticMethodID(env, android_class, "java_method",String_signature);
// call method
jString hyp_java = (*env)->CallIntMethod(env, android_class, android_mid, int samplerate);
EDIT (add these lines) :
const char *hyp_tmp = (*env)->GetStringUTFChars(env, hyp_java, 0);
const char *hyp = hyp_tmp;
printf(hyp_java);
(*env)->ReleaseStringUTFChars(env, hyp_java, hyp_tmp);
//END of EDIT
}
In the java file (the methods process and getHyp are declared as native methods ; the read method is a method from Android):
file.java
String java_method(int samplerate){
int buf[samplerate];
recorder.read(buf);
this.process(buf);
return this.getHyp();
}
Is it possible to do that?
EDIT : In brief, I wonder whether I can declare a global variable "ps" in C and then call a java function that will call other C functions that will use the value of the declared variable "ps".

JNI - java ArrayList conversion to c++ std::string*

I am trying to make a data conversion with JNI in c++. I have come into trouble working with java's ArrayList of strings, since I have not been able to convert such a data into a c++ vector or std::string*.
I would like to know how this conversion can be made without sacrificing too much performance, if possible. Any ideas would be appreciated.
I don't know if this fits your performance requirements, but it may be a good start.
For both options assume that jobject jList; is your ArrayList.
Option 1
Convert the List into an array and iterate on the array (maybe more applicable if you have a LinkedList instead of an ArrayList)
// retrieve the java.util.List interface class
jclass cList = env->FindClass("java/util/List");
// retrieve the toArray method and invoke it
jmethodID mToArray = env->GetMethodID(cList, "toArray", "()[Ljava/lang/Object;");
if(mToArray == NULL)
return -1;
jobjectArray array = (jobjectArray)env->CallObjectMethod(jList, mToArray);
// now create the string array
std::string* sArray = new std::string[env->GetArrayLength(array)];
for(int i=0;i<env->GetArrayLength(array);i++) {
// retrieve the chars of the entry strings and assign them to the array!
jstring strObj = (jstring)env->GetObjectArrayElement(array, i);
const char * chr = env->GetStringUTFChars(strObj, NULL);
sArray[i].append(chr);
env->ReleaseStringUTFChars(strObj, chr);
}
// just print the array to std::cout
for(int i=0;i<env->GetArrayLength(array);i++) {
std::cout << sArray[i] << std::endl;
}
Option 2
An alternative would be to use the List.size() and List.get(int) methods to retrieve the data from the list. As you use an ArrayList this solution would also be okay as the ArrayList is an RandomAccessList.
// retrieve the java.util.List interface class
jclass cList = env->FindClass("java/util/List");
// retrieve the size and the get method
jmethodID mSize = env->GetMethodID(cList, "size", "()I");
jmethodID mGet = env->GetMethodID(cList, "get", "(I)Ljava/lang/Object;");
if(mSize == NULL || mGet == NULL)
return -1;
// get the size of the list
jint size = env->CallIntMethod(jList, mSize);
std::vector<std::string> sVector;
// walk through and fill the vector
for(jint i=0;i<size;i++) {
jstring strObj = (jstring)env->CallObjectMethod(jList, mGet, i);
const char * chr = env->GetStringUTFChars(strObj, NULL);
sVector.push_back(chr);
env->ReleaseStringUTFChars(strObj, chr);
}
// print the vector
for(int i=0;i<sVector.size();i++) {
std::cout << sVector[i] << std::endl;
}
_edited: replaced JNI_FALSE with NULL_
_edited: replaced insert with push_back_
You could also use classic java iterators from within JNI, having the loop equally optimized both for ArrayList and LinkedList.
jobject jIterator = env->CallObjectMethod(jList, env->GetMethodID(env->GetObjectClass(jList), "iterator", "()Ljava/util/Iterator;"));
jmethodID nextMid = env->GetMethodID(env->GetObjectClass(jIterator), "next", "()Ljava/lang/Object;");
jmethodID hasNextMid = env->GetMethodID(env->GetObjectClass(jIterator), "hasNext", "()Z");
while (env->CallBooleanMethod(jIterator, hasNextMid)) {
jobject jItem = env->CallObjectMethod(jIterator, nextMid);
/* do something with jItem */
}

What is the best way to pass a jbyteArray from (objective-c) JNI to Java?

I'm currently retrieving image data from an iSight camera and I'd like to hand it over to Java for processing. I originally tried to put the data in a jbyteArray and return the jbyteArray. This works once per process. Calling the native function a second time will result in an invalid memory access.
Since I'm working with objective-c and Cocoa, I must use the JNF_COCOA_ENTER(...) and JNF_COCOA_EXIT(...) functions. Sadly, if I can't return the jbyteArray because doing so would result in JNF_COCOA_EXIT(...) not getting called. It was suggested to use a direct ByteBuffer to pass the data from JNI land to java land. Unfortunately, all the resources and references I've been using don't outline this straightforwardly enough for my brain to comprehend. I deeply apologize if this is a "duh" moment or has already been asked (I've searched with no luck), but...
1) What is the most efficient way to bring this image data to Java?
2) How should I use the ByteBuffer class to accomplish this? (if relevant)
Thanks!
Code might be helpful here. It's not as difficult as you might imagine once you get an idea of what those macros do, and how to separate the java and cocoa concerns from each other.
What you need to remember is that JNF_COCOA_ENTER and JNF_COCOA_EXIT do two main things for you:
They establish a local scope (so variables defined inside are not available outside - this is to help you not do "dumb" things with variables you shouldn't be touching) and also to set up an auto-release pool, so cocoa objects which are "autoreleased" inside that scope will be gone when the scope disappears. (this is part of why the local scope is useful/helpful) They'll also do some exception parsing so that you can catch Cocoa exceptions in Java.
That said, the following code IS LEGAL, you just need to be quite careful about managing memory access and ownership of data. Avoid mixing Java ownership and Objective-C ownership if possible, or else have your object manage the ownership, and clean up when the java object is GCed.
jbyteArray bytes;
JNF_COCOA_ENTER(env);
// Assign and full the bytes array here, doing any requisite transformations.
// Remember to COPY any data out of COCOA objects, as references will be dead soon!
JNF_COCOA_EXIT(env);
return bytes;
Using Java Objects from C is complex, but not unworkable. Still, the number of method calls is non-trivial, and the jump back and fourth is time consuming, so if you'll be calling this method often, or it's time critical, stick with the basic types as much as possible.
If you need to push data from Cocoa to Java from a delegate, things are a little bit more complicated, but not unworkable. Here's a segment from a project I manage called QTCubed, which does exactly that. didOutputVideoFrame is the delegate method, and this object MUST be initialized with the target Java Object Reference, and Java Environment from a JNI called method. Once initialized, it's then set as a delegate object, and receives updates from the camera.
#implementation QTKitCaptureDecompressedVideoOutput
- (QTKitCaptureDecompressedVideoOutput *)initWithEnv:(JNIEnv *) env javaObject:(jobject) javaObjectRef {
[super init];
// Save a reference to the VM
(*env)->GetJavaVM(env,&g_vm);
// Create a global reference to this object so we can access it later
objectRef = (*env)->NewGlobalRef(env,javaObjectRef);
return self;
}
- (void)captureOutput:(QTCaptureOutput *)captureOutput didOutputVideoFrame:(CVImageBufferRef)videoFrame withSampleBuffer:(QTSampleBuffer *)sampleBuffer fromConnection:(QTCaptureConnection *)connection {
// Move into Java to deliver the data
JNIEnv *env;
(*g_vm)->AttachCurrentThread (g_vm, (void **) &env, NULL);
void * rawData = [sampleBuffer bytesForAllSamples];
int length = [sampleBuffer lengthForAllSamples];
QTFormatDescription * formatDescription = [sampleBuffer formatDescription];
QTTime duration = [sampleBuffer duration];
float frameDuration = duration.timeValue/duration.timeScale;
float fps = 1/frameDuration;
jint format = [formatDescription formatType];
NSValue * pixelSize = [formatDescription attributeForKey:QTFormatDescriptionVideoEncodedPixelsSizeAttribute];
NSSize size = [pixelSize sizeValue];
jint width = size.width;
jint height = size.height;
//NSLog(#"Outputting frame sized %d x %d of length %d with format: %#x",width,height,length,format);
switch (format) {
// 8 bit codecs
case kCVPixelFormatType_1Monochrome:
case kCVPixelFormatType_2Indexed:
case kCVPixelFormatType_4Indexed:
case kCVPixelFormatType_8Indexed:
case kCVPixelFormatType_1IndexedGray_WhiteIsZero:
case kCVPixelFormatType_2IndexedGray_WhiteIsZero:
case kCVPixelFormatType_4IndexedGray_WhiteIsZero:
case kCVPixelFormatType_8IndexedGray_WhiteIsZero:
case kCVPixelFormatType_422YpCbCr8:
case kCVPixelFormatType_4444YpCbCrA8:
case kCVPixelFormatType_4444YpCbCrA8R:
case kCVPixelFormatType_444YpCbCr8:
case kCVPixelFormatType_420YpCbCr8Planar:
case kCVPixelFormatType_422YpCbCr_4A_8BiPlanar:
case kCVPixelFormatType_24RGB:
case kCVPixelFormatType_24BGR:
default:
{
// Re-use the existing array if possible
if (byteFrameData == nil || (*env)->GetArrayLength(env,byteFrameData) < length) {
// Clean up the previously allocated global reference
if (byteFrameData != nil) {
(*env)->DeleteGlobalRef(env,byteFrameData);
byteFrameData = nil;
}
// Create an appropriately sized byte array to hold the data
byteFrameData = (*env)->NewGlobalRef(env,(*env)->NewByteArray(env,length));
}
if (byteFrameData) {
// Copy the raw data into the byteArray
(*env)->SetByteArrayRegion(env,byteFrameData,0,length,rawData);
// Get the class reference for our object
jclass classRef = (*env)->GetObjectClass(env,objectRef);
// Get the pushFrame methodId
jmethodID methodId = (*env)->GetMethodID(env,classRef,"pushFrame","([BIIIF)V");
// Call pushFrame with the byte array
(*env)->CallVoidMethod(env,objectRef,methodId,byteFrameData,format,width,height,fps);
}
break;
}
// 16 bit (short) storage of values
case kCVPixelFormatType_16BE555:
case kCVPixelFormatType_16LE555:
case kCVPixelFormatType_16LE5551:
case kCVPixelFormatType_16BE565:
case kCVPixelFormatType_16LE565:
case kCVPixelFormatType_16Gray:
case kCVPixelFormatType_422YpCbCr16:
case kCVPixelFormatType_422YpCbCr10:
case kCVPixelFormatType_444YpCbCr10:
{
// Re-use the existing array if possible
if (shortFrameData == nil || (*env)->GetArrayLength(env,shortFrameData) < length/2) {
// Clean up the previously allocated global reference
if (shortFrameData != nil) {
(*env)->DeleteGlobalRef(env,shortFrameData);
shortFrameData = nil;
}
// Create an appropriately sized byte array to hold the data
shortFrameData = (*env)->NewGlobalRef(env,(*env)->NewShortArray(env,length/2));
}
if (shortFrameData) {
// Copy the raw data into the byteArray
(*env)->SetShortArrayRegion(env,shortFrameData,0,length/2,rawData);
// Get the class reference for our object
jclass classRef = (*env)->GetObjectClass(env,objectRef);
// Get the pushFrame methodId
jmethodID methodId = (*env)->GetMethodID(env,classRef,"pushFrame","([SIIIF)V");
// Call pushFrame with the short array
(*env)->CallVoidMethod(env,objectRef,methodId,shortFrameData,format,width,height,fps);
}
break;
}
// 32 bit (int) storage of values
case kCVPixelFormatType_32ABGR:
case kCVPixelFormatType_32AlphaGray:
case kCVPixelFormatType_32ARGB:
case kCVPixelFormatType_32BGRA:
case kCVPixelFormatType_32RGBA:
{
// Re-use the existing array if possible
if (intFrameData == nil || (*env)->GetArrayLength(env,intFrameData) < length/4) {
// Clean up the previously allocated global reference
if (intFrameData != nil) {
(*env)->DeleteGlobalRef(env,intFrameData);
intFrameData = nil;
}
// Create an appropriately sized byte array to hold the data
intFrameData = (*env)->NewGlobalRef(env,(*env)->NewIntArray(env,length/4));
}
if (intFrameData) {
// Copy the raw data into the byteArray
(*env)->SetByteArrayRegion(env,intFrameData,0,length/4,rawData);
// Get the class reference for our object
jclass classRef = (*env)->GetObjectClass(env,objectRef);
// Get the pushFrame methodId
jmethodID methodId = (*env)->GetMethodID(env,classRef,"pushFrame","([IIIIF)V");
// Call pushFrame with the int array
(*env)->CallVoidMethod(env,objectRef,methodId,intFrameData,format,width,height,fps);
}
break;
}
}
// Detatch from Java
(*g_vm)->DetachCurrentThread (g_vm);
}
/* Clean up java references so they can be properly GCed in java */
- (void) dealloc {
// Attach to java so we can release references
JNIEnv *env;
(*g_vm)->AttachCurrentThread (g_vm, (void **) &env, NULL);
// release the references we hold
if (objectRef != nil) {
(*env)->DeleteGlobalRef(env,objectRef);
objectRef = nil;
}
if (byteFrameData != nil) {
(*env)->DeleteGlobalRef(env,byteFrameData);
byteFrameData = nil;
}
if (shortFrameData != nil) {
(*env)->DeleteGlobalRef(env,shortFrameData);
shortFrameData = nil;
}
if (intFrameData != nil) {
(*env)->DeleteGlobalRef(env,intFrameData);
intFrameData = nil;
}
// Detatch from Java
(*g_vm)->DetachCurrentThread (g_vm);
g_vm = nil;
[super dealloc];
}
#end
Here's the method signature on the java side for the java object reference that is passed in:
protected void pushFrame(byte[] frameData, int formatInt, int width, int height, float frameRate);
protected void pushFrame(short[] frameData, int formatInt, int width, int height, float frameRate);
protected void pushFrame(int[] frameData, int formatInt, int width, int height, float frameRate);
Once you have a running application, USE INSTRUMENTS TO MAKE SURE YOU'RE DISPOSING OF REFERENCES PROPERLY!

Categories