Facial Expression recognition on Android Java using Tensorflow Lite - java

I am working on facial expression recognition using deep learning algorithm i.e CNN, to identify user's emotions like happy, sad, anger etc. I have trained and tested it in python using pre-trained VGG-16 model altering top 3 layers to train my test images,To speed up the training process i have used Tensorflow. The test accuracy is 62%. I have saved architecture and weights of my model in train_model.h5 file.
Now i have to implement it on Android phone. For that i have used Tensorflow-Lite as it is suitable for android phone. So i had converted my .h5 file in .tflite file using Tensorflow lite converter method.
This is what i did for converting:
from tensorflow.contrib import lite
converter=lite.TFLiteConverter.from_keras_model_file
("train_model.h5")
tflite_model = converter.convert()
open ("model.tflite" , "wb") .write(tflite_model)
I successfully got the tflite file.
Coming to android part i have chose Java language to load tflite file and predict the emotions of new image. I have gone through the example of "image classification" given in the Tensorflow-lite website, but i am confused how to use it. I don't know how to read tflite and use it to predict the output of new image and display the result on android app. Please help me with some good resources with explanation

Here is a good blog post on how to use an image classification TFLite model on Android:
https://medium.com/tensorflow/using-tensorflow-lite-on-android-9bbc9cb7d69d
How you'll run inferences depends largely on how the model was built and what inputs it expects. If the approach in the blog post above doesn't work, you'll have to manually compose the tensor to feed to the model. The code in this codelab does just that.
Another option to consider is the face detection API in ML Kit. It does some of what you are looking for (though not all of it) by detecting the curvature of the smile.

Related

Android Studio: Could I load my Yolov5 model (.pt file) in my phone without converting to TFLite and torchscript?

I search on the Internet, most of people use TFLite and Torchscript for object detection. But, I think my phone has GPU, it is not necessarily to convert to other version for reducing the hardware pressure. So can we just load the .pt file directly? And how to do that? Seem that I cannot just change the code as below.
mModule = LiteModuleLoader.load(MainActivity.assetFilePath(getApplicationContext(), "best.pt"));
Here is some detail about the PyTorch package for Android. Seem that it not yet support android to run the .pt file.
[1]https://pytorch.org/mobile/android/

How can I do face recognition of two images using opencv java in android

I'm trying to create a face recognition application in android studio in java language. I spent considerable amount of time, searching for step by step guides for achieving this function. I found many tutorials for python language and i couldn't find any proper tutorial or atleast a video tutorial for java.
Can someone please provide me with a proper link with the steps to do face recognition with OpenCv in android using java language.
I'm new to OpenCv, so i have no big idea on its functions.
Your help is highly appreciated
Update 1 ::
I'm trying to capture the image of the user and then cross check the taken image with an image that is already available in the gallery.
I want to know if both are the same so that i can allow the user to use the application.
I actually made a Java program that utilised OpenCV for facial recognition earlier this year.
For me it was an utter nightmare. Guides to doing this on Java are basically nonexistent and the documentation for Java OpenCV is very poor.
What I ended up doing to get to grips with OpenCV was I used PyImageSearch to learn how OpenCV works and then just trawled through the OpenCV javadocs until I found the most similar classes to what was being used in PyImageSearch.
There are a few guides for OpenCV recognition using Java (I'll have a look through my notes when I get a chance and see if I saved any links for my own reference) but none of them were for what I needed.
If you edit your post with some more information on the recognition you're trying to do (video or jpeg or png?; frontal faces or side faces?; specific faces or faces in general?) I may be able to help a little more.
The Youtube link that #Varma posted might help you get to grips with OpenCV as a whole, but that series doesn't seem to cover facial recognition.
EDIT #1
Okay, recognising who someone is via an image is definitely doable with OpenCV, but it's not something I've done. Here's a PyImageSearch page on how to do it in Python. The methodology for doing it in Java with OpenCV will most likely be more or less the same even thought the syntax will - of course - make it look very different (should mostly be a matter of finding the equivalent Java classes and methods).
If it proves difficult, there are alternatives to OpenCV you could use, like these or Google's API. Keep in mind though that facial recognition is not very secure and does come with some controversy.

Image Recognition using OpenCv and android

I want to work on making an android app by integrating OpenCV with android Studio. I have a set of 2D hardcopy card images that i want to save as templates with in the app. Then, using the app, when i place my camera on any of the cards, the app should search the directory which contain the templates and look for match and provide feedback if a match is found. If anyone can guide on how to achieve this, it will be highly appreciated.
Also, if not OpenCV, then which SDK or tool should be preferred ?
The question is a general one, so the answer will be general as well, and will make assumptions about what you'd like to accomplish with your application.
Android Studio with OpenCV is probably a reasonable stack to use.
Presuming the library has more than a trivial number of images, you'll probably want to extract matching information for each image in your library in an offline process (on your code-development machine). So for instance, you would configure your development machine with a SQLite database driver and OpenCV, construct a program that extracts feature points and saves those to your a SQLite database. That file can then be loaded into the Android APK assets, and it would be ready upon the application's first use. The application would take frames from the camera and compare those with each item in the database, looking for matches.

ZXing on Android PDF417

Was anybody able to read the PDF417 barcode with use of the ZXing library on the Android OS? They are supporting this - and according to their page it is in 'alpha' stage.
We are not looking for perfect solution - since the PDF417 is pretty complex and needs a very good camera with auto-focus, we can accept that it will be working only on few pre-selected high end devices.
We have tried also the Barcode Scanner + available on the Android Market - it has the PDF417 option in the settings, but whatever we read it always fails.
We were looking also for commercial SDK, also here on stackoverflow, but with no luck.
Any help is appreciated.
Kind Regards,
STeN
It really depends on what you expect. Simple PDF417 reads pretty instantly, like... this or this.
This will never be scanned.
Borderline is stuff that is small or moderately complex: example 1 and example 2.
I can read the first but not the second, even though the first is denser -- size helps.
Make sure to enable PDF 417 decoding; it's off by default
Quiet zone (white space around the code) is required
Focus and light help a lot
You can try PDF417.mobi SDK. It should work on low-end phones if equipped with auto-focus camera. It's a commercial library, but free for developers and non-commercial purposes.
You can try the demo here or play with code directly from GitHub.
Official web site is here http://pdf417.mobi/
Disclaimer: I'm part of the team working on PDF417.mobi
Have used , It can scan PDF417 format. Make sure you give a try with a Gadget containing Auto Focus camera. Have tried It on Samsung Galaxy Tab it works like a charm.
Zxing's solution did not work for me. I used DataSymbol Decoder (turn on 2d codes, by default they are off) on my samsung charge. In less than a second I captured my drivers license...
I got similar results as described by #sean-owen in that only the simple PDF417 were being read. It feels like the ZXing library doesn't have the same error correction for PDF417 that it does for QR Codes. However, with user assistance we were able to eliminate noise and create an artificial quiet zone by:
require the user to hold the phone in landscape mode (this maximizes the pixels captured from the camera, even in 640x480 mode)
require the user to fit the barcode inside a 50:18 clipping rectangle (this ratio seems to best fit the US Driver's License and such a clipping rectangle will empower the user to clip away most of the noise)
allow the user control focus, tilt distortions
By following the above, even some of the notoriously difficult PDF417 images can be scanned.
Google's ML KIT Barcode Scanning which is part of google's Mobile Vision library lists support for PDF-417 Barcodes.
It automatically parses QR Codes, Data Matrix, PDF-417, and Aztec values, for the following supported formats:
URL
Contact information (VCARD, etc.)
Calendar event
Email
Phone
SMS
ISBN
WiFi
Geo-location (latitude and longitude)
AAMVA driver license/ID
Review the Getting Started Page or clone GIT project to get started.

Converting audio files(.3gp) to video with Album cover and uploading to YouTube

I have an audio file in .3gp format on my Android device which I wish
to upload to YouTube. I know that YouTube is a video upload site and
that I need to convert this sound file to video.
I just want an image to display all the time the audio is playing.
Google tells me there are number of tools that can help me. But I want
to do this via java code from my Android device.
Please help.
Thanks.
There are tools such as FFMPEG available for free that allow you to, essentially, mix and convert heterogenous streams. That is you can add a bitmap to a video, create video from slide shows and then add sound etc. (See a related question I asked here).
These programs can be executed from within java applications by making Runtime.exec(..) calls.
Sun has an example for stitching multiple JPEGs together into a movie, you can find it here. You should be able to take this example, (its fairly robust), and add what you need to it.
I recommend looking into the Java Media Framework (FAQ)
You can find a vast collection of sample applets/code at the Sun Solutions page. You can find the API on this page. I do hope this is compatible on the Android platform, as I haven't had any personal experience developing for it. But it might be a good place to start.

Categories