I am making a camera app.
There has been a host of issues getting orientation right because some phones don't write EXIF orientation data. Because of this, I get the bitmap, save it (since I don't think I should read EXIF data from the byte[]), then rotate the bitmap, then save over the original file.
It works, and the the orientation issue is fixed. The problem is its taking me 25 seconds or longer on some of the top of the line phones. Can you advise why my code is so slow or advise me on how I can find the problem?
Note: If I only save the image once (i.e. with the wrong orientation) it only takes a couple seconds.
Here is my image capture callback:
private Camera.PictureCallback pictureCallback = new Camera.PictureCallback()
{
#Override
public void onPictureTaken(byte[] data, Camera camera)
{
File pictureFile = getOutputMediaFile(MEDIA_TYPE_IMAGE);
if (pictureFile == null){
Log.d("EditPhotoFragment", "Error creating media file, check storage permissions");
return;
}
try
{
FileOutputStream fos = new FileOutputStream(pictureFile);
fos.write(data);
fos.flush();
fos.close();
orientPicture(pictureFile);
//TODO async
galleryAddPic(pictureFile);
} catch (FileNotFoundException e) {
Log.d("EditPhotoFragment", "File not found: " + e.getMessage());
} catch (IOException e) {
Log.d("EditPhotoFragment", "Error accessing file: " + e.getMessage());
}
}
};
And here is where I orient and resave the image:
private Bitmap orientPicture(File pictureFile)
{
Bitmap bitmap = BitmapFactory.decodeFile(pictureFile.getAbsolutePath());
Uri uri = Uri.parse(pictureFile.toString());
ExifInterface exif = null;
try{
exif = new ExifInterface(uri.getPath());
}catch (Exception e)
{
e.printStackTrace();
}
int exifOrientation = exif.getAttributeInt(ExifInterface.TAG_ORIENTATION, ExifInterface.ORIENTATION_NORMAL);
Matrix matrix = new Matrix();
int rotationInDegrees = 0;
//If the orientation tag is missing need to manually rotate it by the 'default' camera
//orientation and if its front facing need to do 360 - the camera rotation value
if(exifOrientation == ExifInterface.ORIENTATION_UNDEFINED)//All phones in this bucket can go fuck themselves
{
Camera.CameraInfo info = new Camera.CameraInfo();
if(_cameraPreview.isBackFacing())
{
Camera.getCameraInfo(Camera.CameraInfo.CAMERA_FACING_BACK, info);
}else
{
Camera.getCameraInfo(Camera.CameraInfo.CAMERA_FACING_FRONT, info);
}
rotationInDegrees = info.orientation; //set it to the default camera orientation
}else
{
rotationInDegrees = exifToDegrees(exifOrientation);
if(!_cameraPreview.isBackFacing())//handle mirroring of front camera
{
Camera.CameraInfo info = new Camera.CameraInfo();
Camera.getCameraInfo(Camera.CameraInfo.CAMERA_FACING_BACK, info);
rotationInDegrees = 360 - rotationInDegrees; //For the front camera doing 360 - gets the right orientation
}
}
matrix.preRotate(rotationInDegrees);
if(!_cameraPreview.isBackFacing())//mirror it
{
matrix.preScale(1,-1);
}
Bitmap adjustedBitmap = Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(), bitmap.getHeight(), matrix, true);
//This saves the proper image over top if it
try
{
FileOutputStream fos = new FileOutputStream(pictureFile);
ByteArrayOutputStream stream = new ByteArrayOutputStream();
adjustedBitmap.compress(Bitmap.CompressFormat.PNG, 100, stream);
byte[] byteArray = stream.toByteArray();
fos.write(byteArray);
fos.flush();
fos.close();
}catch(Exception e)
{
e.printStackTrace();
}
return adjustedBitmap;
}
SOLUTION
As advised I SHOULD read the exif data which I was able to do without needing an external library thanks to this:
https://stackoverflow.com/a/13581324/3324388
Can you advise why my code is so slow
Perhaps among other reasons, you are writing the image to a file, re-reading the same image from the file, doing the transform, then writing the image back out to a file. That is going to take a lot of time.
Note: If I only save the image once (i.e. with the wrong orientation) it only takes a couple seconds.
That's because you are doing a lot less work, including only ~33% of the disk I/O, and disk I/O is going to be slow.
since I don't think I should read EXIF data from the byte[]
My apologies if you were viciously attacked by a byte[] as a young child or something. However, if you want better performance, you are going to have to read the EXIF data out of the existing in-memory copy of the image.
Related
Currently, I'm building a camera app using camera2 api. I record the video and that file is sent to another activity to make sure we can correct orientation and watch the video. Then the altered video or picture is saved to the device.
When I use a still image it works, because I can pull the bitmap image and then resave the image like this:
public String saveImage() {
//Getting a new file name and file path
//Should we delete these images after were done with them?
File newImageFile = getExternalFilesDir(Environment.DIRECTORY_PICTURES);
File newImageFolder = new File(newImageFile, "camera2VideoImage");
if (newImageFolder.exists())
{
newImageFolder.mkdirs();
}
String timestamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date());
String prepend = "Image_" + timestamp + "_";
File imageFile = null;
try {
imageFile = File.createTempFile(prepend, ".jpg", newImageFolder);
} catch (IOException e) {
e.printStackTrace();
}
String newFileName = imageFile.getAbsolutePath();
Bitmap bitmap = ((BitmapDrawable) imageView.getDrawable()).getBitmap();
try {
FileOutputStream fos = new FileOutputStream(newFileName);
bitmap.compress(Bitmap.CompressFormat.JPEG, 100, fos);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
//returning the new file path.
return newFileName;
}
This works, they view the image and have the option to change orientation on the image in case it is messed up and then we resave.
I'm trying to do the same with a video, but not sure how to pull the video from the textureView so I can save again with the corrected orientation as the front facing camera is upside down sometimes depending on the phone.
The recording save method:
public String saveVideo() {
//Getting a new file name and file path
//Should we delete these images after were done with them?
File newVideoFile = getExternalFilesDir(Environment.DIRECTORY_MOVIES);
File newVideoFolder = new File(newVideoFile, "camera2VideoImage");
if (newVideoFolder.exists())
{
newVideoFolder.mkdirs();
}
String timestamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date());
String prepend = "Video_" + timestamp + "_";
File videoFile = null;
try {
videoFile = File.createTempFile(prepend, ".jpg", newVideoFolder);
} catch (IOException e) {
e.printStackTrace();
}
String newFileName = videoFile.getAbsolutePath();
textureView.getBitmap();
FileOutputStream fos = new FileOutputStream(newFileName);
//returning the new file path.
return newFileName;
}
How do I get the corrected video from the texture view and then save it, like the Image method above?
This is not a recommended way; the cost of getBitmap on TextureView is high, and not likely suitable for 30fps video recording.
But if you really want to try, you need to feed the Bitmap to a MediaRecorder; you may be able to use MediaRecorder.getSurface() for that, then lock the Surface Canvas and draw your Bitmap into it.
However, I would not be surprised if the performance is poor, or if the MediaRecorder Surface won't accept RGB Bitmaps.
In general, you want to connect the camera API directly to the MediaRecorder or MediaCodec Surface. If you really need to edit frames in the middle, using the GPU is generally the most performant option, though it's a lot of code to write to do that.
I ended up changing the configuration on the Media Recorder, looks like I was trying to correct other orientation problems and caused this. So under my set up mediaRecorder I did this:
private void setupMediaRecorder() throws IOException {
mMediaRecorder.setVideoSource(MediaRecorder.VideoSource.SURFACE);
mMediaRecorder.setAudioSource(MediaRecorder.AudioSource.MIC);
mMediaRecorder.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4);
mMediaRecorder.setOutputFile(mVideoFileName);
mMediaRecorder.setVideoEncodingBitRate(100000000);
mMediaRecorder.setVideoFrameRate(30);
mMediaRecorder.setVideoSize(mVideoSize.getWidth(),mVideoSize.getHeight());
mMediaRecorder.setVideoEncoder(MediaRecorder.VideoEncoder.H264);
mMediaRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AAC);
if (cameraCheck.contains("1") && mTotalRotation == 180){
//This corrects problems for the front facing camera when recording, the default settings work, so we do nothing here.
}else {
mMediaRecorder.setOrientationHint(mTotalRotation);
}
mMediaRecorder.prepare();
}
This checks if it is the front-facing camera and the phone is positioned at 180. If it is do nothing for correction, else use the correction.
I would like to detect a barcode within a captured image. I capture an image using android's camera2. Following this, the image's metadata is retrieved and the image is saved to the device. The metadata is all passed along to the next activity, which is where the application attempts to detect a barcode.
This next activity creates a byte[] from the File saved previously. Next, the relevant FirebaseVision objects are created using the data passed with the intent. Finally, the application attempts to call the detectInImage() method, where an error is thrown:
"java.lang.IllegalArgumentException: Invalid image data size."
I suspect this is from the captured image being too large, however I cannot seem to figure out how to capture a smaller image, and I also cannot find anything in the reference documentation regarding the maximum size allowed. Information regarding this error and how to solve it would be very much appreciated. Below is what I believe to be the relevant code.
private final ImageReader.OnImageAvailableListener onImageAvailableListener
= new ImageReader.OnImageAvailableListener() {
#Override
public void onImageAvailable(ImageReader imageReader) {
try{
// Semaphore ensures date is recorded before starting next activity
storeData.acquire();
Image resultImg = imageReader.acquireNextImage(); // Image from camera
imgWidth = resultImg.getWidth();
imgHeight = resultImg.getHeight();
ByteBuffer buffer = resultImg.getPlanes()[0].getBuffer();
data = new byte[buffer.remaining()]; // Byte array with the images data
buffer.get(data);
String timeStamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date());
// Note: mediaFile directs to Pictures/"ThisProject" folder
File media = new File(mediaFile.getPath() +
File.separator + "IMG_" + timeStamp + ".jpg");
// Saving the image
FileOutputStream fos = null;
try {
fos = new FileOutputStream(media);
fos.write(data);
uri = Uri.fromFile(media);
} catch (IOException e) {
Log.e(TAG, e.getMessage());
} finally {
if (fos != null) {
try {
fos.close();
} catch (IOException e) {
Log.e(TAG, e.getMessage());
}
}
}
resultImg.close();
} catch (InterruptedException e) {
Log.e(TAG, e.getMessage());
}
storeData.release();
}
};
This essentially retrieves the image height & width, then writes it to a file.
The data sent to the next activity consists of the: Image width, Image height, Image rotation, and the Uri directing to the file.
Using this, I try to detect a barcode using Firebase ML Kit:
// uri is the uri referencing the saved image
File f = new File(uri.getPath());
data = new byte[(int) f.length()];
try{
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(f));
DataInputStream dis = new DataInputStream(bis);
dis.readFully(data);
} catch (IOException e) {
Log.e(TAG, e.getMessage());
}
FirebaseVisionBarcodeDetectorOptions options = new FirebaseVisionBarcodeDetectorOptions.Builder().setBarcodeFormats(
FirebaseVisionBarcode.FORMAT_QR_CODE,
FirebaseVisionBarcode.FORMAT_DATA_MATRIX
).build();
FirebaseVisionBarcodeDetector detector = FirebaseVision.getInstance().getVisionBarcodeDetector(options);
FirebaseVisionImage image;
int rotationResult;
switch (imgRotation) {
case 0: {
rotationResult = FirebaseVisionImageMetadata.ROTATION_0;
break;
}
case 90: {
rotationResult = FirebaseVisionImageMetadata.ROTATION_90;
break;
}
case 180: {
rotationResult = FirebaseVisionImageMetadata.ROTATION_180;
break;
}
case 270: {
rotationResult = FirebaseVisionImageMetadata.ROTATION_270;
break;
}
default: {
rotationResult = FirebaseVisionImageMetadata.ROTATION_0;
break;
}
}
FirebaseVisionImageMetadata metadata = new FirebaseVisionImageMetadata.Builder()
.setWidth(imgWidth)
.setHeight(imgHeight)
.setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21)
.setRotation(rotationResult)
.build();
image = FirebaseVisionImage.fromByteArray(data, metadata);
Task<List<FirebaseVisionBarcode>> result = detector.detectInImage(image)
A few things.
Your image format should not be NV21 if you use camera2. See here for all camera2 supported image formats:
https://developer.android.com/reference/android/media/Image#getFormat()
Your byte[] is not NV21 and you specified IMAGE_FORMAT_NV21 and led to the error
Most intuitive integration with camera2 is like below:
Specify JPEG format when you instantiate the ImageReader.
onImageAvailable will give you back an android.media.Image and you can directly use FirebaseVisionImage.fromMediaImage(...) to create a FirebaseVisionImage. (You can find how to compute the rotation info from official doc here)
If you must do two Activities, then you need to work around the fact that android.media.Image is not Parcelable. I'd suggest you convert it to Bitmap first which is Parcelable and you can directly set it as an Intent extra (Up to you. Just thinking from end user's perspective, it's non-common to see the barcode being saved to my image gallery.
So you might want to consider skipping the step of saving it to file). Later, in your 2nd Activity, you can use FirebaseVisionImage.fromBitmap(...).
My Requirement is, after opening the application there will be a button which will open the camera application and takes picture of the two barcodes then that bitmap will be given to my application and the barcode decoder will decode that two barcodes and displays result !!
Is it possible?
if yes then how?
Sure it is possible.
There are a lot of barcode decoding libraries like https://github.com/zxing/zxing/
Integrate the library, follow its tutorial and pass the bitmap to it. Then after decoding display the result.
You Can, Try MultiFormatReader from ZXing Librery
Sample Code
try
{
InputStream inputStream = activity.getContentResolver().openInputStream(uri);
Bitmap bitmap = BitmapFactory.decodeStream(inputStream);
if (bitmap == null)
{
Log.e(TAG, "uri is not a bitmap," + uri.toString());
return null;
}
int width = bitmap.getWidth(), height = bitmap.getHeight();
int[] pixels = new int[width * height];
bitmap.getPixels(pixels, 0, width, 0, 0, width, height);
bitmap.recycle();
bitmap = null;
RGBLuminanceSource source = new RGBLuminanceSource(width, height, pixels);
BinaryBitmap bBitmap = new BinaryBitmap(new HybridBinarizer(source));
MultiFormatReader reader = new MultiFormatReader();
try
{
Result result = reader.decode(bBitmap);
return result;
}
catch (NotFoundException e)
{
Log.e(TAG, "decode exception", e);
return null;
}
}
catch (FileNotFoundException e)
{
Log.e(TAG, "can not open file" + uri.toString(), e);
return null;
}
I have been augmenting the QR scanning library Zxing to save a photo instantly upon scan. I was advised to do so within the onPreviewFrame method within PreviewCallback.java as thus:
public void onPreviewFrame(byte[] data, Camera camera) {
Point cameraResolution = configManager.getCameraResolution();
Handler thePreviewHandler = previewHandler;
YuvImage im = new YuvImage(data, ImageFormat.NV21, 1200,
800, null);
Rect r = new Rect(0, 0, 1200, 800);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
im.compressToJpeg(r, 50, baos);
try {
FileOutputStream output = new FileOutputStream("/sdcard/test_jpg.jpg");
output.write(baos.toByteArray());
output.flush();
output.close();
System.out.println("Attempting to save file");
System.out.println(data);
} catch (FileNotFoundException e) {
System.out.println("Saving to file failed");
} catch (IOException e) {
System.out.println("Saving to file failed");
}
if (cameraResolution != null && thePreviewHandler != null) {
Message message = thePreviewHandler.obtainMessage(previewMessage, cameraResolution.x,
cameraResolution.y, data);
message.sendToTarget();
previewHandler = null;
} else {
Log.d(TAG, "Got preview callback, but no handler or resolution available");
}}
The result of running this code is a corrupt image at the set file directory. I believe this is due to the code being run every frame. Is there a way to limit this to every second or so if that will allow the full image to save, or is there a method I can use to cause the image to only save upon completed scan.
I have a less favourable working alternative, in that I can successfully save the black and white image that is shown upon scan; colour is the preferable option of course.
Update: Code changed to (in theory) accommodate camera resolution on any device. Image is still corrupt.
public void onPreviewFrame(byte[] data, Camera camera) {
Point cameraResolution = configManager.getCameraResolution();
Handler thePreviewHandler = previewHandler;
android.hardware.Camera.Parameters parameters = camera.getParameters();
android.hardware.Camera.Size size = parameters.getPictureSize();
int height = size.height;
int width = size.width;
YuvImage im = new YuvImage(data, ImageFormat.NV21, width,
height, null);
Rect r = new Rect(0, 0, width, height);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
im.compressToJpeg(r, 50, baos);
try {
FileOutputStream output = new FileOutputStream("/sdcard/test_jpg.jpg");
output.write(baos.toByteArray());
output.flush();
output.close();
System.out.println("Attempting to save file");
System.out.println(data);
} catch (FileNotFoundException e) {
System.out.println("Saving to file failed");
} catch (IOException e) {
System.out.println("Saving to file failed");
}
if (cameraResolution != null && thePreviewHandler != null) {
Message message = thePreviewHandler.obtainMessage(previewMessage, cameraResolution.x,
cameraResolution.y, data);
message.sendToTarget();
previewHandler = null;
} else {
Log.d(TAG, "Got preview callback, but no handler or resolution available");
}}
Having checked the width and height variable are correctly set to the Nexus 7's camera width and height of 1280x960. I am confident the issue comes from attempting to save the image every frame, as "Attempting to save to file" appears somewhat rapidly within the logcat; several times a second. It may also be worth noting that the corrupt image saved is square(ish).
Thanks in advance.
1200 x 800? are you sure this is your preview size? check your parameters. Probably it's 1280 x 720
For some reason, my camera app saves all photos rotated 90 degrees (pictures only look right when taken with camera on landscape mode) I believe onPictureTaken should rotate photos automatically but I read there is a problem with Samsung devices (I haven't been able to test it on another brand so I don't know if it's the case). This is my code:
public void onPictureTaken(byte[] data, Camera camera) {
// Generate file name
FileOutputStream outStream = null;
outStream = new FileOutputStream(filePath);
outStream.write(data);
outStream.close();
I was thinking it could be fixed by checking the orientation and rotating the byte array but there must be a more straightforward way to do it since handling byte arrays is a pain.
How can I make sure photos are saved matching the orientation they were taken?
Try something like this:
int orientation = Exif.getOrientation(data);
Log.d("#", "onPictureTaken().orientation = " + orientation);
if(orientation != 0) {
Bitmap bmpSrc = BitmapFactory.decodeByteArray(data, 0, data.length);
Bitmap bmpRotated = CameraUtil.rotate(bmpSrc, orientation);
bmpSrc.recycle();
try {
FileOutputStream localFileOutputStream = new FileOutputStream(filePath);
bmpRotated.compress(Bitmap.CompressFormat.JPEG, 90,localFileOutputStream);
localFileOutputStream.flush();
localFileOutputStream.close();
bmpRotated.recycle();
}
catch (FileNotFoundException e)
{
e.printStackTrace();
}
catch (IOException e)
{
e.printStackTrace();
}
} else {
try {
FileOutputStream localFileOutputStream = new FileOutputStream(filePath);
localFileOutputStream.write(data);
localFileOutputStream.flush();
localFileOutputStream.close();
} catch (IOException localIOException)
{
Log.e("#",localIOException.getMessage());
}
}
There are different ways to get your image, as data, as stream, as file, also it is different for camera and gallery and other apps. For each of them you have another way to access the orientation tag. Once you have the orientation, you can rotate your image.
You - or for that matter everyone - should really get a Nexus, they are nice and rotate the image for you and set orientation to 0, while the lazy ones like Samsung just store the image and set the orientation tag. ;)