I'm trying to convert several pixels from a YUV (nv21) image format to RGB format(yes, just some pixels, not the whole image because the run time constrains)
Currently I'm using the decodeyuv420SP function from internet:
static public void decodeYUV420SP(int[] rgb, byte[] yuv420sp, int width, int height) {
final int frameSize = width * height;
for (int j = 0, yp = 0; j < height; j++) {
int uvp = frameSize + (j >> 1) * width, u = 0, v = 0;
for (int i = 0; i < width; i++, yp++) {
int y = (0xff & ((int) yuv420sp[yp])) - 16;
if (y < 0) y = 0;
if ((i & 1) == 0) {
v = (0xff & yuv420sp[uvp++]) - 128;
u = (0xff & yuv420sp[uvp++]) - 128;
}
int y1192 = 1192 * y;
int r = (y1192 + 1634 * v);
int g = (y1192 - 833 * v - 400 * u);
int b = (y1192 + 2066 * u);
if (r < 0) r = 0; else if (r > 262143) r = 262143;
if (g < 0) g = 0; else if (g > 262143) g = 262143;
if (b < 0) b = 0; else if (b > 262143) b = 262143;
rgb[yp] = 0xff000000 | ((r << 6) & 0xff0000) | ((g >> 2) & 0xff00) | ((b >> 10) & 0xff);
}
}
}
Now I want to make some modification s.t it can return the R,G,B values of a specific pixel(X,Y),
but I can't understand what exactly it does, looks like a YUV pixel corresponds to more than one RGB pixel. Can someone help me with this issue?
Thank you!
This is just quick and dirty, but I think it will work. All I'm doing is removing the loops and replacing them with the given X,Y (represented herein as "row" and "column" because we're using "y" to mean something else).
int uvp = frameSize + (row >> 1) * width, u = 0, v = 0;
int yp = row*width + column;
int y = (0xff & ((int) yuv420sp[yp])) - 16;
if (y < 0) y = 0;
if ((column & 1) == 0) {
v = (0xff & yuv420sp[uvp++]) - 128;
u = (0xff & yuv420sp[uvp++]) - 128;
}
int y1192 = 1192 * y;
int r = (y1192 + 1634 * v);
int g = (y1192 - 833 * v - 400 * u);
int b = (y1192 + 2066 * u);
if (r < 0) r = 0; else if (r > 262143) r = 262143;
if (g < 0) g = 0; else if (g > 262143) g = 262143;
if (b < 0) b = 0; else if (b > 262143) b = 262143;
result = 0xff000000 | ((r << 6) & 0xff0000) | ((g >> 2) & 0xff00) | ((b >> 10) & 0xff);
Hope it helps.
Related
I want to divide the canvas into 8 vertical black and white Stripes. I wrote a method changeValue to change the value of the color everytime the loop is executed but the canvas is complete white. I think my changeValue method doesn't do as supposed but I can't explain why.
So far I got this:
public class Stripe {
public boolean switch = false;
private void stripes(int[] pixels, int width, int height) {
// TODO set some values here
int counter = 1;
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
int pos = y * width + x;
int r, g, b;
// TODO code for the flag.
r = 255;
g = 255;
b = 255;
for (int i = 0; i < 8; i++) {
if (x < (counter * (width - 1)) / 8) {
r = changeValue();
g = changeValue();
b = changeValue();
}
counter++;
switch ^= true;
}
pixels[pos] = 0xFF000000 | (r << 16) | (g << 8) | b;
}
}
}
public int changeValue() {
if (switch) {
return 255;
} else {
return 0;
}
}
}
I am supposed not to write 8 If statements for the 8 stripes but this for example is the code for the Italian flag which are 3 vertical Stripes but which actually works:
private void flagItalian(int[] pixels, int width, int height) {
// TODO set some values here
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
int pos = y * width + x;
int r, g, b;
// TODO code for the flag.
r = 255;
g = 0;
b = 0;
if (x < 2 * (width - 1) / 3) {
r = 255;
g = 255;
b = 255;
}
if (x < (width - 1) / 3) {
r = 0;
g = 255;
b = 0;
}
pixels[pos] = 0xFF000000 | (r << 16) | (g << 8) | b;
}
}
}
If you need alternating color stripes then you could go for the classic odd/even checks. Instead of you below code block
for (int i = 0; i < 8; i++) {
if (x < (counter * (width - 1)) / 8) {
r = changeValue();
g = changeValue();
b = changeValue();
}
counter++;
switch ^= true;
}
Try replacing with this one
if (x%2) {
r = 255;
g = 255;
b = 255
} else {
r = 0;
g = 0;
b = 0;
}
I am trying to encode a video in java.
I have access to the separate frames as I420 yuv frames (these come from a different part of the program that I cannot change).
I basically have 3 bytebuffers for the different planes of a frame (+ dimensions).
As far as I understand, my format has 1 byte for the y-plane, and half a byte for u and v each, per pixel.
What is the best way to encode these into an mp4 video file?
I have tried with the xuggler API, but I can't seem to find a way to use the yuv frames directly.
Right now, I would convert them to a BufferedImage (TYPE_3BYTE_BGR) first before I can use them with the xuggler api to encode them to a video.
But this creates a huge overhead (I have to convert the yuv data to rgb for each pixel) and is unnecessary, as xuggler encodes them to yuv frames again to store them in a video file? (Not sure about this.)
So is there any easier way to encode raw yuv-frames to a video file directly in java?
Thanks for any pointers.
The way your are planning this seems correct (manual conversion), as long as xuggler didn't re-encode the frame after you.
I have done this conversion with both python and C, the process is still the same as yours (frame by frame, looping the pixels). In Java, this could look like :
public class YUV2RGB
{
public static void convert(int[] argb, byte[] yuv, int width, int height)
{
final int frameSize = width * height;
final int ii = 0;
final int ij = 0;
final int di = +1;
final int dj = +1;
int a = 0;
int y, v, u, r, g, b;
for (int i = 0, ci = ii; i < height; ++i, ci += di)
{
for (int j = 0, cj = ij; j < width; ++j, cj += dj)
{
y = (0xff & ((int) yuv[ci * width + cj]));
v = (0xff & ((int) yuv[frameSize + (ci >> 1) * width
+ (cj & ~1) + 0]));
u = (0xff & ((int) yuv[frameSize + (ci >> 1) * width
+ (cj & ~1) + 1]));
y = y < 16 ? 16 : y;
// METHOD 1 [slower, less accurate]
/*
* r = y + (int) 1.402f * v; g = y - (int) (0.344f * u + 0.714f
* * v); b = y + (int) 1.772f * u; r = r > 255 ? 255 : r < 0 ? 0
* : r; g = g > 255 ? 255 : g < 0 ? 0 : g; b = b > 255 ? 255 : b
* < 0 ? 0 : b; argb[a++] = 0xff000000 | (b<<16) | (g<<8) | r;
*/
// METHOD 2
r = (int) (1.164f * (y - 16) + 1.596f * (v - 128));
g = (int) (1.164f * (y - 16) - 0.813f * (v - 128) - 0.391f * (u - 128));
b = (int) (1.164f * (y - 16) + 2.018f * (u - 128));
r = r < 0 ? 0 : (r > 255 ? 255 : r);
g = g < 0 ? 0 : (g > 255 ? 255 : g);
b = b < 0 ? 0 : (b > 255 ? 255 : b);
argb[a++] = 0xff000000 | (r << 16) | (g << 8) | b;
}
}
}
}
Sample code from https://github.com/jyanik/Mocobar/blob/master/Mocobar/src/com/yanik/mocobar/camera/YUV2RGB.java
I can´t remember the source of this code, probably it was from a question here in SO, but comes in handy here as it´s a version of the above using integer maths, I needed it for an Android project!
//Method from Ketai project! Not mine! See below...
void decodeYUV420SP(int[] rgb, byte[] yuv420sp, int width, int height) {
final int frameSize = width * height;
for (int j = 0, yp = 0; j < height; j++) {
int uvp = frameSize + (j >> 1) * width, u = 0, v = 0;
for (int i = 0; i < width; i++, yp++) {
int y = (0xff & ((int) yuv420sp[yp])) - 16;
if (y < 0)
y = 0;
if ((i & 1) == 0) {
v = (0xff & yuv420sp[uvp++]) - 128;
u = (0xff & yuv420sp[uvp++]) - 128;
}
int y1192 = 1192 * y;
int r = (y1192 + 1634 * v);
int g = (y1192 - 833 * v - 400 * u);
int b = (y1192 + 2066 * u);
if (r < 0) r = 0;
else if (r > 262143)
r = 262143;
if (g < 0) g = 0;
else if (g > 262143)
g = 262143;
if (b < 0) b = 0;
else if (b > 262143)
b = 262143;
rgb[yp] = 0xff000000 | ((r << 6) & 0xff0000) | ((g >> 2) & 0xff00) | ((b >> 10) & 0xff);
}
}
}
So I'm trying to blur an image as fast as possible(instant feel like),
as the activity needs to be updated as I press the Blur button.
The problem I am having is that, I cannot find a Blur that works quick enough...
Note: The blur, preferably a Gaussian blur, doesn't need to be the best quality at all..
I tried out the following, but it takes a few seconds, is there anyway this code could be made to run quicker in sacrifice of quality ? Or are there any other alternatives?
I would look into GPU stuff, but this blur is really just an effect related to the UI and only happens when I press open a transparent activity sized as a small box...
Any Ideas?
static Bitmap fastblur(Bitmap sentBitmap, int radius, int fromX, int fromY,
int width, int height) {
// Stack Blur v1.0 from
// http://www.quasimondo.com/StackBlurForCanvas/StackBlurDemo.html
//
// Java Author: Mario Klingemann <mario at quasimondo.com>
// http://incubator.quasimondo.com
// created Feburary 29, 2004
// Android port : Yahel Bouaziz <yahel at kayenko.com>
// http://www.kayenko.com
// ported april 5th, 2012
// This is a compromise between Gaussian Blur and Box blur
// It creates much better looking blurs than Box Blur, but is
// 7x faster than my Gaussian Blur implementation.
//
// I called it Stack Blur because this describes best how this
// filter works internally: it creates a kind of moving stack
// of colors whilst scanning through the image. Thereby it
// just has to add one new block of color to the right side
// of the stack and remove the leftmost color. The remaining
// colors on the topmost layer of the stack are either added on
// or reduced by one, depending on if they are on the right or
// on the left side of the stack.
//
// If you are using this algorithm in your code please add
// the following line:
//
// Stack Blur Algorithm by Mario Klingemann <mario#quasimondo.com>
Bitmap bitmap = sentBitmap.copy(sentBitmap.getConfig(), true);
if (radius < 1) {
return (null);
}
int w = width;
int h = height;
int[] pix = new int[w * h];
bitmap.getPixels(pix, 0, w, fromX, fromY, w, h);
int wm = w - 1;
int hm = h - 1;
int wh = w * h;
int div = radius + radius + 1;
int r[] = new int[wh];
int g[] = new int[wh];
int b[] = new int[wh];
int rsum, gsum, bsum, x, y, i, p, yp, yi, yw;
int vmin[] = new int[Math.max(w, h)];
int divsum = (div + 1) >> 1;
divsum *= divsum;
int dv[] = new int[256 * divsum];
for (i = 0; i < 256 * divsum; i++) {
dv[i] = (i / divsum);
}
yw = yi = 0;
int[][] stack = new int[div][3];
int stackpointer;
int stackstart;
int[] sir;
int rbs;
int r1 = radius + 1;
int routsum, goutsum, boutsum;
int rinsum, ginsum, binsum;
int originRadius = radius;
for (y = 0; y < h; y++) {
rinsum = ginsum = binsum = routsum = goutsum = boutsum = rsum = gsum = bsum = 0;
for (i = -radius; i <= radius; i++) {
p = pix[yi + Math.min(wm, Math.max(i, 0))];
sir = stack[i + radius];
sir[0] = (p & 0xff0000) >> 16;
sir[1] = (p & 0x00ff00) >> 8;
sir[2] = (p & 0x0000ff);
rbs = r1 - Math.abs(i);
rsum += sir[0] * rbs;
gsum += sir[1] * rbs;
bsum += sir[2] * rbs;
if (i > 0) {
rinsum += sir[0];
ginsum += sir[1];
binsum += sir[2];
} else {
routsum += sir[0];
goutsum += sir[1];
boutsum += sir[2];
}
}
stackpointer = radius;
for (x = 0; x < w; x++) {
r[yi] = dv[rsum];
g[yi] = dv[gsum];
b[yi] = dv[bsum];
rsum -= routsum;
gsum -= goutsum;
bsum -= boutsum;
stackstart = stackpointer - radius + div;
sir = stack[stackstart % div];
routsum -= sir[0];
goutsum -= sir[1];
boutsum -= sir[2];
if (y == 0) {
vmin[x] = Math.min(x + radius + 1, wm);
}
p = pix[yw + vmin[x]];
sir[0] = (p & 0xff0000) >> 16;
sir[1] = (p & 0x00ff00) >> 8;
sir[2] = (p & 0x0000ff);
rinsum += sir[0];
ginsum += sir[1];
binsum += sir[2];
rsum += rinsum;
gsum += ginsum;
bsum += binsum;
stackpointer = (stackpointer + 1) % div;
sir = stack[(stackpointer) % div];
routsum += sir[0];
goutsum += sir[1];
boutsum += sir[2];
rinsum -= sir[0];
ginsum -= sir[1];
binsum -= sir[2];
yi++;
}
yw += w;
}
radius = originRadius;
for (x = 0; x < w; x++) {
rinsum = ginsum = binsum = routsum = goutsum = boutsum = rsum = gsum = bsum = 0;
yp = -radius * w;
for (i = -radius; i <= radius; i++) {
yi = Math.max(0, yp) + x;
sir = stack[i + radius];
sir[0] = r[yi];
sir[1] = g[yi];
sir[2] = b[yi];
rbs = r1 - Math.abs(i);
rsum += r[yi] * rbs;
gsum += g[yi] * rbs;
bsum += b[yi] * rbs;
if (i > 0) {
rinsum += sir[0];
ginsum += sir[1];
binsum += sir[2];
} else {
routsum += sir[0];
goutsum += sir[1];
boutsum += sir[2];
}
if (i < hm) {
yp += w;
}
}
yi = x;
stackpointer = radius;
for (y = 0; y < h; y++) {
pix[yi] = 0xff000000 | (dv[rsum] << 16) | (dv[gsum] << 8)
| dv[bsum];
rsum -= routsum;
gsum -= goutsum;
bsum -= boutsum;
stackstart = stackpointer - radius + div;
sir = stack[stackstart % div];
routsum -= sir[0];
goutsum -= sir[1];
boutsum -= sir[2];
if (x == 0) {
vmin[y] = Math.min(y + r1, hm) * w;
}
p = x + vmin[y];
sir[0] = r[p];
sir[1] = g[p];
sir[2] = b[p];
rinsum += sir[0];
ginsum += sir[1];
binsum += sir[2];
rsum += rinsum;
gsum += ginsum;
bsum += binsum;
stackpointer = (stackpointer + 1) % div;
sir = stack[stackpointer];
routsum += sir[0];
goutsum += sir[1];
boutsum += sir[2];
rinsum -= sir[0];
ginsum -= sir[1];
binsum -= sir[2];
yi += w;
}
}
bitmap.setPixels(pix, 0, w, fromX, fromY, w, h);
return (bitmap);
}
Try to scale down the image 2, 4, 8, ... times and then scale it up again. That is fast. Otherwise implement it in renderscript.
If you want more than the scaling you can look at this code snippet in renderscript. It does the same kind of bluring as given in another answer. The same algorithm can be implemented in Java and is an optimization of the other answer. This code blurs one line. To blur a bitmap you should invoke this for all lines and then the same for all columns (you need to reimplement it to handle columns). To get a quick blur just do this once. If you want a better looking blur do it several times. I usually only do it twice.
The reason for doing one line is that I tried to parallelize the algorithm, that gave some improvement and is really simple in renderscript. I invoked the below code for all lines in parallel, and then the same for all columns.
int W = 8;
uchar4 *in;
uchar4 *out;
int N;
float invN;
uint32_t nx;
uint32_t ny;
void init_calc() {
N = 2*W+1;
invN = 1.0f/N;
nx = rsAllocationGetDimX(rsGetAllocation(in));
ny = rsAllocationGetDimY(rsGetAllocation(in));
}
void root(const ushort *v_in) {
float4 sum = 0;
uchar4 *head = in + *v_in * nx;
uchar4 *tail = head;
uchar4 *p = out + *v_in * nx;
uchar4 *hpw = head + W;
uchar4 *hpn = head + N;
uchar4 *hpx = head + nx;
uchar4 *hpxmw = head + nx - W - 1;
while (head < hpw) {
sum += rsUnpackColor8888(*head++);
}
while (head < hpn) {
sum += rsUnpackColor8888(*head++);
*p++ = rsPackColorTo8888(sum*invN);
}
while (head < hpx) {
sum += rsUnpackColor8888(*head++);
sum -= rsUnpackColor8888(*tail++);
*p++ = rsPackColorTo8888(sum*invN);
}
while (tail < hpxmw) {
sum -= rsUnpackColor8888(*tail++);
*p++ = rsPackColorTo8888(sum*invN);
}
}
Here is for the vertical bluring:
int W = 8;
uchar4 *in;
uchar4 *out;
int N;
float invN;
uint32_t nx;
uint32_t ny;
void init_calc() {
N = 2*W+1;
invN = 1.0f/N;
nx = rsAllocationGetDimX(rsGetAllocation(in));
ny = rsAllocationGetDimY(rsGetAllocation(in));
}
void root(const ushort *v_in) {
float4 sum = 0;
uchar4 *head = in + *v_in;
uchar4 *tail = head;
uchar4 *hpw = head + nx*W;
uchar4 *hpn = head + nx*N;
uchar4 *hpy = head + nx*ny;
uchar4 *hpymw = head + nx*(ny-W-1);
uchar4 *p = out + *v_in;
while (head < hpw) {
sum += rsUnpackColor8888(*head);
head += nx;
}
while (head < hpn) {
sum += rsUnpackColor8888(*head);
*p = rsPackColorTo8888(sum*invN);
head += nx;
p += nx;
}
while (head < hpy) {
sum += rsUnpackColor8888(*head);
sum -= rsUnpackColor8888(*tail);
*p = rsPackColorTo8888(sum*invN);
head += nx;
tail += nx;
p += nx;
}
while (tail < hpymw) {
sum -= rsUnpackColor8888(*tail);
*p = rsPackColorTo8888(sum*invN);
tail += nx;
p += nx;
}
}
And here is the Java code that calls into the rs code:
private RenderScript mRS;
private ScriptC_horzblur mHorizontalScript;
private ScriptC_vertblur mVerticalScript;
private ScriptC_blur mBlurScript;
private Allocation alloc1;
private Allocation alloc2;
private void hblur(int radius, Allocation index, Allocation in, Allocation out) {
mHorizontalScript.set_W(radius);
mHorizontalScript.bind_in(in);
mHorizontalScript.bind_out(out);
mHorizontalScript.invoke_init_calc();
mHorizontalScript.forEach_root(index);
}
private void vblur(int radius, Allocation index, Allocation in, Allocation out) {
mHorizontalScript.set_W(radius);
mVerticalScript.bind_in(in);
mVerticalScript.bind_out(out);
mVerticalScript.invoke_init_calc();
mVerticalScript.forEach_root(index);
}
Bitmap blur(Bitmap org, int radius) {
Bitmap out = Bitmap.createBitmap(org.getWidth(), org.getHeight(), org.getConfig());
blur(org, out, radius);
return out;
}
private Allocation createIndex(int size) {
Element element = Element.U16(mRS);
Allocation allocation = Allocation.createSized(mRS, element, size);
short[] rows = new short[size];
for (int i = 0; i < rows.length; i++) rows[i] = (short)i;
allocation.copyFrom(rows);
return allocation;
}
private void blur(Bitmap src, Bitmap dst, int r) {
Allocation alloc1 = Allocation.createFromBitmap(mRS, src);
Allocation alloc2 = Allocation.createTyped(mRS, alloc1.getType());
Allocation hIndexAllocation = createIndex(alloc1.getType().getY());
Allocation vIndexAllocation = createIndex(alloc1.getType().getX());
// Iteration 1
hblur(r, hIndexAllocation, alloc1, alloc2);
vblur(r, vIndexAllocation, alloc2, alloc1);
// Iteration 2
hblur(r, hIndexAllocation, alloc1, alloc2);
vblur(r, vIndexAllocation, alloc2, alloc1);
// Add more iterations if you like or simply make a loop
alloc1.copyTo(dst);
}
Gaussian blur is expensive to do accurately. A much faster approximation can be done by just iteratively averaging the pixels. It's still expensive to blur the image a lot but you can redraw between each iteration to at least give instant feedback and a nice animation of the image blurring.
static void blurfast(Bitmap bmp, int radius) {
int w = bmp.getWidth();
int h = bmp.getHeight();
int[] pix = new int[w * h];
bmp.getPixels(pix, 0, w, 0, 0, w, h);
for(int r = radius; r >= 1; r /= 2) {
for(int i = r; i < h - r; i++) {
for(int j = r; j < w - r; j++) {
int tl = pix[(i - r) * w + j - r];
int tr = pix[(i - r) * w + j + r];
int tc = pix[(i - r) * w + j];
int bl = pix[(i + r) * w + j - r];
int br = pix[(i + r) * w + j + r];
int bc = pix[(i + r) * w + j];
int cl = pix[i * w + j - r];
int cr = pix[i * w + j + r];
pix[(i * w) + j] = 0xFF000000 |
(((tl & 0xFF) + (tr & 0xFF) + (tc & 0xFF) + (bl & 0xFF) + (br & 0xFF) + (bc & 0xFF) + (cl & 0xFF) + (cr & 0xFF)) >> 3) & 0xFF |
(((tl & 0xFF00) + (tr & 0xFF00) + (tc & 0xFF00) + (bl & 0xFF00) + (br & 0xFF00) + (bc & 0xFF00) + (cl & 0xFF00) + (cr & 0xFF00)) >> 3) & 0xFF00 |
(((tl & 0xFF0000) + (tr & 0xFF0000) + (tc & 0xFF0000) + (bl & 0xFF0000) + (br & 0xFF0000) + (bc & 0xFF0000) + (cl & 0xFF0000) + (cr & 0xFF0000)) >> 3) & 0xFF0000;
}
}
}
bmp.setPixels(pix, 0, w, 0, 0, w, h);
}
I have found a way on how to convert raw data to jpeg but I have some issues with it.
My app takes a picture on the current frame (onPreviewFrame) and has the raw data in a bytearray.
First of all, the code I found is only supported by android API 7+ (Android 2.1+). I want this app to be able to use since API 4+ so android 1.6 users can also enjoy the app.
Second thing is that I have found some code to convert raw2jpg, but it is copyright protected so I can't use it.
I want to put it in a bytearray, so I won't use takePicture with it, remember.
Does anyone have an idea or some code snippet that I can use on how to convert raw data taken on a current frame to make a jpeg image in a bytearray from Android version 1.6?
EDIT: Here is the code:
private void raw2jpg(int[] rgb, byte[] raw, int width, int height)
{
final int frameSize = width * height;
for (int j = 0, yp = 0; j < height; j++)
{
int uvp = frameSize + (j >> 1) * width, u = 0, v = 0;
for (int i = 0; i < width; i++, yp++)
{
int y=0;
if( yp < raw.length)
{
y = (0xff & ((int) raw[yp])) - 16;
}
// int y = (0xff & ((int) raw[yp])) - 16;
if (y < 0) y = 0;
if ((i & 1) == 0)
{
if(uvp<raw.length)
{
v = (0xff & raw[uvp++]) - 128;
u = (0xff & raw[uvp++]) - 128;
}
}
int y1192 = 1192 * y;
int r = (y1192 + 1634 * v);
int g = (y1192 - 833 * v - 400 * u);
int b = (y1192 + 2066 * u);
if (r < 0) r = 0; else if (r > 262143) r = 262143;
if (g < 0) g = 0; else if (g > 262143) g = 262143;
if (b < 0) b = 0; else if (b > 262143) b = 262143;
rgb[yp] = 0xff000000 | ((r << 6) &
0xff0000) | ((g >> 2) &
0xff00) | ((b >> 10) &
0xff);
}
}
}
I've implemented a simple application which shows the camera picture on the screen. What I like to do now is grab a single frame and process it as bitmap.
From what I could find out to this point it is not an easy thing to do.
I've tried using the onPreviewFrame method with which you get the current frame as a byte array and tried to decode it with the BitmapFactory class but it returns null.
The format of the frame is a headerless YUV which could be translated to bitmap but it takes too long on a phone. Also I've read that the onPreviewFrame method has contraints on the runtime, if it takes too long the application could crash.
So what is the right way to do this?
Ok what we ended up doing is using the onPreviewFrame method and decoding the data in a seperate Thread using a method which can be found in the android help group.
decodeYUV(argb8888, data, camSize.width, camSize.height);
Bitmap bitmap = Bitmap.createBitmap(argb8888, camSize.width,
camSize.height, Config.ARGB_8888);
...
// decode Y, U, and V values on the YUV 420 buffer described as YCbCr_422_SP by Android
// David Manpearl 081201
public void decodeYUV(int[] out, byte[] fg, int width, int height)
throws NullPointerException, IllegalArgumentException {
int sz = width * height;
if (out == null)
throw new NullPointerException("buffer out is null");
if (out.length < sz)
throw new IllegalArgumentException("buffer out size " + out.length
+ " < minimum " + sz);
if (fg == null)
throw new NullPointerException("buffer 'fg' is null");
if (fg.length < sz)
throw new IllegalArgumentException("buffer fg size " + fg.length
+ " < minimum " + sz * 3 / 2);
int i, j;
int Y, Cr = 0, Cb = 0;
for (j = 0; j < height; j++) {
int pixPtr = j * width;
final int jDiv2 = j >> 1;
for (i = 0; i < width; i++) {
Y = fg[pixPtr];
if (Y < 0)
Y += 255;
if ((i & 0x1) != 1) {
final int cOff = sz + jDiv2 * width + (i >> 1) * 2;
Cb = fg[cOff];
if (Cb < 0)
Cb += 127;
else
Cb -= 128;
Cr = fg[cOff + 1];
if (Cr < 0)
Cr += 127;
else
Cr -= 128;
}
int R = Y + Cr + (Cr >> 2) + (Cr >> 3) + (Cr >> 5);
if (R < 0)
R = 0;
else if (R > 255)
R = 255;
int G = Y - (Cb >> 2) + (Cb >> 4) + (Cb >> 5) - (Cr >> 1)
+ (Cr >> 3) + (Cr >> 4) + (Cr >> 5);
if (G < 0)
G = 0;
else if (G > 255)
G = 255;
int B = Y + Cb + (Cb >> 1) + (Cb >> 2) + (Cb >> 6);
if (B < 0)
B = 0;
else if (B > 255)
B = 255;
out[pixPtr++] = 0xff000000 + (B << 16) + (G << 8) + R;
}
}
}
Link: http://groups.google.com/group/android-developers/browse_thread/thread/c85e829ab209ceea/3f180a16a4872b58?lnk=gst&q=onpreviewframe#3f180a16a4872b58
In API 17+, you can do conversion to RGBA888 from NV21 with the 'ScriptIntrinsicYuvToRGB' RenderScript. This allows you to easily process preview frames without manually encoding/decoding frames:
#Override
public void onPreviewFrame(byte[] data, Camera camera) {
Bitmap bitmap = Bitmap.createBitmap(r.width(), r.height(), Bitmap.Config.ARGB_8888);
Allocation bmData = renderScriptNV21ToRGBA888(
mContext,
r.width(),
r.height(),
data);
bmData.copyTo(bitmap);
}
public Allocation renderScriptNV21ToRGBA888(Context context, int width, int height, byte[] nv21) {
RenderScript rs = RenderScript.create(context);
ScriptIntrinsicYuvToRGB yuvToRgbIntrinsic = ScriptIntrinsicYuvToRGB.create(rs, Element.U8_4(rs));
Type.Builder yuvType = new Type.Builder(rs, Element.U8(rs)).setX(nv21.length);
Allocation in = Allocation.createTyped(rs, yuvType.create(), Allocation.USAGE_SCRIPT);
Type.Builder rgbaType = new Type.Builder(rs, Element.RGBA_8888(rs)).setX(width).setY(height);
Allocation out = Allocation.createTyped(rs, rgbaType.create(), Allocation.USAGE_SCRIPT);
in.copyFrom(nv21);
yuvToRgbIntrinsic.setInput(in);
yuvToRgbIntrinsic.forEach(out);
return out;
}
I actually tried the code given the previous answer found that the Colorvalues are not exact. I checked it by taking both the preview and the camera.takePicture which directly returns a JPEG array. And the colors were very different. After a little bit more searching I found another example to convert the PreviewImage from YCrCb to RGB:
static public void decodeYUV420SP(int[] rgb, byte[] yuv420sp, int width, int height) {
final int frameSize = width * height;
for (int j = 0, yp = 0; j < height; j++) {
int uvp = frameSize + (j >> 1) * width, u = 0, v = 0;
for (int i = 0; i < width; i++, yp++) {
int y = (0xff & ((int) yuv420sp[yp])) - 16;
if (y < 0) y = 0;
if ((i & 1) == 0) {
v = (0xff & yuv420sp[uvp++]) - 128;
u = (0xff & yuv420sp[uvp++]) - 128;
}
int y1192 = 1192 * y;
int r = (y1192 + 1634 * v);
int g = (y1192 - 833 * v - 400 * u);
int b = (y1192 + 2066 * u);
if (r < 0) r = 0; else if (r > 262143) r = 262143;
if (g < 0) g = 0; else if (g > 262143) g = 262143;
if (b < 0) b = 0; else if (b > 262143) b = 262143;
rgb[yp] = 0xff000000 | ((r << 6) & 0xff0000) | ((g >> 2) & 0xff00) | ((b >> 10) & 0xff);
}
}
}
The color values given by this and the takePicture() exactly match. I thought I should post it here.
This is where I got this code from.
Hope this helps.
Tim's RenderScript solution is great. Two comments here though:
Create and reuse RenderScript rs, and Allocation in, out. Creating them every frame will hurt the performance.
RenderScript support library can help you back support to Android 2.3.
I don't see any of the answers better in performance than the built-in way to convert it.
You can get the bitmap using this.
Camera.Parameters params = camera.getParameters();
Camera.Size previewsize = params.getPreviewSize();
YuvImage yuv = new YuvImage(data, ImageFormat.NV21, previewsize.width, previewsize.height, null);
ByteArrayOutputStream stream = new ByteArrayOutputStream();
yuv.compressToJpeg(new Rect(0,0,previewsize.width, previewsize.height), 100, stream);
byte[] buf = stream.toByteArray();
Bitmap bitmap = BitmapFactory.decodeByteArray(buf, 0, buf.length);