I am working on a Rubik's side scanner to determine what state the cube is in. I am quite new to computer vision and using it so it has been a little bit of a challenge. What I have done so far is that I use a video capture and at certain frames capture that frame and save it for image processing. Here is what it looks like.
When the photo is taken the cube is in the same position each time so I don't have to worry about locating the stickers.
What I am having trouble doing is getting a small range of pixels in each square to determine its HSV.
I know the ranges of HSV are roughly
Red = Hue(0...9) AND Hue(151..180)
Orange = Hue(10...15)
Yellow = Hue(16..45)
Green = Hue(46..100)
Blue = Hue(101..150)
White = Saturation(0..20) AND Value(230..255)
So after I have captured the image I then load it and split the HSV values of the image but don't know how to get the certain pixel coordinates of the image. How do I do so?
BufferedImage getOneFrame() {
currFrame++;
//At the 90th frame I capture that frame and save that frame
if (currFrame == 120) {
cap.read(mat2Img.mat);
mat2Img.getImage(mat2Img.mat);
Imgcodecs.imwrite("firstImage.png", mat2Img.mat);
}
cap.read(mat2Img.mat);
return mat2Img.getImage(mat2Img.mat);
}
public void splitChannels() {
IplImage firstShot = cvLoadImage("firstImage.png");
//I split the channels so that I can determine the value of the pixel range
IplImage hsv = IplImage.create( firstShot.width(), firstShot.height(), firstShot.depth(), firstShot.nChannels());
IplImage hue = IplImage.create( firstShot.width(), firstShot.height(), firstShot.depth(), CV_8UC1 );
IplImage sat = IplImage.create( firstShot.width(), firstShot.height(), firstShot.depth(), CV_8UC1 );
IplImage val = IplImage.create( firstShot.width(), firstShot.height(), firstShot.depth(), CV_8UC1 );
cvSplit( hsv, hue, sat, val, null );
//How do I get a small range of pixels of my images to determine get their HSV?
}
If I understand your question right, you know the coordinates of all areas that interest you. Save the information about each area into cvRect objects.
You can traverse the rectangle area by looping. Make a double loop. In outer loop start at rect.y and stop before rect.y + rect.height. In inner loop, do a similar thing in x direction. Inside the loop, use CV_IMAGE_ELEM macro to access individual pixel values and compute whatever you need.
One advice though: There are several advantages to using Mat instead of IplImage when working with OpenCV. I recommend that you start using 'Mat', unless you have some special reasons to do so, of course. Click to see the documentation and take a look at one of constructors that takes one Mat and one Rect as parameters. This constructor is your good friend - you can create a new Mat object (without copying any data) which will only contain the area inside the rectangle.
Related
I am trying to do something I have never done before.
My goal is to be able to manipulate the opacity of a BufferedImage. First off, I do not use Graphics. I am developing a simple Game Engine and I only use pixel data from BufferedImages.
What have I tried?
I made my own "Image" class that takes in a BufferedImage.
BufferedImage image = null;
try {
image = ImageIO.read(Image.class.getResourceAsStream(resourcePath));
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
width = image.getWidth();
height = image.getHeight();
hasAlpha = image.getColorModel().hasAlpha();
pixels = image.getRGB(0, 0, width, height,null,0,width);
After that I proceeded to my rendering code. I have a function called "drawImage()" that takes the image and it's data and then puts it out on a specific spot on the screen depending on the users wishes. I however want the user to choose it's opacity as well.
I started out by taking the pixel data for that pixel and I instead made a Color.
Color c = new Color(image.getPixels()[value2],true);
The "true" statement, what I know of, says that it should "care" about Alpha.
After that I place the pixel by using this code:
pixels[x + y * pWidth] = new Color(c.getRed(),c.getGreen(),c.getBlue(),alphaValue).getRGB();
This doesn't work. It doesnt output any errors and I can change the alphaValue to a number between 0 and infinity. However one issue I found is that it doesnt affect the image at all when I change the alpha from 255 to something like 50. It stays the same. The affect comes at 0-5. Another thing that I noticed is that when I set alphaValue to something like 2 I get an animation instead of the image with lower opacity. Sounds weird, that's why I provided a .GIF of the issue:
I guarantee you that there is no code that changes the alphaValue. During that "animation" it is set to 2 and nothing else.
So something is weird here and I wonder if you guys know the issue?
It might have something to do with the Image class and that I don't set it to BufferedImage.RGBA but I don't know how I can do so when I load an image?
If you need any more information, please leave a comment and I will provide more info!
Thanks in advance!
EDIT
I clear the screen every 1/60 of a second by manipulating the backgrounds pixels (also a bufferedImage):
public void clear() {
for(int i = 0; i < pixels.length; i++) {
pixels[i] = clearColor.getRGB();
}
}
Possible issue
I did some troubleshooting and I found out that it actually somehow increased in opacity each frame as #Joni said. Somehow the opacity or alpha seems to increase each frame but I am clearing the screen between drawings. I will troubleshoot further.
UPDATE
I made so I could move the image around and found this weird behaviour:
Link to Imgur
It seems that the "alpha" isn't reset. I know that the RGB is being reset but something is really off when it comes to the Alpha.
I changed the clear function to this:
public void clear() {
for(int i = 0; i < pixels.length; i++) {
pixels[i] = new Color(clearColor.getRed(),clearColor.getGreen(),clearColor.getBlue(),255).getRGB();
}
}
And I defined clearColor as:
Color clearColor = new Color(Color.black.getRGB(),true);
That did however not work either. Does anyone have a solution?
Your game engine is drawing the image on top of itself in a loop. After drawing it enough many times the effect is the same as not having used transparency at all. The lower the alpha, the longer it takes though: with 50% alpha you need 7 frames to get 99% opacity, with 5% alpha you need about 90 frames.
For example, suppose you are drawing a pixel value of 100 on a screen that's intially 0 (black) with 50% opacity. After the first frame, the output pixel value is .5*100 + .5*0 = 50. The second frame is drawn on top of the first frame, so the output pixel value is .5*100 + .5*50 = 75. The third frame, drawn on top of the second frame, will show .5*100 + .5*75 = 87.5.
To avoid this, you need to fill a solid background color under the image in every frame.
So i have a lot of questions to this but first here is my assignment.
Write a method public Picture authenticate(Picture p) which checks to
see if ”this” picture and the picture p are identical. It returns a
new picture that it the same size as ”this” picture. For each pixel
position in the original picture ”this”, compare the color of that
pixel with the color of the pixel in the same position from second
picture. If the two pixels have the same color (i.e. the original is
not modified in the copy), set the color of this position in the
resulting picture to white. If the two pixels do not have the same
color, set the color of this position in the resulting picture to the
color of the pixel from the original picture. Return the resulting
picture. If the two pictures are not the same size, the resulting
picture should contain all black pixels
This is my code as of right now
public Picture Authenticate(Picture p)
{
Pixel sourcePixel = null;
Pixel targetPixel = null;
Picture result = new Picture(this.getWidth(), this.getHeight());
for(int sourceX = 0, targetX= getWidth()-1; sourceX < getWidth(); sourceX++,targetX++)
{
for(int sourceY = 0, targetY= getHeight(); sourceY< getHeight(); sourceY++,targetY++)
{
sourcePixel = this.getPixel(sourceX,sourceY);
targetPixel = result.getPixel(targetX, targetY);
}
if (sourcePixel.getColor() == targetPixel.getColor())
targetPixel.setColor(Color.WHITE);
else
targetPixel.setColor(Color.BLACK);
}
return result;
}
Here are my questions:
How do i make p in the authenticate parameter fit into this?
Am i going about this the right way within the loop?
With my if and else statement, i feel as if that is not even close
to changing the pixel i need tooa
How do i make p in the authenticate parameter fit into this?
this is your source image; p is the target image; result is where you place the output pixels resulting from comparing the source and target.
Am i going about this the right way within the loop?
Yes, but the if/else statement belongs inside the inner loop
With my if and else statement, i feel as if that is not even close to changing the pixel i need to
You are close. Keep working on it and step through the code in your debugger to see what is actually happening.
Don't get discouraged, this is closer than you think.
I am making a game that has campfire objects. What I want to do is to brighten all pixels in a circle around each campfire. However, looping through every pixel and changing those within the radius is not all that efficient and makes the game run at ~7 fps. Ideas on how to either make this process efficient or simulate light differently?
I haven't written the code for the fires but this is the basic loop to check each pixel/change its brightness based on a number:
public static BufferedImage updateLightLevels(BufferedImage img, float light)
{
BufferedImage brightnessBuffer = new BufferedImage(img.getWidth(), img.getHeight(), BufferedImage.TYPE_4BYTE_ABGR);
brightnessBuffer.getGraphics().drawImage(img, 0, 0, null);
for(int i = 0; i < brightnessBuffer.getWidth(); i++)
{
for(int a = 0; a < brightnessBuffer.getHeight(); a++)
{
//get the color at the pixel
int rgb = brightnessBuffer.getRGB(i, a);
//check to see if it is transparent
int alpha = (rgb >> 24) & 0x000000FF;
if(alpha != 0)
{
//make a new color
Color rgbColor = new Color(rgb);
//turn it into an hsb color
float[] hsbCol = Color.RGBtoHSB(rgbColor.getRed(), rgbColor.getGreen(), rgbColor.getBlue(), null);
//lower it by the certain amount
//if the pixel is already darker then push it all the way to black
if(hsbCol[2] <= light)
hsbCol[2] -= (hsbCol[2]) - .01f;
else
hsbCol[2] -= light;
//turn the hsb color into a rgb color
int rgbNew = Color.HSBtoRGB(hsbCol[0], hsbCol[1], hsbCol[2]);
//set the pixel to the new color
brightnessBuffer.setRGB(i, a, rgbNew);
}
}
}
return brightnessBuffer;
}
I apologize if my code is not clean, I'm self taught.
I can give you lots of approaches.
You're currently rendering on the CPU, and you're checking every single pixel. That's hardcore brute force, and brute force isn't what the CPU is best at. It works, but as you've seen, the performance is abysmal.
I'd point you in two directions that would massively improve your performance:
Method 1 - Culling. Does every single pixel really need to have its lighting calculated? If you could instead calculate a general "ambient light", then you could paint most of the pixels in that ambient light, and then only calculate the really proper lighting for pixels closest to lights; so lights throw a "spot" effect which fades into the ambient. That way you're only ever performing checks on a few of the pixels of the screen at a time (the circle area around each light). The code you posted just looks like it paints every pixel, I'm not seeing where the "circle" dropoff is even applied.
Edit:
Instead, sweep through the lights, and just loop through local offsets of the light position.
for(Light l : Lights){
for(int x = l.getX() -LIGHT_DISTANCE, x< l.getX() + LIGHT_DISTANCE, y++){
for(int y = l.getY() - LIGHT_DISTANCE, y < l.getY() + LIGHT_DISTANCE, y++){
//calculate light
int rgb = brightnessBuffer.getRGB(x, y);
//do stuff
}
}
You may want to add a check with that method so overlapping lights don't cause a bunch of rechecks, unless you DO want that behavior (ideally those pixels would be twice as bright)
Method 2 - Offhand calculation to the GPU. There's a reason we have graphics cards; they're specifically built to be able to number crunch those situations where you really need brute force. If you can offload this process to the GPU as a shader, then it'll run licketysplit, even if you run it on every pixel several times over. This will require you to learn graphics APIs however, but if you're working in java, LibGDX makes it very painless to render using the GPU and pass off a couple shaders to the GPU.
I am uncertain about the way in which you are going about calculating light values, but I do know that using the BufferedImage.getRGB() and BufferedImage.setRGB() methods is very slow.
I would suggest accessing the pixels of the BufferedImage directly from an array (much faster IMO)
to do this:
BufferedImage lightImage = new BufferedImage(width,height,BufferedImage.TYPE_INT_ARGB);
Raster r = lightImage.getRaster();
int[] lightPixels = ((DataBufferInt)r.getDataBuffer()).getData();
Now, changing any pixel in this array will show on your image. Note that the values used in this array are color values in the format of whatever format you defined your image with.
In this case it is TYPE_INT_ARGB meaning you will have to include the alpha value in the number when setting the coloar (RRGGBB*AA*)
Since this array is a 1D array, it is more difficult to access pixels using x and y co-ordinates. The following method is an implementation of accessing pixels from the lightPixels array more easily.
public void setLight(int x, int y,int[] array,int width, int value){
array[width*y+x] = value;
}
*note: width is the width of your level, or the width of the 2D array your level might exist as, if it was a 2D array.
You can also get pixels from the lightPixels array with a similar method, just excluding the value and returning the array[width*y+x].
It is up to you how you use the setLight() and getLight() methods but in the cases that I have encountered, using this method is much faster than using getRGB and setRGB.
Hope this helps
Situation: I have a canvas on an Android game, I have some objects (I will keep it as simple as possible):World(where are storaged all Laser and Block objects), Block and Laser. I can draw all this objects in the canvas.
I would like to 'hide' them behind a black 'background', and then draw a blurry 'transparent' circle, so all objects are hidden behind the black background, except the objects behing the circle.
I have thought about it, but I can't think of an approach to do this.
Images:
This is my actual situation:
This is the expected:
Do something like this:
public void drawBitmapsInCanvas(Canvas c){
c.drawBitmap(block, new Rect(/*coordinates here*/), new Rect(/*More coordinates*/),null);
c.drawBitmap(block2, new Rect(/*coordinates here*/), new Rect(/*More coordinates*/),null);
c.drawBitmap(laser, new Rect(/*coordinates here*/), new Rect(/*More coordinates*/),null);
c.drawColor(Color.BLACK);//this hides everything under your black background.
c.drawBitmap(circle, new Rect(/*coordinates here*/), new Rect(/*More coordinates*/),null);
}
If you want transparency:
Paint paint =new Paint();
paint.setARGB(120,0,0,0); //for the "120" parameter, 0 is completely transparent, 255 is completely opaque.
paint.setAntiAlias(true);
c.drawBitmap(bmp,Rect r,Rect rr, paint);
or if you are trying to change the opacity of individual pixels, the approach is a bit more complicated (I have not tested the code, but you get the gist of it):
public static final Bitmap getNewBitmap(Bitmap bmp, int circleCenterX,
int circleCenterY,int circleRadius){
//CIRCLE COORDINATES ARE THE DISTANCE IN RESPECT OF (0,0) of the bitmap
//, not (0,0) of the canvas itself. The circleRadius is the circle's radius.
Bitmap temp=bmp.copy(Bitmap.Config.ARGB_8888, true);
int[]pixels = new int[temp.getWidth()*temp.getHeight()];
temp.getPixels(pixels,0 ,temp.getWidth(),0,0,temp.getWidth(), temp.getHeight());
int counter=0;
for(int i=0;i<pixels.length;i++){
int alpha=Color.alpha(pixels[i]);
if(alpha!=0&&!((Math.pow(counter/temp.getWidth()-circleCenterY,2.0)+
Math.pow(counter%temp.getWidth()-circleCenterX,2.0))<Math.pow(circleRadius,2.0))){
//if the pixel itself is not completely transparent and the pixel is NOT within range of the circle,
//set the Alpha value of the pixel to 0.
pixels[i]=Color.argb(0,Color.red(pixels[i]),Color.green(pixels[i]),Color.blue(pixels[i]));
}
counter++;
}
temp.setPixels(pixels,0, temp.getWidth(),0,0,temp.getWidth(),temp.getHeight());
return temp;
}
and then draw temp.
I'm not completely sure what you are trying to ask, so you may have to modify as necessary.
If you try the second answer of qwertyuiop5040, you will get a ver low - perfomance when you try to apply it to a large image. Let's say a 1000*800 pixels image. Then you will have a loop:
for (int i = 0 ; i < 1000*800; i++)
You could create an image that's a black rectangle with a transparent hole in it. The hole would be the circle that you can see through, and the image would be rendered over the spot you want to be visible. Then, you can draw four black rectangles around the image to cover the rest of the screen.
What is the most efficient way to do lighting for a tile based engine in Java?
Would it be putting a black background behind the tiles and changing the tiles' alpha?
Or putting a black foreground and changing alpha of that? Or anything else?
This is an example of the kind of lighting I want:
There are many ways to achieve this. Take some time before making your final decision. I will briefly sum up some techiques you could choose to use and provide some code in the end.
Hard Lighting
If you want to create a hard-edge lighting effect (like your example image),
some approaches come to my mind:
Quick and dirty (as you suggested)
Use a black background
Set the tiles' alpha values according to their darkness value
A problem is, that you can neither make a tile brighter than it was before (highlights) nor change the color of the light. Both of these are aspects which usually make lighting in games look good.
A second set of tiles
Use a second set of (black/colored) tiles
Lay these over the main tiles
Set the new tiles' alpha value depending on how strong the new color should be there.
This approach has the same effect as the first one with the advantage, that you now may color the overlay tile in another color than black, which allows for both colored lights and doing highlights.
Example:
Even though it is easy, a problem is, that this is indeed a very inefficent way. (Two rendered tiles per tile, constant recoloring, many render operations etc.)
More Efficient Approaches (Hard and/or Soft Lighting)
When looking at your example, I imagine the light always comes from a specific source tile (character, torch, etc.)
For every type of light (big torch, small torch, character lighting) you
create an image that represents the specific lighting behaviour relative to the source tile (light mask). Maybe something like this for a torch (white being alpha):
For every tile which is a light source, you render this image at the position of the source as an overlay.
To add a bit of light color, you can use e.g. 10% opaque orange instead of full alpha.
Results
Adding soft light
Soft light is no big deal now, just use more detail in light mask compared to the tiles. By using only 15% alpha in the usually black region you can add a low sight effect when a tile is not lit:
You may even easily achieve more complex lighting forms (cones etc.) just by changing the mask image.
Multiple light sources
When combining multiple light sources, this approach leads to a problem:
Drawing two masks, which intersect each other, might cancel themselves out:
What we want to have is that they add their lights instead of subtracting them.
Avoiding the problem:
Invert all light masks (with alpha being dark areas, opaque being light ones)
Render all these light masks into a temporary image which has the same dimensions as the viewport
Invert and render the new image (as if it was the only light mask) over the whole scenery.
This would result in something similar to this:
Code for the mask invert method
Assuming you render all the tiles in a BufferedImage first,
I'll provide some guidance code which resembles the last shown method (only grayscale support).
Multiple light masks for e.g. a torch and a player can be combined like this:
public BufferedImage combineMasks(BufferedImage[] images)
{
// create the new image, canvas size is the max. of all image sizes
int w, h;
for (BufferedImage img : images)
{
w = img.getWidth() > w ? img.getWidth() : w;
h = img.getHeight() > h ? img.getHeight() : h;
}
BufferedImage combined = new BufferedImage(w, h, BufferedImage.TYPE_INT_ARGB);
// paint all images, preserving the alpha channels
Graphics g = combined.getGraphics();
for (BufferedImage img : images)
g.drawImage(img, 0, 0, null);
return combined;
}
The final mask is created and applied with this method:
public void applyGrayscaleMaskToAlpha(BufferedImage image, BufferedImage mask)
{
int width = image.getWidth();
int height = image.getHeight();
int[] imagePixels = image.getRGB(0, 0, width, height, null, 0, width);
int[] maskPixels = mask.getRGB(0, 0, width, height, null, 0, width);
for (int i = 0; i < imagePixels.length; i++)
{
int color = imagePixels[i] & 0x00ffffff; // Mask preexisting alpha
// get alpha from color int
// be careful, an alpha mask works the other way round, so we have to subtract this from 255
int alpha = (maskPixels[i] >> 24) & 0xff;
imagePixels[i] = color | alpha;
}
image.setRGB(0, 0, width, height, imagePixels, 0, width);
}
As noted, this is a primitive example. Implementing color blending might be a bit more work.
Raytracing might be the simpliest approach.
you can store which tiles have been seen (used for automapping, used for 'remember your map while being blinded', maybe for the minimap etc.)
you show only what you see - maybe a monster of a wall or a hill is blocking your view, then raytracing stops at that point
distant 'glowing objects' or other light sources (torches lava) can be seen, even if your own light source doesn't reach very far.
the length of your ray gives will be used to check amount light (fading light)
maybe you have a special sensor (ESP, gold/food detection) which would be used to find objects that are not in your view? raytrace might help as well ^^
how is this done easy?
draw a line from your player to every point of the border of your map (using Bresehhams Algorithm http://en.wikipedia.org/wiki/Bresenham%27s_line_algorithm
walk along that line (from your character to the end) until your view is blocked; at this point stop your search (or maybe do one last final iteration to see what did top you)
for each point on your line set the lighning (maybe 100% for distance 1, 70% for distance 2 and so on) and mark you map tile as visited
maybe you won't walk along the whole map, maybe it's enough if you set your raytrace for a 20x20 view?
NOTE: you really have to walk along the borders of viewport, its NOT required to trace every point.
i'm adding the line algorithm to simplify your work:
public static ArrayList<Point> getLine(Point start, Point target) {
ArrayList<Point> ret = new ArrayList<Point>();
int x0 = start.x;
int y0 = start.y;
int x1 = target.x;
int y1 = target.y;
int sx = 0;
int sy = 0;
int dx = Math.abs(x1-x0);
sx = x0<x1 ? 1 : -1;
int dy = -1*Math.abs(y1-y0);
sy = y0<y1 ? 1 : -1;
int err = dx+dy, e2; /* error value e_xy */
for(;;){ /* loop */
ret.add( new Point(x0,y0) );
if (x0==x1 && y0==y1) break;
e2 = 2*err;
if (e2 >= dy) { err += dy; x0 += sx; } /* e_xy+e_x > 0 */
if (e2 <= dx) { err += dx; y0 += sy; } /* e_xy+e_y < 0 */
}
return ret;
}
i did this whole lightning stuff some time ago, a* pathfindin feel free to ask further questions
Appendum:
maybe i might simply add the small algorithms for raytracing ^^
to get the North & South Border Point just use this snippet:
for (int x = 0; x <map.WIDTH; x++){
Point northBorderPoint = new Point(x,0);
Point southBorderPoint = new Point(x,map.HEIGHT);
rayTrace( getLine(player.getPos(), northBorderPoint), player.getLightRadius()) );
rayTrace( getLine(player.getPos(), southBorderPoint, player.getLightRadius()) );
}
and the raytrace works like this:
private static void rayTrace(ArrayList<Point> line, WorldMap map, int radius) {
//int radius = radius from light source
for (Point p: line){
boolean doContinue = true;
float d = distance(line.get(0), p);
//caclulate light linear 100%...0%
float amountLight = (radius - d) / radius;
if (amountLight < 0 ){
amountLight = 0;
}
map.setLight( p, amountLight );
if ( ! map.isViewBlocked(p) ){ //can be blockeb dy wall, or monster
doContinue = false;
break;
}
}
}
I've been into indie game development for about three years right now. The way I would do this is first of all by using OpenGL so you can get all the benefits of the graphical computing power of the GPU (hopefully you are already doing that). Suppose we start off with all tiles in a VBO, entirely lit. Now, there are several options of achieving what you want. Depending on how complex your lighting system is, you can choose a different approach.
If your light is going to be circular around the player, no matter the fact if obstacles would block the light in real life, you could choose for a lighting algorithm implemented in the vertex shader. In the vertex shader, you could compute the distance of the vertex to the player and apply some function that defines how bright things should be in function of the computed distance. Do not use alpha, but just multiply the color of the texture/tile by the lighting value.
If you want to use a custom lightmap (which is more likely), I would suggest to add an extra vertex attribute that specifies the brightness of the tile. Update the VBO if needed. Same approach goes here: multiply the pixel of the texture by the light value. If you are filling light recursively with the player position as starting point, then you would update the VBO every time the player moves.
If your lightmap depends on where the sunlight hits your level, you could combine two sort of lighting techniques. Create one vertex attribute for the sun brightness and another vertex attribute for the light emitted by light points (like a torch held by the player). Now you can combine those two values in the vertex shader. Suppose the your sun comes up and goes down like the day and night pattern. Let's say the sun brightness is sun, which is a value between 0 and 1. This value can be passed to the vertex shader as a uniform. The vertex attribute that represents the sun brightness is s and the one for light, emitted by light points is l. Then you could compute the total light for that tile like this:
tileBrightness = max(s * sun, l + flicker);
Where flicker (also a vertex shader uniform) is some kind of waving function that represents the little variants in the brightness of your light points.
This approach makes the scene dynamic without having to recreate continuously VBO's. I implemented this approach in a proof-of-concept project. It works great. You can check out what it looks like here: http://www.youtube.com/watch?v=jTcNitp_IIo. Note how the torchlight is flickering at 0:40 in the video. That is done by what I explained here.