Related
I would like to build a kind of image morphing tool in Processing. Similar to what you can see in this link:
https://giphy.com/gifs/painting-morph-oil-c8ygOpL64UDuw
My first step to achieve this was to build a two-dimensional grid of pixels. The pixels are filled with colour. The fill colour is created by reading colour from an image (PImage img1;) with the get(); function. This is how I recreated an image with my pixels. In the second step, I thought I would use the lerp(); function to give the respective pixels the colour of a second image (PImage img2;) - I thought this would create the desired morph effect. But I was wrong! The whole thing works - but the effect is only that a fade-in takes place between the two images. And no morphing. What exactly happens to pixels while this morph effect? How could I recreate it in Processing?
float pixel;
float pixelsize;
PImage img1;
PImage img2;
float counter;
void setup() {
size(1080, 1080);
pixel = 100;
pixelsize = width/pixel;
noStroke();
img1 = loadImage("0.jpg");
img2 = loadImage("1.jpg");
counter = 0;
}
void draw() {
background(255);
for (int y = 0; y < pixel; y++) {
for (int x = 0; x < pixel; x++) {
color c1 = img1.get(int(pixelsize*x), int(pixelsize*y));
color c2 = img2.get(int(pixelsize*x), int(pixelsize*y));
color from = c1;
color to = c2;
color interA = lerpColor(from, to, counter);
pushMatrix();
translate(pixelsize*x, pixelsize*y);
fill(interA);
rect(0, 0, pixelsize, pixelsize);
popMatrix();
}
}
counter= counter + 0.01;
}
Indeed it is not a straight forward task.
You're approach is not a bad start: it would result in a nice crossfade between the two images.
Bare in mind get() can be costly on the CPU.
You can however use the pixels[]:
PImage img1;
PImage img2;
// transition image
PImage imgT;
void setup() {
size(1080, 1080);
img1 = loadImage("0.jpg");
img2 = loadImage("1.jpg");
// copy the 1st image (copies width/height as well)
imgT = img1.get();
}
void draw() {
background(255);
// map transition amount to mouse X position
float t = map(mouseX, 0, width, 0.0, 1.0);
// make all pixels readable
imgT.loadPixels();
// lerp each pixel
for(int i = 0 ; i < imgT.pixels.length; i++){
imgT.pixels[i] = lerpColor(img1.pixels[i], img2.pixels[i], t);
}
// update all pixels in one go
imgT.updatePixels();
// display result
image(imgT, 0, 0);
}
Implementing a full morph image is non-trivial.
I can recomend two options to make use of existing algorithms, however these options are also not beginner friendly:
ImageMagick implements shepards distortion and there is a java library that interfaces with imagemagick: im4java. Note that you'd need to download the precompiled java library and drop the .jar file on top of your sketch and processing the output might take time: probably not feasible for realtime (however it should be possible to save individual frames to disk and assemble them as a gif/movie/etc.)
Using OpenCV: there's an OpenCV Face Morph tutorial with source code in c++ or Python and there is a Processing OpenCV library. It would be a matter of porting the c++/Python OpenCV calls to the Java OpenCV API.
--------------read edit below---------------
I am trying to detect the edge of the pupils and iris within various images. I am altering parameters and such but I can only manage to ever get one iris/pupil outline correct, or get unnecessary outlines in the background, or none at all. Is the some specific parameters that I should try to try and get the correct outlines. Or is there a way that I can crop the image just to the eyes, so the system can focus on that part?
This is my UPDATED method:
private void findPupilIris() throws IOException {
//converts and saves image in grayscale
Mat newimg = Imgcodecs.imread("/Users/.../pic.jpg");
Mat des = new Mat(newimg.rows(), newimg.cols(), newimg.type());
Mat norm = new Mat();
Imgproc.cvtColor(newimg, des, Imgproc.COLOR_BGR2HSV);
List<Mat> hsv = new ArrayList<Mat>();
Core.split(des, hsv);
Mat v = hsv.get(2); //gets the grey scale version
Imgcodecs.imwrite("/Users/Lisa-Maria/Documents/CapturedImages/B&Wpic.jpg", v); //only writes mats
CLAHE clahe = Imgproc.createCLAHE(2.0, new Size(8,8) ); //2.0, new Size(8,8)
clahe.apply(v,v);
// Imgproc.GaussianBlur(v, v, new Size(9,9), 3); //adds left pupil boundary and random circle on 'a'
// Imgproc.GaussianBlur(v, v, new Size(9,9), 13); //adds right outer iris boundary and random circle on 'a'
Imgproc.GaussianBlur(v, v, new Size(9,9), 7); //adds left outer iris boundary and random circle on left by hair
// Imgproc.GaussianBlur(v, v, new Size(7,7), 15);
Core.addWeighted(v, 1.5, v, -0.5, 0, v);
Imgcodecs.imwrite("/Users/.../after.jpg", v); //only writes mats
if (v != null) {
Mat circles = new Mat();
Imgproc.HoughCircles( v, circles, Imgproc.CV_HOUGH_GRADIENT, 2, v.rows(), 100, 20, 20, 200 );
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
System.out.println("circles.cols() " + circles.cols());
if(circles.cols() > 0) {
System.out.println("1");
for (int x = 0; x < circles.cols(); x++) {
System.out.println("2");
double vCircle[] = circles.get(0, x);
if(vCircle == null) {
break;
}
Point pt = new Point(Math.round(vCircle[0]), Math.round(vCircle[1]));
int radius = (int) Math.round(vCircle[2]);
//draw the found circle
Imgproc.circle(v, pt, radius, new Scalar(255,0,0),2); //newimg
//Imgproc.circle(des, pt, radius/3, new Scalar(225,0,0),2); //pupil
Imgcodecs.imwrite("/Users/.../Houghpic.jpg", v); //newimg
//draw the mask: white circle on black background
// Mat mask = new Mat( new Size( des.cols(), des.rows() ), CvType.CV_8UC1 );
// Imgproc.circle(mask, pt, radius, new Scalar(255,0,0),2);
// des.copyTo(des,mask);
// Imgcodecs.imwrite("/Users/..../mask.jpg", des); //newimg
Imgproc.logPolar(des, norm, pt, radius, Imgproc.WARP_FILL_OUTLIERS);
Imgcodecs.imwrite("/Users/..../Normalised.jpg",norm);
}
}
}
}
Result: hough pic
Following discussion in comments, I am posting a general answer with some results I got on the worst case image uploaded by the OP.
Note : The code I am posting is in Python, since it is the fastest for me to write
Step 1. As you ask for a way to crop the image, so as to focus on the eyes only, you might want to look at Face Detection. Since, the image essentially requires to find eyes only, I did the following:
eye_cascade = cv2.CascadeClassifier('haarcascade_eye.xml')
eyes = eye_cascade.detectMultiScale(v) // v is the value channel of the HSV image
// The results "eyes" gives you the dimensions of the rectangle where the eyes are detected as [x, y, w, h]
// Just for drawing
cv2.rectangle(v, (x1, y1), (x1+w1, y1+h1), (0, 255, 0), 2)
cv2.rectangle(v, (x2, y2), (x2+w2, y2+h2), (0, 255, 0), 2)
Now, once you have the bounding rectangles, you can crop the rectangles from the image like:
crop_eye1 = v[y1:y1+h1, x1:x1+w1]
crop_eye2 = v[y2:y2+h2, x2:x2+w2]
After you obtain the rectangles, I would suggest looking into different color spaces instead of RGB/BGR, HSV/Lab/Luv in particular.
Because the R, G, and B components of an object’s color in a digital image are all correlated with the amount of light hitting the object, and therefore with each other, image descriptions in terms of those components make object discrimination difficult. Descriptions in terms of hue/lightness/chroma or hue/lightness/saturation are often more relevant
Then, once, you have the eyes, its time to equalize the contrast of the image, however, I suggest using CLAHE and play with the parameters for clipLimit and tileGridSize. Here is a code which I implemented a while back in Java:
private static Mat clahe(Mat image, int ClipLimit, Size size){
CLAHE clahe = Imgproc.createCLAHE();
clahe.setClipLimit(ClipLimit);
clahe.setTilesGridSize(size);
Mat dest_image = new Mat();
clahe.apply(image, dest_image);
return dest_image;
}
Once you are satisfied, you should sharpen the image so that HoughCircle is robust. You should look at unsharpMask. Here is the code in Java for UnsharpMask I implemented in Java:
private static Mat unsharpMask(Mat input_image, Size size, double sigma){
// Make sure the {input_image} is gray.
Mat sharpend_image = new Mat(input_image.rows(), input_image.cols(), input_image.type());
Mat Blurred_image = new Mat(input_image.rows(), input_image.cols(), input_image.type());
Imgproc.GaussianBlur(input_image, Blurred_image, size, sigma);
Core.addWeighted(input_image, 2.0D, Blurred_image, -1.0D, 0.0D, sharpened_image);
return sharpened_image;
}
Alternatively, you could use bilateral filter, which is edge preserving smoothing, or read through this for defining a custom kernel for sharpening image.
Hope it helps and best of luck!
I want to draw some (filled) polygons with libGDX. It shoudn't be filled with a graphic/texture. I have only the vertices of the polygon (closed path) and tried to visualize with meshes but at some point this is not the best solution, I think.
My code for an rectangle is:
private Mesh mesh;
#Override
public void create() {
if (mesh == null) {
mesh = new Mesh(
true, 4, 0,
new VertexAttribute(Usage.Position, 3, "a_position")
);
mesh.setVertices(new float[] {
-0.5f, -0.5f, 0
0.5f, -0.5f, 0,
-0.5f, 0.5f, 0,
0.5f, 0.5f, 0
});
}
}
// ...
#Override
public void render() {
Gdx.gl.glClear(GL10.GL_COLOR_BUFFER_BIT);
mesh.render(GL10.GL_TRIANGLE_STRIP, 0, 4);
}
is there a function or something to draw filled polygons in an easier way?
Since recent updates of LibGDX, #Rus answer is using deprecated functions. However, I give him/her credits for the new updated version below:
PolygonSprite poly;
PolygonSpriteBatch polyBatch = new PolygonSpriteBatch(); // To assign at the beginning
Texture textureSolid;
// Creating the color filling (but textures would work the same way)
Pixmap pix = new Pixmap(1, 1, Pixmap.Format.RGBA8888);
pix.setColor(0xDEADBEFF); // DE is red, AD is green and BE is blue.
pix.fill();
textureSolid = new Texture(pix);
PolygonRegion polyReg = new PolygonRegion(new TextureRegion(textureSolid),
new float[] { // Four vertices
0, 0, // Vertex 0 3--2
100, 0, // Vertex 1 | /|
100, 100, // Vertex 2 |/ |
0, 100 // Vertex 3 0--1
}, new short[] {
0, 1, 2, // Two triangles using vertex indices.
0, 2, 3 // Take care of the counter-clockwise direction.
});
poly = new PolygonSprite(polyReg);
poly.setOrigin(a, b);
polyBatch = new PolygonSpriteBatch();
For good triangulating algorithms if your polygon is not convex, see the almost-linear earclipping algorithm from Toussaint (1991)
Efficient triangulation of simple polygons, Godfried Toussaint, 1991
Here is a libGDX example which draws a 2D concave polygon.
Define class members for PolygonSprite PolygonSpriteBatch
PolygonSprite poly;
PolygonSpriteBatch polyBatch;
Texture textureSolid;
Create instances, 1x1 size texture used with red pixel as workaround. An array of coordinates (x, y) is used for initialization of the polygon.
ctor() {
textureSolid = makeTextureBox(1, 0xFFFF0000, 0, 0);
float a = 100;
float b = 100;
PolygonRegion polyReg = new PolygonRegion(new TextureRegion(textureSolid),
new float[] {
a*0, b*0,
a*0, b*2,
a*3, b*2,
a*3, b*0,
a*2, b*0,
a*2, b*1,
a*1, b*1,
a*1, b*0,
});
poly = new PolygonSprite(polyReg);
poly.setOrigin(a, b);
polyBatch = new PolygonSpriteBatch();
}
Draw and rotate polygon
void draw() {
super.draw();
polyBatch.begin();
poly.draw(polyBatch);
polyBatch.end();
poly.rotate(1.1f);
}
I believe the ShapeRenderer class now has a polygon method for vertex defined polygons:
ShapeRenderer.polygon()
You can use the ShapeRenderer API to draw simple, solid-color shapes with Libgdx.
The code you've given is a reasonable way to draw solid color polygons too. Its much more flexible than ShapeRenderer, but is a good bit more complicated. You'll need to use glColor4f to set the color, or add a Usage.Color attribute to each vertex. See the SubMeshColorTest example for more details on the first approach and the MeshColorTexture example for details on the second approach.
Another option to think about is using sprite textures. If you're only interested in simple solid colors objects, you can use very simple 1x1 textures of a single color and let the system stretch that across the sprite. Much of Libgdx and the underlying hardware are really optimized for rendering textures, so you may find it easier to use even if you're not really taking advantage of the texture contents. (You can even use a 1x1 white texture, and then use a SpriteBatch with setColor and draw()
to draw different color rectangles easily.)
You can also mix and match the various approaches, too.
Use triangulation algorithm and then draw all triangles as GL_TRIANGLE_STRIP
http://www.personal.psu.edu/cxc11/AERSP560/DELAUNEY/13_Two_algorithms_Delauney.pdf
just wanted to share my related solution with you, namely for implementing and drawing a walkZone with scene2d. I basically had to put together the different suggestions of the others' posts:
1) The WalkZone:
import com.badlogic.gdx.graphics.Pixmap;
import com.badlogic.gdx.graphics.Texture;
import com.badlogic.gdx.graphics.g2d.PolygonRegion;
import com.badlogic.gdx.graphics.g2d.TextureRegion;
import com.badlogic.gdx.math.EarClippingTriangulator;
import com.badlogic.gdx.math.Polygon;
import com.mygdx.game.MyGame;
public class WalkZone extends Polygon {
private PolygonRegion polygonRegion = null;
public WalkZone(float[] vertices) {
super(vertices);
if (MyGame.DEBUG) {
Pixmap pix = new Pixmap(1, 1, Pixmap.Format.RGBA8888);
pix.setColor(0x00FF00AA);
pix.fill();
polygonRegion = new PolygonRegion(new TextureRegion(new Texture(pix)),
vertices, new EarClippingTriangulator().computeTriangles(vertices).toArray());
}
}
public PolygonRegion getPolygonRegion() {
return polygonRegion;
}
}
2) The Screen:
you can then add a listener in the desired Stage:
myStage.addListener(new InputListener() {
#Override
public boolean touchDown(InputEvent event, float x, float y, int pointer, int button) {
if (walkZone.contains(x, y)) player.walkTo(x, y);
// or even directly: player.addAction(moveTo ...
return super.touchDown(event, x, y, pointer, button);
}
});
3) The implementation:
The array passed to te WZ constructor is a set of x,y,x,y... points. If you put them counter-clockwise, it works (I didn't check the other way, nor know how it exactly works); for example this generates a 100x100 square:
yourScreen.walkZone = new WalkZone(new int[]{0, 0, 100, 0, 100, 100, 0, 100});
In my project it works like a charm, even with very intricated polygons. Hope it helps!!
Most answers suggest triangulation, which is fine, but you can also do it using the stencil buffer. It handles both convex and concave polygons. This may be a better solution if your polygon changes a lot, since otherwise you'd have to do triangulation every frame. Also, this solution properly handles self intersecting polygons, which EarClippingTriangulator does not.
FloatArray vertices = ... // The polygon x,y pairs.
Color color = ... // The color to draw the polygon.
ShapeRenderer shapes = ...
ImmediateModeRenderer renderer = shapes.getRenderer();
Gdx.gl.glClearStencil(0);
Gdx.gl.glClear(GL20.GL_STENCIL_BUFFER_BIT);
Gdx.gl.glEnable(GL20.GL_STENCIL_TEST);
Gdx.gl.glStencilFunc(GL20.GL_NEVER, 0, 1);
Gdx.gl.glStencilOp(GL20.GL_INVERT, GL20.GL_INVERT, GL20.GL_INVERT);
Gdx.gl.glColorMask(false, false, false, false);
renderer.begin(shapes.getProjectionMatrix(), GL20.GL_TRIANGLE_FAN);
renderer.vertex(vertices.get(0), vertices.get(1), 0);
for (int i = 2, n = vertices.size; i < n; i += 2)
renderer.vertex(vertices.get(i), vertices.get(i + 1), 0);
renderer.end();
Gdx.gl.glColorMask(true, true, true, true);
Gdx.gl.glStencilOp(GL20.GL_ZERO, GL20.GL_ZERO, GL20.GL_ZERO);
Gdx.gl.glStencilFunc(GL20.GL_EQUAL, 1, 1);
Gdx.gl.glEnable(GL20.GL_BLEND);
shapes.setColor(color);
shapes.begin(ShapeType.Filled);
shapes.rect(-9999999, -9999999, 9999999 * 2, 9999999 * 2);
shapes.end();
Gdx.gl.glDisable(GL20.GL_STENCIL_TEST);
To use the stencil buffer, you must specify the number of bits for the stencil buffer when your app starts. For example, here is how to do that using the LWJGL2 backend:
LwjglApplicationConfiguration config = new LwjglApplicationConfiguration();
config.stencil = 8;
new LwjglApplication(new YourApp(), config);
For more information on this technique, try one of these links:
http://commaexcess.com/articles/7/concave-polygon-triangulation-shortcut
http://glprogramming.com/red/chapter14.html#name13
http://what-when-how.com/opengl-programming-guide/drawing-filled-concave-polygons-using-the-stencil-buffer-opengl-programming/
I have created Polygon object to wrap my airplane (size of airplane's TextureRegion is 256x74, but size of this one in a game is 70x20). So:
TextureRegion[] texRegsAirplane = TextureRegion.split(textureAirplane, 256, 74);
Rectangle bounds = new Rectangle(0, 0, 70, 20);
Polygon polygon = new Polygon(new float[]{0,0,bounds.width,0,bounds.width,bounds.height,0,bounds.height,0,0});
After that in my update function I update position of it:
public void update(float delta){
Vector2 v = getPosition();
v.add(velocity);
polygon.setPosition(v.x, v.y);
}
Then I render polygon to know where it is:
public void render(SpriteBatch spriteBatch, float pixelPerUnitX, float pixelPerUnitY){
spriteBatch.draw(testTexture,polygon.getX()*pixelPerUnitX, polygon.getY()*pixelPerUnitY,
polygon.getBoundingRectangle().width*pixelPerUnitX,polygon.getBoundingRectangle().height*pixelPerUnitY);
}
At the end I create 2 airplanes and make them fly to each other and every iteration I try to detect collision like below:
public void detectCollision(){
for(Airplane airplane1 : Airplanes){
for(Airplane airplane2 : Airplanes){
if(Intersector.overlapConvexPolygons(airplane1.getPolygon(), airplane2.getPolygon())){
//COLLISION DON'T HAPPEN!!!
}
}
}
I see that 2 rectangles move to each other and intersect, but overlapConvexPolygons function doesn't work! Why?
I've solved this problem. I incorrectly specified vertices. I needed to get rectangular polygon, so I had to use following:
polygon = new Polygon(new float[]{0,0,bounds.width,0,bounds.width,bounds.height,0,bounds.height});
and do not forget set origin if you are going to rotate polygon object:
polygon.setOrigin(bounds.width/2, bounds.height/2);
Now it works perfect!
Hey all I'm trying to implement 3D picking into my program, and it works perfectly if I don't move from the origin. It is perfectly accurate. But if I move the model matrix away from the origin (the viewmatrix eye is still at 0,0,0) the picking vectors are still drawn from the original location. It should still be drawing from the view matrix eye (0,0,0) but it isn't. Here's some of my code to see if you can find out why..
Vector3d near = unProject(x, y, 0, mMVPMatrix, this.width, this.height);
Vector3d far = unProject(x, y, 1, mMVPMatrix, this.width, this.height);
Vector3d pickingRay = far.subtract(near);
//pickingRay.z *= -1;
Vector3d normal = new Vector3d(0,0,1);
if (normal.dot(pickingRay) != 0 && pickingRay.z < 0)
{
float t = (-5f-normal.dot(mCamera.eye))/(normal.dot(pickingRay));
pickingRay = mCamera.eye.add(pickingRay.scale(t));
addObject(pickingRay.x, pickingRay.y, pickingRay.z+.5f, Shape.BOX);
//a line for the picking vector for debugging
PrimProperties a = new PrimProperties(); //new prim properties for size and center
Prim result = null;
result = new Line(a, mCamera.eye, far);//new line object for seeing look at vector
result.createVertices();
objects.add(result);
}
public static Vector3d unProject(
float winx, float winy, float winz,
float[] resultantMatrix,
float width, float height)
{
winy = height-winy;
float[] m = new float[16],
in = new float[4],
out = new float[4];
Matrix.invertM(m, 0, resultantMatrix, 0);
in[0] = (winx / width) * 2 - 1;
in[1] = (winy / height) * 2 - 1;
in[2] = 2 * winz - 1;
in[3] = 1;
Matrix.multiplyMV(out, 0, m, 0, in, 0);
if (out[3]==0)
return null;
out[3] = 1/out[3];
return new Vector3d(out[0] * out[3], out[1] * out[3], out[2] * out[3]);
}
Matrix.translateM(mModelMatrix, 0, this.diffX, this.diffY, 0); //i use this to move the model matrix based on pinch zooming stuff.
Any help would be greatly appreciated! Thanks.
I wonder which algorithm you have implemented. Is it a ray casting approach to the problem?
I didn't focus much on the code itself but this looks a way too simple implementation to be a fully operational ray casting solution.
In my humble experience, i would like to suggest you, depending on the complexity of your final project (which I don't know), to adopt a color picking solution.
This solution is usually the most flexible and the easiest to be implemented.
It consist in the rendering of the objects in your scene with unique flat colors (usually you disable lighting as well in your shaders) to a backbuffer...a texture, then you acquire the coordinates of the click (touch) and you read the color of the pixel in that specific coordinates.
Having the color of the pixel and the tables of the colors of the different objects you rendered, makes possible for you to understand what the user clicked from a logical perspective.
There are other approaches to the object picking problem, this is probably universally recognized as the fastest one.
Cheers
Maurizio