Related
I have been trying to add mouse click detection on the triangles of the mesh, but it seems that I am doing something wrong and I cannot figure out how to solve the problem.
So before explaining the problem I will define the environment(the full code is available at http://pastebin.com/TxfNuYXZ):
Camera position
cam = new OrthographicCamera(10, 9);
cam.position.set(0, 5.35f, 2f);
cam.lookAt(0, 0, 0);
cam.near = 0.5f;
cam.far = 12f;
Mesh renders 4 vertices.
mesh = new Mesh(true, NUM_COLUMNS * NUM_LINES, (NUM_COLUMNS * 6 - 6) * (NUM_LINES - 1), VertexAttribute.Position(), VertexAttribute.ColorUnpacked());
mesh.setVertices(new float[] {
0, 0, 0, 0, 1, 0, 1,
1, 0, 0, 0, 1, 0, 1,
0, 0, 1, 0, 1, 0, 1,
1, 0, 1, 0, 1, 0, 1 });
mesh.setIndices(new short[] { 2, 0, 1, 2, 3, 1 });
So when I run the application I try to check if the click was done inside some of the triangles of the mesh. Now the result depends on the position of the camera. When the camera has almost top view(like in the following picture), corresponding to around 6 on Y axes, the click point is being correctly translated to the coordinates and corresponds to what is actually being seen.
When I move camera on the Y axes to lower position (around 2 or 3), so the image looks like the following one
the click is being detected in the completely wrong positions (the red line shows the place where the click is detected).. Which seems to be right according to the coordinates, but not according to what is being seen..
I would like to understand what an I missing to be able to detect clicks on what actually is being seen? The code I use to detect the click is the following:
#Override
public boolean touchDown(int screenX, int screenY, int pointer, int button) {
Ray ray = cam.getPickRay(screenX, screenY);
Vector3 intersection = new Vector3();
float[] v = new float[NUM_COLUMNS * NUM_LINES * VERTEX_SIZE];
short[] i = new short[(NUM_COLUMNS * 6 - 6) * (NUM_LINES - 1)];
mesh.getIndices(i);
if (Intersector.intersectRayTriangles(ray, mesh.getVertices(v), i, VERTEX_SIZE, intersection)) {
System.out.println(intersection);
}
return false;
}
Thanks a lot for your help!
Basically, after several days of drawing and some math I have found the source of the problem. The Vertex Shader, in order to determine the position of the vertices, was performing a_position * u_projectionViewMatrix multiplication, which was resulting on what was looking fine on the screen, but actually when you compare with actual coordinates of the mesh it was wrong. Now if you check the examples at enter link description here, you can see, that gl_Position is being calculated by multiplying u_projectionViewMatrix * a_position. Making the correct calculation made the trick.
I also had to change the camera to perspective, since the mesh was not rendered how I wanted it to.
I have two objects base and weapon and I need to set rotate point of weapon to position of base.
public Test(){
position1 = new Vector3(0,0,0);
baseModel = modelLoader.loadModel(Gdx.files.getFileHandle("data/models/tower/bases/base1.g3db", FileType.Internal));
base = new Base(baseModel, position1);
position2 = new Vector3(3,10,5);
weaponModel = modelLoader.loadModel(Gdx.files.getFileHandle("data/models/tower/weapons/weapon2.g3db", FileType.Internal));
weapon = new Weapon(weaponModel, position2);
}
Here is update method
public void update(float delta){
weapon.transform.rotate(0, 1, 0, 45*(delta/2));
base.transform.rotate(0, 1, 0, 45*(delta/2));
}
Thank you for answer
A rotation around a point is the same as translating to that point, rotating and then translating back.
So this process consists of 3 steps:
Translate to the rotationPoint, for example translate(3, 0, 0)
Rotate arround the center (which is now the rotationPoint), for example rotate(0,1,0, 45*delta)
Translate back (the translation is relative to the rotation), for example translate(-3, 0, 0);
In this case the code then looks like this:
weapon.transform.translate(3, 0, 0).rotate(0,1,0, 45*delta).translate(-3, 0, 0);
I want to draw some (filled) polygons with libGDX. It shoudn't be filled with a graphic/texture. I have only the vertices of the polygon (closed path) and tried to visualize with meshes but at some point this is not the best solution, I think.
My code for an rectangle is:
private Mesh mesh;
#Override
public void create() {
if (mesh == null) {
mesh = new Mesh(
true, 4, 0,
new VertexAttribute(Usage.Position, 3, "a_position")
);
mesh.setVertices(new float[] {
-0.5f, -0.5f, 0
0.5f, -0.5f, 0,
-0.5f, 0.5f, 0,
0.5f, 0.5f, 0
});
}
}
// ...
#Override
public void render() {
Gdx.gl.glClear(GL10.GL_COLOR_BUFFER_BIT);
mesh.render(GL10.GL_TRIANGLE_STRIP, 0, 4);
}
is there a function or something to draw filled polygons in an easier way?
Since recent updates of LibGDX, #Rus answer is using deprecated functions. However, I give him/her credits for the new updated version below:
PolygonSprite poly;
PolygonSpriteBatch polyBatch = new PolygonSpriteBatch(); // To assign at the beginning
Texture textureSolid;
// Creating the color filling (but textures would work the same way)
Pixmap pix = new Pixmap(1, 1, Pixmap.Format.RGBA8888);
pix.setColor(0xDEADBEFF); // DE is red, AD is green and BE is blue.
pix.fill();
textureSolid = new Texture(pix);
PolygonRegion polyReg = new PolygonRegion(new TextureRegion(textureSolid),
new float[] { // Four vertices
0, 0, // Vertex 0 3--2
100, 0, // Vertex 1 | /|
100, 100, // Vertex 2 |/ |
0, 100 // Vertex 3 0--1
}, new short[] {
0, 1, 2, // Two triangles using vertex indices.
0, 2, 3 // Take care of the counter-clockwise direction.
});
poly = new PolygonSprite(polyReg);
poly.setOrigin(a, b);
polyBatch = new PolygonSpriteBatch();
For good triangulating algorithms if your polygon is not convex, see the almost-linear earclipping algorithm from Toussaint (1991)
Efficient triangulation of simple polygons, Godfried Toussaint, 1991
Here is a libGDX example which draws a 2D concave polygon.
Define class members for PolygonSprite PolygonSpriteBatch
PolygonSprite poly;
PolygonSpriteBatch polyBatch;
Texture textureSolid;
Create instances, 1x1 size texture used with red pixel as workaround. An array of coordinates (x, y) is used for initialization of the polygon.
ctor() {
textureSolid = makeTextureBox(1, 0xFFFF0000, 0, 0);
float a = 100;
float b = 100;
PolygonRegion polyReg = new PolygonRegion(new TextureRegion(textureSolid),
new float[] {
a*0, b*0,
a*0, b*2,
a*3, b*2,
a*3, b*0,
a*2, b*0,
a*2, b*1,
a*1, b*1,
a*1, b*0,
});
poly = new PolygonSprite(polyReg);
poly.setOrigin(a, b);
polyBatch = new PolygonSpriteBatch();
}
Draw and rotate polygon
void draw() {
super.draw();
polyBatch.begin();
poly.draw(polyBatch);
polyBatch.end();
poly.rotate(1.1f);
}
I believe the ShapeRenderer class now has a polygon method for vertex defined polygons:
ShapeRenderer.polygon()
You can use the ShapeRenderer API to draw simple, solid-color shapes with Libgdx.
The code you've given is a reasonable way to draw solid color polygons too. Its much more flexible than ShapeRenderer, but is a good bit more complicated. You'll need to use glColor4f to set the color, or add a Usage.Color attribute to each vertex. See the SubMeshColorTest example for more details on the first approach and the MeshColorTexture example for details on the second approach.
Another option to think about is using sprite textures. If you're only interested in simple solid colors objects, you can use very simple 1x1 textures of a single color and let the system stretch that across the sprite. Much of Libgdx and the underlying hardware are really optimized for rendering textures, so you may find it easier to use even if you're not really taking advantage of the texture contents. (You can even use a 1x1 white texture, and then use a SpriteBatch with setColor and draw()
to draw different color rectangles easily.)
You can also mix and match the various approaches, too.
Use triangulation algorithm and then draw all triangles as GL_TRIANGLE_STRIP
http://www.personal.psu.edu/cxc11/AERSP560/DELAUNEY/13_Two_algorithms_Delauney.pdf
just wanted to share my related solution with you, namely for implementing and drawing a walkZone with scene2d. I basically had to put together the different suggestions of the others' posts:
1) The WalkZone:
import com.badlogic.gdx.graphics.Pixmap;
import com.badlogic.gdx.graphics.Texture;
import com.badlogic.gdx.graphics.g2d.PolygonRegion;
import com.badlogic.gdx.graphics.g2d.TextureRegion;
import com.badlogic.gdx.math.EarClippingTriangulator;
import com.badlogic.gdx.math.Polygon;
import com.mygdx.game.MyGame;
public class WalkZone extends Polygon {
private PolygonRegion polygonRegion = null;
public WalkZone(float[] vertices) {
super(vertices);
if (MyGame.DEBUG) {
Pixmap pix = new Pixmap(1, 1, Pixmap.Format.RGBA8888);
pix.setColor(0x00FF00AA);
pix.fill();
polygonRegion = new PolygonRegion(new TextureRegion(new Texture(pix)),
vertices, new EarClippingTriangulator().computeTriangles(vertices).toArray());
}
}
public PolygonRegion getPolygonRegion() {
return polygonRegion;
}
}
2) The Screen:
you can then add a listener in the desired Stage:
myStage.addListener(new InputListener() {
#Override
public boolean touchDown(InputEvent event, float x, float y, int pointer, int button) {
if (walkZone.contains(x, y)) player.walkTo(x, y);
// or even directly: player.addAction(moveTo ...
return super.touchDown(event, x, y, pointer, button);
}
});
3) The implementation:
The array passed to te WZ constructor is a set of x,y,x,y... points. If you put them counter-clockwise, it works (I didn't check the other way, nor know how it exactly works); for example this generates a 100x100 square:
yourScreen.walkZone = new WalkZone(new int[]{0, 0, 100, 0, 100, 100, 0, 100});
In my project it works like a charm, even with very intricated polygons. Hope it helps!!
Most answers suggest triangulation, which is fine, but you can also do it using the stencil buffer. It handles both convex and concave polygons. This may be a better solution if your polygon changes a lot, since otherwise you'd have to do triangulation every frame. Also, this solution properly handles self intersecting polygons, which EarClippingTriangulator does not.
FloatArray vertices = ... // The polygon x,y pairs.
Color color = ... // The color to draw the polygon.
ShapeRenderer shapes = ...
ImmediateModeRenderer renderer = shapes.getRenderer();
Gdx.gl.glClearStencil(0);
Gdx.gl.glClear(GL20.GL_STENCIL_BUFFER_BIT);
Gdx.gl.glEnable(GL20.GL_STENCIL_TEST);
Gdx.gl.glStencilFunc(GL20.GL_NEVER, 0, 1);
Gdx.gl.glStencilOp(GL20.GL_INVERT, GL20.GL_INVERT, GL20.GL_INVERT);
Gdx.gl.glColorMask(false, false, false, false);
renderer.begin(shapes.getProjectionMatrix(), GL20.GL_TRIANGLE_FAN);
renderer.vertex(vertices.get(0), vertices.get(1), 0);
for (int i = 2, n = vertices.size; i < n; i += 2)
renderer.vertex(vertices.get(i), vertices.get(i + 1), 0);
renderer.end();
Gdx.gl.glColorMask(true, true, true, true);
Gdx.gl.glStencilOp(GL20.GL_ZERO, GL20.GL_ZERO, GL20.GL_ZERO);
Gdx.gl.glStencilFunc(GL20.GL_EQUAL, 1, 1);
Gdx.gl.glEnable(GL20.GL_BLEND);
shapes.setColor(color);
shapes.begin(ShapeType.Filled);
shapes.rect(-9999999, -9999999, 9999999 * 2, 9999999 * 2);
shapes.end();
Gdx.gl.glDisable(GL20.GL_STENCIL_TEST);
To use the stencil buffer, you must specify the number of bits for the stencil buffer when your app starts. For example, here is how to do that using the LWJGL2 backend:
LwjglApplicationConfiguration config = new LwjglApplicationConfiguration();
config.stencil = 8;
new LwjglApplication(new YourApp(), config);
For more information on this technique, try one of these links:
http://commaexcess.com/articles/7/concave-polygon-triangulation-shortcut
http://glprogramming.com/red/chapter14.html#name13
http://what-when-how.com/opengl-programming-guide/drawing-filled-concave-polygons-using-the-stencil-buffer-opengl-programming/
Hey all I'm trying to implement 3D picking into my program, and it works perfectly if I don't move from the origin. It is perfectly accurate. But if I move the model matrix away from the origin (the viewmatrix eye is still at 0,0,0) the picking vectors are still drawn from the original location. It should still be drawing from the view matrix eye (0,0,0) but it isn't. Here's some of my code to see if you can find out why..
Vector3d near = unProject(x, y, 0, mMVPMatrix, this.width, this.height);
Vector3d far = unProject(x, y, 1, mMVPMatrix, this.width, this.height);
Vector3d pickingRay = far.subtract(near);
//pickingRay.z *= -1;
Vector3d normal = new Vector3d(0,0,1);
if (normal.dot(pickingRay) != 0 && pickingRay.z < 0)
{
float t = (-5f-normal.dot(mCamera.eye))/(normal.dot(pickingRay));
pickingRay = mCamera.eye.add(pickingRay.scale(t));
addObject(pickingRay.x, pickingRay.y, pickingRay.z+.5f, Shape.BOX);
//a line for the picking vector for debugging
PrimProperties a = new PrimProperties(); //new prim properties for size and center
Prim result = null;
result = new Line(a, mCamera.eye, far);//new line object for seeing look at vector
result.createVertices();
objects.add(result);
}
public static Vector3d unProject(
float winx, float winy, float winz,
float[] resultantMatrix,
float width, float height)
{
winy = height-winy;
float[] m = new float[16],
in = new float[4],
out = new float[4];
Matrix.invertM(m, 0, resultantMatrix, 0);
in[0] = (winx / width) * 2 - 1;
in[1] = (winy / height) * 2 - 1;
in[2] = 2 * winz - 1;
in[3] = 1;
Matrix.multiplyMV(out, 0, m, 0, in, 0);
if (out[3]==0)
return null;
out[3] = 1/out[3];
return new Vector3d(out[0] * out[3], out[1] * out[3], out[2] * out[3]);
}
Matrix.translateM(mModelMatrix, 0, this.diffX, this.diffY, 0); //i use this to move the model matrix based on pinch zooming stuff.
Any help would be greatly appreciated! Thanks.
I wonder which algorithm you have implemented. Is it a ray casting approach to the problem?
I didn't focus much on the code itself but this looks a way too simple implementation to be a fully operational ray casting solution.
In my humble experience, i would like to suggest you, depending on the complexity of your final project (which I don't know), to adopt a color picking solution.
This solution is usually the most flexible and the easiest to be implemented.
It consist in the rendering of the objects in your scene with unique flat colors (usually you disable lighting as well in your shaders) to a backbuffer...a texture, then you acquire the coordinates of the click (touch) and you read the color of the pixel in that specific coordinates.
Having the color of the pixel and the tables of the colors of the different objects you rendered, makes possible for you to understand what the user clicked from a logical perspective.
There are other approaches to the object picking problem, this is probably universally recognized as the fastest one.
Cheers
Maurizio
I'm trying to detect the positions of billiards balls on a table from an image taken at a perspective angle. I'm using the getPerspectiveTransform() method to find the transformation matrix and I want to apply that to only the circles I detect using HoughCircles. I'm trying to go from a rather large trapezoidal shape to a smaller rectangular shape. I don't want to do the transformation on the image first and then find the HoughCircles because the image gets too warped for houghcircles to provide useful results.
Here's my code:
CvMat mmat = cvCreateMat(3,3,CV_32FC1);
double srcX1 = 462;
double srcX2 = 978;
double srcX3 = 1440;
double srcX4 = 0;
double srcY = 241;
double srcHeight = 772;
double dstX = 56.8;
double dstY = 33.5;
double dstWidth = 262.4;
double dstHeight = 447.3;
CvSeq seq = cvHoughCircles(newGray, circles, CV_HOUGH_GRADIENT, 2.1d, (double)newGray.height()/40, 85d, 65d, 5, 50);
JavaCV.getPerspectiveTransform(new double[]{srcX1, srcY, srcX2,srcY, srcX3, srcHeight, srcX4, srcHeight},
new double[]{dstX, dstY, dstWidth, dstY, dstWidth, dstHeight, dstX, dstHeight}, mmat);
cvWarpPerspective(seq, seq, mmat);
for(int j=0; j<seq.total(); j++){
CvPoint3D32f point = new CvPoint3D32f(cvGetSeqElem(seq, j));
float xyr[] = {point.x(),point.y(),point.z()};
CvPoint center = new CvPoint(Math.round(xyr[0]), Math.round(xyr[1]));
int radius = Math.round(xyr[2]);
cvCircle(gray, center, 3, CvScalar.GREEN, -1, 8, 0);
cvCircle(gray, center, radius, CvScalar.BLUE, 3, 8, 0);
}
The problem is I get this error on the warpPerspective() method:
error: (-215) seq->total > 0 && CV_ELEM_SIZE(seq->flags) == seq->elem_size in function cv::Mat cv::cvarrToMat(const CvArr*, bool, bool, int)
Also I guess it's worth mentioning that I'm using JavaCV, in case the method calls look a bit different than what you're used to. Thanks for any help.
Answer:
the problem with what you want to do (besides the obvious, opencv wont let you) is that the radius cant really be warped correctly. AFAIK the xy coordinates are pretty easy to calculate x'=((m00x+m01y+m02)/(m20x+m21y+m22)) y'=((m10x+m11y+m12)/(m20x+m21y_m22)) when m is the transformation matrix. the radius you can hack by transforming all the points of the original circle and then find the max distance between x'y' and those points (atleast if the radius in the warped image is expected to cover all those points)
btw, mIJx = m(i,j)*x (just to clarify)
End Answer.
Everything i write is according to the c++ version, i've never used JavaCV but from what i could see its just a wrapper that calls the native c++ lib.
CvSeq is a sequance data structure that behaves like a linked list.
the assert your application crushes at is
CV_Assert(seq->total > 0 && CV_ELEM_SIZE(seq->flags) == seq->elem_size);
which means that either your seq instance is empty (total is the number of elements in the sequence) or somehow the inner seq flags are corrupted.
I'd recommend that you'd check the total member of your CvSeq, and the cvHoughCircles call.
all of this occurs before the actual implementation of cvWarpPerspective (its the first line in the implementation, that only converts your CvSeq to cv::Mat).. so its not the warping but what you're doing before that.
anyway, to understand whats wrong with cvHoughCircles we'll need more info about the creation of newGray and circles.
here is an example i've found on the javaCV page (Link)
IplImage gray = cvCreateImage( cvSize( img.width, img.height ), IPL_DEPTH_8U, 1 );
cvCvtColor( img, gray, CV_RGB2GRAY );
// smooth it, otherwise a lot of false circles may be detected
cvSmooth(gray,gray,CV_GAUSSIAN,9,9,2,2);
CvMemStorage circles = CvMemStorage.create();
CvSeq seq = cvHoughCircles(gray, circles.getPointer(), CV_HOUGH_GRADIENT,
2, img.height/4, 100, 100, 0, 0);
for(int i=0; i<seq.total; i++){
float xyr[] = cvGetSeqElem(seq,i).getFloatArray(0, 3);
CvPoint center = new CvPoint(Math.round(xyr[0]), Math.round(xyr[1]));
int radius = Math.round(xyr[2]);
cvCircle(img, center.byValue(), 3, CvScalar.GREEN, -1, 8, 0);
cvCircle(img, center.byValue(), radius, CvScalar.BLUE, 3, 8, 0);
}
from what i've seen in the implementation of cvHoughCircles, the answer is saved in the circles buff and at the end they create from it the CvSeq to return, so if you've allocated the circles buff wrong, it wont work.
EDIT:
as you can see, the CvSeq instance in case of the return from cvHoughCircles is a list of point-values, that is probably why the assertion failed. you cannot convert this CvSeq into a cv::Mat.. because its just not a cv::Mat. to get only the circles returned from cvHoughCircles in an cv::Mat instance, you'll need to create a new cv::Mat instance and than draw onto it all the circles in the CvSeq - as seen in the provided example above.
than the warping will work (you'll have a cv::Mat instance, and that is what the function expect - a cv::Mat as the only element in the CvSeq)
END EDIT
here is the c++ reference for CvSeq
and if you want to fiddle with the source code than
cvarrToMat is in matrix.cpp
CV_ELEM_SIZE is in types_c.h
cvWarpPerspective is in imgwarp.cpp
cvHoughCircles is in hough.cpp
I hope that will help.
BTW, your next error will probably be:
cv::warpPerspective in the C++ OpencCv asserts that dst.data != src.data
thus
cvWarpPerspective(seq, seq, mmat);
wont work cause your source mat and destination mat referencing the same data.
Not all the functions in OpenCV (and image processing in general) work in-situ (because there is no in-situ algorithm or because its slower then the other version eg. transpose of an n*n mat will work in-situ, but n*m where n!=m will be harder to do in-situ and might be slower)
you cant assume the using the src matrix as the dst will work.