I am implementing 3D modeling with LibGDX, and I want to manually rotate objects with the mouse, but I just I can't find any tutorials and examples that are right.
EDIT: Originally I only asked this question regarding rotating models, but I've discovered the same problem exists when rotating the camera.
Click here for the source code of a fully functional demo in github.com
Here's a snapshot of rotating the view and a model:
I want the object to rotate in the direction it is dragged by the mouse, no matter which direction it happens to be orientated at the time. As it is now, when I first drag the mouse to the right, the object rotates to the right about the screen Y axis as expected; but then when I drag the mouse upward I want the object to rotate upward about the screen X axis, but instead it spins to the left about the screen Z axis. Think of it like a floating ball in a bowl of water - whichever way you swipe at it, it rotates in that direction.
It seems to me that the mouse movement is transforming the objects directly in their local coordinate system; but instead I think it needs to transform the axis of rotation itself from the Screen Coordinate System into the Object Coordinate System before applying it to the object. I don't know, but it may be even more complicated than that.
I would really appreciate any insight or help to resolve this; I'm running out of hair to pull out... Thanks in advance.
LibGDX implements OpenGL. The terminology we use in OpenGL can help us to know how LibGDX works behind the scene. The other technology that implements OpenGL is WebGL, WebGL uses JavaScript. LibGDX uses Java. Once we know how OpenGL works, drawing objects and rotating objects should be fun. Of course depending on what we are drawing. OpenGL is well
documented. OpenGL itself always works the same. The first question, should be what are we drawing? And what are the objectives of the project. So, you want to draw the cube and rotate it. Cool. Once we can draw one cube and rotate it, we can add more objects in the scene. Cool. Strategically you can divide your project into parts.
Draw your object.
Rotate it.
Add more objects.
Rotate them.
We are done.
If you want to rotate the view as well, you can use the same process as above with some modifications. For example:
Draw the view.
Draw objects inside the view.
Rotate the objects.
Rotate the view.
On the other hand you can just use the camera and move it around the scene.
Done.
To make things worse LibGDX can extend many different classes, the programmer have to implement all abstract methods. Your code might look differently or some functions behave differently depending on which class you extend or implement in your project. The documentation about those classes is stochastic. Each and every abstract class comes with its abstract methods. The programmer should release any other resources allocated by LibGDX by using dispose() method. With just few changes your code should work as expected.
For Example:
//
package com.mygdx.game;
import com.badlogic.gdx.ApplicationAdapter;
import com.badlogic.gdx.ApplicationListener;
import com.badlogic.gdx.Input.Buttons;
import com.badlogic.gdx.Input.Keys;
import com.badlogic.gdx.InputProcessor;
//import etc...
public class Space extends ApplicationAdapter implements ApplicationListener, InputProcessor {
SpriteBatch batch;
BitmapFont font;
float backgroundColor[];
Cube cubes[];
ModelBatch modelBatch;
int selectedCube;
PerspectiveCamera camera;
Environment environment;
int xCubes = 27;
int touchedButton;
int lastX, lastY;
float Theta, Phi, dX, dY;
Vector3 screenAOR;
float screenAng;
float point[];
int side[];
int front[];
float w;
float h;
Model viewM;
ModelInstance viewMin;
Vector3 position;
ColorAttribute attrib;
Vector3 cubePositions[];
boolean drag;
#Override
public void create () {
drag = false;
Theta = 0;
Phi = 0;
batch = new SpriteBatch();
font = new BitmapFont();
font.setColor(Color.FOREST);
w = Gdx.graphics.getWidth();
h = Gdx.graphics.getHeight();
modelBatch = new ModelBatch();
screenAOR = new Vector3();
camera = new PerspectiveCamera(67f, 3f, 2f);
camera.position.set(10f, -10f, 70f);
camera.lookAt(Vector3.Zero);
camera.up.set(Vector3.Y);
camera.near = 1f;
camera.far = 500f;
camera.update();
backgroundColor = new float[]{.9f, .9f, .7f};
environment = new Environment();
environment.set(new ColorAttribute( ColorAttribute.AmbientLight, .6f, .6f, .6f, 1f));
environment.add(new DirectionalLight().set(.8f, .8f, .8f, 50f, 50f, 50f));
environment.add(new DirectionalLight().set(.5f, .5f, .5f, -50f, -50f, 50f));
spaceModel();
Gdx.input.setInputProcessor(this);
//Gdx.graphics.requestRendering();
}
#Override
public void render () {
Gdx.gl.glClearColor(1, 0, 0, 1);
//Gdx.gl.glClear(GL20.GL_COLOR_BUFFER_BIT);
//Gdx.gl.glClearColor(backgroundColor[0], backgroundColor[1], backgroundColor[2], 1f);
Gdx.gl.glClear(GL20.GL_COLOR_BUFFER_BIT | GL20.GL_DEPTH_BUFFER_BIT);
Gdx.gl.glEnable(GL20.GL_DEPTH_TEST);
Gdx.gl.glEnable(GL20.GL_CULL_FACE);
batch.begin();
modelBatch.begin(camera);
modelBatch.render(viewMin, environment);
for(int i = 0; i < cubes.length; i++){
modelBatch.render(cubes[i].modelInstance, environment);
}
font.draw(batch, "Space pro...", 10, 100);
batch.end();
modelBatch.end();
}
#Override
public void dispose () {
batch.dispose();
modelBatch.dispose();
font.dispose();
viewM.dispose();
}
/*///////////////////////////////////
//Implements all abstract methods.
//
*/////////////////////////////////
#Override
public boolean touchDragged(int screenX, int screenY, int pointer) {
//lastX -= screenX;
//lastY -= screenY;
//float aspRatio = w/h;
//float angle = 40.0f;
moveModel(screenX, screenY);
// distance of mouse movement
//screenAng = (float) Math.sqrt( ((lastX * lastX) + (lastY * lastY)) );
//screenAng = (float) Math.tan((angle * 0.5)* (Math.PI/180));
// direction vector of the AOR
//screenAOR.set((lastY/screenAng), (lastX/screenAng), 0f );
//screenAOR.set(projection(angle,aspRatio,h,w));
//public Vector3 set(float x, float y, float z)
screenAOR.set(dX, dY, 0f);
if ( touchedButton == 0 ){
//public Matrix4 rotate(Vector3 axis, float degrees)
//cubes[ selectedCube ].modelInstance.transform.rotate( screenAOR, screenAng );
//public Matrix4 rotate(float axisX, float axisY, float axisZ, float degrees)
cubes[ selectedCube ].modelInstance.transform.rotate(dX, dY, 0f, Theta);
cubes[ selectedCube ].modelInstance.transform.rotate(dX, dY, 0f, Phi);
}
else{
//public void rotateAround(Vector3 point, Vector3 axis, float angle)
//camera.rotateAround( Vector3.Zero, screenAOR, (-screenAng/5.5f) );
//public void rotate(float angle, float axisX, float axisY, float axisZ)
//camera.rotate(Theta, dX, dY, 0f);
//camera.rotate(Phi, dX, dY, 0f);
//camera.rotateAround(position, screenAOR, Theta);
camera.rotateAround(Vector3.Zero, screenAOR, Theta);
camera.update();
//camera.rotateAround(position, screenAOR, Phi);
camera.rotateAround(Vector3.Zero, screenAOR, Phi);
camera.update();
viewMin.transform.rotate(dX, dY, 0f, Theta);
viewMin.transform.rotate(dX, dY, 0f, Phi);
}
//Gdx.graphics.requestRendering();
//Gdx.app.log("touchDragged:", screenAng+" : "+screenAOR+" : "+touchedButton);
return true;
}
#Override
public boolean touchDown(int screenX, int screenY, int pointer, int button) {
drag = true;
if(button == Buttons.LEFT){
touchedButton = 0;
}
else{
touchedButton = button;
}
Gdx.app.log("touchDown:", button+" : "+screenX+" : "+screenY+" : "+pointer);
return true;
}
#Override
public boolean keyDown(int i) {
float move = 1.0f;
float pX = w/10;
float pY = h/10;
if(i == Keys.LEFT){
pX -= move;
//public void rotate(float angle, float axisX, float axisY, float axisZ)
//camera.rotate(-45, pX, 0f, 0f);
camera.rotate(-45, 0f, pY, 0f);
//camera.update();
//public void translate(float x, float y, float z)
//camera.translate(move, 0f, 0f);
}
if(i == Keys.RIGHT){
pX += move;
//camera.rotate(45, pX, 0f, 0f);
camera.rotate(45, 0f, pY, 0f);
//camera.update();
//camera.translate(-move, 0f, 0f);
}
if(i == Keys.DOWN){
pY -= move;
//camera.rotate(-45, 0f, pY, 0f);
//camera.rotate(-45, pX, 0f, 0f);
camera.rotate(45, pX, 0f, 0f);
//camera.update();
//camera.translate(0f, 0f, move);
//camera.update();
//camera.translate(0f, move, 0f);
}
if(i == Keys.UP){
pY += move;
//camera.rotate(45, 0f, pY, 0f);
//camera.rotate(45, pX, 0f, 0f);
camera.rotate(-45, pX, 0f, 0f);
//camera.update();
//camera.translate(0f, 0f, -move);
//camera.update();
//camera.translate(0f, -move, 0f);
}
camera.update();
Gdx.app.log("KeyDown: ", pX+" : "+pY+" : "+i);
return true;
}
#Override
public boolean keyUp(int i) {
Gdx.app.log("KeyUp: ",i+" : ");
return false;
}
#Override
public boolean keyTyped(char c) {
//Gdx.app.log("KeyTyped: ",c+" : ");
return false;
}
#Override
public boolean touchUp(int i, int i1, int i2, int i3) {
drag = false;
Gdx.app.log("touchUp: ",i+" : "+i1+" : "+i2+" : "+i3);
return true;
}
#Override
public boolean mouseMoved(int i, int i1) {
if(!drag)
{
dX *= 0.96;
dY *= 0.96;
Theta += dX;
Phi += dY;
return false;
}
Gdx.app.log("mouseMoved: ", i+" : "+i1);
return false;
}
#Override
public boolean scrolled(int i) {
return false;
}
public void moveModel(int x2, int y2){
dX = (float) ((x2 - lastX)*2*(Math.PI/w));
dY = (float) ((y2 - lastY)*2*(Math.PI/h));
Theta += dX;
Phi += dY;
lastX = x2;
lastY = y2;
}
public void spaceModel(){
xCubes = 27;
selectedCube = 14;
ModelBuilder modelB = new ModelBuilder();
attrib = new ColorAttribute(1,Color.VIOLET);
Material m = new Material();
m.set(attrib);
//public Model createXYZCoordinates(float axisLength, Material material, long attributes)
viewM = modelB.createXYZCoordinates(w, m , 1);
cubePositions = new Vector3[xCubes];
for(int i = 0; i < xCubes; i++){
cubePositions[i] = new Vector3((i/9), ((i%9)/3), (i%3)).scl(20f).add(-20f, -20f, -20f);
}
cubes = new Cube[xCubes];
for(int i = 0; i < xCubes; i++){
cubes[i] = new Cube(cubePositions[i], (i == selectedCube));
}
viewMin = new ModelInstance(viewM);
position = cubePositions[0];
viewMin.transform.setTranslation(position);
Gdx.app.log("viewModel: ", w+" : "+h+" : "+w/h);
}
float[] projection(float angle, float a, float z1, float z2){
float ang = (float) Math.tan((angle * 0.5)* (Math.PI/180));
float[] proj = {
(float)0.5/ang, 0, 0, 0,
0, (float)0.5*a/ang, 0, 0,
0, 0, -(z2+z1)/(z2-z1), -1,
0, 0, (-2*z2*z1)/(z2-z1), 0
};
return proj;
}
}
/*////////////////////
//Draw cubes.
//
*/////////////////
class Cube{
Vector3 position;
Model model;
ModelInstance modelInstance;
Cube(Vector3 cubePosition, boolean selected) {
position = cubePosition;
compose(selected);
}
//etc...
}
//
When you are rotating the camera and rotating the objects, the direction can change or sometimes reversed. Depending at what angle is the object and the camera are at, at that point in time. It's like looking at the review mirror facing the opposite direction. So user's position and orientation in the scene is also important.
//
//
When you are looking at an object spinning “ < ---” around the circle, logically it is going to change direction “ ---> ”. When it reaches the far end “ <---> ”. i.e. From right to left and from left to right. Of course the user will still be using the same button. When you press the other buttons the same logic follows. Different sequences of rotations can result in different images as well. The other option which is time consuming is: translate(rotate(scale(geometry))). Eventually the entire image will be a single whole that is composed of its various parts. This process mighty help you to debug your code, and figuring out what caused the errors. With a little bit of maths, things can't get any worse. The science behind the object you are rotating is also important. When you are looking at the object rotating, the beauty is on the observers eye. e.g. Are you looking at the front side or the back side? Finally you have to use maths to get your model to behave the way you want it to behave.
Enjoy.
I asked the same question on one of the other sister forums, and got an answer that I was able to implement.
See the discussion here.
Here is the change to the code that made the whole thing work correctly:
Here is the change to the code that made the whole thing work correctly:
#Override public boolean touchDragged( int screenX, int screenY, int pointer )
{
lastX -= screenX;
lastY -= screenY;
// distance of mouse movement
screenAng = Vector3.len( lastX, lastY, 0f );
// direction vector of the AOR
screenAOR.set( lastY/screenAng, lastX/screenAng, 0f );
if ( touchedButton == 0 )
{ // rotate the part
// transform the screen AOR to a model rotation
Matrix4 camT, camR, camRi, modT, modR;
camT = new Matrix4();
camR = new Matrix4();
modT = new Matrix4();
modR = new Matrix4();
decompose( camera.view, camT, camR );
camRi = camR.cpy().inv();
decompose( cubes[ selectedCube ].modelInstance.transform, modT, modR );
tempMat.idt()
.mul( modT )
.mul( camRi )
.rotate( screenAOR, -screenAng )
.mul( camR )
.mul( modR );
cubes[ selectedCube ].modelInstance.transform.set( tempMat );
}
else if ( touchedButton == 1 )
{ // rotate the camera
// transform the AOR from screen CS to camera CS
// get the camera transformation matrix
tempMat.set( camera.view );
tempMat.translate( camera.position );
tempMat.inv();
// transform the screen AOR to a world AOR
worldAOR = transform( tempMat, screenAOR, worldAOR ).nor();
// apply the rotation of the angle about the world AOR to the camera
camera.rotateAround( Vector3.Zero, worldAOR, screenAng/5.5f );
camera.update();
}
lastX = screenX;
lastY = screenY;
Gdx.graphics.requestRendering();
return true;
}
Vector3 transform( Matrix4 mat, Vector3 from, Vector3 to )
{
// transform a vector according to a transformation matrix
to.x = from.dot( mat.val[ Matrix4.M00 ], mat.val[ Matrix4.M01 ],
mat.val[ Matrix4.M02 ] )+mat.val[ Matrix4.M03 ];
to.y = from.dot( mat.val[ Matrix4.M10 ], mat.val[ Matrix4.M11 ],
mat.val[ Matrix4.M12 ] )+mat.val[ Matrix4.M13 ];
to.z = from.dot( mat.val[ Matrix4.M20 ], mat.val[ Matrix4.M21 ],
mat.val[ Matrix4.M22 ] )+mat.val[ Matrix4.M23 ];
return to;
}
void decompose( Matrix4 m, Matrix4 t, Matrix4 r )
{
Matrix4 I4 = new Matrix4(); // Identity
for ( int i = 0; i < 4; i++ )
{
for ( int j = 0; j < 4; j++ )
{
if (i == 3 || j == 3)
{
r.val[ i*4+j ] = I4.val[ i*4+j ];
t.val[ i*4+j ] = m.val[ i*4+j ];
}
else
{
r.val[ i*4+j ] = m.val[ i*4+j ];
t.val[ i*4+j ] = I4.val[ i*4+j ];
}
}
}
}
I'm trying to convert a 3d point in OpenGL to a 2d point on screen to render a healthbar for a little game I'm writing. However, I'm having some trouble retrieving the x coordinate of where to draw the healthbar. Basically, the healthbar must appear to be above a player, but must always have the same width/height relative to the screen.
I tweaked a snippet of code I found from the accepted answer at Convert a 3D location to a 2D on-screen point. (XYZ => XY) and I now have this
public static int[] getScreenCoords(double x, double y, double z) {
FloatBuffer screenCoords = BufferUtils.createFloatBuffer(4);
IntBuffer viewport = BufferUtils.createIntBuffer(16);
FloatBuffer modelView = BufferUtils.createFloatBuffer(16);
FloatBuffer projection = BufferUtils.createFloatBuffer(16);
// int[] screenCoords = new double[4];
// int[] viewport = new int[4];
// double[] modelView = new double[16];
// double[] projection = new double[16];
GL11.glGetFloat(GL11.GL_MODELVIEW_MATRIX, modelView);
GL11.glGetFloat(GL11.GL_PROJECTION_MATRIX, projection);
GL11.glGetInteger(GL11.GL_VIEWPORT, viewport);
boolean result = GLU.gluProject((float) x, (float) y, (float) z, modelView, projection, viewport, screenCoords);
if (result) {
return new int[] { (int) screenCoords.get(3), (int) screenCoords.get(1) };
}
return null;
}
It seems to work fine with the y coordinate, however, x always returns 0 no matter what the angle is.
Many thanks in advance!
screenCoords.get(3) should be screenCoords.get(0) because the x position is stored at index 0. You also only actually need the capacity of screenCoords to be 3 floats, not 4.
I'm trying to use gluProject with my game. I've found a way to use it but I don't know how to deal with rotation.
Here is the code I made:
public static Vector2 getScreenCoordinates(float x, float y, float z, int height, int width)
{
FloatBuffer screen_coords = GLAllocation.createDirectFloatBuffer(4);
IntBuffer viewport = GLAllocation.createDirectIntBuffer(16);
FloatBuffer modelview = GLAllocation.createDirectFloatBuffer(16);
FloatBuffer projection = GLAllocation.createDirectFloatBuffer(16);
GL11.glGetFloat(2982, modelview);
GL11.glGetFloat(2983, projection);
GL11.glGetInteger(2978, viewport);
boolean result = GLU.gluProject(x, y, z, modelview, projection, viewport, screen_coords);
if (result)
{
float screen_x = screen_coords.get(0);
float screen_y = screen_coords.get(1);
float scrren_z = screen_coords.get(2);
screen_y -= height / 2;
screen_y = -screen_y;
screen_y += height / 2;
return new Vector2(screen_x, screen_y);
}
else
{
System.out.printf("Failed to convert 3D coords to 2D screen coords");
return null;
}
}
So x, y and z are coords in my 3D map. height and width are my window size.
How can I modify it to deal with rotation (yaw and pitch)?
Thanks.
Here is my new code:
public Vector2 worldToScreen(double x, double y, double z, int yaw, int pitch)
{
// Initialize the result buffer
FloatBuffer screen_coords = GLAllocation.createDirectFloatBuffer(4);
// Init the OpenGL buffers
IntBuffer viewport = GLAllocation.createDirectIntBuffer(16);
FloatBuffer modelview = GLAllocation.createDirectFloatBuffer(16);
FloatBuffer projection = GLAllocation.createDirectFloatBuffer(16);
// Add the rotation
GL11.glRotatef(yaw, 0, 0, 1);
GL11.glRotatef(pitch, 1, 0, 0);
// Get the OpenGL data
GL11.glGetFloat(2982, modelview);
GL11.glGetFloat(2983, projection);
GL11.glGetInteger(2978, viewport);
// Remove the rotation
GL11.glRotatef(-yaw, 0, 0, 1);
GL11.glRotatef(-pitch, 1, 0, 0);
// Calculate the screen position
boolean result = GLU.gluProject(x, y, z, modelview, projection, viewport, screen_coords);
if (result)
{
if ( (screen_coords.get(0) < -1.0f) || (screen_coords.get(0) > 1.0f) || (screen_coords.get(1) < -1.0f) || (screen_coords.get(1) > 1.0f) )
return null;
int window_half_width = getWidth() / 2;
int window_half_height = getHeight() / 2;
int screen_x = window_half_width + (window_half_width * -screen_coords.get(0));
int screen_y = window_half_height + (window_half_height * screen_coords.get(1));
System.out.printf("(Full Screen / No bounds) [" +x+ ", " +y+ ", " +z+ "] gives [" +screen_coords.get(0)+ ", " +screen_coords.get(1)+ "]");
System.out.printf("(Every Time) [" +x+ ", " +y+ ", " +z+ "] gives [" +screen_x+ ", " +screen_y+ "]");
return new Vector2(screen_x, screen_y);
}
else
{
System.out.printf("Failed to convert 3D coords to 2D screen coords");
return null;
}
}
Is that correct?
Yaw is a rotation about the y axis (pointing upwards), and pitch is a rotation about the x axis.
GL11.glRotatef(pitch, 1, 0, 0);
GL11.glRotatef(yaw, 0, 1, 0);
It may also be faster to push/pop the the projection matrix, and you should be in the correct matrix mode. (Both your model-view matrix and your projection matrix have to be correct).
GL11.glPushMatrix();
// Camera rotations
GL11.glPopMatrix();
But it's questionable as to why you need to undo the camera tranformations. You should probably just apply them once per frame or viewport. Render your scene with that view, then get the projected coordinate and then change to your 2D rendering mode.
Update
I'd prefer it if you used roll, pitch and yaw of the camera to mean what they actually mean in OpenGL's coordinate system - with respect to the camera. The point is, as long as you have the camera model-view and projection matrices setup correctly, you can project the point. You must already be setting up the camera somewhere. You will be able to see if this is correct, because you can SEE the results. Apply that transformation and you will know it's correct.
Is this correct? It might be? You have to apply all of the transformations. The exact same transformations that give you the camera view you see in game. This includes all translations and rotations. You will know if you are right, because you should already be applying these transformations in-game.
However, What you need to think about is what is the camera projection matrix before hand? If you don't know, call glLoadIdentity() to clear it, and then call your camera transformation logic. You need an accurate model-view and projection matrix - the exact same camera setup you are using in-game. If your model-view or projection matrices are in the wrong state, it just won't work.