We Keep Coding

Shader testgrid - lines jagged and breaking

I need to write a simple shader for the testgrid ground surface. I want to basically draw parallel lines in shader code.
Problem: as the lines grow more distant from the camera, they begin to break up and there're gaps in them. I understand why that happens with my code - because OpenGL approximates fragment's position as being too far from the point that I calculate, so it marks it as not belonging to a line.
I am passing the actual world positions of the plane surface vectors to my shader - that's how I can calculate it.
I've been playing with the algorithm for an hour, but can't seem to get good results.
The best idea I've tried was to include a small coefficient that grows the further the line gets from the camera - but the results are underwhelming. I calculated the coefficient linearly, but I guess I need some smarter formula to go that route, because the rate at which the lines grow thinner on the screen isn't linear. I can't figure this out so far though. Currently it either makes close lines too thick, which is undesirable, or still has the same problem for distant lines.
To simplify, I'm currently only drawing X-axis lines
I'm including a piece of shader code and a screenshot of the problem.
#version 300 es
precision highp float;
precision highp int;
in highp vec3 vertexPosition;
out mediump vec4 fragColor;
void main()
{
highp float lineWidth = 0.2;
highp float squareSize = 5.0f;
highp int roundX = int(vertexPosition.x / squareSize);
highp int roundY = int(vertexPosition.z / squareSize);
highp float remainderX = vertexPosition.x - float(roundX)*squareSize;
highp float remainderY = vertexPosition.x - float(roundY)*squareSize;
// this is the small coefficient I was trying to add to linewidth
highp float test = abs(0.08 * float(roundX));
if (abs(remainderX) <= (lineWidth))
{
fragColor = vec4(1,0,0, 1);
}
else
{
fragColor = vec4(0.8,0.8,0.8, 1);
}
}
The first answer fixes the main problem with lines breaking, but introduces a visual bug. Gonna go and try to find out why. Anyway, this is already a good idea! But as you can see the lines get wider towards the end.
Edit: Found it. Just removed the Z coordinate from vertexPosition before doing dFdy. Now all I need it a way to make the lines smoother and not staircase-like.
p.s. Don't look at how optimized the code is - I'm currently just searching for the right idea
p.p.s. If someone can tell me how to do simple antialiasing for this example - this also would be most welcome.

It is important that roundX is rounded (round) to the nearest integer, rather than truncated:
highp int roundX = int(round(vertexPosition.x / squareSize));
or
highp int roundX = int(vertexPosition.x / squareSize + 0.5 * sign(vertexPosition.x));
A possible solution is to get the partial derivative of vertexPosition.xy along the y axis of the viewport by dFdy.
The length of the partial derivative of vertexPosition.xy gives the distance between 2 fragments in model space. Thus the minimum thickness of a line can be defined:
vec2 dy = dFdy(vertexPosition.xy);
float minWidth = length(dy);
float w = step(max(lineWidth, minWidth), abs(remainderX));
fragColor = mix(vec4(1.0, 0.0, 0.0, 1.0), vec4(0.8, 0.8, 0.8, 1.0), w);
For smoother lines, you have to interpolate the line color and the ground color. Interpolate if abs(remainderX) is between min(lineWidth, minWidth) and max(lineWidth, minWidth). Use smoothstep for the interpolation. e.g.:
highp int roundX = int(round(vertexPosition.x / squareSize));
highp float remainderX = vertexPosition.x - float(roundX)*squareSize;
vec2 dy = dFdy(vertexPosition.xy);
float minWidth = length(dy);
float w = smoothstep(min(lineWidth, minWidth), max(lineWidth, minWidth), abs(remainderX));
fragColor = mix(vec4(1.0, 0.0, 0.0, 1.0), vec4(0.8, 0.8, 0.8, 1.0), w);
See the Three.js example, which uses the shader:
(function onLoad() {
var camera, scene, renderer, orbitControls;
init();
animate();
function init() {
renderer = new THREE.WebGLRenderer({
antialias: true,
alpha: true
});
renderer.setPixelRatio(window.devicePixelRatio);
renderer.setSize(window.innerWidth, window.innerHeight);
renderer.shadowMap.enabled = true;
document.body.appendChild(renderer.domElement);
camera = new THREE.PerspectiveCamera(70, window.innerWidth / window.innerHeight, 1, 300);
camera.position.set(10, 15, -60);
loader = new THREE.TextureLoader();
loader.setCrossOrigin("");
scene = new THREE.Scene();
scene.background = new THREE.Color(0xffffff);
scene.add(camera);
window.onresize = resize;
orbitControls = new THREE.OrbitControls(camera, renderer.domElement);
var helper = new THREE.GridHelper(400, 10);
helper.material.opacity = 0.25;
helper.material.transparent = true;
scene.add(helper);
var axis = new THREE.AxesHelper(1000);
scene.add(axis);
var material = new THREE.ShaderMaterial({
vertexShader: document.getElementById('vertex-shader').textContent,
fragmentShader: document.getElementById('fragment-shader').textContent,
});
material.extensions = {
derivatives: true
}
var geometry = new THREE.BoxGeometry( 100, 0.1, 100 );
var mesh = new THREE.Mesh(geometry, material);
scene.add(mesh);
}
function resize() {
var aspect = window.innerWidth / window.innerHeight;
renderer.setSize(window.innerWidth, window.innerHeight);
camera.aspect = aspect;
camera.updateProjectionMatrix();
}
function animate() {
requestAnimationFrame(animate);
orbitControls.update();
render();
}
function render() {
renderer.render(scene, camera);
}
})();
<script type='x-shader/x-vertex' id='vertex-shader'>
varying vec3 vertexPosition;
void main() {
vertexPosition = position.zyx;
gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
}
</script>
<script type='x-shader/x-fragment' id='fragment-shader'>
precision highp float;
varying vec3 vertexPosition;
int round(float x)
{
return int(x + 0.5 * sign(x));
}
void main(){
vec4 fragColor;
highp float lineWidth = 0.2;
highp float squareSize = 5.0;
highp int roundX = round(vertexPosition.x / squareSize);
highp float remainderX = vertexPosition.x - float(roundX)*squareSize;
vec2 dy = dFdy(vertexPosition.xy);
float minWidth = length(dy);
float w = smoothstep(min(lineWidth, minWidth), max(lineWidth, minWidth), abs(remainderX));
//float w = step(max(lineWidth, minWidth), abs(remainderX));
fragColor = mix(vec4(1.0, 0.0, 0.0, 1.0), vec4(0.8, 0.8, 0.8, 1.0), w);
gl_FragColor = fragColor;
}
</script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/110/three.min.js"></script>
<script src="https://threejs.org/examples/js/controls/OrbitControls.js"></script>

trouble with entity coordinates with lwjgl

I've a trouble with moving my entities in a OpenGL context:
when I try to place an entity, the position seems correct, but when the entity starts to move, everything is going wrong, and collisions don't work. I'm new to OpenGL, and I suspect my world matrix or model matrix to be wrong.
Here's the code of the vertex shader:
#version 330 core
layout (location=0) in vec3 position;
out vec3 extColor;
uniform mat4 projectionMatrix;
uniform mat4 modelMatrix;
uniform vec3 inColor;
void main()
{
gl_Position = projectionMatrix * modelMatrix * vec4(position, 1.0);
extColor = inColor;
}
Here is the class that computes most of the Matrix:
public class Transformations {
private Matrix4f projectionMatrix;
private Matrix4f modelMatrix;
public Transformations() {
projectionMatrix = new Matrix4f();
modelMatrix = new Matrix4f();
}
public final Matrix4f getOrthoMatrix(float width, float height, float zNear, float zFar) {
projectionMatrix.identity();
projectionMatrix.ortho(0.0f, width, 0.0f, height, zNear, zFar);
return projectionMatrix;
}
public Matrix4f getModelMatrix(Vector3f offset, float angleZ, float scale) {
modelMatrix.identity().translation(offset).rotate(angleZ, 0, 0, 0).scale(scale);
return modelMatrix;
}
}
Here's the test for collisions:
public boolean isIn(Pos p) {
return (p.getX() >= this.pos.getX() &&
p.getX() <= this.pos.getX() + DIMENSION)
&& (p.getY() >= this.pos.getY() &&
p.getY() <= this.pos.getY() + DIMENSION);
}
Also, there's a link to the github project: https://github.com/ShiroUsagi-san/opengl-engine.
I'm really new to OpenGL 3 so I could have done some really big mistakes.
I'm also running i3 as WM, I don't really know if this could lead to this kind of issues.

I fixes the issues after thinking about how openGL and VBO work: Indeed, I was setting a new reference for each entity, so I had to change the line
Mesh fourmiMesh = MeshBuilder.buildRect(this.position.getX(), this.position.getY(), 10, 10);
to
Mesh fourmiMesh = MeshBuilder.buildRect(0, 0, 10, 10);
It was a confusion that I made between the positions of the vertex in a VBO and the positions in my world.
Hope that misunderstood helps people to understand.

Adding ProjectionMatrix to entity results in wierd behavior when rendering

Introduction to the problem:
I'm working on a game engine using the LWJGL library following this tutorial. However, I'm trying to make it so that there is a real division between the main engine and the game itself. I've therefore complicated the project a whole lot and I think this is causing some problems as the ProjectionMatrix doesn't work as explained in the video.
What am I doing:
Creating the ProjectionMatrix:
In order to create a ProjectionMatrix I created the a method which creates it for me:
public static Matrix4f createProjectionMatrix(float aspectRatio, float fov, float nearPlane, float farPlane) {
float y_scale = (float) ((1f / Math.tan(Math.toRadians(fov / 2f))) * aspectRatio);
float x_scale = y_scale / aspectRatio;
float frustum_length = nearPlane - farPlane;
Matrix4f projectionMatrix = new Matrix4f();
projectionMatrix.m00 = x_scale;
projectionMatrix.m11 = y_scale;
projectionMatrix.m22 = -((farPlane + nearPlane) / frustum_length);
projectionMatrix.m23 = -1;
projectionMatrix.m32 = -((2 * nearPlane * farPlane) / frustum_length);
projectionMatrix.m33 = 0;
return projectionMatrix;
}
I create the ProjectionMatrix with the following values:
aspectRatio = width/height = 640/480 = 1.33333
fov = 100
nearPlane = -0.5
farPlane = 100
This results in the following values for my ProjectionMatrix:
0.83909965 0.0 0.0 0.0
0.0 0.83909965 0.0 0.0
0.0 0.0 0.9990005 -0.9995003
0.0 0.0 -1.0 0.0
Using the ProjectionMatrix:
In order to use the ProjectionMatrix I created the following shaders:
vertex.vs:
#version 150
in vec3 position;
in vec2 textureCoordinates;
out vec2 passTextureCoordinates;
uniform mat4 transformationMatrix;
uniform mat4 projectionMatrix;
uniform int useProjectionMatrix;
void main(void){
if (useProjectionMatrix == 1) {
gl_Position = projectionMatrix * transformationMatrix * vec4(position,1.0);
} else {
gl_Position = transformationMatrix * vec4(position,1.0);
}
passTextureCoordinates = textureCoordinates;
}
fragment.fs:
#version 150
in vec2 passTextureCoordinates;
out vec4 out_Color;
uniform sampler2D textureSampler;
void main(void){
out_Color = texture(textureSampler,passTextureCoordinates);
}
Finally in order to render the entity I've created the following renderer class:
public class TexturedEntityRenderer extends AbstractEntityRenderer{
private float aspectRatio;
private float fov;
private float nearPlane;
private float farPlane;
public void prepare() {
GL11.glClearColor(0,0,0,1);
GL11.glClear(GL11.GL_COLOR_BUFFER_BIT);
}
public void render (AbstractEntity entity, AbstractShader shader) {
if(shader instanceof TexturedEntityShader) {
if(entity.getModel() instanceof TexturedModel) {
TexturedModel model = (TexturedModel)entity.getModel();
GL30.glBindVertexArray(model.getVaoID());
GL20.glEnableVertexAttribArray(0);
GL20.glEnableVertexAttribArray(1);
Matrix4f transformationMatrix = MatrixMaths.createTransformationMatrix(entity.getPosition(), entity.getRx(), entity.getRy(), entity.getRz(), entity.getScale());
((TexturedEntityShader)shader).loadTransformationMatrix(transformationMatrix);
GL13.glActiveTexture(GL13.GL_TEXTURE0);
GL11.glBindTexture(GL11.GL_TEXTURE_2D, ((TexturedModel)entity.getModel()).getTexture().getTextureID());
GL15.glBindBuffer(GL15.GL_ARRAY_BUFFER, model.getVaoID());
GL11.glDrawElements(GL11.GL_TRIANGLES, model.getVertexCount(), GL11.GL_UNSIGNED_INT, 0);
GL20.glDisableVertexAttribArray(0);
GL20.glDisableVertexAttribArray(1);
GL30.glBindVertexArray(0);
} else {
ExceptionThrower.throwException(new ModelInvalidException());
}
} else {
ExceptionThrower.throwException(new ShaderIncompatableException(shader.toString()));
}
}
public void setup(AbstractShader shader) {
nearPlane = Float.parseFloat(OptionHandler.getProperty(GraphicOptions.WINDOWNEARPLANE_KEY, OptionHandler.GRAPHIC_OPTION_ID));
farPlane = Float.parseFloat(OptionHandler.getProperty(GraphicOptions.WINDOWFARPLANE_KEY, OptionHandler.GRAPHIC_OPTION_ID));
aspectRatio = DisplayManager.getWidth() / DisplayManager.getHeight();
fov = Float.parseFloat(OptionHandler.getProperty(GraphicOptions.WINDOWFOV_KEY, OptionHandler.GRAPHIC_OPTION_ID));
((TexturedEntityShader)shader).loadProjectionMatrix(MatrixMaths.createProjectionMatrix(aspectRatio, fov, nearPlane, farPlane));
((TexturedEntityShader)shader).loadUseProjectionMatrix();
}
}
The Optionhandler.getProperty() function in the setup() returns the property for a given key(like fov or nearPlane value) from a text file. (I've checked that this works by printing all loaded options.) Also, the DisplayManager.getWidth() and DisplayManager.getHeight() functions, obviously, obtain the width and height for calculating the aspectRatio variable.
Updating the entity:
Last but not least, I'm updating the entity using a class called EntityModifier which looks like this:
public class EntityModifier {
private Vector3f dposition;
private float drx;
private float dry;
private float drz;
private float dscale;
public BasicEntityModifier(Vector3f dposition, float drx, float dry, float drz, float dscale) {
this.dposition = dposition;
this.drx = drx;
this.dry = dry;
this.drz = drx;
this.dscale = dscale;
}
public Vector3f getDposition() {
return dposition;
}
public float getDrx() {
return drx;
}
public float getDry() {
return dry;
}
public float getDrz() {
return drz;
}
public float getDscale() {
return dscale;
}
#Override
public String toString() {
return "BasicEntityModifier [dposition=" + dposition + ", drx=" + drx + ", dry=" + dry + ", drz=" + drz + ", dscale=" + dscale + "]";
}
}
Each entity I create has one of these classes and I cal call an update method which adds the values to the entity's transformation:
public void update() {
increasePosition(modifier.getDposition().getX(),modifier.getDposition().getY(),modifier.getDposition().getZ());
increaseRotation(modifier.getDrx(), modifier.getDry(), modifier.getDrz());
increaseScale(modifier.getDscale());
}
private void increasePosition(float dx, float dy, float dz) {
position.x += dx;
position.y += dy;
position.z += dz;
}
private void increaseRotation(float drx, float dry, float drz) {
rx += drx;
ry += dry;
rz += drz;
}
private void increaseScale(float dscale) {
scale += dscale;
}
The problem:
I'm able to change the position of the x and y values of the entity normally but whenever I change the z position, using an EntityModifier, the entity loads but then dissapears from the screen. It's loaded for about 60 frames before dissapearing and changing dz's value doesn't seem to affect the speed at which it dissapears in any way(It does, see EDIT 2). Also there the entity doesn't have the scale effect as shown in the tutorial here (same link but with timestamp).
Changing the dz value to 0 stops the dissapearing of the entity.
What is going on here? How can I fix this?
EDIT:
I've been pointed out in the comments that the nearPlane value should be positive so I changed it to 0.5 but I still get the same results. I also changed: float frustum_length = nearPlane - farPlane; to float frustum_length = farPlane - nearPlane; which was also suggested there (this also did not solve the problem).
EDIT 2:
After some more investigation I found a few intresting things:
1. Changing the speed at which the z value changes does affect how long it takes for the entity to dissapear. After finding this out I tried timing a few different dz(with dz being the change per frame of z) values and I got this:
`for dz = -0.002 -> frames before dissapear: 515 frames.`
`for dz = -0.001 -> frames before dissapear: 1024 frames.`
`for dz = 0.02 -> frames before dissapear: 63 frames.`
If we take into account reaction times (I made the program output the total ammount of rendered frames on closure and just closed it as fast as possible when the entity dissapeared) we can calculate the values for z at which the entity dissapears.
-0.002 * 515 ≈ -1
-0.001 * 1024 ≈ -1
0.02 * 63 ≈ 1
This probably has to do with the way the coordinate system works in OpenGL but it still doesn't explain why the entity isn't becoming smaller as it does in the tutorial mentioned above.
2. Removing the code which adds the ProjectionMatrix to the renderer class does not change the behavior. This means the error is elsewere.
New Problem:
I think there is no problem with the ProjectionMatrix (or at least not a problem that is causing this behavior) but the problem is with the entity's position surpassing 1 or -1 on the z axes. However this still doesn't explain why there is no "zoom effect". Therefor I don't think that restricting the z movement between -1 and 1 will solve this problem, infact, I think it will work against us as the entity should not be rendered anyway if it's totaly "zoomed" out or in.
What can cause this problem if it isn't the ProjectionManager?
EDIT 3:
Someone on reddit pointed out that the following classes might also be of intrest for solving the problem:
AbstractShader: contains basic shader functionality common for all shader classes.
TexturedEntityShader: used to render a texturedEntity (shown above)
DisplayManager: class which handles rendering.
EDIT 4:
After some more discussion on reddit about this problem we've come across a problem and were able to fix it: The value for useProjectionMatrix was not loaded because the shader was stopped when I tried to load it. Changing the loadUseProjectionMatrix() method to:
public void loadUseProjectionMatrix() {
super.start();
super.loadBoolean(useProjectionMatrixLocation, useProjectionMatrix);
System.out.println("loaded useProjectionMatrix: " + useProjectionMatrix + "\n\n");
super.stop();
}
seems to partially solve the problem as the projectionMatrix now can be used inside the shader (before it would not be used due to the fact that the useProjectionMatrix value would always be 0 as we did not load a value for it.).
However, this did not fix the entire problem as there is still an issue with the projectionMatrix I think. The entity does not want to render at all when using the projectionMatrix but it renders fine when not using it. I've tried hardcoding the values of the projectionMatrix by using the following shader:
#version 150
in vec3 position;
in vec2 textureCoordinates;
out vec2 passTextureCoordinates;
uniform mat4 transformationMatrix;
uniform mat4 projectionMatrix;
uniform int useProjectionMatrix;
mat4 testMat;
void main(void){
testMat[0] = vec4(0.83909965, 0, 0, 0);
testMat[1] = vec4(0, 0.83909965, 0, 0);
testMat[2] = vec4(0, 0, 0.9990005, -0.9995003);
testMat[3] = vec4(0, 0, -1, 0);
if (true) {
gl_Position = testMat * transformationMatrix * vec4(position,1.0);
} else {
gl_Position = transformationMatrix * vec4(position,1.0);
}
passTextureCoordinates = textureCoordinates;
}
However that does not seem to work. Are these values OK?
Fow who wants to see it here are the 2 posts I created on reddit about this problem: post 1, post 2.

Opengl nothing is being written to depth buffer

I'm trying to implement depth testing for 2D isometric game. To get something working, I started off with this sample, but I cannot get it to work correctly.
I'm trying to draw 2 images in a specific order.
first.png
second.png
first.png is drawn first, and second.png is drawn on top. Using fragment shader, I compute that red color has lower depth than green color, hence green fragments should be discarded when drawn on top of red fragments. The end result is that when second.png is drawn directly on top of first.png, the resulting square colored only red.
At the end of render function, I get the pixels of depth buffer, and looping over them I check if the values have been changed from defaults ones. It seems that no matter what I do, the values in depth buffer never change.
The depth test itself is working, if I set green fragments to depth=1.0, red fragments to depth=0.0 and my depth function is GL_LESS, only red fragments are drawn, but the depth buffer is not changed.
The code is in Java, but OpenGL functions are the same.
private SpriteBatch mBatch;
private Texture mTexture1;
private Texture mTexture2;
#Override
public void create() {
mBatch = new SpriteBatch();
mBatch.setShader(new ShaderProgram(Gdx.files.internal("test.vsh"), Gdx.files.internal("test.fsh")));
mTexture1 = new Texture("first.png");
mTexture2 = new Texture("second.png");
Gdx.gl20.glEnable(GL20.GL_DEPTH_TEST);
Gdx.gl20.glDepthFunc(GL20.GL_LESS);
Gdx.gl20.glDepthMask(true);
}
#Override
public void render() {
Gdx.gl20.glClear(GL20.GL_COLOR_BUFFER_BIT | GL20.GL_DEPTH_BUFFER_BIT);
mBatch.begin();
float scale = 4.0f;
float x = Gdx.graphics.getWidth() / 2;
float y = Gdx.graphics.getHeight() / 2;
mBatch.draw(mTexture1, x - mTexture1.getWidth() / 2 * scale, y - mTexture1.getHeight() / 2 * scale,
mTexture1.getWidth() * scale, mTexture1.getHeight() * scale);
mBatch.flush();
mBatch.draw(mTexture2, x - mTexture2.getWidth() / 2 * scale, y - mTexture2.getHeight() / 2 * scale,
mTexture2.getWidth() * scale, mTexture2.getHeight() * scale);
mBatch.end();
int width = Gdx.graphics.getWidth();
int height = Gdx.graphics.getHeight();
FloatBuffer buffer = BufferUtils.newFloatBuffer(width * height);
Gdx.gl20.glReadPixels(0, 0, width, height, GL20.GL_DEPTH_COMPONENT, GL20.GL_FLOAT,
buffer);
for (int i = 0; i < width * height; i++) {
float pixel = buffer.get(i);
if (pixel != 1.0f && pixel != 0.0f) {
// why is this never thrown??
// it means depth buffer wasn't changed.
throw new IllegalStateException("OMG IT WORKS!! " + pixel);
}
}
if (Gdx.gl20.glGetError()!=0) {
throw new Error("OPENGL ERROR: " + Gdx.gl20.glGetError());
}
}
Vertex shader
#ifdef GL_ES
precision mediump float;
#endif
attribute vec3 a_position;
attribute vec4 a_color;
attribute vec2 a_texCoord0;
uniform mat4 u_projTrans;
varying vec4 v_color;
varying vec2 v_texCoord;
void main()
{
gl_Position = u_projTrans * vec4(a_position, 1);
v_color = a_color * 2.0;
v_texCoord = a_texCoord0;
}
Fragment shader
#ifdef GL_ES
precision mediump float;
#endif
uniform sampler2D u_texture;
varying vec4 v_color;
varying vec2 v_texCoord;
void main()
{
vec4 texel = v_color * texture2D(u_texture, v_texCoord);
if (texel.r > texel.g)
{
gl_FragDepth = 0.0;
}
else
{
gl_FragDepth = 0.5;
}
gl_FragColor = texel;
}

Ok, I found the problem.
SpriteBatch.begin() does
glDepthMask(false)
Setting glDepthMask to false prevents OpenGL from writing to depth buffer.
The solution is to call glDepthMask(true) after SpriteBatch.begin()

OpenGL Rotation Skewed Issues

I am having a bit of an issue with any rotation around the x-axis inside of OpenGL. What I have is a basic cube that is being rendered inside of a shader, with normals for lighting calculations. The transformation of the cube is the projection matrix multiplied by the model matrix. This concatenation is done inside the shader, while the rotation and translation calculations are done inside a Matrix class I wrote. When any rotations are done around the y-axis, everything rotates as expected as shown by these pictures:
The problems start when any rotations occur around the x-axis as shown here:
Apparently the x-axis rotations skew the scene and cause everything to be shown out of proportion. Here is the code the calculates the matrices and passes is to the shader every time this is rendered:
glClear(GL_DEPTH_BUFFER_BIT | GL_COLOR_BUFFER_BIT);
viewMatrix.setIdentity();
viewMatrix.translate(new Vector4(translation));
viewMatrix.rotate(1, 0, 0, -rotateX);
viewMatrix.rotate(0, 1, 0, rotateY);
Matrix4 mvMatrix = new Matrix4(viewMatrix);
Matrix4 pMatrix = new Matrix4(projectionMatrix);
lightShader.useShader();
lightShader.setUniformVector("vColor", new Vector4(.4f,0,0,1f));
lightShader.setUniformMatrix("mvMatrix", mvMatrix);
lightShader.setUniformMatrix("pMatrix", pMatrix);
triangleBatch.draw(lightShader.getAttributes());
Display.update();
The shader code is as follows:
uniform mat4 mvMatrix;
uniform mat4 pMatrix;
uniform vec4 vColor;
varying vec4 outFragColor;
attribute vec4 inVertex;
attribute vec4 inNormal;
void main(void) {
vec3 newNorm = vec3(inNormal);
mat3 mNormalMatrix;
mNormalMatrix[0] = mvMatrix[0].xyz;
mNormalMatrix[1] = mvMatrix[1].xyz;
mNormalMatrix[2] = mvMatrix[2].xyz;
vec3 vNorm = normalize(mNormalMatrix * newNorm);
vec3 vLightDir = vec3(0.0, 0.0, 1.0);
float fDot = max(0.0, dot(vNorm, vLightDir));
outFragColor.rgb = vColor.rgb * fDot;
outFragColor.a = vColor.a;
mat4 mvpMatrix;
mvpMatrix = pMatrix * mvMatrix;
gl_Position = mvpMatrix * inVertex;
}
Finally the matrix math code for rotations is as follows:
public Matrix4 rotate(Vector4 a, float radians){
Matrix4 temp = new Matrix4().setToRotate(a, radians);
this.multiply(temp);
return this;
}
public Matrix4 setToRotate(Vector4 a, float radians){
a = new Vector4(a).normalise();
float c = GameMath.cos(radians);
float s = GameMath.sin(radians);
float t = 1-c;
float x = a.x;
float y = a.y;
float z = a.z;
float x2 = a.x*a.x;
float y2 = a.y*a.y;
float z2 = a.z*a.z;
this.setIdentity();
this.matrix[0] = c + t*x2;
this.matrix[1] = t*x*y + s*z;
this.matrix[2] = t*x*z - s*y;
this.matrix[4] = t*x*y - s*z;
this.matrix[5] = c + t*y2;
this.matrix[6] = t*y*z + s*x;
this.matrix[8] = t*x*z + s*y;
this.matrix[9] = t*y*z + s*x;
this.matrix[10] = c + t*z2;
return this;
}
The matrices are stored in column major.

I found out what the issue was. apparently one of the operations in setToRotate() need to be changed from addition to subtraction. It works as expected now.