We Keep Coding

Fastest Performance Filtering an Image

Context:
I'm trying to create an animation in java.
The animation is simply take an image and make it appear from the darkest pixels to the lightest.
The Problem:
The internal algorithm defining the pixels transformations is not my issue.
I'm new to Java and Computing in general. I've done a bit of research, and know that there are plenty of APIs that helps with image filters/transformations.
My problem is performance, understanding it.
To the implementation i've created a method that do the following:
Receives an BufferedImage.
Get the WritableRaster of the BufferedImage.
Using setSample and getSample, process and change pixel by pixel.
Return the BufferedImage.
After that, i use a Timer to call the method.
The returned BufferedImage is attached to a JButton via setIcon after each call.
With a 500x500 image my machine takes around 3ms to process each call.
For standard 1080p images it takes around 30ms, wich is about 33 frames per second.
My goal is to process/animate FullHD images at 30fps... And i will not be able to with the path I'm following. Not in most computers.
What i'm getting wrong? How i can make it faster? Using getDataBuffer or getPixels instead of getSample can improve it?
Thanks in advance! And sorry my english.
Partial Conclusions:
Thanks to some help here. I've changed concept. Instead of using getSample and setSample I've stored the pixels ARGB informations of the BufferedImage into an array. So i process the array and copy it all at once into a Raster of another BufferedImage.
The process time reduced from 30ms ( get/set sample ) to 1ms. ( measured poorly, but in the same machine, enviroment and code ).
Below is a little class i coded to implement it. The class can filter pixels only below a Brightness level, the other pixels become transparent ( alpha = 0 ).
Hope it help's who search for the same solution in the future. Be wary that I'm below rookie level in Java, so the code might be poorly organized/optimized.
import java.awt.Graphics2D;
import java.awt.image.*;
/**
* #author Psyny
*/
public class ImageAppearFX {
//Essencial Data
BufferedImage imgProcessed;
int[] RAWoriginal;
int[] RAWprocessed;
WritableRaster rbgRasterProcessedW;
//Information about the image
int x,y;
int[] mapBrightness;
public ImageAppearFX(BufferedImage inputIMG) {
//Store Dimensions
x = inputIMG.getWidth();
y = inputIMG.getHeight();
//Convert the input image to INT_ARGB and store it.
this.imgProcessed = new BufferedImage(x, y, BufferedImage.TYPE_INT_ARGB);
Graphics2D canvas = this.imgProcessed.createGraphics();
canvas.drawImage(inputIMG, 0, 0, x, y, null);
canvas.dispose();
//Create an int Array of the pixels informations.
//p.s.: Notice that the image was converted to INT_ARGB
this.RAWoriginal = ((DataBufferInt) this.imgProcessed.getRaster().getDataBuffer()).getData();
//Dupplication of original pixel array. So we can make changes based on original image
this.RAWprocessed = this.RAWoriginal.clone();
//Get Raster. We will need the raster to write pixels on
rbgRasterProcessedW = imgProcessed.getRaster();
//Effect Information: Store brightness information
mapBrightness = new int[x*y];
int r,g,b,a,greaterColor;
// PRocess all pixels
for(int i=0 ; i < this.RAWoriginal.length ; i++) {
a = (this.RAWoriginal[i] >> 24) & 0xFF;
r = (this.RAWoriginal[i] >> 16) & 0xFF;
g = (this.RAWoriginal[i] >> 8) & 0xFF;
b = (this.RAWoriginal[i] ) & 0xFF;
//Search for Stronger Color
greaterColor = r;
if( b > r ) {
if( g > b ) greaterColor = g;
else greaterColor = b;
} else if ( g > r ) {
greaterColor = g;
}
this.mapBrightness[i] = greaterColor;
}
}
//Effect: Show only in a certain percent of brightness
public BufferedImage BrightnessLimit(float percent) {
// Adjust input values
percent = percent / 100;
// Pixel Variables
int hardCap = (int)(255 * percent);
int r,g,b,a,bright;
// Process all pixels
for(int i=0 ; i < this.RAWoriginal.length ; i++) {
//Get information of a pixel of the ORIGINAL image
a = (this.RAWoriginal[i] >> 24) & 0xFF;
r = (this.RAWoriginal[i] >> 16) & 0xFF;
g = (this.RAWoriginal[i] >> 8) & 0xFF;
b = (this.RAWoriginal[i] ) & 0xFF;
//Brightness information of that same pixel
bright = this.mapBrightness[i];
//
if( bright > hardCap ) {
a = 0;
}
this.RAWprocessed[i] = ((a << 24) + (r << 16) + (g << 8) + ( b )); //Write ARGB in byte format
}
//Copy the processed array into the raster of processed image
rbgRasterProcessedW.setDataElements(0, 0, x, y, RAWprocessed);
return imgProcessed;
}
//Return reference to the processed image
public BufferedImage getImage() {
return imgProcessed;
}
}

While the time difference resulting from the change doesn't prove that the repeated searching is the bottleneck, it does strongly implicate it.
If you are willing/able to trade memory for time, I would first sort a list of all the pixel locations by brightness. Next, I would use the sorted list during the animation to look up the next pixel to copy.
An extra piece of advice: use one of Java's built in sorting methods. It's educational to make your own, but learning how to sort doesn't seem to be your goal here. Also, if my guess about the bottleneck is wrong, you'll want to minimize your time pursuing this answer.

Efficient 2D Tile based lighting system

What is the most efficient way to do lighting for a tile based engine in Java?
Would it be putting a black background behind the tiles and changing the tiles' alpha?
Or putting a black foreground and changing alpha of that? Or anything else?
This is an example of the kind of lighting I want:

There are many ways to achieve this. Take some time before making your final decision. I will briefly sum up some techiques you could choose to use and provide some code in the end.
Hard Lighting
If you want to create a hard-edge lighting effect (like your example image),
some approaches come to my mind:
Quick and dirty (as you suggested)
Use a black background
Set the tiles' alpha values according to their darkness value
A problem is, that you can neither make a tile brighter than it was before (highlights) nor change the color of the light. Both of these are aspects which usually make lighting in games look good.
A second set of tiles
Use a second set of (black/colored) tiles
Lay these over the main tiles
Set the new tiles' alpha value depending on how strong the new color should be there.
This approach has the same effect as the first one with the advantage, that you now may color the overlay tile in another color than black, which allows for both colored lights and doing highlights.
Example:
Even though it is easy, a problem is, that this is indeed a very inefficent way. (Two rendered tiles per tile, constant recoloring, many render operations etc.)
More Efficient Approaches (Hard and/or Soft Lighting)
When looking at your example, I imagine the light always comes from a specific source tile (character, torch, etc.)
For every type of light (big torch, small torch, character lighting) you
create an image that represents the specific lighting behaviour relative to the source tile (light mask). Maybe something like this for a torch (white being alpha):
For every tile which is a light source, you render this image at the position of the source as an overlay.
To add a bit of light color, you can use e.g. 10% opaque orange instead of full alpha.
Results
Adding soft light
Soft light is no big deal now, just use more detail in light mask compared to the tiles. By using only 15% alpha in the usually black region you can add a low sight effect when a tile is not lit:
You may even easily achieve more complex lighting forms (cones etc.) just by changing the mask image.
Multiple light sources
When combining multiple light sources, this approach leads to a problem:
Drawing two masks, which intersect each other, might cancel themselves out:
What we want to have is that they add their lights instead of subtracting them.
Avoiding the problem:
Invert all light masks (with alpha being dark areas, opaque being light ones)
Render all these light masks into a temporary image which has the same dimensions as the viewport
Invert and render the new image (as if it was the only light mask) over the whole scenery.
This would result in something similar to this:
Code for the mask invert method
Assuming you render all the tiles in a BufferedImage first,
I'll provide some guidance code which resembles the last shown method (only grayscale support).
Multiple light masks for e.g. a torch and a player can be combined like this:
public BufferedImage combineMasks(BufferedImage[] images)
{
// create the new image, canvas size is the max. of all image sizes
int w, h;
for (BufferedImage img : images)
{
w = img.getWidth() > w ? img.getWidth() : w;
h = img.getHeight() > h ? img.getHeight() : h;
}
BufferedImage combined = new BufferedImage(w, h, BufferedImage.TYPE_INT_ARGB);
// paint all images, preserving the alpha channels
Graphics g = combined.getGraphics();
for (BufferedImage img : images)
g.drawImage(img, 0, 0, null);
return combined;
}
The final mask is created and applied with this method:
public void applyGrayscaleMaskToAlpha(BufferedImage image, BufferedImage mask)
{
int width = image.getWidth();
int height = image.getHeight();
int[] imagePixels = image.getRGB(0, 0, width, height, null, 0, width);
int[] maskPixels = mask.getRGB(0, 0, width, height, null, 0, width);
for (int i = 0; i < imagePixels.length; i++)
{
int color = imagePixels[i] & 0x00ffffff; // Mask preexisting alpha
// get alpha from color int
// be careful, an alpha mask works the other way round, so we have to subtract this from 255
int alpha = (maskPixels[i] >> 24) & 0xff;
imagePixels[i] = color | alpha;
}
image.setRGB(0, 0, width, height, imagePixels, 0, width);
}
As noted, this is a primitive example. Implementing color blending might be a bit more work.

Raytracing might be the simpliest approach.
you can store which tiles have been seen (used for automapping, used for 'remember your map while being blinded', maybe for the minimap etc.)
you show only what you see - maybe a monster of a wall or a hill is blocking your view, then raytracing stops at that point
distant 'glowing objects' or other light sources (torches lava) can be seen, even if your own light source doesn't reach very far.
the length of your ray gives will be used to check amount light (fading light)
maybe you have a special sensor (ESP, gold/food detection) which would be used to find objects that are not in your view? raytrace might help as well ^^
how is this done easy?
draw a line from your player to every point of the border of your map (using Bresehhams Algorithm http://en.wikipedia.org/wiki/Bresenham%27s_line_algorithm
walk along that line (from your character to the end) until your view is blocked; at this point stop your search (or maybe do one last final iteration to see what did top you)
for each point on your line set the lighning (maybe 100% for distance 1, 70% for distance 2 and so on) and mark you map tile as visited
maybe you won't walk along the whole map, maybe it's enough if you set your raytrace for a 20x20 view?
NOTE: you really have to walk along the borders of viewport, its NOT required to trace every point.
i'm adding the line algorithm to simplify your work:
public static ArrayList<Point> getLine(Point start, Point target) {
ArrayList<Point> ret = new ArrayList<Point>();
int x0 = start.x;
int y0 = start.y;
int x1 = target.x;
int y1 = target.y;
int sx = 0;
int sy = 0;
int dx = Math.abs(x1-x0);
sx = x0<x1 ? 1 : -1;
int dy = -1*Math.abs(y1-y0);
sy = y0<y1 ? 1 : -1;
int err = dx+dy, e2; /* error value e_xy */
for(;;){ /* loop */
ret.add( new Point(x0,y0) );
if (x0==x1 && y0==y1) break;
e2 = 2*err;
if (e2 >= dy) { err += dy; x0 += sx; } /* e_xy+e_x > 0 */
if (e2 <= dx) { err += dx; y0 += sy; } /* e_xy+e_y < 0 */
}
return ret;
}
i did this whole lightning stuff some time ago, a* pathfindin feel free to ask further questions
Appendum:
maybe i might simply add the small algorithms for raytracing ^^
to get the North & South Border Point just use this snippet:
for (int x = 0; x <map.WIDTH; x++){
Point northBorderPoint = new Point(x,0);
Point southBorderPoint = new Point(x,map.HEIGHT);
rayTrace( getLine(player.getPos(), northBorderPoint), player.getLightRadius()) );
rayTrace( getLine(player.getPos(), southBorderPoint, player.getLightRadius()) );
}
and the raytrace works like this:
private static void rayTrace(ArrayList<Point> line, WorldMap map, int radius) {
//int radius = radius from light source
for (Point p: line){
boolean doContinue = true;
float d = distance(line.get(0), p);
//caclulate light linear 100%...0%
float amountLight = (radius - d) / radius;
if (amountLight < 0 ){
amountLight = 0;
}
map.setLight( p, amountLight );
if ( ! map.isViewBlocked(p) ){ //can be blockeb dy wall, or monster
doContinue = false;
break;
}
}
}

I've been into indie game development for about three years right now. The way I would do this is first of all by using OpenGL so you can get all the benefits of the graphical computing power of the GPU (hopefully you are already doing that). Suppose we start off with all tiles in a VBO, entirely lit. Now, there are several options of achieving what you want. Depending on how complex your lighting system is, you can choose a different approach.
If your light is going to be circular around the player, no matter the fact if obstacles would block the light in real life, you could choose for a lighting algorithm implemented in the vertex shader. In the vertex shader, you could compute the distance of the vertex to the player and apply some function that defines how bright things should be in function of the computed distance. Do not use alpha, but just multiply the color of the texture/tile by the lighting value.
If you want to use a custom lightmap (which is more likely), I would suggest to add an extra vertex attribute that specifies the brightness of the tile. Update the VBO if needed. Same approach goes here: multiply the pixel of the texture by the light value. If you are filling light recursively with the player position as starting point, then you would update the VBO every time the player moves.
If your lightmap depends on where the sunlight hits your level, you could combine two sort of lighting techniques. Create one vertex attribute for the sun brightness and another vertex attribute for the light emitted by light points (like a torch held by the player). Now you can combine those two values in the vertex shader. Suppose the your sun comes up and goes down like the day and night pattern. Let's say the sun brightness is sun, which is a value between 0 and 1. This value can be passed to the vertex shader as a uniform. The vertex attribute that represents the sun brightness is s and the one for light, emitted by light points is l. Then you could compute the total light for that tile like this:
tileBrightness = max(s * sun, l + flicker);
Where flicker (also a vertex shader uniform) is some kind of waving function that represents the little variants in the brightness of your light points.
This approach makes the scene dynamic without having to recreate continuously VBO's. I implemented this approach in a proof-of-concept project. It works great. You can check out what it looks like here: http://www.youtube.com/watch?v=jTcNitp_IIo. Note how the torchlight is flickering at 0:40 in the video. That is done by what I explained here.

Drawing Video Frames in OpenGL on Android

I'm currently working on a system which displays video frames (as bitmaps) via OpenGL ES 1.0 on Android. My issue is that I have not been able to get more than about 10 fps.
After doing some testing, I've determined that one of the biggest bottlenecks is the need for the bitmap to have its width and height both be a power of 2. A 640x480 video has to be scaled up to 1024x1024, for example. Without the scaling, I've been able to get about 40-50fps, but the texture just appears white, which does me no good.
I know that OpenGL ES 2.0 supports using non-power of two textures, but I have no experience with shaders / anything else new in 2.0
Is there any way I can get around this issue? How do other video plays get such good performance in comparison to what I have? I have included some code for reference.
private Bitmap makePowerOfTwo(Bitmap bitmap)
{
// If one of the bitmap's resolutions is not a power of two
if(!isPowerOfTwo(bitmap.getWidth()) || !isPowerOfTwo(bitmap.getHeight()))
{
int newWidth = nextPowerOfTwo(bitmap.getWidth());
int newHeight = nextPowerOfTwo(bitmap.getHeight());
// Generate a new bitmap which has the needed padding
return Bitmap.createScaledBitmap(bitmap, newWidth, newHeight, true);
}
else
{
return bitmap;
}
}
private static boolean isPowerOfTwo(int num)
{
// Check if a bitwise and of the number and itself minus one is zero
return (num & (num - 1)) == 0;
}
private static int nextPowerOfTwo(int num)
{
if(num <= 0)
{
return 0;
}
int nextPowerOfTwo = 1;
while(nextPowerOfTwo < num)
{
nextPowerOfTwo <<= 1; // Bitwise shift left one bit
}
return nextPowerOfTwo;
}

Just because a texture has to be a power of two, doesn't mean that your data has to be a power of two.
You're free to create a 1024x1024 (or 1024x512) texture during initialization with glTexImage, fill in the lower 640x480 with your bitmap data with glTexSubImage, and then display the lower 640x480 of the texture with some intelligent texcoords (0,0) to (640/1024, 480/1024). The remainder of the texture will just contain empty space that's never seen.

Image Pixel Replacing

I am looking to replace pixels in an image that are black to some degree (semi-black) to become fully black.
The method to do this is setRGB(int x, int y, int rgb). I know this. What I do not know is how to detect pixels that are semi-black.
I have tried (i is a BufferedImage):
final int rgb = i.getRGB(x, y);
if (rgb == -16777216) {
i.setRGB(x, y, -16777216);
}
To do this, but it only replaces the pixels that are pure black with pure black.
I have also tried dimming the image, but that does not work either.
Any ideas on how I test for generic blackness?
My goal: the image I am reading is thin text. I wish to make this bolder text by this.

The integer that you receive represents the combined red, green, blue and alpha values. Essentially, you need to:
break that integer down into its component red, green, blue values
from those values, assess the overall "brightness" of the pixel
As a rough implementation, you could do something like this:
int pixVal = ... getRGB() as you have
int red = (pixVal >>> 16);
int green = (pixVal >>> 8) & 0xff;
int blue = pixVal & 0xff;
int brightness = (red + green + blue) / 3;
if (brightness < 16) {
// pixel is black
}
Now, the value 16 is a rough value: ideally, you would tailor this to the particular image.
Purists might also whinge that the perceived "brightness" of a pixel isn't literally the mean of the red/green/blue pixels (because the human eye is not equally sensitive to these components). But that's the rough idea to work from.

Java - how to make an image represent which tile to draw in a game?

I am trying to figure out how to make my game draw a certain tile in a specific spot using an image to represent each spot. So if a pixel of that image was the color red a specified picture(tile) would be draw in the game, and each pixel that was green stood for a different specified image. I have seen people who make games do this but I dont know how to do it and I dont know the name for it.
If you need more info I can try to explain what I want to do more. Could someone please help?

That may actually be slower in the long run. I would definitely recommend you use a byte array to represent the tiles, i.e. byte[width][height]. It will be faster, easier to manage and easier to extend to something like spriteData[width][height] if a single byte does not supply enough information anymore.
However, if you insist on using an image to store game data, you can use something like the following:
File file = new File("mygamedata.jpg");
BufferedImage image = ImageIO.read(file);
// Getting pixel color at position x, y (width, height)
int colour = image.getRGB(x ,y);
int red = (colour & 0x00ff0000) >> 16;
int green = (colour & 0x0000ff00) >> 8;
int blue = colour & 0x000000ff;
System.out.println("Red colour component = " + red);
System.out.println("Green colour component = " + green);
System.out.println("Blue colour component = " + blue);
Each component will be in the range (0...255) and you can use that to determine the correct tile, i.e.
Graphics2D gfx = (Graphics2D) offScreenImage.getImage();
if (red == 120 && green == 0 && blue == 0) {
gc.drawImage(tile[whichTileYouWantForRed], x, y, null); // where x and y is the pixel you read.
}
Alternatively, you can skip extracting the components altogether, and simply use colour, i.e.
if (colour == 0x00ff0000) {
gc.drawImage(tile[whichTileYouWantForRed], x, y, null); // where x and y is the pixel you read.
}
(Which will be slightly faster anyway and actually what you want.)