For a project I'm working on, I need to animate an ocean through salinity colormaps. Essentially, I have salinity data for n timesteps and I need to update the pixel colors to match the current salinity values.
Right now I'm going through every pixel on a canvas in a nested for-loop and, for each pixel, computing the current salinity value at that pixel and updating the pixel's color to match the salinity value.
I keep track of the frames with a AnimationTimer object. The problem is that I'm getting around 20 fps and the application is lagging on events like resizing the window.
private void drawHeatMap(int depth, float t) {
float meanSalt = model.getMeanSalinity(depth);
float stdSalt = model.getSalinityStdDev(depth);
Colorbar colorbar = new Colorbar(SALINITY_COLORMAP);
colorbar.setValueRange(meanSalt - stdSalt, meanSalt + (stdSalt / 2));
GraphicsContext graphicsContext = canvas.getGraphicsContext2D();
graphicsContext.clearRect(0, 0, canvas.getWidth(), canvas.getHeight());
PixelWriter pixelWriter = graphicsContext.getPixelWriter();
for (int x = 0; x < canvas.getWidth(); ++x) {
for (int y = 0; y < canvas.getHeight(); ++y) {
GeoCoordinate coordinate = getMapCoordinates(x, y);
if (model.isOverWater(coordinate)) {
float salt = model.computeSalinity(coordinate, depth, t);
pixelWriter.setColor(x, y, colorbar.getColor(salt));
}
}
}
}
This is the code that runs every frame, but even a simpler version like the one below still runs at 30 fps.
private void drawHeatMapBenchmark(int depth, float t) {
GraphicsContext graphicsContext = canvas.getGraphicsContext2D();
graphicsContext.clearRect(0, 0, canvas.getWidth(), canvas.getHeight());
PixelWriter pixelWriter = graphicsContext.getPixelWriter();
for (int x = 0; x < canvas.getWidth(); ++x) {
for (int y = 0; y < canvas.getHeight(); ++y) {
pixelWriter.setColor(x, y, Color.gray(Math.random()));
}
}
}
Anyone knows a better approach to this? I'm not familiar with computer graphics, thank you.
You are doing all this on the (single) JavaFX application thread. It might help if you first render your heat map into an image which you then draw onto the canvas because for your particular case it seems to be very simple to do all the image rendering on parallel threads which should speed up the whole process.
Related
I have a picture then used a flashlight type of light to only show where the mouse is hovering over. That part of the code works, but now I want to use if/else statements to zoom in on the selected area and then click again to zoom back out. Any other way to zoom in on specific area then back out of that area also helps. Really any help will be appreciated!
PImage ispy;
void setup () {
size(1024,768);
ispy = loadImage("ispy2.jpeg");
}
void draw () {
loadPixels();
ispy.loadPixels();
for (int x = 0; x < width; x++) {
for (int y = 0; y < height; y++) {
int loc = x+y*width;
float r = red(ispy.pixels[loc]);
float g = green(ispy.pixels[loc]);
float b = blue(ispy.pixels[loc]);
float d = dist(mouseX, mouseY, x, y); //
float factor = map(d, 0, 200, 2, 0);
pixels[loc] = color(r*factor, g*factor, b*factor);
}
}
updatePixels();
}
Here is my interpretation of what you are talking about. We store a isClicked boolean to store the state of whether we should zoom in or not. When we are going to draw the image, we translate() to the mouse, then we scale(), then we translate() back the same amount that we moved before, but in the opposite direction. What this does is it does the scale transform around the mouse position.
One thing that I couldn't find a way around way your way of updating the pixels directly from the image and the flashlight effect. What the program is doing instead is using your method to make a mask image and applying that to a PGraphics object. Another thing that I noticed is that when just rendering straight to the screen, there is considerable lag from the scaling. Instead, I have moved the drawing to a PGraphics object. This improves the performance.
In the end, to render, the program is drawing everything on the PGraphics object, then applying the mask to that object to get the flashlight effect.
Here is the code that I have:
PImage ispy, distMask;
boolean isClicked = false;
PGraphics renderer;
void createDistanceMask(PImage distMask){ //Takes image and changes its pixels to "alpha" for the PGraphics renderer
distMask.loadPixels();
for (int x = 0; x < width; x++) {
for (int y = 0; y < height; y++) {
int loc = x+(height-y-1)*width;
float d = dist(mouseX, mouseY, x, y); //
int factor = int(map(d, 0, 200, 400, 0)); //Pixel data will be between 0 and 255, must scale down later.
if (factor > 255)
factor = 255;
distMask.pixels[loc] = color(factor,factor,factor);
}
}
distMask.updatePixels();
}
void setup () {
size(1024,768, P2D);
ispy = loadImage("ispy2.jpeg");
distMask = new PImage(width,height);
renderer = createGraphics(width,height,P2D);
mouseX = width/2; //Not necessary, but will have black screen until mouse is moved
mouseY = height/2;
}
void draw () {
background(0);
pushMatrix();
createDistanceMask(distMask);
renderer.beginDraw(); //Starts processing stuff into PGraphics object
renderer.background(0);
if(isClicked){ //This is to get the zoom effect
renderer.translate(mouseX, mouseY);
renderer.scale(2);
renderer.translate(-mouseX, -mouseY);
}
renderer.image(ispy,0,0); //Render Image
renderer.endDraw();
renderer.mask(distMask); //Apply Distance mask for flashlight effect
image(renderer,0,0); //Draw renderer result to screen
popMatrix();
}
void mouseClicked(){
isClicked = !isClicked;
}
In my comment, I asked about having the screen move to the mouse, which is what this is doing. If you want to "freeze" the screen in one position, what you can do is store a lastMouseClickPosition PVector or simply just ints. Then, when translating, translate to the position instead of the PVector.
Here's the code that would change:
PVector lastClickPos = new PVector(); //Make the position
if(isClicked){ //When Rendering
renderer.translate(lastClickPos.x, lastClickPos.y);
renderer.scale(scalingFactor);
renderer.translate(-lastClickPos.x, -lastClickPos.y);
}
void mouseClicked(){ //At the bottom
isClicked = !isClicked;
lastClickPos.set(mouseX, mouseY);
}
I have been doing a small little project using Processing, and the effect I wanted to achieve was a kind of "mountains" forming and moving, using Perlin Noise with the noise() function, with 2 parameters.
I was originally using a image for the background, but for illustrational purposes, I made the background black, and it's basically the same effect.
My issue is that I want to have a "history" of the mountains because they should fade away after some time, and so I made a history of PShapes, and draw the history and update it each frame.
Updating it is no issue, but drawing the PShapes seems to take a lot of time, reducing the frame rate from 60 to 10 when the length of the history is 100 elements.
Below is the code I used :
float noise_y = 0;
float noise_increment = 0.01;
// increment x in the loop by this amount instead of 1
// makes the drawing faster, since the PShapes have less vertices
// however, mountains look sharper, not as smooth
// bigger inc = better fps
final int xInc = 1;
// maximum length of the array
// bigger = less frames :(
final int arrLen = 100;
int lastIndex = 0;
PShape[] history = new PShape[arrLen];
boolean full = false;
// use this to add shapes in the history
PShape aux;
void setup() {
size(1280, 720);
}
void draw() {
background(0);
// create PShape object
aux = createShape();
aux.beginShape();
aux.noFill();
aux.stroke(255);
aux.strokeWeight(0.5);
for (float x = 0; x < width + xInc; x = x + xInc) {
float noise = noise(x / 150, noise_y) ;
// get the actual y coordinate
float y = map(noise, 0, 1, height / 2, 0);
// create vertex of shape at x, y
aux.vertex(x, y);
}
aux.endShape();
// push the current one in the history
history[lastIndex++] = aux;
// if it reached the maximum length, start it over ( kinda works like a queue )
if (lastIndex == arrLen) {
lastIndex = 0;
full = true;
}
// draw the history
// this part takes the MOST TIME to draw, need to fix it.
// without it is running at 60 FPS, with it goes as low as 10 FPS
if (full) {
for (int i = 0; i < arrLen; i++) {
shape(history[i]);
}
} else {
for (int i = 0; i < lastIndex; i++) {
shape(history[i]);
}
}
noise_y = noise_y - noise_increment;
println(frameRate);
}
I have tried to use different ways of rendering the "mountains" : I tried writing my own class of a curve and draw lines that link the points, but I get the same performance. I tried grouping the PShapes into a PShape group object like
PShape p = new PShape(GROUP);
p.addChild(someShape);
and I got the same performance.
I was thinking of using multiple threads to render each shape individually, but after doing some research, there's only one thread that is responsible with rendering - the Animation Thread, so that won't do me any good, either.
I really want to finish this, it seems really simple but I can't figure it out.
One possible solution would be, not to draw all the generated shapes, but to draw only the new shape.
To "see" the shapes of the previous frames, the scene can't be cleared at the begin of the frame, of course.
Since the scene is never cleared, this would cause, that the entire view is covered, by shapes over time. But if the scene would be slightly faded out at the begin of a new frame, instead of clearing it, then the "older" shapes would get darker and darker by time. This gives a feeling as the "older" frames would drift away into the depth by time.
Clear the background at the initlization:
void setup() {
size(1280, 720);
background(0);
}
Create the scene with the fade effect:
void draw() {
// "fade" the entire view
blendMode(DIFFERENCE);
fill(1, 1, 1, 255);
rect(0, 0, width, height);
blendMode(ADD);
// create PShape object
aux = createShape();
aux.beginShape();
aux.stroke(255);
aux.strokeWeight(0.5);
aux.noFill();
for (float x = 0; x < width + xInc; x = x + xInc) {
float noise = noise(x / 150, noise_y) ;
// get the actual y coordinate
float y = map(noise, 0, 1, height / 2, 0);
// create vertex of shape at x, y
aux.vertex(x, y);
}
aux.endShape();
// push the current one in the history
int currentIndex = lastIndex;
history[lastIndex++] = aux;
if (lastIndex == arrLen)
lastIndex = 0;
// draw the newes shape
shape(history[currentIndex]);
noise_y = noise_y - noise_increment;
println(frameRate, full ? arrLen : lastIndex);
}
See the preview:
Hi I am in need of some help. I need to write a convolution method from scratch that takes in the following inputs: int[][] and BufferedImage inputImage. I can assume that the kernel has size 3x3.
My approach is to do the follow:
convolve inner pixels
convolve corner pixels
convolve outer pixels
In the program that I will post below I believe I convolve the inner pixels but I am a bit lost at how to convolve the corner and outer pixels. I am aware that corner pixels are at (0,0), (width-1,0), (0, height-1) and (width-1,height-1). I think I know to how approach the problem but not sure how to execute that in writing though. Please to aware that I am very new to programming :/ Any assistance will be very helpful to me.
import java.awt.*;
import java.awt.image.BufferedImage;
import com.programwithjava.basic.DrawingKit;
import java.util.Scanner;
public class Problem28 {
// maximum value of a sample
private static final int MAX_VALUE = 255;
//minimum value of a sample
private static final int MIN_VALUE = 0;
public BufferedImage convolve(int[][] kernel, BufferedImage inputImage) {
}
public BufferedImage convolveInner(double center, BufferedImage inputImage) {
int width = inputImage.getWidth();
int height = inputImage.getHeight();
BufferedImage inputImage1 = new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB);
//inner pixels
for (int x = 1; x < width - 1; x++) {
for (int y = 1; y < height - 1; y ++) {
//get pixels at x, y
int colorValue = inputImage.getRGB(x, y);
Color pixelColor = new Color(colorValue);
int red = pixelColor.getRed() ;
int green = pixelColor.getGreen() ;
int blue = pixelColor.getBlue();
int innerred = (int) center*red;
int innergreen = (int) center*green;
int innerblue = (int) center*blue;
Color newPixelColor = new Color(innerred, innergreen, innerblue);
int newRgbvalue = newPixelColor.getRGB();
inputImage1.setRGB(x, y, newRgbvalue);
}
}
return inputImage1;
}
public BufferedImage convolveEdge(double edge, BufferedImage inputImage) {
int width = inputImage.getWidth();
int height = inputImage.getHeight();
BufferedImage inputImage2 = new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB);
//inner pixels
for (int x = 0; x < width - 1; x++) {
for (int y = 0; y < height - 1; y ++) {
//get pixels at x, y
int colorValue = inputImage.getRGB(x, y);
Color pixelColor = new Color(colorValue);
int red = pixelColor.getRed() ;
int green = pixelColor.getGreen() ;
int blue = pixelColor.getBlue();
int innerred = (int) edge*red;
int innergreen = (int) edge*green;
int innerblue = (int) edge*blue;
Color newPixelColor = new Color(innerred, innergreen, innerblue);
int newRgbvalue = newPixelColor.getRGB();
inputImage2.setRGB(x, y, newRgbvalue);
}
}
return inputImage2;
}
public BufferedImage convolveCorner(double corner, BufferedImage inputImage) {
int width = inputImage.getWidth();
int height = inputImage.getHeight();
BufferedImage inputImage3 = new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB);
//inner pixels
for (int x = 0; x < width - 1; x++) {
for (int y = 0; y < height - 1; y ++) {
//get pixels at x, y
int colorValue = inputImage.getRGB(x, y);
Color pixelColor = new Color(colorValue);
int red = pixelColor.getRed() ;
int green = pixelColor.getGreen() ;
int blue = pixelColor.getBlue();
int innerred = (int) corner*red;
int innergreen = (int) corner*green;
int innerblue = (int) corner*blue;
Color newPixelColor = new Color(innerred, innergreen, innerblue);
int newRgbvalue = newPixelColor.getRGB();
inputImage3.setRGB(x, y, newRgbvalue);
}
}
return inputImage3;
}
public static void main(String[] args) {
DrawingKit dk = new DrawingKit("Compositor", 1000, 1000);
BufferedImage p1 = dk.loadPicture("image/pattern1.jpg");
Problem28 c = new Problem28();
BufferedImage p5 = c.convolve();
dk.drawPicture(p5, 0, 100);
}
}
I changed the code a bit but the output comes out as black. What did I do wrong:
import java.awt.*;
import java.awt.image.BufferedImage;
import com.programwithjava.basic.DrawingKit;
import java.util.Scanner;
public class Problem28 {
// maximum value of a sample
private static final int MAX_VALUE = 255;
//minimum value of a sample
private static final int MIN_VALUE = 0;
public BufferedImage convolve(int[][] kernel, BufferedImage inputImage) {
int width = inputImage.getWidth();
int height = inputImage.getHeight();
BufferedImage inputImage1 = new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB);
//for every pixel
for (int x = 0; x < width; x ++) {
for (int y = 0; y < height; y ++) {
int colorValue = inputImage.getRGB(x,y);
Color pixelColor = new Color(colorValue);
int red = pixelColor.getRed();
int green = pixelColor.getGreen();
int blue = pixelColor.getBlue();
double gray = 0;
//multiply every value of kernel with corresponding image pixel
for (int i = 0; i < 3; i ++) {
for (int j = 0; j < 3; j ++) {
int imageX = (x - 3/2 + i + width) % width;
int imageY = (x -3/2 + j + height) % height;
int RGB = inputImage.getRGB(imageX, imageY);
int GRAY = (RGB) & 0xff;
gray += (GRAY*kernel[i][j]);
}
}
int out;
out = (int) Math.min(Math.max(gray * 1, 0), 255);
inputImage1.setRGB(x, y, new Color(out,out,out).getRGB());
}
}
return inputImage1;
}
public static void main(String[] args) {
int[][] newArray = {{1/9, 1/9, 1/9}, {1/9, 1/9, 1/9}, {1/9, 1/9, 1/9}};
DrawingKit dk = new DrawingKit("Problem28", 1000, 1000);
BufferedImage p1 = dk.loadPicture("image/pattern1.jpg");
Problem28 c = new Problem28();
BufferedImage p2 = c.convolve(newArray, p1);
dk.drawPicture(p2, 0, 100);
}
}
Welcome ewuzz! I wrote a convolution using CUDA about a week ago, and the majority of my experience is with Java, so I feel qualified to provide advice for this problem.
Rather than writing all of the code for you, the best way to solve this large program is to discuss individual elements. You mentioned you are very new to programming. As the programs you write become more complex, it's essential to write small working snippets before combining them into a large successful program (or iteratively add snippets). With this being said, it's already apparent you're trying to debug a ~100 line program, and this approach will cost you time in most cases.
The first point to discuss is the general approach you mentioned. If you think about the program, what is the simplest and most repeated step? Obviously this is the kernel/mask step, so we can start from here. When you convolute each pixel, you are performing a similar option, regardless of the position (corner, edge, inside). While there are special steps necessary for these edge cases, they share similar underlying steps. If you try to write code for each of these cases separately, you will have to update the code in multiple (three) places with each adjustment and it will make the whole program more difficult to grasp.
To support my point above, here's what happened when I pasted your code into IntelliJ. This illustrates the (yellow) red flag of using the same code in multiple places:
The concrete way to fix this problem is to combine the three convolve methods into a single one and use if statements for edge-cases as necessary.
Our pseudocode with this change:
convolve(kernel, inputImage)
for each pixel in the image
convolve the single pixel and check edge cases
endfor
end
That seems pretty basic right? If we are able to successfully check edge cases, then this extremely simple logic will work. The reason I left it so general above to show how convolve the single pixel and check edge cases is logically grouped. This means it's a good candidate for extracting a method, which could look like:
private void convolvePixel(int x, int y, int[][] kernel, BufferedImage input, BufferedImage output)
Now to implement our method above, we will need to break it into a few steps, which we may then break into more steps if necessary. We'll need to look at the input image, if possible for each pixel accumulate the values using the kernel, and then set this in the output image. For brevity I will only write pseudocode from here.
convolvePixel(x, y, kernel, input, output)
accumulation = 0
for each row of kernel applicable pixels
for each column of kernel applicable pixels
if this neighboring pixel location is within the image boundaries then
input color = get the color at this neighboring pixel
adjusted value = input color * relative kernel mask value
accumulation += adjusted value
else
//handle this somehow, mentioned below
endif
endfor
endfor
set output pixel as accumulation, assuming this convolution method does not require normalization
end
The pseudocode above is already relatively long. When implementing you could write methods for the if and the else cases, but it you should be fine with this structure.
There are a few ways to handle the edge case of the else above. Your assignment probably specifies a requirement, but the fancy way is to tile around, and pretend like there's another instance of the same image next to this input image. Wikipedia explains three possibilities:
Extend - The nearest border pixels are conceptually extended as far as necessary to provide values for the convolution. Corner pixels are extended in 90° wedges. Other edge pixels are extended in lines.
Wrap - (The method I mentioned) The image is conceptually wrapped (or tiled) and values are taken from the opposite edge or corner.
Crop - Any pixel in the output image which would require values from beyond the edge is skipped. This method can result in the output image being slightly smaller, with the edges having been cropped.
A huge part of becoming a successful programmer is researching on your own. If you read about these methods, work through them on paper, run your convolvePixel method on single pixels, and compare the output to your results by hand, you will find success.
Summary:
Start by cleaning-up your code before anything.
Group the same code into one place.
Hammer out a small chunk (convolving a single pixel). Print out the result and the input values and verify they are correct.
Draw out edge/corner cases.
Read about ways to solve edge cases and decide what fits your needs.
Try implementing the else case through the same form of testing.
Call your convolveImage method with the loop, using the convolvePixel method you know works. Done!
You can look up pseudocode and even specific code to solve the exact problem, so I focused on providing general insight and strategies I have developed through my degree and personal experience. Good luck and please let me know if you want to discuss anything else in the comments below.
Java code for multiple blurs via convolution.
I'm attempting to take a picture as input, then manipulate said picture (I specifically want to make it greyscale) and then output the new image. This is a snippet of the code that I'm editing in order to do so but I'm getting stuck. Any ideas of what I can change/do next. Greatly appreciated!
public boolean recieveFrame (Image frame) {
int width = frame.width();
int height = frame.height();
for (int i = 0; i < width; i++) {
for (int j = 0; j < height; j++) {
Color c1 = frame.get(i, j);
double greyScale = (double) ((Color.red *.3) + (Color.green *.59) + (Color.blue * .11));
Color newGrey = Color.greyScale(greyScale);
frame.set(i, j, newGrey);
}
}
boolean shouldStop = displayImage(frame);
return shouldStop;
}
I'm going to try to stick as close as possible to what you already have. So, I'll assume that you are looking for how to do pixel-level processing on an Image, rather than just looking for a technique that happens to work for converting to greyscale.
The first step is that you need the image to be a BufferedImage. This is what you get by default from ImageIO, but if you have some other type of image, you can create a BufferedImage and paint the other image into it first:
BufferedImage buffer = new BufferedImage(w, h, BufferedImage.TYPE_INT_RGB);
Graphics2D g = buffer.createGraphics();
g.drawImage(image, 0, 0);
g.dispose()
Then, you can operate on the pixels like this:
public void makeGrey(BufferedImage image) {
for(int x = 0; x < image.getWidth(); ++x) {
for(int y = 0; y < image.getHeight(); ++y) {
Color c1 = new Color(image.getRGB(x, y));
int grey = (int)(c1.getRed() * 0.3
+ c1.getGreen() * 0.59
+ c1.getBlue() * .11
+ .5);
Color newGrey = new Color(grey, grey, grey);
image.setRGB(x, y, newGrey.getRGB());
}
}
}
Note that this code is horribly slow. A much faster option is to extract all the pixels from the BufferedImage into an int[], operate on that, and then set it back into the image. This uses the other versions of the setRGB()/getRGB() methods that you'll find in the javadoc.
I'm trying to get a small section of image on the screen and read any pixel to compare the other pixels.The code to get screen image is:
Rectangle captureSize = new Rectangle(x, y, height, width);
BufferedImage image = robot.createScreenCapture(captureSize);
And, to read pixel by pixel I used
for (int y = 0; y < image.getHeight(); y = y + 1) {
for (int x = 0; x < image.getWidth(); x = x + 1) {
color = image.getRGB(x, y);
// Some methods etc
{
{
However, when I ran it I was shocked. Because createScreenCapture took about 40 ms and using getRGB of each pixel took around 350 ms which is very inefficient to create an application for 60 fps. By the way, my image is 800x400 pixels size. I didn't try
rgbArray = image.getRGB(startX, startY, w, h, rgbArray,offset, scansize) ;
method because I don't know how efficient it is and to reorder my code would be a bit difficult. So, any help would be appreciated.
Use
rgbArray = image.getRGB(startX, startY, w, h, rgbArray,offset, scansize) ;
It will be much faster to read the pixel values from the array than to do the method call to get each pixel value, and the single call to getRGB to fetch the array is not slow.