I am trying to implement a painting tool with layer support. For each layer I create a WritableRaster. To display the canvas, the layers have to be merged. What's the most efficient way to do this? I could use a bufferedImage, set the raster and draw each layer to the Graphics context with a alpha composite. But isn't it better/faster to merge it on data level and then draw a single image?
Edit:
I did some profiling. This is what my current approach looks like:
//tmp.getRaster().setRect(layer.getRaster()); // VERY slow
tmp.getRaster().setDataElements(0, 0, layer.getRaster()); //slow
g2.drawImage(tmp, 0, 0, scaledDim.width, scaledDim.height, null);
I used BufferedImages instead of WritableRasters for each layer before and there was no delay noticeable.
My next step would be to cache top and bottom layers, so there would be only 3 images to be drawn. That would probably solve the problem.
My sense is that drawing a raster is roughly equivalent to "merging it on a data level".
I wouldn't worry about this unless you notice it's a bottleneck.
maybe you can use JAI:
ParameterBlock pb = new ParameterBlock();
pb.add(bufImage1);
pb.add(bufImage2);
pb.add(....
RenderedImage ri = JAI.create("xor",pb);
If you blit (draw) them each onto an image buffer, you are probably using graphics-card acceleration.
If you loop through the indivual buffers and merge them by hand (I assume this is what you mean by "Do it on the data level"), you will be using the CPU.
So actually, doing it by hand will be significantly slower than blitting.
So, your original solution is best. Let the graphics card do its job, it's very good at it.
Ok, so I introduced a upper and a lower layer cache and now I got a pretty good performance. I can add as many layers as my memory can handle without affecting rendering time.
Related
Are there any in-built methods, or flags I can set, with a Java Graphics2D object such that I can draw polygons or other shapes with gaussian blur automatically applied?
Or if anyone knows of a computationally efficient shortcut/hack, that would be greatly appreciated.
AFAIK - no.
Most blur operations operate on a BufferedImage as they need to manipulate the underlying pixel data. A typical approach is to render the content to an image, blur it, then paint that to whatever you want
I prefer to use JHLabs filters generally, there not the fastest, but generally give a high quality result.
This example is a little involved, but demonstrates the basic idea.
As another example and if your really want to make your eyes bleed ;)
I am developing an isometric game in Java2D. I.e, note that I do not have direct access to hardware pixel shaders (real-time software pixel shaders aren't practical. I can do a single pass on every entity texture without a noticeable hit on performance)
I know the typical method would be to somehow encode the depth of the individual pixels into a depth buffer and look that up. However, I don't know how I can do that efficiently in Java2D. How would I store the depth buffer? How would I filter out the alpha in an image? Etc.
Up until now I have just been reversing the projection matrix I use to calculate the tile-coordinates. However, that doesn't work well when you have entities that render outside of those tile's bounds.
Another method I considered was using a color-map, however I have the same problems with this as I do with the depth buffer (and if I can get the depth buffer working I'd much rather use that.)
Here is a picture of what I am working with:
I've resolved this quite nicely. The solution is actually very simple, just unconventional.
The graphics are depth sorted via a TreeMap, and then rendered to the screen. One can simply traverse this TreeMap in reverse (and keep it until the next render cycle) to translate the cursor location to the proper image it falls over (by testing the pixels [in reverse render order] and checking if they are transparent.)
The solution is in the open-source project, under the io.github.jevaengine.world.World class, pick method. https://github.com/JeremyWildsmith/JevaEngine/blob/master/jevaengine/src/main/java/io/github/jevaengine/world/World.java
You see that ?
Each brick of each house is an image of 16x16 pixels.
What you can see here a version based on simple JavaFX, with some Imageview moved on X and Y to give the effect of "construction".
I just adapt this to Swing using paintComponent.
The problem:
- With JavaFX: my computer has trouble. What you see on the picture took 2 seconds to load and it is then moving very slow and jerky.
- With Swing: I do not know how to adapt each block according to brightness, shadows, etc.. So it looks like this:
What method should I chose? Both have major drawbacks. I would have liked to keep the JavaFX method but also to find something else than Imageview. It should not be a good idea.
For information on Swing optimizations and implementations, see other answers as my answer is JavaFX specific.
Should you stick with a JavaFX implementation, here are a couple of things to try:
Use node.setCache(true) to switch on node caching.
Use node.setCacheHint(CacheHint.SPEED) to enable high speed node transforms.
Provide an implementation with no effects and another with effects and see if the one without effects performs noticeably better. (If that's the case you may need to use a more efficient effect chain or forgo some effects altogether).
Check the JavaFX system requirements to ensure that your setup meets the minimum requirements for hardware acceleration in terms of hardware/os/driver combinations.
If you are using Regions and css based processing be careful that the css system is not causing too much overhead (by, for example, having a version of the code which does not use css and comparing it's performance).
If needed, implement a level of detail zoom, (e.g. when zoomed out provide a single image for the entire house rather than a separate image for each house tile).
Make sure you are only loading a given Image once and reusing it in multiple ImageViews.
Use a profiler to identify bottlenecks.
Vary the test cpu or graphics card to see if either of those are a bottleneck.
Try a Java 8 preview, which features many internal performance optimizations for JavaFX.
If you are loading a lot of images, use background loading and have an animation or progress bar playing in front while the images are loading.
If you are loading lots of images, scale them upfront in the Image constructor so that they don't take up too much memory or require (potentially) extra processing power for the extra pixels (only really a consideration if you are using a lot of different very high resolution textures - which you don't seem to be doing).
For example:
Image tile = new Image("tile.png");
Group house = new Group();
house.setCache(true);
house.setCacheHint(CacheHint.SPEED);
Effect lighting = new Lighting();
for (int i = 0; i < houseWidth; i++) {
// here is the critical part => don't do new ImageView(new Image("tile.png"))
ImageView tileView = new ImageView(tile));
tileView.setEffect(lighting);
tileView.setCache(true);
tileView.setCacheHint(CacheHint.SPEED);
house.add(tileView);
}
Mantrid's suggestions are interesting. I do believe that, with JavaFX you don't need to implement a dirty rectangle algorithm yourself (as the underlying platform is able to take care of dirty region processing for you). It may be that, because it is a generic mechanism, it isn't providing the level of optimization required for your specific case, in which case you need to handle the dirty processing yourself (e.g. by removing nodes from the scenegraph and re-adding them as appropriate).
Additionally, pre-calculating the blur/brightness/etc on the images could be done by defining effects in JavaFX, applying the effects to an offscreen ImageView node, then taking a snapshot of the offscreen ImageView node to get a pre-calculated image. This technique would allow you to reuse your existing JavaFX effects pipeline without the need to re-implement it using the ConvolveOp mechanism. Perhaps you may get the same performance level by just setting cache to true and cacheHint to speed on the ImageView nodes, as I believe this does a similar kind of thing behind the scene (i.e. increases speed at the cost of increased memory usage).
The JavaFX scenegraph is pretty efficient and can handle thousands of nodes. However, you may have more than that in your application. If the above optimization points don't help you, you may want to calculate your node count and post a reference to your question (and perhaps some source) to the open-jfx mailing list where the JavaFX developers who know the details of JavaFX application optimization are.
JavaFX 8 has far better support for 3D scenegraphs in than the (mostly useless) 3D scenegraph in JavaFX 2.2. It seems that your current application is psuedo-3d where you transform 2D objects, individually apply lighting effects and adjust the brightness of each tile to get a 3D look. If that's the case, using a fully hardware accelerated 3D scenegraph with a unified 3D lighting model may end up performing better, looking better and being easier to work with - you would have to evaluate it in the context of your application to see if it is worth switching your app from a 2D JavaFX scenegraph to a 3D JavaFX scenegraph or from JavaFX to Swing or some other tech like libgdx.
Addendum
Answers to some of Mizur's additional questions:
So why are they so power-consuming?
As can be seen from my answer, there are many aspects that go into performance, so singling out one particular reason as to why something is power-consuming is sometimes difficult and often not possible. Sometimes it is a combination of things which contribute to performance issues. The biggest wins usually come from optimizing stuff executed in inner most loops or by changing the strategy used to solve a problem.
If they need so much, please let me move tiles, or please just be displayed, no?
In terms of only consuming resources for the stuff you move, the JavaFX scenegraph has some optimizations for that and things like the cachehints can further improve performance for that. Perhaps they aren't enough for your particular use case or the way in which you are using the system. You may need to change your algorithms to reduce the system load.
Is there a way to generate images of each building with this technique of Imageview, then to integrate these images of buildings in the final scene?
Yes - you could render a building to an offscreen scene using many images and nodes, snapshot the offscreen scene to create a single image, then overlay that image over your final scene.
modifying blur/brightness/etc on the images before they are drawn
I haven't used it myself, but here's a hint - use Swing/AWT Kernel+ConvolveOp mechanism: http://www.java-tips.org/java-se-tips/java.awt.image/styling-digital-images-with-convolveop.html
BufferedImage biSrc = ...
BufferedImage biDest = ...
float[] data = new float[] { 0.0625f, 0.125f, 0.0625f, 0.125f, 0.25f, 0.125f, 0.0625f, 0.125f, 0.0625f };
Kernel kernel = new Kernel(3, 3, data);
ConvolveOp convolve = new ConvolveOp(kernel, ConvolveOp.EDGE_NO_OP, null);
convolve.filter(biSrc, biDest);
operations are defined via a 3x3 matrix (data). you can do a search for examples for brigthness, blur etc.
the picture took 2 seconds to load but it is moving very slow and jerky after
use Dirty Rectangles algorithm: Dirty Rectangles
I'm making a 2D game in Java and one of the main issues causing low FPS (on my slow laptop) is having to re-draw complex structures to a Graphics instance, such as dials with markings.
The dial and its markings will never change unless the window is resized, so I thought it would be a good idea to draw to a BufferedImage and just re-draw the image rather than re-drawing the details. The position of the needle obviously changes, so this can just be drawn on top.
I've never heard about this being done to improve the FPS of 2D games so I'm wondering if it's actually good practice to store a cache of images or if there's a better way to solve this sort of problem? Are there any issues associated with this that I haven't considered?
Caching images isn't a bad idea: you can rely on raster rendering to be pretty well optimised on most any platform. In my experience (which is admittedly mostly on mobile devices where 2D graphics are concerned) the Graphics.drawXXX() methods are often considerably slower than Graphics.drawImage().
In my experience the vast majority of 2D games out there make use of sprites (i.e. images) for rendering just about everything. Often that's true even when the graphics look like they are rendered using primitives!
Another useful technique to think about is not redrawing regions at all unless you really need to!
EDIT:
As others have mentioned, the major tradeoff is that you're going to be using more memory. You're also going to have to make sure you free up those images once you no longer need them.
Is it good practice to cache parts of a 2D drawing?
You're making a trade-off between drawing speed and storage space. Only you can determine which is more important.
You might consider rendering your dials in advance and saving the images as GIF, JPG, or PNG files. You would have to scale these images to your window size before you draw them.
Are you using double buffering for your Graphics panel?
Yes, that is a good practice, and it's done all the time. Drawing to an image first before displaying it on the screen is called double buffering, and that method can be used in different ways according to the needs of the program.
The downside of double buffering is memory, since it takes more memory to store the second image, but that sounds like a trade-off you'll need to make.
I'm writing a game in Java, LJGWL (OpenGL). I'm using a library that handles a lot of messy details for me, but need to find a lot faster way to do this.
Basically I want to set every pixel on the screen to say a random color as fast a possible. The "random colors" is just an Array [][] that gets updated every 2-3 seconds. I've tried drawing rects and using images, both are pretty slow for what I want to do.
I think I want to learn how to write a GPU shader? That is the fastest way to do this? LJGWL exposes OpenGL api to java. Any basic tutorials on how to get started with OpenGL shaders? Or should I dynamically create a texture of some sort and then just throw up the entire texture, would that be faster?
If it were the case that you were statically displaying the same image, than using a texture or display list would suffice. But as you want to frequently update it, shaders really are the best option. Shader code executes on the GPU and modifies data in GRAM, so you have no bottle neck transferring from CPU to GPU. The next best thing would probably be a Pixel or Frame Buffer Object. Buffer Objects let you read/write to GRAM via DMA (without having to go through the CPU) so they can be pretty fast.
I haven't written any shaders yet, so I can't recommend any good resources. But SongHo's OpenGL pages are a good place to learn about Buffer Objects. (His examples are in C++ though)
Textures are the fastest way to draw something on screen, draw a texture mapped quad into the screen, it should be fast enough. When you need to reupload the texture data, use glTexSubimage2D to update it.
No need to use shaders.
I've yet to do any work with shaders in OpenGL, but given the same scenario in multiple occasions, I handled it with a texture I threw up across the screen on top, and it worked quite effectively.
I don't know how you are drawing your pixels exactly, but this limit you hit could be because of the amount of data you transfer (inefficiently?). Updating a screen full of pixels every 2-3 seconds shouldn't be hard at all. Although shaders bring you closer to the graphics card, they will never make inefficient methods fast, so...
Why is your code so slow?
What code? What code exactly did you try? What texture did you use, render to, ...?
Is it slow? How slow? How fast do you expect it to be?
How quickly can one get 1920x1080(?) pixels in video ram, what's your hardware, drivers, OS?
I think you need to edit/repost before we can help you solve your problem. Just because it is slow, is no guarantee at all that shaders will even be one bit faster.