I am writing a JavaFX application that receives data points on a socket and visualizes them in real time. The problem is that the JavaFX rendering is too slow. I have a Swing implementation that runs fast enough but I need to use JavaFX instead.
The constraints that I am working within are:
The control for the visualization must only be updated by the JavaFX application thread (I believe this is required for all JavaFX and Swing applications).
The visualization should be updated smoothly from the perspective of the human eye. Around 10 updates per second would be sufficient. Once every second would not be sufficient.
The incoming data rate is high enough (about 50 events per second which is not that high in other contexts) and the per event processing is expensive enough that the incoming data must be received and processed in a thread other than the JavaFX application thread so that the GUI doesn't block (I believe this is a somewhat common requirement for many GUI applications).
My approach so far has been to use a Canvas JavaFX node as the visualization control and for the reception thread to schedule updates to the Canvas to run later in the JavaFX application thread, like this.
public void onEvent(Event event) {
....do processing...
Platform.runLater(new Runnable() {
#Override
public void run() {
graphics.setFill(...);
graphics.fillRect(...);
}});
}
I have thought of a couple of approaches that might speed this up:
Use a WritableImage instead of a Canvas for the visualization. The downside is that WritableImage/PixelWriter doesn't seem to have many drawing methods, for example it doesn't even have fillRect. I think I would have to implement my own versions and my versions would probably be slower.
Have a Canvas object owned by the thread that processes incoming data. Copy from that canvas to the canvas that is a node in the scene graph in the JavaFX application thread. The copy would probably be done with code along these lines sceneCanvas.getGraphicsContext2D().drawImage(processingCanvas.snapshot(SnapshotParameters(), null) 0, 0);. The downside of this is that I think it isn't thread safe and it seems that the snapshot call is relatively expensive.
Render to an AWT BufferedImage in the thread that processes incoming data and then copy from the BufferedImage to the Canvas using SwingFXUtils.toFXImage(). The downside of this is that the threading semantics seem unclear and it seems a little silly to use AWT.
Would you be able to suggest some potential approaches?
Thank you!
I assume, the main problem is that your code pushes too many drawing-tasks into the queue of your FX Application thread. Usually, it is sufficient to have 60 drawing operations per second, which is equal to the refresh rate of your monitor. If you get more "incoming data" events than that, you'll draw more often than necessary, wasting CPU. So you must decouple data processing from painting.
One solution is to use an AnimationTimer. Its handle method will be called in every animation frame, so usually 60 times per second. The animation timer handles redrawing in case new data has been processed.
// generic task that redraws the canvas when new data arrives
// (but not more often than 60 times per second).
public abstract class CanvasRedrawTask<T> extends AnimationTimer {
private final AtomicReference<T> data = new AtomicReference<T>(null);
private final Canvas canvas;
public CanvasRedrawTask(Canvas canvas) {
this.canvas = canvas;
}
public void requestRedraw(T dataToDraw) {
data.set(dataToDraw);
start(); // in case, not already started
}
public void handle(long now) {
// check if new data is available
T dataToDraw = data.getAndSet(null);
if (dataToDraw != null) {
redraw(canvas.getGraphicsContext2D(), dataToDraw);
}
}
protected abstract void redraw(GraphicsContext context, T data);
}
// somewhere else in your concrete canvas implementation
private final RedrawTask<MyData> task = new RedrawTask<MyData>(this) {
void redraw(GraphicsContext context, MyData data) {
// TODO: redraw canvas using context and data
}
}
// may be called by a different thread
public void onDataReceived(...) {
// process data / prepare for redraw task
// ...
// handover data to redraw task
task.requestRedraw(dataToDraw);
}
Related
I have a java App and I am Loading 3 Gif files into BufferedImage Arrays and want to display them.
I have a simple Thread that is executed every e.g. 16ms (~ 60fps) using Thread.sleep()
In this method, I am calling the paintcomponent method to draw on the GUI component.
But I don't want the Thread (Runnable) to wait for the Method to end rendering because the Delay is more than 16ms afterward - even if it is not measurable.
Here's a snippet inside my Runnable JLabel (Currently the fast/efficient method)
#Override
public void run() {
while (active) {
repaint(); //Takes 0ms
i++; // Counter used inside paintComponent()
Thread.sleep(speed * multiplier));
if (i + 1 == frames.length) {
i = 0;
}
}
}
#Override
public void paintComponent(Graphics g) {
g.drawImage(frames[i], 0, 0, x, y, observerFrame);
g.dispose();
}
I tested it and the repaint methods take 0ms. Probably not measurable.
But isn't it still unprecise? Or maybe inefficient?
So the (simple) math is done inside the Thread. But how do I ensure that the timing is precise? I mean it is still precise enough but it still depends on the speed of the pc
After the Answer of #Reto Höhener I did some research and tried another method.
But it is still inefficient and lags when rendering the images (BufferedImages btw):
This is one big attempt I did where I thought that it would be better:
Timer timer = new Timer(16, this);
Executor executor = Executors.newCachedThreadPool();
//called by executor
#Override
public void run() {
i++; // Counter used inside paintComponent()
Thread.sleep(speed * multiplier));
if (i + 1 == frames.length) {
i = 0;
}
}
//called by timer & runs on EDT
#Override
public void actionPerformed(ActionEvent e) {
repaint();
executor.execute(this); //Costs no time and has enought time to complete
}
#Override
public void paintComponent(Graphics g) {
g.drawImage(frames[i], 0, 0, x, y, observerFrame);
g.dispose();
}
So this way I ensure that
The repaint is done in the EDT
The math is done by another Thread using executors
Even if I was proud after compiling and running it the results made me cry...
With the first method, the JVM uses ~10% of the CPU. Everything is fine and running on the fps for all 3 Frames.
The second try uses less CPU but is bad performing.
I don't know why but the GUI is responsive but the rendering is slow almost 3x slower.
The goal is to render images in 60 frames per second without lag.
The point here is that I tried an attempt where I thought that it is conceptually better but it did even worse.
So my question is if there is a better and more precise way to achieve this.
Otherwise, I will stay at the Simple Thread based solution.
regards
Nur1
First it would probably be helpful to learn just a little bit about the Java Event Dispatching Thread (or AWT Event Queue) concept.
My suggestion would be to use one thread that is only responsible to call a .repaint()-method on your component. Instead of rolling your own thread, consider using javax.swing.Timer for this, it already uses a thread behind the scenes, and is able to coalesce multiple events if they arrive too quickly.
Then use another thread that is only responsible for calculating the information required to paint. One approach is to render into a buffer image. The .paint()- or .paintComponent()-method of your display component then only has to paint that image.
I am making a game in java using the MVC design pattern (with swing) and I do not know how to make the Controller class separate from the view Class.
until now I have a model that contains all the data, a controller which in charge of all the logic and now I think about how to separate the view.
I have the GameView which extends the Jpanel and has a paintComonent:
#Override
public void paintComponent(Graphics g){
super.paintComponent(g);
Graphics2D g2D = (Graphics2D) g;
draw(g2D);
}
public void draw(Graphics2D g2D){
drawComponent(background, g2D);
drawComponent(arenaController.getArena().getPlane(), g2D);
drawComponent(arenaController.getArena().getPlayer().getBoat(), g2D);
ArrayList<PlaneDrop> planeDrops = arenaController.getArena().getPlaneDrops();
for(PlaneDrop planeDrop : planeDrops){
drawComponent(planeDrop, g2D);
}
g2D.drawString("Life: " + arenaController.getArena().getPlayer().getLife(), 10, 30);
g2D.drawString("Score: " + arenaController.getArena().getPlayer().getScore(), GAME_WIDTH - 50, 30);
}
but on the other hand, I have the GameEngine which in charge of the configuration and run
#Override
public void run() {
arenaController.init();
long waitTime = 0;
Graphics g = this.getGraphics();
gameViewer.paintComponent(g);
while(arenaController.isRunning()){
long startTime = System.currentTimeMillis();
gameViewer.paintComponent(g);
update(); // update game
gameViewer.repaint();
long endTime = System.currentTimeMillis() - startTime;
waitTime = (MILLISECOND / FPS) - endTime / MILLISECOND;
try{
Thread.sleep(waitTime);
} catch (Exception e){}
}
}
the run() method in the engine invokes the paintComponent() method of the view (which for me sounds like the controller --> invokes the viewer) but I find it is not the best way to do that and it is not recommended to invoke the paintComponent() directly.
So I want a clear separation of the controller and the view but I having trouble finding the appropriate way to do that.
Okay, so you're running into a series of problems.
First
Swing is already a MVC
Second
The form of "MVC" which is taught and discussed today isn't actually how the original intent of MVC was meant to be used. It was considered more of guide line or recommendation than a stead fast rule or paradigm.
The problem is, you end up with "purests" who insist on an absolute separation of all models, controllers and views, in some cases, to the detriment of the API or solution (this when you end up with "massive view controller" problems).
Why? Consider a button. It's a view, it has a model and it can act as a controller.
You don't need a seperate "controller" to control the button. It can register the keyboard and mouse actions it needs to monitor with the underlying system. It can coordinate changes between the model and itself (the view) and it becomes a self contained unit of work.
Seriously, could you imagine having to create a seperate controller every time you created a button or any other UI component ... 😱😓
When you add the button to another container (view), the button is acting as it's own view/controller. The parent container may then have a larger model and a number of sub views which it is "controlling" or coordinating.
The container may also be controlled via an external controller which is acting to coordinate the changes between the container and the large model, allowing it to be decoupled, but in this case, I would design it in such away so that the implementation logic was decoupled, so that the outter controller didn't care about "how" stuff got done, just that there was a contract in place to allow of the passage of information between these layers.
So, what's the point?
Don't get hung up on trying to produce a "clean" MVC, especially in Swing, you're going to be tearing your hear out.
Also remember, you can have a controller talk to another controller, this allows it to disseminate information where sub controllers don't need or want all the information which might be available to the parent controller. It also allows it to control distinctly different controllers, allowing for a greater decoupling of elements and a nice plug and play approach.
So, how does this help you?
This should provide you with some places to jump off from.
If you don't want the GameEngine to talk directly to the view, you could establish a "rendering" controller. Who's responsibility would be to coordinate changes made by the game engine to the model with the underlying view (ie, schedule a rendering pass).
This would allow the GameEngine to "control" one or more "rendering" controllers and allow for a separation in how the underlying rendering is actual done from the GameEngine
You can do this because a controller can be responsible for other controllers - it's just a coordinator between the model and the next layer (typically a view)
In this example, you could change the rendering system simply by changing the implementation of the "rendering" controller (and if you're using interfaces, this becomes much simpler) without effecting the GameEngie, which, at the end of the day is the benefit of following this type of design - separation of responsibility through de-coupling.
Passing thoughts...
Don't, ever, call paintComponent directly. There is a reason why it's protected
Don't use getGraphics. This isn't how custom painting is done in Swing. getGraphics is at best just a snapshot of the last paint pass made by Swing. Remember, in Swing, you don't control the painting process, this is taken care of for you, instead, you schedule a paint pass (ie repaint) and let the system take care of the reset
If you want/need control over the painting process, then you need to use a java.awt.Canvas and a BufferStrategy instead.
Solution:
A good way to split data and logic in a Game is using an Entity Component System.
LibGDX's Ashley is what I use.
These Systems help keeping even the largest games relatively clean and understandable.
However there is some initial work required, so for small games you might just be better off with mixing logic and data in classes like Player or Enemy, put all instances of your Game Objects into a List and have a Canvas that paints them all ordered by their Z-Index.
Brief explanation of an Entity Component System:
In an Entity Component System you Entities as your Game Objects. These Entites are like empty objects, but with the capability to take posess an indefinite amout of Components (one per Component Type). These Components give the Entity the ability to be something (like extending an Class, but not limited to one).
So if your Entity has a TextureComponent it is displayable, but without giving it a TransformComponent it cannot yet be drawn at a specific location. If you now also give it a BodyComponent it can have a physical body in your world and can be pushed around. The TransformComponent will be updated everytime the Body is moved.
All the Logic is performed by Systems. The process all Entites with specific Component Types.
A self drawn visualization (I apologize for the looks, I'm not an artist):
This question already has answers here:
How do i get a graphics object in Java?
(2 answers)
Closed 9 years ago.
Im making a basic space invaders game. I got all the resources from the LWJGL .zip file (Im not using LWJGL librarys to create my game, just got the pictures, etc. from it.) Anyway, whenever i press "space" on my keyboard, my KeyListener creates a new bullet that my ship fires. However, i dont know how to draw the bullets image, since my KeyListener doesnt pass a graphics object, and you need one to draw a image. The code thats causing the problem is the "drawImage" method in the "Shot" constructor. Heres my code:
public class KeyTyped{
public void keyESC(){
Screen.isRunning = false;
}
public void keyLEFT() {
Screen.shipPosX -= 10;
}
public void keyRIGHT() {
Screen.shipPosX += 10;
}
//This is automatically called from my KeyListener whenever i
//press the spacebar
public void keySPACE(){
if(!spacePressed){
ShotHandler.scheduleNewShot();
}else{
return;
}
}
}
public class ShotHandler {
public static int shotX = Screen.shipPosX;
public static int shotY = Screen.shipPosY + 25;
public static void scheduleNewShot() {
//All this does is set a boolean to 'false', not allowing you to fire any more shots until a second has passed.
new ShotScheduler(1);
new Shot(25);
}
}
public class Shot extends ShotHandler{
public Shot(int par1){
//This is my own method to draw a image. The first parameter is the graphics object that i need to draw.
GUI.drawImage(*????????*, "res/spaceinvaders/shot.gif", ShotHandler.shotX, ShotHandler.shotY);
}
//Dont worry about this, i was just testing something
for(int i = 0; i <= par1; i++){
ShotHandler.shotY++;
}
}
}
Thanks guys! Any help will be appreciated!
(This is a re-post because the last time i posted this, i didnt get a sufficient answer, at least for my skill level)
Start by taking a look at
Painting in AWT and Swing
Performing Custom Painting
You may also find Concurrency in Swing of some usefulness.
The basic gist is you should have some kind of model which describes the state of the game play at any moment in time. This model is updated outside the content of the Event Dispatching Thread, normally in some kind of background thread, where the time it takes to perform the updates doesn't effect the ability for Swing to continue running and remain responsive to the user.
You will need to override the JPanel's paintComponent method. In here, you would paint the model state to the screen.
There are a bunch of other techniques which could be discussed, but lets get this started
Examples...
Java Bouncing Ball
Drawing 2 Balls to move in different direction on Java but one disappeared
Multiple bouncing balls thread issue
I am trying to make ball gradually move
Java ball object doesn't bounce off of drawn rectangles like it's supposed to.
https://stackoverflow.com/questions/15858623/how-to-use-a-swing-timer-to-animate/15859932#15859932
the images are not loading
Swing animation running extremely slow
How to make line animation smoother?
I'm planning to write a simple spaceshooter. I have read that the repaint() method is only a request, and it doesn't execute every time it's called. I believe I'm noticing the effects of this, as my spaceship tends to lag ever so slightly when I'm moving it. Currently I'm simply drawing my ship in a a JPanel's paintComponent() method, and keep calling repaint() on regular intervals (my panel's also Runnable). Seeing as repaint() may potentially screw me over, I'm trying to find a way to work arround it, however I've ran out of ideas. The code I have so far:
private void renderGraphics() {
if (MyImage == null) {
MyImage = new BufferedImage(getPreferredSize().width,
getPreferredSize().height, BufferedImage.TYPE_INT_RGB);
}
MyGraphics = MyImage.getGraphics();
MyGraphics.setColor(Color.BLACK);
MyGraphics.fillRect(0, 0, getPreferredSize().width, getPreferredSize().height);
MyGraphics.drawImage(ship.getImage(), ship.getCurrentX(), ship.getCurrentY(), null);
}
The idea was to create my own graphics, and then make the JPanel draw it, and keep calling this instead of repaint() in my run() method, however I have no idea how to do that. I'd appriciate any input on the matter.
There are multiple ways to approach this.
The best is probably to use BufferStrategy and draw to that, of which I have included a code snippet that should work for you.
You can take this one step further and abandon Swing altogether, just using Frame/BufferStrategy. There is a fully working example (from which the code snippet was taken and adapted) in my question here:
AWT custom rendering - capture smooth resizes and eliminate resize flicker
Anyway, here is an implementation BufferStrategy that you should be able to just drop in:
// you should be extending JFrame
public void addNotify() {
super.addNotify();
createBufferStrategy(2);
}
private synchronized void render() {
BufferStrategy strategy = getBufferStrategy();
if (strategy==null) return;
sizeChanged = false;
// Render single frame
do {
// The following loop ensures that the contents of the drawing buffer
// are consistent in case the underlying surface was recreated
do {
MyGraphics draw = strategy.getDrawGraphics();
draw.setColor(Color.BLACK);
draw.fillRect(0, 0, getPreferredSize().width, getPreferredSize().height);
draw.drawImage(ship.getImage(), ship.getCurrentX(), ship.getCurrentY(), null);
draw.dispose();
// Repeat the rendering if the drawing buffer contents
// were restored
} while (strategy.contentsRestored());
// Display the buffer
strategy.show();
// Repeat the rendering if the drawing buffer was lost
} while (strategy.contentsLost());
}
any drawing will still be performed in the Swing Thread, so no matter what you try work around, it wont help.
Make sure you are not doing any lengthy calculations in the swing thread, this may be stopping repaint from being executed as soon as it needs to be executed
Separate all the logic into 2 parts. Static and Dynamic. (e.g. sea and moving ship. Ship changes shape/location on a static image of sea)
Draw static content in an image once and use the image in your paintComponent(). Call dynamic parts painting after the static image.
Use setClip() to restrict repainting areas.
Calling repaint without any arguments means that the whole panel is repainted.
If you need to repaint parts of the screen (the spaceship has moved to a different location) you should make shure that only those parts of the screen are repainted. The areas that stay the same should not be touched.
Repaint takes coordinates of a rectangle that should be repainted. When moving the ship you should know the old coordinates of the ship and the coordinates the ship should move to.
repaint( oldShipCoordinateX, oldShipCoordinateY, shipWidth, shipLength );
repaint( newShipCoordinateX, newShipCoordinateY, shipWidth, shipLength );
This is usually much faster than calling repaint() without arguments. However you have extra effort to remember the last position of the ship and must be able to calculate the new position of the ship.
See also: http://download.oracle.com/javase/tutorial/uiswing/painting/index.html - especially step 3
Just for code that you post here:
1/ if you want to display Image/ImageIcon, then the best and easiest way is to Use Labels
2/ as you mentioned Runnable{...}.start(); Swing is simple threaded and all output to GUI must be done on EDT; you have to look at Concurrency in Swing, result is that all output from BackGround Task(s) must be wrapped into invokeLater(), and if is there problem with perfomancie then into invokeAndWait()
3/ if you be switch (between JComponents)/add/delete/change Layout then you have to call revalidate() + repaint() as last lines in concrete code block
EDIT:
dirty hack would be paintImmediately()
I have read that the repaint() method is only a request, and it doesn't execute every time it's called
It consolidates multiple repaint() requests into one to be more efficient.
I believe I'm noticing the effects of this, as my spaceship tends to lag ever so slightly when I'm moving it.
Then post your SSCCE that demonstrates this problem. I suspect the problem is your code.
Regarding the solution you accepted, take a look at Charles last posting: Swing/JFrame vs AWT/Frame for rendering outside the EDT comparing Swing vs AWT solutions.
I am looking into using Buffer Strategy and the following technique described on the Javadoc:
// Main loop
while (!done) {
// Prepare for rendering the next frame
// ...
// Render single frame
do {
// The following loop ensures that the contents of the drawing buffer
// are consistent in case the underlying surface was recreated
do {
// Get a new graphics context every time through the loop
// to make sure the strategy is validated
Graphics graphics = strategy.getDrawGraphics();
// Render to graphics
// ...
// Dispose the graphics
graphics.dispose();
// Repeat the rendering if the drawing buffer contents
// were restored
} while (strategy.contentsRestored());
// Display the buffer
strategy.show();
// Repeat the rendering if the drawing buffer was lost
} while (strategy.contentsLost());
}
It would be great to avoid EDT and invokeLater or invokeAndWait when performing an animation.
My questions:
If this is in a Swing application don't we need to worry about putting the call to show on the EDT?
Can anyone else see any problems using this in a Swing app?
This was inspired by this interesting answer to game programming.
In general, no. Painting on a thread other than the event dispatch thread (EDT) results in undesirable artifacts, although the results may be acceptable. This example demonstrates some of the trade-offs. I prefer arranging to draw on the EDT using javax.swing.Timer, but other approaches are possible. Also, your chosen top-level Container may already implement a Buffer Strategy.
As mentioned by trashdog it is not good idea to do so. Create own animation thread which will draw off-screen image. Then push it to some holder. If paintComponent is called fetch current image and draw it on component. Done. Look at rendezvous pattern.