I want to show users a heatmap with 10000 * 400 tiles. Current state in my program is that I have a class which is a JComponent. In this JComponent, I have instances of Image for the tiles. If I had a 300 * 300 matrix, the program uses between 800 and 1000 MB ram. This is too much.
What can I do else or has someone a control which can show such a big heatmap?
Today I have tested JHeatChart, which creates a BufferedImage. There the size for a 10000 * 400 matrix is under 400 MB, but the class needs over 15 Minutes to create it. This is too long.
Has anybody some idee or knows an control which can handle the data?
Since this will be a map, you do not need to create a component for each tile. many of them can be re-used. What you want to do is during initialization of the app, create a map of all possible components, and then on each tile pass a reference to that component.
Here would be an example.
public class HeatMap{
private static List<JComponent> tiles = new ArrayList<JComponent>();
private List<JComponent> heatmap = new ArrayList<JComponent>();
private Random rand = new Random();
static{
tiles.add(new JLabel("Cold"));
tiles.add(new JLabel("Hot"));
tiles.add(new JLabel("Warm"));
}
public HeatMap(){
for(int i=0; i<10000; i++){
for(int j=0; j<400; j++){
heatmap.add(tiles.get(rand.nextInt(3)));
}
}
}
}
In the above case we only have to create 3 tiles, the rest are references to those. This type of approach should help reduce your memory usage.
You could try scaling your map. So instead of priniting every single tile to your screen, if your zoomed far away, use bigger tiles. One could also try to merging the single tiles to one object instead of keeping references for every single tile.
Obviously, 10000 * 400 tiles is too many to view at once. Instead, use the fly-weight pattern to render only the tiles that are visible in the viewport of a JScrollPane. JTable is an example.
Does a tile really have to be represented as an image or would a colored box be sufficient?
If a colored box would be sufficient you might just draw the visible tiles in your paint(...) method. There would be no need to load those images.
If it has to be images, I assume that not every tile is unique, thus instead of loading an image per tile you might try and load all needed images once and have the tiles reference the corresponding image. Thus if you have 100 different images, instead of having 4000000 copies you'd have 100 and 4000000 quite small references.
Any way you present it, the user will never be able to perceive four million data points simultaneously. Therefore you should never have four million graphical widgets. Either the display should be scaled down, or (when zoomed in) you should display only a subset of all tiles at a time. Either way you will have to map the notional 4,000,000 to far fewer graphical widgets and reuse those you already have.
As other answers have specified, you should also look into using the simplest possible widget type that will do the job.
Related
I was working on my own procedural generator and finished making it.
Here is how I did it,
generate the entire world at the beginning,
store info about spawn positions and tiles positions into arrays and data structures
Now obviously, I have to loop through all the arrays, to render/update stuff,
this is creating problems,
I tried dealing with this problem by applying a condition that if any object is visible, only then render update else don't, but it does not make much of a difference.
my TILE_SIZE is 1 world units
so when my world size was
a)500 by 500,fps was 60
b)2000 by 2000, fps was 50
c)5000 by 5000 fps decreased further and so on
If only I could know how Terraria does it, or any way it done, would really help thanks.
I have to loop through all the arrays, to render/update stuff
You don't have to do that.
For rendering stuff you really only need to consider the blocks that are close to the player. Here is some code(x, y are the player coordinates, width, height are the dimensions of your world, render(x, y) is a placeholder for your rendering functionality, and for simplicity I assumed the player can see 100 blocks in each direction):
for (int i = Math.max(x-100, 0); i ≤ Math.min(x+100, width); i++) {
for (int j = Math.max(y-100, 0); j ≤ Math.min(y+100, height); j++) {
render(i, j);
}
}
The process of updating can also be simplified:
For npcs you only need to check a few blocks around it.
For block updates(liquid flowing, sand falling, corruption spread, …) there are actually two possibilities:
Iterate through a different part of the array each update:
Block updates don't have to be that frequent. For fluid flow it might be good enough to update only every 50th block in each update.
You can store the blocks that might need to be updated in an ArrayList(I suggest storing only the coordinates, not the blocks itself).
Then when the update function is called you go through that list and check wether the block really needs an update. If that's the case, you add the blocks neighbors to the list, which will then get updated in the next iteration.
Make sure to remove the blocks you just updated from the list and avoid adding duplicates!
While the second approach is normally faster(ususally the list is only determined by player actions and even if the player decides to drain the entire ocean the whole list would probably take less than ¹⁄₂₀ of the entire world).
But the first approach is certainly easier to implement and will be faster if for whatever reason you have lots of block updates.
Anything you do not see on screen is stored as values. This means that all tiles that are not visible in your viewbox do not get rendered. They are stored as objects (e.g. {x, y, type}); Make sure you count how many tiles are on screen VS how many you are rendering each loop in your code. Never render something that is not visible to the user as this will eat up your memory.
Any tile groups that cannot be rendered soon (i.e. by moving left/right) are not looped through. So say you are using tile sectors to group your tiles, and you're in sector D. Moving left will put you in C, and moving right will put you in E. So these sectors should be loaded in to arrays (but not rendered) There is no need for you to loop through any other sectors (e.g. A or B). So these can be stored externally (e.g. as text files). So they are available, but not taking up space in your games memory.
Good luck. :)
I would like to draw a map consisting of a limited set of tiles in different zoom levels, just like google maps for instance. The reason why I consider using tiles is because the map is very large and in the highest magnification the whole map is very large (about 15000 to 20000 pixels).
Now I thought about creating a quadtree to organize the tiles. Each time a frame is drawn, the quadtree logic would evaluate which tiles to render, so that the rendering time is minimal.
However, how would I organize the map tile objects itself? When would I best load the tiles?
Creating a sprite for each tile seems to be overkill, as the dimensions of the tiles and position remains static. A SpriteCache doesn't seem to help in this case, as I am only drawing a subset of the map each frame.
Also, there are quite a few pros and cons in several cases:
When I would load all tiles when launching the application, then it would take a long time to load all of them and cache them in the GPU (Not even considered that all the tiles would fit in the memory).
When I would load the necessary tiles on runtime, I suppose to experience kind of a lag when tiles are being rendered.
Is there a "best practice" for a case like this?
I know there is not a right or a wrong answer to this, but I would like to hear other suggestions to solving this problem.
Thanks!!!!
'how would I organize the map tile objects itself?':
a) You could group tiles into Areas;
b) Then your viewer/camera draws only Areas in view (like frustrum culling, but not for each tile, because it's too expensive to ask each tile if it is on view);
c) Each drawn Area draws its Tiles on correct position (relatively to Area.position).
Here question about 'When would I best load the tiles?' could be solved with answer:
- the best approach is to load sprites before draw call AND only for Area.Tiles on view;
- load sprites from pool.
I tested such approach on Map 1000 x 1000 tiles,
where my Viewer (active tiles) was 3 x 3 Area (90 x 90 tiles). And with such approach it doesn't matter for performance how big is map (while you computer have memory:) ), only viewer size matters...
If viewer could have bigger dimension - then you could pre-render whole Area.Tiles to texture and draw directly one Area.texture for each visible Area on each frame.
Textures should be loaded initially when app starts.
Textures could be linked to tiles when Viewer created and when it moves one area up/down/left/right.
for (Area area : Viewer.areas) {
for (Tile tile : area.tiles) {
tile.texture = <loaded_texture> OR tile.image = new Libgdx.Image(<loaded_texture>) OR any other appropriate for your tile object way
}
}
I'm making a map editor for my 2d tile based RPG game, and I'm running into a roadblock. The problem is actually how to draw the grid lines efficiently for very large maps like, say 300x300 tiles.
Originally I called redrew the entire grid panel every time you moved the mouse or clicked(since it allows you to drag and also highlights the tile you are hovering over). This was horrible for large maps and was very slow.
So my solution was to create a bufferedImage for the grid of a specified size right when a new empty map was created, and redraw that image every time paint was called. This worked very well, except that for anything above 200x200ish java would run out of heap space in creating that bufferedImage. I can't really have a preloaded image because the user should be able to specify a new map to be any size. Also, I like the grid to be drawn over the tiles, otherwise the tiles cover up the lines and it looks messier. That means the grid has to be drawn again every time tiles are added right?
My question is, is there another way to draw a very large grid that will repaint each time you move the mouse?
The program Tiled can easily create maps of 1000x1000 or more, how is it done?? Thanks
EDIT:
I mean the actual grid LINES, not the grid content such as tiles. Right now when the grid is drawn it iterates through the 2D array and draws a line at every length of a tile.
EDIT:
I got it to work by painting only the visible portion of the screen. Turns out it's not that hard to find the coordinates of the view pane in a JScrollPane. :)
Ok so this is similair I guess to what im working on at the moment. I am currently writing a bit of simulation software with a 3D frontend in Ogre3D. I have a tile based map that can be easily 3000 x 3000 sometimes they may be even larger.
Now im not sure how you are going about drawing your grid, if you are just doing 2D/3D, but the main problem for me was how to apply different textures to each tile. If this isnt relevant then hopefully this may help anyway for future lol. The problem for me is that creating a seperate object for eacch tile is a no no, due to speed. If there are even 1024 x 1024 your looking at over 1 million tiles, each with attributes. This worked great on my first tests where I only had 10 x 10 maps but on larger maps it just ground to a halt.
What I am currently doing is rewriting it, the new approach is as follows. (You may be able to adapt this for what you are doing if it helps). The grid is now only 1 object, its a mesh in my case as im working in 3d. Each vertex holds a couple of values, one is the column and one is the row. I have a seperate texture which is the same size as my map, so for a map with 100 columns and 100 rows i create a texture 100 x 100 (this is created dynamically).
I then pass this texture and the vertices to my cg shader (this is where tyou may have to think of an equivalent way of doing things, but I think it should be doable). By looking up the colour at the pixel in this texture relating to the grid slot I want to texture I can find a colour value, these colour values relate to my texture atlas which holds all my possible textures for my map. All you then have to do is use that particular part of the texture atlas for that particular part of the mesh / grid.
I hope that makes some sort of sense, if not then id be happy to explain a bit more. I think even though im using CG shaders, you should be able to come up with an equivalent for non CG based stuff in java. The advantage of this is that you end up with one lookup texture, one object in memory and a buffer holding your vertices which is very quick for lookup.
Hope this helps.
I'm a bit stumped on an Android game I am developing, so maybe someone can help me. Don't bother suggesting AndEngine; I've made it this far on my own and I'd prefer to continue on this path if possible.
The game I'm working on is a simple platformer. Originally the levels were loaded as massive bitmaps, but a few out of memory errors made it obvious that I had to come up with a more efficient system. Now I'm parsing out .tmx files into a 2d array and using a method that creates a bitmap slightly bigger than the device's screen and fills it with the appropriate tiles.
The issue is that accessing this 2d array 1500 times (using 16x16 tiles) is very expensive, and I'm having trouble figuring out a way to not do that so much. Currently I'm having to redraw the image every time I pass over 16 pixels. Current ideas include using larger tiles (smaller array, still limits level size though), attempting to shift the contents of my bitmap and only load in the new row about to be displayed, or using a larger bitmap so that new images don't need to be loaded as often. Any suggestions?
Unless already done or impossible; consider separating the background from the foreground.
If you use a simple background (think Super Mario plain blue backgrounds) then you can most likely reuse those tiles/that bitmap/color/gradient much longer than the foreground elements.
This will also make your array/s contain lots of empty spaces where no foreground exists, meaning you could collapse the array and only store the parts containing tiles. Collapsing the arrays will come with the drawback that you will have to store a height attribute for the tiles to know where they are supposed to go.
(A background separate from the foreground also give the possibility of parallax-effects (background moving in different speed than foreground))
An example of how this changes your lookup:
Consider the game is being run on a 480x800 screen in landscape-mode.
This means 50*30 (1500) tiles to look up.
Every tile is saved in a 2d-array.
The current visible area looks like this:
A ground area, 4 tiles high.
A platform, 5 tiles high and 20 tiles wide, somewhere above the ground.
Another platform 3 tiles high and 5 tiles wide, also above ground.
A wall extending from the ground to the top of the screen, 5 tiles wide.
Removing the background tiles and collapsing the arrays we only have to look up the tiles:
Ground tiles: 50*4
Platform1 tiles: 5*20
Platform2 tiles: 3*5
Wall tiles: 26*5
Total lookups: 50*4 + 5*20 + 3*5 + 26*5 = 445 which is less than a third of the original 1500.
Now, of course there is some extra work to be done to figure out the position of each tile and drawing the background, but assuming a badly optimized case where this takes as much time as the lookup it is still less time than 1000 lookups.
Of course, this is just one of possibly many approaches to the problem.
Also remember small things like that 2d arrays are much slower than 'flat' arrays. If you cannot put the data in a 1d-array, try to minimize the nested calls, e.g. get one row or column at a time and perform the lookups from that array before switching to the next:
//Really bad:
for(int x=0; x<array.length; x++){
for(int y=0; y<array[x].length; y++){
Object o = array[x][y];
}
}
//Better (can be made even better by using the for(Object o : array)-syntax):
for(int x=0; x<array.length; x++){
Object[] yArray = array[x];
for(int y=0; y<yArray.length; y++){
Object o = yArray[y];
}
}
I had to develop this project where we needed to download tiles of image from the web and display them as one whole image. The thing was a casual scroll would end up crashing the entire app because the VM would throw a OutOfMemory Exception. So I developed a view using a TwoDScrollView and MxN ImageViews, which downloaded only those views which were in the users view. As the user scrolls, we maintain a cache and start de allocating images that are "far" away from the viewing region. It was slower but it stopped crashing.
I won't be able to share my part of the code. But here's a Github repo that uses the same approach. Good Luck :)
I can't vouch for them but here's a project specifically fits your needs
I have a large number of data points which are two dimensional coordinates with non-integer values (floats). I am looking for a Java library that can help me plot these points, allowing for custom point size, color, and labels. Further, I would like the user to be able to interact with the points with panning and zooming, and I want to be able to capture KeyEvents from the user.
Processing looks great for what I want, but I don't want to do everything from scratch. Is there a better solution?
Thanks in advance.
Edit: There are about 2k points.
Depends. I recently did app that displays large 2d datasets with JFreechart, but I ended making datasets smaller to have performance.
I displayed matrices of points, that changed in time (when new data arrives) with 1 second refresh time (so every one second chart is repainted).
For matries 256 x 256 it is OK on normal user computer. If the matrix is ~350 pts it gets rough (user sees lags in the GUI), but it is usable, if the matrix is 1024 x 1024 app is unusable.
I did chart drawing ind EDT, but still even if I took it into different thread --- rendering would still eat processor power.
So depending on data set size --- you might want to use JFreeChart.
I haven't found a good library that works well for large data sets.
When you say large what do you mean? 1k, 100k, or 1 million points?
I just rolled my own since my data sets were at least 100k points. I've tried all the charting libraries I could find but none of them were as fast the one I rolled on my own.
This example easily handles thousands of nodes.
GraphPanel() {
...
Random rnd = new Random();
for (int i = 0; i < 2000; i++) {
Point p = new Point(rnd.nextInt(WIDE), rnd.nextInt(HIGH));
nodes.add(new Node(p, 4, Color.blue, kind));
}
}