JavaFX (8) - Improving scatter chart performance - java

Question:
In what ways can I improve render-time in scenario described below?
Description:
The task I'm facing is plotting quite a few points on a scatter chart in JavaFX. I'm getting the data from an Oracle DB.
I think my current bottleneck is scatterChart.getData().add(series);
The series contains all the datapoints needed in the chart.
Executing this, and rendering the chart on screen may take anywhere from several seconds to several minutes. During this time the GUI freezes up.
We're running some Lenovo Thinkpads with 4 Cores 4 Threads, Intel HD4000 Graphics.
Some examples of finished charts are below. The latter being the current worst case, but as more data is collected, more data will be displayed.
(due to images being impractically large, only links)
Chart with ~5.5K points
Chart with ~75K points
More info
I followed this tutorial because I'm new to JavaFX, so I'm not sure exactly how rendering happens.
Code sample
This isn't the actual code I'm using, it's just to illustrate how I'm doing things.
//Reference to the chart in the fxml file.
#FXML
private ScatterChart<Date, Number> scatterChart
= new ScatterChart<>(new DateAxis(), new NumberAxis());
private void handleSelectionUpdate(PerfResultSet newValue){
//newValue is an object containing all the data from the database
XYChart.Series<Date, Number> series = new XYChart.Series<>();
/**
* adding all the data from newValue to series as
* XYChart.Data<Date, Number>
*
* this is handled by multiple threads and does not take long
*/
series.setName("myName");
//this statement is where the gui freezes
//can I optimize this in any way?
scatterChart.getData().add(series);
}

I faced similar problem - I was drawing route consisting of many points to google map with Javascript. I eliminated points taking to account current scale. I saw your png files, it seems you really need ~30-50 points for expressing the following message - hey, this vertical/horizontal segment contains so many points you may assume it solid line!
Next problem is max scale - I think, each solid line in 75K chart is actually not solid. I mean, in small scale, I see a lot points like solid line ____ but in max scale I see them as . . . ... . .. But usually, max scale required only for small piece of chart and you may eliminate not by scale but by borders. Again reference to google map - when you zoom the map, you are not seeing whole globe in cool resolution but some small piece of map. Other borders eliminated automatically.
Post here your code, some information about scale, zoom (do you have such feature?). If you haven't and your chart is always as in png files in link, you should iterate over each vertical or horizontal line, look for sequental points and replace, for example, 100 sequantal points to middle point of that 100 points. Instead of 100 experimentally find such X, that replacing sequental X points to one middle-value-point does not affect the resulting chart.
I repeat the main idea - There exist some X, such that drawing X sequental points equal to drawing 1 point in some fixed scale - many points does not give precision but simply "overwrites" each other due to scale and human eye limitations.
Some pseudocode for finding X if your scale is constant;
int MAGIC = 50; //change it on each run of your program! You have to find the best value
int counter=1; // points[0] taken as included in accumulating solid segment
int startIndex=0,endIndex;
ArrayList<Integer> compressedDatesArray = new ArrayList<Integer>();
for(int i=1;i<datesArray.length;i++){
if(deltaBetweenDates(datesArray[i]-datesArray[i-1])==1){
counter++;
endIndex=i;
if(counter==MAGIC){
counter=0;
compressedDatesArray.add(datesArray[{endIndex-startIndex)/2]);
startIndex=endIndex+1;
}
}
}
Render compressedArray, see png. If result is bad - chnage MAGIC and repeat
UPDATE
You may binary search MAGIC - try 5000, if still solid after comperssing 5000 points to one, try bigger value if not, try 2500 and so on... Humans also may binary search something=)))
Also, do not forget to remember Y value of point representing big segment. I mean, also declare compressedY and add correspinding Y of datesArray[(endIndex-startIndex/2)]

Related

Java: how to detect colorclusters in an image

I´m looking forward to writing a method in Java wich detects and also stores "colorclusters" in seperate files.
For instance: a color cluster could be a green rectangle or any other section of a picture which contains very similar pixelcolors in range:
Unfortunately, I´ve already tried thought a thousand ways of how to solve this issue but nothing worked so far. Does anyone know if there is already such a method or how to solve this problem?
Look into the concept of a "Graphics Kernel"
Basically it is a relocatable array representing a pixel and its neighbors, coupled with an algorithm to determine some interesting quality about the pixel.
Kernels are evaluated against each pixel to give a value. An pseudocode example of a kernel might be.
value = sum of color_distance_between pixel and all neighbors
If the value is zero, then the pixel should be exactly the same as it's neighbors. If the value is not zero, then it had differing neighbors. Take care that all color distances are positive, or you might have color differences that cancel each other out.
Then your program runs through each pixel, determining if it is similar to it's neighbors. Large regions of pixels with no color distance would be grouped and any pixel within that group has roughly the same color.

Tiled Terrain Generation with Perlin Noise - Java

I'm currently generating 2d Perlin noise to a 2d array and (after interpolation) rendering the results held as a height map (essentially array[x][z] = y).
This is fine for one array, but not for tile loading based on the camera position, and I'm having difficulty where the tiles should meet at the seam.
[Noise Tiles]
My best attempt has been to generate a large 2d array of the base noise (-1 to 1), then have each tile as an object that stores an offset value for the base grid (which part of the noise to read).
This has allowed me to sample (for interpolation) areas of the base grid that are much larger than the tile array, but it still not lining up!
My objective is to have a flycam that loads tiles as it moves, and can travel very far without repeating.
Is this approach of 2d arrays solid enough to handle the objective?
If so, how can it be implemented to tile or wrap correctly?
I have had similar problems before and after alot of tweaking and testing I've come to the conclusion that just plain 2D perlin noise as is will never look like natural terrain, it's essentially white noise(ie no huge mountains or valleys, just hills close together)
What I recently found as the way to go is by making a fractal out of multiple Perlin Noises but with different "resolutions" -if you will, added together to get custom terrain resolution using different octaves(ie higher octave = higher noise in your terrain). I usually go about using gain = 0.5(this really do not have to be changed much, it looks pretty good as is), and octaves = ~16
Note; this is made in processing, Java might have some different syntax, but it should be quite the same
float fractal(int x,int y,float gridSubs,float gain,int octaves){
float total = 0;
float freq = 1.0/gridSubs;
float amp = gain;
for(int f=0;f<octaves;f++){
total += noise(x*freq,y*freq)*amp;
freq *= 2.0;
amp *= gain;
}
return total;
}
Website where I found it: here
If you replace your noise() function with this you might get better looking results in the end.
As for your problem with your seams you probably have the coordinate offset in the noise function call for each chunk set wrongly, it should look somewhat like this:
noise( localX * tileSize + (chunkX * chunkSize) , localZ * tileSize + (chunkZ * chunkSize) );
You might have to add some resolution koefficent to make the noise somewhat smooth.
Now You said you are storing the heightvalues in a 2D heightMap, now that is fine and makes the heightvalues easy to access if you need to update them often or need to access them easily.
The problem with this is the size of the array easily get very large, and by that I mean, Very large. With my past experiences I could get a small map(can't remember size, but smaller than yours) to eat up 4Gb of my RAM just by loading it. I did use a float array though so using an integer array could have reduced the memory usage slightly.
How I do it now is I just recalculate each point in the terrain whenever I need to update the geometry, how I currently have set it up it does introduce a slight lagspike each time the terrain is changed when moving around. The benefit is that I can have a 4times larger map with greater detail and still use about 1-2Gb of RAM.
Since what I understood, you simply want to move around the terrain and look at it. This would benefit from not storing the heightmap in an array, since you do not really need the values after you have generated the terrain(this may differ if you are manually smoothing out the terrain).
One last thing to note; I am not an professional programmer, this is just what I have learned from my past experiences using randomly generated noise.

Finding All 3D Objects within a Certain Distance from Point

I have a set of objects (let's call it points) that contain the x- y- and z- components of their positions within some definite space. I would like to model the interactions between the objects in points, however, I cannot do so unless I can quickly find the objects in the set that are less than a certain distance away from one of the objects in this set.
This undoubtedly sounds a bit unclear, so let me put it another way: if the first point in points has coordinates <x, y, z>, I would like to figure out which of the objects in points has a distance that is less than [some arbitrary value] from the first point.
I was considering an implementation of an R-Tree to do this in Java, yet I feel as though this is a common-enough problem that a simpler solution exists. If there is not one, I would appreciate a simple explanation of the method by which one queries an R-Tree in order to find objects that are within some distance x from an object in the tree, where x is already known.
Edit: note that the position values of these objects will be changing
The R*-tree is a pretty good data structure for this, in particular when points are changing. It is designed for changes, actually.
The k-d-tree is simpler, but it doesn't support changes very well. It is designed for a one-time bulk construction.
However, as your data is only three dimensional: if your data is small enough to fit into memory, and the maximum and minimum values of x,y,z are known, an octree or a simple grid may be the tradeoff of simplicity and performance you need.
In particular if you fix your query radius beforehand, a grid-file is hard to beat. R*-trees get attractive when you need to support multiple radiuses, window queries, nearest-neighbor queries and all this.
EDIT : Square = Cube (however imagining it in 2D space would be maybe better, then you can convert it into 3D easily)
I was thinking and I think I solved it. However this is just "my" solution, I have no reference for it.
You create class "Square", which has position, width and list of points in that object.
All squares will be stored in array or hashmap based on their position, so they can be accessed quickly, if you know position you seeks.
All squares will be distributed regularly, so - from the point of view of "point instance" - you dont have to know all the existing squares to figure out in constant time in which one you belong. (example : I know there are squares with width of 40 and they are distributed by distance of 20. I am in position 10001, so I know I belong into squares in position 9980 and 10000)
Squares will be crossed by each other, therefore one point can be in more squares.
When you do something, for each point, you only check points, which are stored in squares that point belongs to. Of course - squares have to be large enough and crossed enough to achieve your goal.
When points moving, they are responsible for registering and unregistering into the squares.
1D EXAMPLE :
Classes : Line segment and Point
Attrributes:
Line segment : int position, List<Points> points
Point : int position, List<LineSegment> lineSegments
I want to interact only with points in distance of 20.
So I create instances of Line segments with width 40 and I put them one by one in distance of 20.
So they will be at positions 0, 20, 40, 60 ....
The frist one will cover area 0-40, second 20-60 etc.
I put them into the array and with known position, I can access them quickly : arrayOfLineSegments[position/20]
When I create point, I add him to the line segments it belongs to.
When I update, each point only interacts with points in lineSegments.
When I move point, it register and unregister lineSegments it belongs to.
You can use a for loop to check through the the array of object.
use the following formula: d = sqrt[(x1-x2)^2 + (y1-y2)^2 + (z1-z2)^2]
x1,y1,z1 being the first point in Points and the x2,y2,z2 being the points of the object you are checking. This will check your known point vs all other points. If the distance (d) is less than your desired distance x then do whatever you want you program to do.

JFreeChart Histogram with constant number of bins while zooming

I'd like to make a JFreeChart Histogram that maintains a constant number of bins while zooming in and out. For example, maybe zoomed way out you can see 12 years, and there would be 1 bin for each year. When you zoom in a little more you might see 12 months and there would be a bin for each month. Zooming in even further maybe there are 1 bin for each day, each hour, etc. They don't have to be such clean bin sizes, but you get the idea.
I'm using HistogramDataset.addSeries to add my data to the histogram. The bins parameter, is the number of bins for the entire set and doesn't take zooming into consideration.
I've observed that I can use a ChartPanel and override its zoom method. In here I could potentially alter the number of bins. However, I'm having a hard time figuring out where I can alter the number of bins and what it should be altered to.
EDIT:
There are a few parts of this problem that I am interested in that I believe are critical to a solution:
How best to tell that the user has zoomed. My current approach is to override zoom.
How to determine the bounds that the user has zoomed to. I'm still not sure how to do this.
How to change the number of bins in the data model so that zooming maintains the same number of shown bins.
For some context, my end goal is to create a Google finance style control:
Google Finance style control for Java?
Overloading the zoom feature is appealing, but perhaps confusing. As an alternative, consider adding a control that updates the data model, as shown in this example. Instances of an enum are particularly convenient for populating a JComboBox, as shown here.
To answer my individual questions above:
I continued to override zoom on the ChartPanel class.
In order to determine the range that the user zoomed to I used the windowToRatio and ratioToModel functions below, as the zoom function is given window coordinates.
To change the number of bins, I actually kept the number of bins the same and simply changed the dataset passed into the chart. I used a SortedMap for my data model that I could easily get subMap's from.
Functions:
private double windowToRatio(double window) {
Rectangle2D scaledDataArea = getScreenDataArea();
return (window - scaledDataArea.getMinX()) / scaledDataArea.getWidth();
}
private double ratioToModel(double ratio) {
Range domainRange = getChart().getXYPlot().getDomainAxis().getRange();
return domainRange.getLowerBound() + ratio * domainRange.getLength();
}

Plotting Points in Java with Interaction

I have a large number of data points which are two dimensional coordinates with non-integer values (floats). I am looking for a Java library that can help me plot these points, allowing for custom point size, color, and labels. Further, I would like the user to be able to interact with the points with panning and zooming, and I want to be able to capture KeyEvents from the user.
Processing looks great for what I want, but I don't want to do everything from scratch. Is there a better solution?
Thanks in advance.
Edit: There are about 2k points.
Depends. I recently did app that displays large 2d datasets with JFreechart, but I ended making datasets smaller to have performance.
I displayed matrices of points, that changed in time (when new data arrives) with 1 second refresh time (so every one second chart is repainted).
For matries 256 x 256 it is OK on normal user computer. If the matrix is ~350 pts it gets rough (user sees lags in the GUI), but it is usable, if the matrix is 1024 x 1024 app is unusable.
I did chart drawing ind EDT, but still even if I took it into different thread --- rendering would still eat processor power.
So depending on data set size --- you might want to use JFreeChart.
I haven't found a good library that works well for large data sets.
When you say large what do you mean? 1k, 100k, or 1 million points?
I just rolled my own since my data sets were at least 100k points. I've tried all the charting libraries I could find but none of them were as fast the one I rolled on my own.
This example easily handles thousands of nodes.
GraphPanel() {
...
Random rnd = new Random();
for (int i = 0; i < 2000; i++) {
Point p = new Point(rnd.nextInt(WIDE), rnd.nextInt(HIGH));
nodes.add(new Node(p, 4, Color.blue, kind));
}
}

Categories