Project Background:
I am writing a map tile overlay class for java that can use gdal2tile.py tiles. Basically I will end up with thousands of jpg files that are in a file structure like
"Zoom Level/X coordinate/Y coordinate"
The coordinates are ints but will not necessarily start at 0 or 1.
I will have to search for tiles that are within a certain range to find out which ones I need to render.
My Problem:
I tried iterating using the file structure itself but it is wicked slow (not surprising).
I tried iterating using an ArrayList of strings of the file structure and .contains() but it seems to be even slower (not too surprising).
Optimally I would like to use a data structure that would let me choose a range on multiple dimensions so that I can call something like.
Tiles.getWhere(Zoom Level,min X,max X,min Y,maxY);
I assume that some sort of Collection or TreeMap would be the right choice but I'm not experienced enough with Java to know for sure and I'd prefer not to have to benchmark a lot of different approaches.
I could use SQLite to do it but that seems like overkill.
My Question:
What is the most efficient way to check for the existence of datasets given multiple dimensional constraints?
May be you are looking for a map with multiple keys.
Commons-collections provides a map with multiple lookup keys:
http://commons.apache.org/collections/apidocs/org/apache/commons/collections/map/MultiKeyMap.html
a map guarantees a O(1) insertion and O(1) selection timings.
Thinking of your problem I could find out three directions to which you could aim your search next (this is not a hand-by-hand guide but rather a out-of-the-box brain opener for a stucked situation you have faced):
1) Usage of Java built in structures. Yes, indeed, a list is the worst case of a searching method. A Map, as the name suggests, is far more convenient for maps. It is not only the name, but the indexing to a Map is signifigantly less time consuming compared to a List. You can imagine your map as a cube, where you have to handle about half of the dots inside it, if you use List and probably only a narrow layer of it when you search by indexing a Map. There is a magnitude of difference. So, my answer here: Map is a key word towards the correct direction (assuming you want to do it in this way after reading on my answer).
2) Usage of a Map Server solution. This is probably too far from your approach, but entire frameworks are made for solving your type of question. An example is GeoServer. It has a ready made solution for the entire problem. It is a stable solution for the great big problem possibly in your hand: showing a map to a user from a source.
3) Sticking to the GDAL framework you were using, you could select slightly different py-file, like gdal_proximity.py and - wow! - you have a searching possibility in your hand! This particular one searches by a center point and a distance, but will do the stuff you need =)
There is a starting point, how I would make it. Could this serve for something?
Sounds to me like you are looking for something like an Interval Tree.
http://en.wikipedia.org/wiki/Interval_tree
I have implemented one of these in the past but only in one dimension. The Wikipedia reference mentions extensions to more dimensions.
Paul
Related
I'm tring to use a efficient way to search if one Coordinate (that is not in arrys) is in arrays of coordinates. I wouldn't read the entire arrays I would a better solution. Anyone can help me?
As Chief Two Pencils commented, this only works if you have some kind of ordering principle on your array. There are lots of good data types you could use to help you with this: Range trees, Quadtrees, and k-d Trees are a few that spring to mind.
If you can't change the structure of your data, you still have options. I can imagine an algorithm where you filter by x-coordinate, and then filter those by y-coordinate, and the performance wouldn't even be terrible.
need help with an optimized solution for the following problem http://acm.ro/prob/probleme/B.pdf.
Depending on the cost i either traverse the graph using only new edges, or using only
old edges, both of them work, but i need to pass test in a limited number of milliseconds,
and the algorithm for the old edges is dragging me down.
I need a way to optimize this, any suggestions are welcome
EDIT: for safety reasons i am taking the algorithm down, i'm sorry, i'm new so I don't
know what i need to do to delete the post now that it has answers
My initial algorithmic suggestion relied on an incorrect reading of the problem. Further, a textbook breadth-first search or Dijkstra on a graph of this size is unlikely to finish in a reasonable amount of time. There's likely an early-termination trick that you can employ for large cases; the thread Niklas B. linked to suggests several (as well as some other approaches). I couldn't find an early-termination trick that I could prove worked.
These micro-optimisation notes are still relevant, however:
I'd suggest not using Java's built-in Queue container for this (or any other built-in Java containers for anything else in a programming contest). It turns your 4-byte int into a gargantual Integer structure. This is very probably where your blowup is coming from. You can use a 500000-long int[] to store the data in your queue and two ints for the front and back of the queue instead. In general, you want to avoid instantiating Objects in Java contest programming because of their overhead.
Likewise, I'd suggest representing the edges of the graph as either a single big int[] or a 500000-long int[][] to cut down on that piece of overhead.
I only saw one queue in you code. That means you are searching from one direction only.
You may want to take a look at
Bidirectional Search
Explanation:
EDIT3: MASSIVE CLEAN UP as this was not clearly explained.
I'm trying to build up a 2D level out of tiles and entities. Where the entities are for example trees that can be cut. I need to store the data (how many chops are left for example) for each entity. I want them to have a more dynamic position (doubles) and a more dynamic sprite-width and height. My tiles are 32x32 pixels whilst my trees are not going to be one tile but a sprite with greater height than width.
I want objects that are closer to the top of the level to be drawn before the other objects. In this case a character behind the tree will cannot be rendered in or in front of the tree. This case also applies to other objects of the same kind (like trees).
I think it might be too inefficient to loop through the entities and calculate each entity's position since there may be a LOT of entites in the level.
As I've done some research I found that certain libraries allow the storage of both the object and it's position in a MAP (BiMap in google's Guava).
Questions:
Is this an inefficient manner.. but are there some changes that can
be applied to make the rendering more efficient (if so, what could be
optimized)?
Or is this an inefficient manner to render the entities and is
there a better way (if so, what other methods are there in Java)?
Or is there something else that I haven't listed?
EDIT2: I looked through the link I've posted in the edit below.
It seems that Google's Guava (I think that's all correct) has BiMaps. Is there an equivalent to this in regular Java? Otherwise Google's Library will probably be able to fix this for me. But I'd rather not install such a huge library for this one interface.
At last:
It's very much possible that the answer has been right in front of my nose here on StackOverflow or somewhere else on the internet. I've tried my best searching but found nothing.
If you've got any suggestions for search queries or any relevant links that might be of use to me I would appriciate it if you'd post them in the comments.
Thanks for taking the time to read through this/helping me ;)
EDIT:
I have looked at; Efficient mapping of game entity positions in Java .
I think it's narrowly related to this question. But I think it's just not what I'm looking for. I am going to look through the second answer very closely since that might be able to solve this for me.. but I'm not sure.
SOLUTION
The solution is to have an array, arraylist or another manner to keep track of your entities. Every tick/update you'll take all the object's Y coordinates and store them in another array/arraylist/map/other with the same size as where the entities are stored in. On every equivalent position to the entity you'll store it's Y. Then you'll order it with another loop or using http://www.leepoint.net/notes-java/data/arrays/70sorting.html .
Then when rendering:
for(int i = 0; i < entityArray.length; i++)
entityArray[i].render();
Off course you'll render it more efficiently by rendering only whats on or near your screen.
But that's basically how one does this in 2D top-view/front-view.
In my own 2d game attempts I come up with the following solution:
use an enum to specify different types of objects in game and give them priorities (sample order: grass, rivers, trees, critters, characters, clouds, birds, GUI)
make all visual objects implement interface which allows for getting this DrawPriority enum
use a sorted implementation of list with comparator based on the enum
use the list to draw all elements
That way the order computing is not very expensive, because it is done only on Visual Object insertion (which is in my case done while loading a level).
.. And since you will already using a comparator, do a x/y comparison when the enum priority values are the same. This should solve your y-order draw problem.
After several hours I've searched information from internet, I still feel not sure anything. My problem is: i want to implement a dictionary on android devices (java base), my requirements are speed and then memory-efficiency, but I couldn't make a decision on which data structure to use for searching.
I have a list of data structures, help me understands them and choose one:
Ternary tree
TRIE
Aho–Corasick tree
[...your suggest DS...]
And will be very kind if somebody can guide me about getting results (many fields: pronounce, mean, example sentence...) of word after we found it? We will save these info on another data file?
You need to list the major concerns of your design before searching data structures. What functions does this dictionary offer? What are the major features of it? Fast search? Space compactness? Insertion/deletion friendly? Cross-referencing friendly? Only when you have these in your mind you may measure how good a candidate structure is.
It can be implemented in several ways, one of them is Trie. The route is represented by the digits and the nodes point to collection of words. Usage of trie is explained here
Agree with Hunter Mcmillen's comment. In case you need the words to be sorted alphabetically like a regular dictionary you can use Java TreeMap which is a SortedMap.
I've building a tree pagination in JSF1.2 and Richfaces 3.3.2, because I have a lot of tree nodes (something like 80k), and it's slow..
So, as first attempt, I create a HashMap with the page and the list of nodes of the page.
But, the performance isn't good enough...
So I was wondering if is something faster than a HashMap, maybe a List of Lists or something.
Someone have some experience with this? What can I do?
Thanks in advance.
EDIT.
The big problem is that I have to validate permissions of users in the childnodes of the tree. I knew that this is the big problem: this validation is slow, because I have to go inside the nodes, I don't have a good way to know if the user have permission in a 10th level node without iterate all of them. Plus to this, the same three has used in more places...
The basic reason for why I was doing this pagination, is that the client side will be much slow, because of the structure generated by richfaces, a lot of tr's and td's, the browser just going crazy with this.
So, unfortunatelly, I have to load all the nodes, and paginate just client side, and I need to know what of them is faster to iterate...
Sorry my bad english.
A hash map is the fastest data structure if you want to get all nodes for a page. The list of nodes can be fetched in constant time (O(1)) while with lists the time is O(n) (n=number of pages, faster on sorted lists but never getting near O(1))
What operations on your datastructure are too slow. That's what you have to analyse before you start optimization.
It's probably more due to the fact that JSF is a performance pig than a data structure choice. The one attempt I've seen to create a JSF app could be timed with a sundial.
You're making a mistake by guessing about solutions without more knowledge about the root cause. I'd recommend that you profile your app to see where the time is being spent.
The data structure to use always depends on how you need to store the data and how you need to access it. HashMap<K, V> is supposed to have constant time complexity in accessing the value, provided the key. When you call get(key), the hashCode() for key is computed and it's used to retrieve the related value. Unless you've got different keys that have the same hashcode (in which case you may have been doing something wrong, as while is not mandatory different objects should have different hash codes, at least in the majority of cases), this is usually fast.
Searching an element in a plain list requires scanning of the list, which will (almost) always be slower than computing an hashcode.
If you need to associate values with keys, a Map is the way. And HashMap should be fast enough.
I don't know too much about JSF, but I think - if the data structure and access pattern is the one that a Map is designed for - the problem is not the HashMap itself.
I would solve this with a javascript/ajax calls method that fetches childnodes.