We Keep Coding

Dynamic Sorting Queue Java [duplicate]

I need to implement a priority queue where the priority of an item in the queue can change and the queue adjusts itself so that items are always removed in the correct order. I have some ideas of how I could implement this but I'm sure this is quite a common data structure so I'm hoping I can use an implementation by someone smarter than me as a base.
Can anyone tell me the name of this type of priority queue so I know what to search for or, even better, point me to an implementation?

Priority queues such as this are typically implemented using a binary heap data structure as someone else suggested, which usually is represented using an array but could also use a binary tree. It actually is not hard to increase or decrease the priority of an element in the heap. If you know you are changing the priority of many elements before the next element is popped from the queue you can temporarily turn off dynamic reordering, insert all of the elements at the end of the heap, and then reorder the entire heap (at a cost of O(n)) just before the element needs to be popped. The important thing about heaps is that it only costs O(n) to put an array into heap order but O(n log n) to sort it.
I have used this approach successfully in a large project with dynamic priorities.
Here is my implementation of a parameterized priority queue implementation in the Curl programming language.

A standard binary heap supports 5 operations (the example below assume a max heap):
* find-max: return the maximum node of the heap
* delete-max: removing the root node of the heap
* increase-key: updating a key within the heap
* insert: adding a new key to the heap
* merge: joining two heaps to form a valid new heap containing all the elements of both.
As you can see, in a max heap, you can increase an arbitrary key. In a min heap you can decrease an arbitrary key. You can't change keys both ways unfortunately, but will this do? If you need to change keys both ways then you might want to think about using a a min-max-heap.

I would suggest first trying the head-in approach, to update a priority:
delete the item from the queue
re-insert it with the new priority
In C++, this could be done using a std::multi_map, the important thing is that the object must remember where it is stored in the structure to be able to delete itself efficiently. For re-insert, it's difficult since you cannot presume you know anything about the priorities.
class Item;
typedef std::multi_map<int, Item*> priority_queue;
class Item
{
public:
void add(priority_queue& queue);
void remove();
int getPriority() const;
void setPriority(int priority);
std::string& accessData();
const std::string& getData() const;
private:
int mPriority;
std::string mData;
priority_queue* mQueue;
priority_queue::iterator mIterator;
};
void Item::add(priority_queue& queue)
{
mQueue = &queue;
mIterator = queue.insert(std::make_pair(mPriority,this));
}
void Item::remove()
{
mQueue.erase(mIterator);
mQueue = 0;
mIterator = priority_queue::iterator();
}
void Item::setPriority(int priority)
{
mPriority = priority;
if (mQueue)
{
priority_queue& queue = *mQueue;
this->remove();
this->add(queue);
}
}

I am looking for just exactly the same thing!
And here is some of my idea:
Since a priority of an item keeps changing,
it's meaningless to sort the queue before retrieving an item.
So, we should forget using a priority queue. And "partially" sort the
container while retrieving an item.
And choose from the following STL sort algorithms:
a. partition
b. stable_partition
c. nth_element
d. partial_sort
e. partial_sort_copy
f. sort
g. stable_sort
partition, stable_partition and nth_element are linear-time sort algorithms, which should be our 1st choices.
BUT, it seems that there is no those algorithms provided in the official Java library. As a result, I will suggest you to use java.util.Collections.max/min to do what you want.

Google has a number of answers for you, including an implementation of one in Java.
However, this sounds like something that would be a homework problem, so if it is, I'd suggest trying to work through the ideas yourself first, then potentially referencing someone else's implementation if you get stuck somewhere and need a pointer in the right direction. That way, you're less likely to be "biased" towards the precise coding method used by the other programmer and more likely to understand why each piece of code is included and how it works. Sometimes it can be a little too tempting to do the paraphrasing equivalent of "copy and paste".

Double-ended priority queue using dual structure approach

I'm trying to understand how to build a double-ended priority queue using two heaps: a min heap and a max heap. My thinking so far is that I'll need one array to store the min heap, and another to store the max heap, and then I need to figure out how to connect the relevant entries in the two arrays to each other. E.g., I need to make sure that wherever the value "12" ends up in the min heap somehow points to where the value "12" is in the max heap, and vice versa. I understand that in theory, but I have no idea how to go about actually implementing it.
How can I make elements in one array point to elements in another array in an efficient and flexible way? Especially since each array is going to be continually re-shuffled throughout the program.
Not sure if that made sense, but any help is most appreciated. Thanks.

How can I make elements in one array point to elements in another
array in an efficient and flexible way?
Use a pointer to each element that knows what object is its counter-part e.g.
public class Element<T> {
T otherElement;
public void setOther(T element) {
this.otherElement = element;
}
}
// when you create the objects
Element<String> one = new Element();
Element<String> two = new Element();
// now both elements know about each other and they can be to whatever list/array etc they want
one.setOther(two);
two.setOther(one);
If your requirements are that each object knows its position (i.e. index) in each list this might require a bit of more work depending on how you implement the heaps. You should make sure that they set the location of each element, every time they change its position. So the Element object would become location aware.

you will be ending up with creating wrapper object and store in an array or a map(if you want to retrive it later with an id). I did not understand for what is the purpose of referencing each other. if you want to add and remove you have to write the logic for that.

How to resize ArrayList

I come from a C++ background and I want to have a matrix of
ArrayList<arrayList<E>> javamatrix
In C++ I would just do
std::vector<std::vector<T> > cppmatrix;
std::vector<T>vcol(cols);
cppmatrix.resize(rows,vcol);
I can't seem to find a built-in resize() function for ArrayLists for this task, so should I use another collection? Is no way to do this except using for loops with javamatrix.add()?
P.S I want it to be initialized in the constructor with its size as that size might be queried before I edit elements or add or remove.

There is no resize equivalent that automatically constructs and adds elements. You must do this yourself. However, ensureCapacity is equivalent to vector's reserve. It will ensure you have room, but not change the actual size.

You shouldn't need to resize arraylists. The size you initially pass in is just its starting size. If you attempt to add items beyond its current size, it will automatically resize.
From the documentation:
Each ArrayList instance has a capacity. The capacity is the size of the array used to store the elements in the list. It is always at least as large as the list size. As elements are added to an ArrayList, its capacity grows automatically. The details of the growth policy are not specified beyond the fact that adding an element has constant amortized time cost.

Mostly, a 'resize()' operation is not needed because (a) ArrayList's auto-resize as you add elements, and (b) it's unclear what values you would store in the ArrayList<>, e.g. 'null' is not very useful. E.g. in your case you'd probably need a loop anyway to create MatrixCell objects.
For those readers who want to know how to resize an ArrayList to make it smaller, it mystifies me why ArrayList was designed without a 'resize()' method. Perhaps it's because novice programmers are likely to see that method and then not realise that ArrayList<> auto-resizes.
In Java this idiom works to reduce the size of an ArrayList<>:
list.subList(n,list.size()).clear();
It works because the 'subList' returns a List backed by the original ArrayList<>, so therefore the 'clear()' operates on the original 'ArrayList<>'.

I know this question is very old already but this link may help java arraylist ensureCapacity not working , The code adds "Null" value in order to adjust the current size.
Instead of using purely ensureCapacity you can have ensureSize
public static void ensureSize(ArrayList<?> list, int size) {
list.ensureCapacity(size);
while (list.size() < size) {
list.add(null);
}
}

Java ArrayList<Double> IndexOutOfBoundsException Problem

I've got a Problem with ArrayList. I need it to store a result. Because I want to start with element n I tried to give the ArrayList a capacity with ensureCapacity(n+1) to use set(n,x) but I get an IndexOutOfBoundsException.
I tried to store n add(x) before the use of set and this works.
So I'd like to know why it doesn't work on my way and how to solve this because put n times a add(x) isn't a good style ;-)

When you change the capacity of an ArrayList it doesn't create any elements, it just reserves memory where there could be elements. You can check the size before and after adjusting the capacity and you will see that it does not change.
The purpose of changing the capacity is if you know in advance how many elements you will have, then you can avoid unnecessary repeated resizing as you add new elements, and you can avoid memory wastage from excess unused capacity.

If you don't like using your own loop and the list add method directly then there is another way. Create your ArrayList with the number of elements you want it directly like this:
final int MAX_ELEMENTS = 1000;
List<Integer> myList = new ArrayList<Integer>(
Collections.<Integer>nCopies(MAX_ELEMENTS, null));
Or, if you already have a list that you want to expand the size by n elements:
myList.addAll(Collections.<Integer>nCopies(n, null));
(Note, I assumed here that the list would be holding Integer objects, but you can change this to your custom type. If you are working with raw/pre-Java 5 types then just drop the generic declarations.)
As for your actual question: capacity != contents. An ArrayList internally has both a physical array and a count of what is actually in it. Increasing the capacity, changes the internal array so it can hold that many elements, however, the count does not change. You need to add elements to increase that count.
On the other hand, if you are just trying to set specific elements and know the maximum that you want to use, why not use an array directly? If you then need to pass this array to an API that takes Lists, then use Arrays.asList. The other classes could still change contents of your backing array but it would not be able to increase the size or capacity of it.

As others have answered, ensureCapacity() is just related to performance, is not frequently used by the common user.
From Bruce Eckel's Thinking in Java book:
In a private message, Joshua Bloch
wrote: "... I believe that we erred by
allowing implementation details (such
as hash table size and load factor)
into our APIs. The client should
perhaps tell us the maximum expected
size of a collection, and we should
take it from there. Clients can easily
do more harm than good by choosing
values for these parameters. As an
extreme example, consider Vector's
capacityIncrement. No one should ever
set this, and we shouldn't have
provided it. If you set it to any
non-zero value, the asymptotic cost of
a sequence of appends goes from linear
to quadratic. In other words, it
destroys your performance. Over time,
we're beginning to wise up about this
sort of thing. If you look at
IdentityHashMap, you'll see that it
has no low-level tuning parameters"

You are getting this exception because ensureCapacity() only makes sure that there is enough memory allocated for adding objects to an ArrayList, I believe this is in case you want to add multiple objects at once, without having to relocate memory.
To do what you want you would have to initiate the ArrayList with null elements first...
int n = 10; //capacity required
ArrayList foo = new ArrayList();
for( int i=0; i<=n; i++ ) {
foo.add(null);
}
Then you have objects in the List that you can reference via index and you wont receive the exception.

Perhaps you should rethink the choice of using List<Double>. It might be that a Map<Integer,Double> would be more appropriate if elements are to be added in an odd order.
Whether this is appropriate depends on knowledge about your usage that I don't have at the moment though.
Is the data structure eventually going to be completely filled, or is the data sparse?

what other people said about ensureCapacity() ...
you should write a class like DynamicArrayList extends ArrayList. then just overrride add(n,x) to do with for loop add(null) logic specified about.

ensureCapacity() has another purpose. It should be used in cases when you get to know the required size of the List after it has been constructed. If you know the size before it is constructor, just pass it as a an argument to the constructor.
In the former case use ensureCapacity() to save multiple copying of the backing array on each addition. However, using that method leaves the structure in a seemingly inconsistent state
the size of the backing array is increased
the size field on the ArrayList isn't.
This, however, is normal, since the capacity != size
Use the add(..) method, which is the only one that is increasing the size field:
ArrayList list = new ArrayList();
list.ensureCapacity(5); // this can be done with constructing new ArrayList(5)
for (int i = 0; i < list.size - 1; i ++) {
list.add(null);
}
list.add(yourObject);

Updating Java PriorityQueue when its elements change priority

I'm trying to use a PriorityQueue to order objects using a Comparator.
This can be achieved easily, but the objects class variables (with which the comparator calculates priority) may change after the initial insertion. Most people have suggested the simple solution of removing the object, updating the values and reinserting it again, as this is when the priority queue's comparator is put into action.
Is there a better way other than just creating a wrapper class around the PriorityQueue to do this?

You have to remove and re-insert, as the queue works by putting new elements in the appropriate position when they are inserted. This is much faster than the alternative of finding the highest-priority element every time you pull out of the queue. The drawback is that you cannot change the priority after the element has been inserted. A TreeMap has the same limitation (as does a HashMap, which also breaks when the hashcode of its elements changes after insertion).
If you want to write a wrapper, you can move the comparison code from enqueue to dequeue. You would not need to sort at enqueue time anymore (because the order it creates would not be reliable anyway if you allow changes).
But this will perform worse, and you want to synchronize on the queue if you change any of the priorities. Since you need to add synchronization code when updating priorities, you might as well just dequeue and enqueue (you need the reference to the queue in both cases).

I don't know if there is a Java implementation, but if you're changing key values alot, you can use a Fibonnaci heap, which has O(1) amortized cost to decrease a key value of an entry in the heap, rather than O(log(n)) as in an ordinary heap.

One easy solution that you can implement is by just adding that element again into the priority queue. It will not change the way you extract the elements although it will consume more space but that also won't be too much to effect your running time.
To proof this let's consider dijkstra algorithm below
public int[] dijkstra() {
int distance[] = new int[this.vertices];
int previous[] = new int[this.vertices];
for (int i = 0; i < this.vertices; i++) {
distance[i] = Integer.MAX_VALUE;
previous[i] = -1;
}
distance[0] = 0;
previous[0] = 0;
PriorityQueue<Node> pQueue = new PriorityQueue<>(this.vertices, new NodeComparison());
addValues(pQueue, distance);
while (!pQueue.isEmpty()) {
Node n = pQueue.remove();
List<Edge> neighbours = adjacencyList.get(n.position);
for (Edge neighbour : neighbours) {
if (distance[neighbour.destination] > distance[n.position] + neighbour.weight) {
distance[neighbour.destination] = distance[n.position] + neighbour.weight;
previous[neighbour.destination] = n.position;
pQueue.add(new Node(neighbour.destination, distance[neighbour.destination]));
}
}
}
return previous;
}
Here our interest is in line
pQueue.add(new Node(neighbour.destination, distance[neighbour.destination]));
I am not changing priority of the particular node by removing it and adding again rather I am just adding new node with same value but different priority.
Now at the time of extracting I will always get this node first because I have implemented min heap here and the node with value greater than this (less priority) always be extracted afterwards and in this way all neighboring nodes will already be relaxed when less prior element will be extracted.

Without reimplementing the priority queue yourself (so by only using utils.PriorityQueue) you have essentially two main approaches:
1) Remove and put back
Remove element then put it back with new priority. This is explained in the answers above. Removing an element is O(n) so this approach is quite slow.
2) Use a Map and keep stale items in the queue
Keep a HashMap of item -> priority. The keys of the map are the items (without their priority) and the values of the map are the priorities.
Keep it in sync with the PriorityQueue (i.e. every time you add or remove an item from the Queue, update the Map accordingly).
Now when you need to change the priority of an item, simply add the same item to the queue with a different priority (and update the map of course). When you poll an item from the queue, check if its priority is the same than in your map. If not, then ditch it and poll again.
If you don't need to change the priorities too often, this second approach is faster. Your heap will be larger and you might need to poll more times, but you don't need to find your item.
The 'change priority' operation would be O(f(n)log n*), with f(n) the number of 'change priority' operation per item and n* the actual size of your heap (which is n*f(n)).
I believe that if f(n) is O(n/logn)(for example f(n) = O(sqrt(n)), this is faster than the first approach.
Note : in the explanation above, by priority I means all the variables that are used in your Comparator. Also your item need to implement equals and hashcode, and both methods shouldn't use the priority variables.

It depends a lot on whether you have direct control of when the values change.
If you know when the values change, you can either remove and reinsert (which in fact is fairly expensive, as removing requires a linear scan over the heap!).
Furthermore, you can use an UpdatableHeap structure (not in stock java though) for this situation. Essentially, that is a heap that tracks the position of elements in a hashmap. This way, when the priority of an element changes, it can repair the heap. Third, you can look for an Fibonacci heap which does the same.
Depending on your update rate, a linear scan / quicksort / QuickSelect each time might also work. In particular if you have much more updates than pulls, this is the way to go. QuickSelect is probably best if you have batches of update and then batches of pull opertions.

To trigger reheapify try this:
if(!priorityQueue.isEmpty()) {
priorityQueue.add(priorityQueue.remove());
}

Something I've tried and it works so far, is peeking to see if the reference to the object you're changing is the same as the head of the PriorityQueue, if it is, then you poll(), change then re-insert; else you can change without polling because when the head is polled, then the heap is heapified anyways.
DOWNSIDE: This changes the priority for Objects with the same Priority.

Is there a better way other than just creating a wrapper class around the PriorityQueue to do this?
It depends on the definition of "better" and the implementation of the wrapper.
If the implementation of the wrapper is to re-insert the value using the PriorityQueue's .remove(...) and .add(...) methods,
it's important to point out that .remove(...) runs in O(n) time.
Depending on the heap implementation,
updating the priority of a value can be done in O(log n) or even O(1) time,
therefore this wrapper suggestion may fall short of common expectations.
If you want to minimize your effort to implement,
as well as the risk of bugs of any custom solution,
then a wrapper that performs re-insert looks easy and safe.
If you want the implementation to be faster than O(n),
then you have some options:
Implement a heap yourself. The wikipedia entry describes multiple variants with their properties. This approach is likely to get your the best performance, at the same time the more code you write yourself, the greater the risk of bugs.
Implement a different kind of wrapper: handlee updating the priority by marking the entry as removed, and add a new entry with the revised priority.
This is relatively easy to do (less code), see below, though it has its own caveats.
I came across the second idea in Python's documentation,
and applied it to implement a reusable data structure in Java (see caveats at the bottom):
public class UpdatableHeap<T> {
private final PriorityQueue<Node<T>> pq = new PriorityQueue<>(Comparator.comparingInt(node -> node.priority));
private final Map<T, Node<T>> entries = new HashMap<>();
public void addOrUpdate(T value, int priority) {
if (entries.containsKey(value)) {
entries.remove(value).removed = true;
}
Node<T> node = new Node<>(value, priority);
entries.put(value, node);
pq.add(node);
}
public T pop() {
while (!pq.isEmpty()) {
Node<T> node = pq.poll();
if (!node.removed) {
entries.remove(node.value);
return node.value;
}
}
throw new IllegalStateException("pop from empty heap");
}
public boolean isEmpty() {
return entries.isEmpty();
}
private static class Node<T> {
private final T value;
private final int priority;
private boolean removed = false;
private Node(T value, int priority) {
this.value = value;
this.priority = priority;
}
}
}
Note some caveats:
Entries marked removed stay in memory until they are popped
This can be unacceptable in use cases with very frequent updates
The internal Node wrapped around the actual values is an extra memory overhead (constant per entry). There is also an internal Map, mapping all the values currently in the priority queue to their Node wrapper.
Since the values are used in a map, users must be aware of the usual cautions when using a map, and make sure to have appropriate equals and hashCode implementations.