We Keep Coding

What java array structure is best for cycling data through? [duplicate]

I have a streaming time series, of which I am interested in keeping the last 4 elements, which means I want to be able to pop the first, and add to the end. Essentially what I need is a ring buffer.
Which Java Collection is the best for this? Vector?

Consider CircularFifoBuffer from Apache Common.Collections. Unlike Queue you don't have to maintain the limited size of underlying collection and wrap it once you hit the limit.
Buffer buf = new CircularFifoBuffer(4);
buf.add("A");
buf.add("B");
buf.add("C");
buf.add("D"); //ABCD
buf.add("E"); //BCDE
CircularFifoBuffer will do this for you because of the following properties:
CircularFifoBuffer is a first in first out buffer with a fixed
size that replaces its oldest element if full.
The removal order of a CircularFifoBuffer is based on the insertion
order; elements are removed in the same order in which they were
added. The iteration order is the same as the removal order.
The add(Object), BoundedFifoBuffer.remove() and
BoundedFifoBuffer.get() operations all perform in constant time.
All other operations perform in linear time or worse.
However you should consider it's limitations as well - for example, you can't add missing timeseries to this collection because it doens't allow nulls.
NOTE: When using current Common Collections (4.*), you have to use Queue. Like this:
Queue buf = new CircularFifoQueue(4);

Since Guava 15.0 (released September 2013) there's EvictingQueue:
A non-blocking queue which automatically evicts elements from the head
of the queue when attempting to add new elements onto the queue and it
is full. An evicting queue must be configured with a maximum size.
Each time an element is added to a full queue, the queue automatically
removes its head element. This is different from conventional bounded
queues, which either block or reject new elements when full.
This class is not thread-safe, and does not accept null elements.
Example use:
EvictingQueue<String> queue = EvictingQueue.create(2);
queue.add("a");
queue.add("b");
queue.add("c");
queue.add("d");
System.out.print(queue); //outputs [c, d]

Since Java 1.6, there is ArrayDeque, which implements Queue and seems to be faster and more memory efficient than a LinkedList and doesn't have the thread synchronization overhead of the ArrayBlockingQueue: from the API docs: "This class is likely to be faster than Stack when used as a stack, and faster than LinkedList when used as a queue."
final Queue<Object> q = new ArrayDeque<Object>();
q.add(new Object()); //insert element
q.poll(); //remove element

If you need
O(1) insertion and removal
O(1) indexing to interior elements
access from a single thread only
generic element type
then you can use this CircularArrayList for Java in this way (for example):
CircularArrayList<String> buf = new CircularArrayList<String>(4);
buf.add("A");
buf.add("B");
buf.add("C");
buf.add("D"); // ABCD
String pop = buf.remove(0); // A <- BCD
buf.add("E"); // BCDE
String interiorElement = buf.get(i);
All these methods run in O(1).

I had the same problem some time ago and was disappointed because I couldn't find any solution that suites my needs so I wrote my own class.
Honestly, I did found some code back then, but even that wasn't what I was searching for so I adapted it and now I'm sharing it, just like the author of that piece of code did.
EDIT: This is the original (although slightly different) code: CircularArrayList for java
I don't have the link of the source because it was time ago, but here's the code:
import java.util.AbstractList;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.RandomAccess;
public class CircularArrayList<E> extends AbstractList<E> implements RandomAccess {
private final int n; // buffer length
private final List<E> buf; // a List implementing RandomAccess
private int leader = 0;
private int size = 0;
public CircularArrayList(int capacity) {
n = capacity + 1;
buf = new ArrayList<E>(Collections.nCopies(n, (E) null));
}
public int capacity() {
return n - 1;
}
private int wrapIndex(int i) {
int m = i % n;
if (m < 0) { // modulus can be negative
m += n;
}
return m;
}
#Override
public int size() {
return this.size;
}
#Override
public E get(int i) {
if (i < 0 || i >= n-1) throw new IndexOutOfBoundsException();
if(i > size()) throw new NullPointerException("Index is greater than size.");
return buf.get(wrapIndex(leader + i));
}
#Override
public E set(int i, E e) {
if (i < 0 || i >= n-1) {
throw new IndexOutOfBoundsException();
}
if(i == size()) // assume leader's position as invalid (should use insert(e))
throw new IndexOutOfBoundsException("The size of the list is " + size() + " while the index was " + i
+". Please use insert(e) method to fill the list.");
return buf.set(wrapIndex(leader - size + i), e);
}
public void insert(E e)
{
int s = size();
buf.set(wrapIndex(leader), e);
leader = wrapIndex(++leader);
buf.set(leader, null);
if(s == n-1)
return; // we have replaced the eldest element.
this.size++;
}
#Override
public void clear()
{
int cnt = wrapIndex(leader-size());
for(; cnt != leader; cnt = wrapIndex(++cnt))
this.buf.set(cnt, null);
this.size = 0;
}
public E removeOldest() {
int i = wrapIndex(leader+1);
for(;;i = wrapIndex(++i)) {
if(buf.get(i) != null) break;
if(i == leader)
throw new IllegalStateException("Cannot remove element."
+ " CircularArrayList is empty.");
}
this.size--;
return buf.set(i, null);
}
#Override
public String toString()
{
int i = wrapIndex(leader - size());
StringBuilder str = new StringBuilder(size());
for(; i != leader; i = wrapIndex(++i)){
str.append(buf.get(i));
}
return str.toString();
}
public E getOldest(){
int i = wrapIndex(leader+1);
for(;;i = wrapIndex(++i)) {
if(buf.get(i) != null) break;
if(i == leader)
throw new IllegalStateException("Cannot remove element."
+ " CircularArrayList is empty.");
}
return buf.get(i);
}
public E getNewest(){
int i = wrapIndex(leader-1);
if(buf.get(i) == null)
throw new IndexOutOfBoundsException("Error while retrieving the newest element. The Circular Array list is empty.");
return buf.get(i);
}
}

A very interesting project is disruptor. It has a ringbuffer and is used from what I know in financial applications.
See here: code of ringbuffer
I checked both Guava's EvictingQueue and ArrayDeque.
ArrayDeque does not limit growth if it's full it will double size and hence is not precisely acting like a ringbuffer.
EvictingQueue does what it promises but internally uses a Deque to store things and just bounds memory.
Hence, if you care about memory being bounded ArrayDeque is not fullfilling your promise. If you care about object count EvictingQueue uses internal composition (bigger object size).
A simple and memory efficient one can be stolen from jmonkeyengine.
verbatim copy
import java.util.Iterator;
import java.util.NoSuchElementException;
public class RingBuffer<T> implements Iterable<T> {
private T[] buffer; // queue elements
private int count = 0; // number of elements on queue
private int indexOut = 0; // index of first element of queue
private int indexIn = 0; // index of next available slot
// cast needed since no generic array creation in Java
public RingBuffer(int capacity) {
buffer = (T[]) new Object[capacity];
}
public boolean isEmpty() {
return count == 0;
}
public int size() {
return count;
}
public void push(T item) {
if (count == buffer.length) {
throw new RuntimeException("Ring buffer overflow");
}
buffer[indexIn] = item;
indexIn = (indexIn + 1) % buffer.length; // wrap-around
count++;
}
public T pop() {
if (isEmpty()) {
throw new RuntimeException("Ring buffer underflow");
}
T item = buffer[indexOut];
buffer[indexOut] = null; // to help with garbage collection
count--;
indexOut = (indexOut + 1) % buffer.length; // wrap-around
return item;
}
public Iterator<T> iterator() {
return new RingBufferIterator();
}
// an iterator, doesn't implement remove() since it's optional
private class RingBufferIterator implements Iterator<T> {
private int i = 0;
public boolean hasNext() {
return i < count;
}
public void remove() {
throw new UnsupportedOperationException();
}
public T next() {
if (!hasNext()) {
throw new NoSuchElementException();
}
return buffer[i++];
}
}
}

None of the previously given examples were meeting my needs completely, so I wrote my own queue that allows following functionality: iteration, index access, indexOf, lastIndexOf, get first, get last, offer, remaining capacity, expand capacity, dequeue last, dequeue first, enqueue / add element, dequeue / remove element, subQueueCopy, subArrayCopy, toArray, snapshot, basics like size, remove or contains.
EjectingQueue
EjectingIntQueue

Use a Queue
Queue<String> qe=new LinkedList<String>();
qe.add("a");
qe.add("b");
qe.add("c");
qe.add("d");
System.out.println(qe.poll()); //returns a
System.out.println(qe.poll()); //returns b
System.out.println(qe.poll()); //returns c
System.out.println(qe.poll()); //returns d
There's five simple methods of a Queue
element() -- Retrieves, but does not remove, the head of this
queue.
offer(E o) -- Inserts the specified element into this queue, if
possible.
peek() -- Retrieves, but does not remove, the head of this
queue, returning null if this queue is empty.
poll() -- Retrieves and removes the head of this queue, or
null if this queue is empty.
remove() -- Retrieves and removes the head of this queue.

How to remove(pop) element from a stack implementation?

Currently working on my home assignment, a simulation code to make stack with a support of dynamic array.
The method pop() is not working to it is function, I wrote some of the code but I need a completion. My stack should be simulates like this, you insert a numbers to be called top then remove that number(top).
import java.util.NoSuchElementException;
public class MyStack implements IntStack {
int[] heltal;
public MyStack() {
heltal = new int[0];
}
public void push(int tal) {
int[] temp = new int[heltal.length + 1];
for (int x = 0; x < heltal.length; x++) {
temp[x] = heltal[x] + tal;
}
heltal = temp;
for (int i = 0; i < heltal.length; i++) {
heltal[i] = tal;
}
}
#Override
public int pop() {
if (Isempty()) {
throw new NoSuchElementException("The stack is empty, there is nothing to pop");
} else {
int[] temp = new int[heltal.length - 1];
for (int x = 0; x < heltal.length - 1; x++) {
temp[x] = heltal[x];
}
int etttal = heltal[0];
heltal = temp;
return etttal;
}
}
#Override
public int peek() {
if (Isempty()) {
throw new NoSuchElementException("The stack is empty");
} else {
return heltal[0];
}
}
public boolean Isempty() {
return heltal.length == 0;
}
}

You appear to be making this more difficult than required. For a stack implementation, push, pop, peek, and others are nothing more than index manipulators that return or store a value. The stack can be backed by an array or a list. And pushing et al are abstract terms. So when you push a value, you do not need to copy everything down by one. Just add it to the end of your data structure.
pop - check the index and if valid, return the element at the current index, update the index.
push - store the value at the next location. Probably index + 1 but it depends on how you implement it.
peek - return the top value (at the index) but don't update the index.
If you are using an array, you need to add method(s) to increase it's capacity.
For more information, check out Stack
Here is a simple push method, backed by an array called stack and an index field. It presumes ints are being used.
public void push(int v) {
if (index == stack.length-1) {
// no more room, increase array size
// while retaining current values.
}
stack[++index] = v;
}

Why do I get this null pointer exception? [duplicate]

This question already has an answer here:
NullPointerException Error with JPanel and CardLayout
(1 answer)
Closed 8 years ago.
So I'm getting this nullpointerexception. I know the line it's being caused at, I just am having trouble seeing what's causing it in my code.
import java.util.*;
public class NoDuplicatesQueueWilson<T> implements NoDuplicatesQueueInterfaceWilson<T>
{
private int MAX_QUEUE = 5; // Default array size, small for testing purposes
private T[] items; // The array of the queue.
private int front; // The first entered item of a queue.
private int back; // The last entered item of a queue.
private int count; // A counter.
public NoDuplicatesQueueWilson() // Default constructor
{
T [] items = (T[]) new Object[MAX_QUEUE];
front = 0;
back = MAX_QUEUE-1;
count = 0;
}
// Begin Queue Operations
// isEmpty() checks the array to determine if there is an items in it.
public boolean isEmpty()
{
return count == 0;
}
// isFull() checks to see if the array is full or not.
public boolean isFull()
{
return count == MAX_QUEUE;
}
// enqueue(Object newItem) adds items to the back of the queue if it is not full.
// If it is full, it will double the size of the array of items,
// and re-create the array, adding the item onto the new array.
public void enqueue(T newItem) {
if(!isFull())
{
back = (back+1) % (MAX_QUEUE);
items[back] = newItem;
++count;
}
else
{
MAX_QUEUE = MAX_QUEUE * 2;
System.out.println("The array was full. We've doubled the size.");
T [] items = (T[]) new Object[MAX_QUEUE];
back = (back+1) % (MAX_QUEUE);
items[back] = newItem;
++count;
} // End if
} // End Enqueue
When I run it in my driver program (with test data), the exception occurs at line 43 of the given code (my main class containing the methods and constructors), which is in the middle of my enqueue method. Specifically this line:
items[back] = newItem;
Any suggestions on what I might need to do or be looking for to see where my mistake was?

In constructor you are not initializing T [] items but you are creating new local variable. It should be
public NoDuplicatesQueueWilson() // Default constructor
{
items = (T[]) new Object[MAX_QUEUE]; // notice the difference
front = 0;
back = MAX_QUEUE-1;
count = 0;
}
Edit: Also please check else part of code in enqueue method.

In your constructor, this line assigns to a local variable, instead of an instance variable:
T [] items = (T[]) new Object[MAX_QUEUE];
You can access the instance variable items with this.items:
this.items = (T[]) new Object[MAX_QUEUE];
You could also just use items = ... as there is no ambiguity in this case.
The same bug is also in your enqueue(T newItem) method at T [] items = ....

Java Stack with elements limit

I know this question has asked many times but after seaching for an hour i still have problem.
I want to use a lifo stack which has a max number of elements it can store.After it reach the max number is deletes the element at first place and replace it with the new so in first pop i can get this element and in second i have to get the element at size-1.
What i tried:
1) Using a modified Stack ,as described here .The problem is that it always returning the first 5 elements(if the size is 5) i added.
class StackSizable<E> extends Stack<E>{
int maxSize;
StackSizable(int size)
{
super();
this.maxSize=size;
}
#Override
public E push(E elt) {
super.push(elt);
while (this.size() > this.maxSize) {
this.removeElementAt(this.size() - 1);
}
return null;
}
}
2)Using an ArrayDeque ,i dont see any diference from a simple Stack , its not setting any limit(am i using it wrong?)
ArrayDeque<State> lifo = new ArrayDeque<State>(5);
lifo.pop();
lifo.push(state);
I want to use this in a puzzle game for undo-redo functionality
Solved: I ended using a fixed size stack as tom said ,mainly for the performance
public class FixedStack<T> {
private T[] stack;
private int size;
private int top;
private int popBalance = 0;//its used to see if all the elements have been popped
public FixedStack(T[] stack) {
this.stack = stack;
this.top = 0;
this.size = stack.length;
}
public void push(T obj) {
if (top == stack.length)top = 0;
stack[top] = obj;
top++;
if (popBalance < size - 1)popBalance++;
}
public T pop() {
if (top - 1 < 0)top = size;
top--;
T ob = stack[top];
popBalance--;
return ob;
}
public void clear() {
top = 0;
}
public int size() {
return size;
}
public boolean poppedAll() {
if (popBalance == -1)return true;
return false;
}
}

I think the most efficient way to this is with a fixed array, with size equal to your max # of elements, and an index that points to the element that is currently the 'top' of the queue.
When you add a new element you add it at index+1 (wrapping back to element 0 if necessary) and possibly overwriting an element that no longer fits. When you pop an element you do the reverse.
This way your data structure never has to be re-ordered, and you can use an array which is more light-weight then a collection.

When the maximum size has been reached, your line
this.removeElementAt(this.size() - 1);
then immediately removes the last pushed element (which you just pushed), which is the top of the stack. You need to remove the first element instead (bottom of the stack):
this.removeElementAt(0);

Stack overflow error for large inputs in Java

I'm writing a Java program that searches for and outputs cycles in a graph. I am using an adjacency list for storing my graph, with the lists stored as LinkedLists. My program takes an input formatted with the first line as the number of nodes in the graph and each subsequent line 2 nodes that form an edge e.g.:
3
1 2
2 3
3 1
My problem is that when the inputs get very large (the large graph I am using has 10k nodes and I don't know how many edges, the file is 23mb of just edges) I am getting a java.lang.StackOverflowError, but I don't get any errors with small inputs. I'm wondering if it would be better to use another data structure to form my adjacency lists or if there is some method I could use to avoid this error, as I'd rather not just have to change a setting on my local installation of Java (because I have to be sure this will run on other computers that I can't control the settings on as much). Below is my code, the Vertex class and then my main class. Thanks for any help you can give!
Vertex.java:
package algorithms311;
import java.util.*;
public class Vertex implements Comparable {
public int id;
public LinkedList adjVert = new LinkedList();
public String color = "white";
public int dTime;
public int fTime;
public int prev;
public Vertex(int idnum) {
id = idnum;
}
public int getId() {
return id;
}
public int compareTo(Object obj) {
Vertex vert = (Vertex) obj;
return id-vert.getId();
}
#Override public String toString(){
return "Vertex # " + id;
}
public void setColor(String newColor) {
color = newColor;
}
public String getColor() {
return color;
}
public void setDTime(int d) {
dTime = d;
}
public void setFTime(int f) {
fTime = f;
}
public int getDTime() {
return dTime;
}
public int getFTime() {
return fTime;
}
public void setPrev(int v) {
prev = v;
}
public int getPrev() {
return prev;
}
public LinkedList getAdjList() {
return adjVert;
}
public void addAdj(int a) { //adds a vertex id to this vertex's adj list
adjVert.add(a);
}
}
CS311.java:
package algorithms311;
import java.util.*;
import java.io.*;
public class CS311 {
public static final String GRAPH= "largegraph1";
public static int time = 0;
public static LinkedList[] DFS(Vertex[] v) {
LinkedList[] l = new LinkedList[2];
l[0] = new LinkedList();
l[1] = new LinkedList(); //initialize the array with blank lists, otherwise we get a nullpointerexception
for(int i = 0; i < v.length; i++) {
v[i].setColor("white");
v[i].setPrev(-1);
}
time = 0;
for(int i = 0; i < v.length; i++) {
if(v[i].getColor().equals("white")) {
l = DFSVisit(v, i, l);
}
}
return l;
}
public static LinkedList[] DFSVisit(Vertex[] v, int i, LinkedList[] l) { //params are a vertex of nodes and the node id you want to DFS from
LinkedList[] VOandBE = new LinkedList[2]; //two lists: visit orders and back edges
VOandBE[0] = l[0]; // l[0] is visit Order, a linked list of ints
VOandBE[1] = l[1]; // l[1] is back Edges, a linked list of arrays[2] of ints
VOandBE[0].add(v[i].getId());
v[i].setColor("gray"); //color[vertex i] <- GRAY
time++; //time <- time+1
v[i].setDTime(time); //d[vertex i] <- time
LinkedList adjList = v[i].getAdjList(); // adjList for the current vertex
for(int j = 0; j < adjList.size(); j++) { //for each v in adj[vertex i]
if(v[(Integer)adjList.get(j)].getColor().equals("gray") && v[i].getPrev() != v[(Integer)adjList.get(j)].getId()) { // if color[v] = gray and Predecessor[u] != v do
int[] edge = new int[2]; //pair of vertices
edge[0] = i; //from u
edge[1] = (Integer)adjList.get(j); //to v
VOandBE[1].add(edge);
}
if(v[(Integer)adjList.get(j)].getColor().equals("white")) { //do if color[v] = WHITE
v[(Integer)adjList.get(j)].setPrev(i); //then "pi"[v] <- vertex i
DFSVisit(v, (Integer)adjList.get(j), VOandBE); //DFS-Visit(v)
}
}
VOandBE[0].add(v[i].getId());
v[i].setColor("black");
time++;
v[i].setFTime(time);
return VOandBE;
}
public static void main(String[] args) {
try {
// --Read First Line of Input File
// --Find Number of Vertices
FileReader file1 = new FileReader("W:\\Documents\\NetBeansProjects\\algorithms311\\src\\algorithms311\\" + GRAPH);
BufferedReader bReaderNumEdges = new BufferedReader(file1);
String numVertS = bReaderNumEdges.readLine();
int numVert = Integer.parseInt(numVertS);
System.out.println(numVert + " vertices");
// --Make Vertices
Vertex vertex[] = new Vertex[numVert];
for(int k = 0; k <= numVert - 1; k++) {
vertex[k] = new Vertex(k);
}
// --Adj Lists
FileReader file2 = new FileReader("W:\\Documents\\NetBeansProjects\\algorithms311\\src\\algorithms311\\" + GRAPH);
BufferedReader bReaderEdges = new BufferedReader(file2);
bReaderEdges.readLine(); //skip first line, that's how many vertices there are
String edge;
while((edge = bReaderEdges.readLine()) != null) {
StringTokenizer ST = new StringTokenizer(edge);
int vArr[] = new int[2];
for(int j = 0; ST.hasMoreTokens(); j++) {
vArr[j] = Integer.parseInt(ST.nextToken());
}
vertex[vArr[0]-1].addAdj(vArr[1]-1);
vertex[vArr[1]-1].addAdj(vArr[0]-1);
}
for(int i = 0; i < vertex.length; i++) {
System.out.println(vertex[i] + ", adj nodes: " + vertex[i].getAdjList());
}
LinkedList[] l = new LinkedList[2];
l = DFS(vertex);
System.out.println("");
System.out.println("Visited Nodes: " + l[0]);
System.out.println("");
System.out.print("Back Edges: ");
for(int i = 0; i < l[1].size(); i++) {
int[] q = (int[])(l[1].get(i));
System.out.println("[" + q[0] + "," + q[1] + "] ");
}
for(int i = 0; i < l[1].size(); i++) { //iterate through the list of back edges
int[] q = (int[])(l[1].get(i)); // q = pair of vertices that make up a back edge
int u = q[0]; // edge (u,v)
int v = q[1];
LinkedList cycle = new LinkedList();
if(l[0].indexOf(u) < l[0].indexOf(v)) { //check if u is before v
for(int z = l[0].indexOf(u); z <= l[0].indexOf(v); z++) { //if it is, look for u first; from u to v
cycle.add(l[0].get(z));
}
}
else if(l[0].indexOf(v) < l[0].indexOf(u)) {
for(int z = l[0].indexOf(v); z <= l[0].indexOf(u); z++) { //if it is, look for u first; from u to v
cycle.add(l[0].get(z));
}
}
System.out.println("");
System.out.println("Cycle detected! : " + cycle);
if((cycle.size() & 1) != 0) {
System.out.println("Cycle is odd, graph is not 2-colorable!");
}
else {
System.out.println("Cycle is even, we're okay!");
}
}
}
catch (IOException e) {
System.out.println("AHHHH");
e.printStackTrace();
}
}
}

The issue is most likely the recursive calls in DFSVisit. If you don't want to go with the 'easy' answer of increasing Java's stack size when you call the JVM, you may want to consider rewriting DFSVisit to use an iterative algorithm instead of recursive. While Depth First Search is more easily defined in a recursive manner, there are iterative approaches to the algorithm that can be used.
For example: this blog post

The stack is a region in memory that is used for storing execution context and passing parameters. Every time your code invokes a method, a little bit of stack is used, and the stack pointer is increased to point to the next available location. When the method returns, the stack pointer is decreased and the portion of the stack is freed up.
If an application uses recursion heavily, the stack quickly becomes a bottleneck, because if there is no limit to the recursion depth, there is no limit to the amount of stack needed. So you have two options: increase the Java stack (-Xss JVM parameter, and this will only help until you hit the new limit) or change your algorithm so that the recursion depth is not as deep.
I am not sure if you were looking for a generic answer, but from a brief glance at your code it appears that your problem is recursion.

If you're sure your algorithm is correct and the depth of recursive calls you're making isn't accidental, then solutions without changing your algorithm are:
add to the JVM command line e.g. -Xss128m to set a 128 MB stack size (not a good solution in multi-threaded programs as it sets the default stack size for every thread not just the particular thread running your task);
run your task in its own thread, which you can initialise with a stack size specific to just that thread (and set the stack size within the program itself)-- see my example in the discussion of fixing StackOverflowError, but essentially the stack size is a parameter to the Thread() constructor;
don't use recursive calls at all-- instead, mimic the recursive calls using an explicit Stack or Queue object (this arguably gives you a bit more control).