Issue resolved.
I've implemented an A* Search algorithm for path-finding in a simple grid-based game. It's my first time doing so, and the implementation works great most of the time. However, sometimes (albeit very rarely) it will get stuck when there is a path available. Of course, the fact that it gets stuck at all makes it not fit for purpose. I assume that I am missing something from my implementation.
I have looked for the issue for several days, to no avail. I have a fast-approaching deadline and a lot of things to do, I'd rather not waste any more time trying to fix this bug.
Edit: I've created a quick video to demonstrate the issue, that way you can see exactly what's going on. It includes captions.
Edit: The getPath method:
/**
* #param currentPosition - the vector the avatar currently occupies.
* #param targetPosition - the vector the avatar is aiming to reach.
* #param levelMap - a clip map of the level.
*
* #return an {#code ArrayList} of {#link ACTIONS} that the avatar can follow to reach its destination.
**/
public static ActionPath getPath(Vector2d currentPosition, Vector2d targetPosition, LevelMap levelMap) {
openTiles = new ArrayList<AStarTile>();
closedTiles = new ArrayList<AStarTile>();
targetMet = false;
AStarTile originTile = AStarTile.fromVector(currentPosition, levelMap.getBlockSize()),
targetTile = AStarTile.fromVector(targetPosition, levelMap.getBlockSize()),
currentTile = null,
parentTile = null;
ActionPath actionPath = new ActionPath(targetTile);
if (originTile.equals(targetTile)) {
targetMet = true;
return null;
}
GVGLogger.logInfo("Creating path from tile " + originTile + " to tile " + targetTile + " (" + currentPosition + " to " + targetPosition + ").");
/*
* Start the search.
*/
openTile(originTile);
originTile.calculateGeneration();// The origin tile will always be generation 0.
closeTile(originTile);
parentTile = originTile;
while(!targetMet) {
for (int i = 0; i != 4; i++) {
currentTile = parentTile.move(i);// Checks an adjacent tile - up, down, left, and right respectively
if (levelMap.inBounds(currentTile) && levelMap.isAccessible(currentTile) && !isClosed(currentTile)) {
if (isOpen(currentTile)) {
// Check to see if this path to this tile is a better one.
currentTile = getOpen(currentTile);
if (currentTile.getGeneration() > parentTile.getGeneration() + 1) {
// The open version's generation is higher than this version's generation - it's a better path
currentTile.setParentTile(parentTile);
currentTile.calculateGeneration();
currentTile.calculateFinalScore();
}
}
else {
currentTile.setParentTile(parentTile);
currentTile.setHeuristic(currentTile.distanceSquared(targetTile));
currentTile.calculateGeneration();
currentTile.calculateFinalScore();
openTile(currentTile);
}
}
}
if (openTiles.size() > 0) {
parentTile = getBestOption();
closeTile(parentTile);
if (parentTile.equals(targetTile)) {
targetMet = true;
}
}
else {
GVGLogger.logWarning("Target unreachable!");
return null;
}
}
//Convert the path of tiles into ACTIONS that the avatar can take to reach it.
for (int i = 0; i != closedTiles.size(); i++) {
Vector2i difference = getDifference(closedTiles.get(i), (i != closedTiles.size() - 1 ? closedTiles.get(i + 1) : targetTile));
if (difference.equals(1, 0)) {
actionPath.add(ACTIONS.ACTION_LEFT);
}
else if (difference.equals(-1, 0)) {
actionPath.add(ACTIONS.ACTION_RIGHT);
}
else if (difference.equals(0, -1)) {
actionPath.add(ACTIONS.ACTION_DOWN);
}
else if (difference.equals(0, 1)) {
actionPath.add(ACTIONS.ACTION_UP);
}
else if (difference.equals(0, 0)) {
return actionPath;
}
else {
GVGLogger.logWarning("Error in path-finding - found a difference of " + difference + "!");
}
}
return null;
}
private static Vector2i getDifference(AStarTile tileA, AStarTile tileB) {
return new Vector2i(tileA.getX() - tileB.getX(), tileA.getY() - tileB.getY());
}
public static boolean targetMet() {
return targetMet;
}
private static void openTile(AStarTile toOpen) {
if (isClosed(toOpen)) {
closedTiles.remove(getOpen(toOpen));
}
if (!isOpen(toOpen)) {
openTiles.add(toOpen);
}
}
private static void closeTile(AStarTile toClose) {
if (isOpen(toClose)) {
openTiles.remove(getOpen(toClose));
}
if (!isClosed(toClose)) {
closedTiles.add(toClose);
}
}
private static boolean isClosed(AStarTile toCheck) {
return getClosed(toCheck) != null;
}
private static boolean isOpen(AStarTile toCheck) {
return getOpen(toCheck) != null;
}
/**
* #return the open tile with the lowest 'final' score.
**/
private static AStarTile getBestOption() {
try {
Collections.sort(openTiles);
return openTiles.get(0);
}
catch(Exception e) {
}
return null;
}
private static AStarTile getClosed(AStarTile t) {
for (AStarTile p : closedTiles) {
if (p.equals(t)) {
return t;
}
}
return null;
}
private static AStarTile getOpen(AStarTile t) {
for (AStarTile p : openTiles) {
if (p.equals(t)) {
return t;
}
}
return null;
}
}
This method returns a list of 'ACTIONS' that the avatar can take to rich the destination tile. If you wish to see any other methods, please ask.
I wrote this implementation after reading an explanation/tutorial by Patrick Lester found at policyalmanac.org ("A* Pathfinding for Beginners").
I'd really appreciate it if you could glance over my implementation and point out any issues, especially if you are experienced with the A* Search algorithm. I think the code is pretty self-documenting, but please feel free to ask me to elaborate on anything if necessary.
Thanks for your time.
A couple of things look suspicious:
One
currentTile.getGeneration() > parentTile.getGeneration() + 1
Should this be >= instead of >?
Two
currentTile.setHeuristic(currentTile.distanceSquared(targetTile));
Squared distance is not an admissible heuristic as it can over-estimate the distance. Try using Manhattan distance (or simply setting the heuristic to 0 for a less efficient but more reliable search).
I've sorted it - I added a check on the generation when closing a tile. It now backtracks slightly instead of getting stuck.
Solution:
if (!closedTiles.isEmpty() && toClose.getGeneration() != lastClosed().getGeneration() + 1) {
openTiles.remove(getOpen(toClose));
return;
}
I'm so glad I've fixed it. Big thanks to anybody who took the time to read and/or answer the question! :)
Related
I am attempting to find the shortest path in a route between two points. My adjacency list seems to be correct but the breadth-first search seems to be messing up somewhere. If no path exists between two points in the graph, "Not found" is printed. My code never seems to enter this for some reason, even if it should. My basic understanding of the BST algorithm is making this problem extremely hard to diagnose. I've spent countless hours modifying the code and watching videos but have remained unsuccessful.
I am reading the route data from a text. This part is working perfectly, therefore I feel like it would be redundant to include. What I will say is that the adjacency list my graph code creates looks correct, so it's likely an issue with my BFS function.
class Graph {
// We use Hashmap to store the edges in the graph
private Map<String, List<String> > map = new HashMap<>();
public void BFS(String start, String stop) {
Queue<String> queue = new ArrayDeque<>();
HashSet<String> seen = new HashSet<>();
ArrayList<String> network = new ArrayList<>();
queue.add(start);
while(0 != queue.size()){
String vertex = queue.poll();
if(!seen.contains(vertex)){
network.add(vertex);
queue.addAll(map.get(vertex)); // Add all neighbors of 'vertex' to the queue
seen.add(vertex);
}
}
if (network.contains(stop)) {
System.out.println("Route Path: ");
for(String location: network) {
if (location.equals(stop)) {
System.out.println(location);
break;
} else {
System.out.println(location + " -> ");
}
}
} else {
System.out.println("Not found.");
}
}
public void printMap() {
for(String item: map.keySet()) {
System.out.println(map.get(item));
}
}
// This function adds a new vertex to the graph
public void addVertex(String s)
{
map.put(s, new LinkedList<String>());
}
// This function adds the edge
// between source to destination
public void addEdge(String source,
String destination,
boolean bidirectional)
{
if (!map.containsKey(source))
addVertex(source);
if (!map.containsKey(destination))
addVertex(destination);
map.get(source).add(destination);
if (bidirectional == true) {
map.get(destination).add(source);
}
}
// This function gives the count of vertices
public void getVertexCount()
{
System.out.println("The graph has "
+ map.keySet().size()
+ " vertex");
}
// This function gives the count of edges
public void getEdgesCount(boolean bidirection)
{
int count = 0;
for (String v : map.keySet()) {
count += map.get(v).size();
}
if (bidirection == true) {
count = count / 2;
}
System.out.println("The graph has "
+ count
+ " edges.");
}
// This function gives whether
// a vertex is present or not.
public boolean hasVertex(String s)
{
if (map.containsKey(s)) {
return true;
}
else {
return false;
}
}
// This function gives whether an edge is present or not.
public boolean hasEdge(String s, String d)
{
if (map.get(s).contains(d)) {
return true;
}
else {
return false;
}
}
// Prints the adjancency list of each vertex.
#Override
public String toString()
{
StringBuilder builder = new StringBuilder();
for (String v : map.keySet()) {
builder.append(v.toString() + ": ");
for (String w : map.get(v)) {
builder.append(w.toString() + " ");
}
builder.append("\n");
}
return (builder.toString());
}
If my question is lacking in any way, please provide constructive feedback so I can make better posts in the future. I can refine my post and supply further information if needed.
Is it possible to convert the function go into the non-recursive function? Some hints or a start-up sketch would be very helpful
public static TSPSolution solve(CostMatrix _cm, TSPPoint start, TSPPoint[] points, long seed) {
TSPSolution sol = TSPSolution.randomSolution(start, points, seed, _cm);
double t = initialTemperature(sol, 1000);
int frozen = 0;
System.out.println("-- Simulated annealing started with initial temperature " + t + " --");
return go(_cm, sol, t, frozen);
}
private static TSPSolution go(CostMatrix _cm, TSPSolution solution, double t, int frozen) {
if (frozen >= 3) {
return solution;
}
i++;
TSPSolution bestSol = solution;
System.out.println(i + ": " + solution.fitness() + " " + solution.time() + " "
+ solution.penalty() + " " + t);
ArrayList<TSPSolution> nHood = solution.nHood();
int attempts = 0;
int accepted = 0;
while (!(attempts == 2 * nHood.size() || accepted == nHood.size()) && attempts < 500) {
TSPSolution sol = nHood.get(rand.nextInt(nHood.size()));
attempts++;
double deltaF = sol.fitness() - bestSol.fitness();
if (deltaF < 0 || Math.exp(-deltaF / t) > Math.random()) {
accepted++;
bestSol = sol;
nHood = sol.nHood();
}
}
frozen = accepted == 0 ? frozen + 1 : 0;
double newT = coolingSchedule(t);
return go(_cm, bestSol, newT, frozen);
}
This is an easy one, because it is tail-recursive: there is no code between the recursive call & what the function returns. Thus, you can wrap the body of go in a loop while (frozen<3), and return solution once the loop ends. And replace the recursive call with assignments to the parameters: solution=bestSol; t=newT;.
You need to thinkg about two things:
What changes on each step?
When does the algorithm end?
Ans the answer should be
bestSol (solution), newT (t), frozen (frozen)
When frozen >= 3 is true
So, the easiest way is just to enclose the whole function in something like
while (frozen < 3) {
...
...
...
frozen = accepted == 0 ? frozen + 1 : 0;
//double newT = coolingSchedule(t);
t = coolingSchedule(t);
solution = bestSol;
}
As a rule of thumb, the simplest way to make a recursive function iterative is to load the first element onto a Stack, and instead of calling the recursion, add the result to the Stack.
For instance:
public Item recursive(Item myItem)
{
if(myItem.GetExitCondition().IsMet()
{
return myItem;
}
... do stuff ...
return recursive(myItem);
}
Would become:
public Item iterative(Item myItem)
{
Stack<Item> workStack = new Stack<>();
while (!workStack.isEmpty())
{
Item workItem = workStack.pop()
if(myItem.GetExitCondition().IsMet()
{
return workItem;
}
... do stuff ...
workStack.put(workItem)
}
// No solution was found (!).
return myItem;
}
This code is untested and may (read: does) contain errors. It may not even compile, but should give you a general idea.
I'm creating a program in Java that solves the n-puzzle, without using heuristics, simply just with depth-first and breadth-first searches of the state space. I'm struggling a little bit with my implementation of depth-first search. Sometimes it will solve the given puzzle, but other times it seems to give up early.
Here's my DFS class. DepthFirstSearch() is passed a PuzzleBoard, which is initially generated by shuffling a solved board (to ensure that the board is in a solvable state).
public class DepthFirst {
static HashSet<PuzzleBoard> usedStates = new HashSet<PuzzleBoard>();
public static void DepthFirstSearch(PuzzleBoard currentBoard)
{
// If the current state is the goal, stop.
if (PuzzleSolver.isGoal(currentBoard)) {
System.out.println("Solved!");
System.exit(0);
}
// If we haven't encountered the state before,
// attempt to find a solution from that point.
if (!usedStates.contains(currentBoard)) {
usedStates.add(currentBoard);
PuzzleSolver.print(currentBoard);
if (PuzzleSolver.blankCoordinates(currentBoard)[1] != 0) {
System.out.println("Moving left");
DepthFirstSearch(PuzzleSolver.moveLeft(currentBoard));
}
if (PuzzleSolver.blankCoordinates(currentBoard)[0] != PuzzleSolver.n-1) {
System.out.println("Moving down");
DepthFirstSearch(PuzzleSolver.moveDown(currentBoard));
}
if (PuzzleSolver.blankCoordinates(currentBoard)[1] != PuzzleSolver.n-1) {
System.out.println("Moving right");
DepthFirstSearch(PuzzleSolver.moveRight(currentBoard));
}
if (PuzzleSolver.blankCoordinates(currentBoard)[0] != 0) {
System.out.println("Moving up");
DepthFirstSearch(PuzzleSolver.moveUp(currentBoard));
}
return;
} else {
// Move up a level in the recursive calls
return;
}
}
}
I can assert that my moveUp(), moveLeft(), moveRight(), and moveDown() methods and logic work correctly, so the problem must lie somewhere else.
Here's my PuzzleBoard object class with the hashCode and equals methods:
static class PuzzleBoard {
short[][] state;
/**
* Default constructor for a board of size n
* #param n Size of the board
*/
public PuzzleBoard(short n) {
state = PuzzleSolver.getGoalState(n);
}
public PuzzleBoard(short n, short[][] initialState) {
state = initialState;
}
#Override
public int hashCode() {
final int prime = 31;
int result = 1;
result = prime * result + Arrays.deepHashCode(state);
return result;
}
#Override
public boolean equals(Object obj) {
if (this == obj) {
return true;
}
if (obj == null) {
return false;
}
if (getClass() != obj.getClass()) {
return false;
}
PuzzleBoard other = (PuzzleBoard) obj;
for (int i = 0; i < n; i++) {
for (int j = 0; j < n; j++) {
if (state[i][j] != other.state[i][j])
return false;
}
}
return true;
}
}
As previously stated, sometimes the search works properly and finds a path to the solution, but other times it stops before it finds a solution and before it runs out of memory.
Here is a snippet of the output, beginning a few moves before the search stops searching.
...
Moving down
6 1 3
5 8 2
0 7 4
Moving right
6 1 3
5 8 2
7 0 4
Moving left
Moving right
Moving up
6 1 3
5 0 2
7 8 4
Moving left
Moving down
Moving right
Moving up
Moving up
Moving right
Moving down
Moving up
Moving down
Moving up
Moving down
Moving up
Moving down
Moving up
Moving down
...
I truncated it early for brevity, but it ends up just moving up and down dozens of times and never hits the solved state.
Can anyone shed light on what I'm doing wrong?
Edit: Here is MoveUp(). The rest of the move methods are implemented in the same way.
/**
* Move the blank space up
* #return The new state of the board after the move
*/
static PuzzleBoard moveUp(PuzzleBoard currentState) {
short[][] newState = currentState.state;
short col = blankCoordinates(currentState)[0];
short row = blankCoordinates(currentState)[1];
short targetCol = col;
short targetRow = row;
newState[targetCol][targetRow] = currentState.state[col - 1][row];
newState[targetCol - 1][targetRow] = 0;
return new PuzzleBoard(n, newState);
}
I have had many problems with hashset in the past best thing to try is not to store object in hashset but try to encode your object into string.
Here is a way to do it:-
StringBuffer encode(PuzzleBoard b) {
StringBuffer buff = new StringBuffer();
for(int i=0;i<b.n;i++) {
for(int j=0;j<b.n;j++) {
// "," is used as separator
buff.append(","+b.state[i][j]);
}
}
return buff;
}
Make two changes in the code:-
if(!usedStates.contains(encode(currentBoard))) {
usedStates.add(encode(currentBoard));
......
}
Note:- Here no need to write your own hashcode function & also no need to implement equals function as java has done it for you in StringBuffer.
I got one of the problems in your implementation:-
In th following code:-
static PuzzleBoard moveUp(PuzzleBoard currentState) {
short[][] newState = currentState.state;
short col = blankCoordinates(currentState)[0];
short row = blankCoordinates(currentState)[1];
short targetCol = col;
short targetRow = row;
newState[targetCol][targetRow] = currentState.state[col - 1][row];
newState[targetCol - 1][targetRow] = 0;
return new PuzzleBoard(n, newState);
}
Here you are using the reference of same array as newState from currentState.state so when you make changes to newState your currentState.state will also change which will affect DFS when the call returns. To prevent that you should initialize a new array. Heres what to be done:-
static PuzzleBoard moveUp(PuzzleBoard currentState) {
short[][] newState = new short[n][n];
short col = blankCoordinates(currentState)[0];
short row = blankCoordinates(currentState)[1];
short targetCol = col;
short targetRow = row;
for(int i=0;i<n;i++) {
for(int j=0;j<n;j++) {
newState[i][j] = currentState.state[i][j];
}
}
newState[targetCol][targetRow] = currentState.state[col - 1][row];
newState[targetCol - 1][targetRow] = 0;
return new PuzzleBoard(n, newState);
}
Do this change for all moveup,movedown....
Moreover I donot think your hashset is working properly because if it was then you would always find your new state in hashset and your program would stop. As in equals you comparing the state arrays with same reference hence will always get true. Please try and use my encode function as hash.
I don't know if this is possible but i'm trying to make an Hashtable of where Interval is a class with 2 integer / long values, a start and an end and i wanted to make something like this:
Hashtable<Interval, WhateverObject> test = new Hashtable<Interval, WhateverObject>();
test.put(new Interval(100, 200), new WhateverObject());
test.get(new Interval(150, 150)) // returns the new WhateverObject i created above because 150 is betwwen 100 and 200
test.get(new Interval(250, 250)) // doesn't find the value because there is no key that contains 250 in it's interval
So basically what i want is that a key between a range of values in an Interval object give the correspondent WhateverObject. I know i have to override equals() and hashcode() in the interval object, the main problem i think is to somehow have all the values between 100 and 200 (in this specific example) to give the same hash.
Any ideias if this is possible?
Thanks
No need to reinvent the wheel, use a NavigableMap. Example Code:
final NavigableMap<Integer, String> map = new TreeMap<Integer, String>();
map.put(0, "Cry Baby");
map.put(6, "School Time");
map.put(16, "Got a car yet?");
map.put(21, "Tequila anyone?");
map.put(45, "Time to buy a corvette");
System.out.println(map.floorEntry(3).getValue());
System.out.println(map.floorEntry(10).getValue());
System.out.println(map.floorEntry(18).getValue());
Output:
Cry Baby
School Time
Got a car yet?
You could use an IntervalTree. Here's one I made earlier.
public class IntervalTree<T extends IntervalTree.Interval> {
// My intervals.
private final List<T> intervals;
// My center value. All my intervals contain this center.
private final long center;
// My interval range.
private final long lBound;
private final long uBound;
// My left tree. All intervals that end below my center.
private final IntervalTree<T> left;
// My right tree. All intervals that start above my center.
private final IntervalTree<T> right;
public IntervalTree(List<T> intervals) {
if (intervals == null) {
throw new NullPointerException();
}
// Initially, my root contains all intervals.
this.intervals = intervals;
// Find my center.
center = findCenter();
/*
* Builds lefts out of all intervals that end below my center.
* Builds rights out of all intervals that start above my center.
* What remains contains all the intervals that contain my center.
*/
// Lefts contains all intervals that end below my center point.
final List<T> lefts = new ArrayList<T>();
// Rights contains all intervals that start above my center point.
final List<T> rights = new ArrayList<T>();
long uB = Long.MIN_VALUE;
long lB = Long.MAX_VALUE;
for (T i : intervals) {
long start = i.getStart();
long end = i.getEnd();
if (end < center) {
lefts.add(i);
} else if (start > center) {
rights.add(i);
} else {
// One of mine.
lB = Math.min(lB, start);
uB = Math.max(uB, end);
}
}
// Remove all those not mine.
intervals.removeAll(lefts);
intervals.removeAll(rights);
uBound = uB;
lBound = lB;
// Build the subtrees.
left = lefts.size() > 0 ? new IntervalTree<T>(lefts) : null;
right = rights.size() > 0 ? new IntervalTree<T>(rights) : null;
// Build my ascending and descending arrays.
/** #todo Build my ascending and descending arrays. */
}
/*
* Returns a list of all intervals containing the point.
*/
List<T> query(long point) {
// Check my range.
if (point >= lBound) {
if (point <= uBound) {
// In my range but remember, there may also be contributors from left or right.
List<T> found = new ArrayList<T>();
// Gather all intersecting ones.
// Could be made faster (perhaps) by holding two sorted lists by start and end.
for (T i : intervals) {
if (i.getStart() <= point && point <= i.getEnd()) {
found.add(i);
}
}
// Gather others.
if (point < center && left != null) {
found.addAll(left.query(point));
}
if (point > center && right != null) {
found.addAll(right.query(point));
}
return found;
} else {
// To right.
return right != null ? right.query(point) : Collections.<T>emptyList();
}
} else {
// To left.
return left != null ? left.query(point) : Collections.<T>emptyList();
}
}
private long findCenter() {
//return average();
return median();
}
protected long median() {
// Choose the median of all centers. Could choose just ends etc or anything.
long[] points = new long[intervals.size()];
int x = 0;
for (T i : intervals) {
// Take the mid point.
points[x++] = (i.getStart() + i.getEnd()) / 2;
}
Arrays.sort(points);
return points[points.length / 2];
}
/*
* What an interval looks like.
*/
public interface Interval {
public long getStart();
public long getEnd();
}
/*
* A simple implemementation of an interval.
*/
public static class SimpleInterval implements Interval {
private final long start;
private final long end;
public SimpleInterval(long start, long end) {
this.start = start;
this.end = end;
}
public long getStart() {
return start;
}
public long getEnd() {
return end;
}
#Override
public String toString() {
return "{" + start + "," + end + "}";
}
}
}
A naive HashTable is the wrong solution here. Overriding the equals() method doesn't do you any good because the HashTable compares a key entry by the hash code first, NOT the equals() method. The equals() method is only checked AFTER the hash code is matched.
It's easy to make a hash function on your interval object, but it's much more difficult to make one that would yield the same hashcode for all possible intervals that would be within another interval. Overriding the get() method (such as here https://stackoverflow.com/a/11189075/1261844) for a HashTable completely negates the advantages of a HashTable, which is very fast lookup times. At the point where you are scanning through each member of a HashTable, then you know you are using the HashTable incorrectly.
I'd say that Using java map for range searches and https://stackoverflow.com/a/11189080/1261844 are better solutions, but a HashTable is simply not the way to go about this.
I think implementing a specialized get-method would be much easier.
The new method can be part of a map-wrapper-class.
The key-class: (interval is [lower;upper[ )
public class Interval {
private int upper;
private int lower;
public Interval(int upper, int lower) {
this.upper = upper;
this.lower = lower;
}
public boolean contains(int i) {
return i < upper && i >= lower;
}
#Override
public boolean equals(Object obj) {
if (obj == null) {
return false;
}
if (getClass() != obj.getClass()) {
return false;
}
final Interval other = (Interval) obj;
if (this.upper != other.upper) {
return false;
}
if (this.lower != other.lower) {
return false;
}
return true;
}
#Override
public int hashCode() {
int hash = 5;
hash = 61 * hash + this.upper;
hash = 61 * hash + this.lower;
return hash;
}
}
The Map-class:
public class IntervalMap<T> extends HashMap<Interval, T> {
public T get(int key) {
for (Interval iv : keySet()) {
if (iv.contains(key)) {
return super.get(iv);
}
}
return null;
}
}
This is just an example and can surely be optimized, and there are a few flaws as well:
For Example if Intervals overlap, there's no garantee to know which Interval will be used for lookup and Intervals are not garanteed to not overlap!
OldCurmudgeon's solution works perfectly for me, but is very slow to initialise (took 20 mins for 70k entries).
If you know your incoming list of items is already ordered (ascending) and has only non overlapping intervals, you can make it initialise in milliseconds by adding and using the following constructor:
public IntervalTree(List<T> intervals, boolean constructorFlagToIndicateOrderedNonOverlappingIntervals) {
if (intervals == null) throw new NullPointerException();
int centerPoint = intervals.size() / 2;
T centerInterval = intervals.get(centerPoint);
this.intervals = new ArrayList<T>();
this.intervals.add(centerInterval);
this.uBound = centerInterval.getEnd();
this.lBound = centerInterval.getStart();
this.center = (this.uBound + this.lBound) / 2;
List<T> toTheLeft = centerPoint < 1 ? Collections.<T>emptyList() : intervals.subList(0, centerPoint);
this.left = toTheLeft.isEmpty() ? null : new IntervalTree<T>(toTheLeft, true);
List<T> toTheRight = centerPoint >= intervals.size() ? Collections.<T>emptyList() : intervals.subList(centerPoint+1, intervals.size());
this.right = toTheRight.isEmpty() ? null : new IntervalTree<T>(toTheRight, true);
}
This depends on your hashCode implementation. You may have two Objects with the same hashCode value.
Please use eclipse to generate a hashCode method for your class (no point to re-invent the wheel
For Hastable or HashMap to work as expected it's not only a equal hashcode, but also the equals method must return true. What you are requesting is that Interval(x, y).equals(Interval(m, n)) for m, n within x,y. As this must be true for any overlapping living instance of Interval, the class has to record all of them and needs to implement what you are trying to achieve, indeed.
So in short the answer is no.
The Google guava library is planning to offer a RangeSet and Map: guava RangeSet
For reasonable small ranges an easy approach would be to specialize HashMap by putting and getting the indivual values of the intervals.
Consider a few web server instances running in parallel. Each server holds a reference to a single shared "Status keeper", whose role is keeping the last N requests from all servers.
For example (N=3):
Server a: "Request id = ABCD" Status keeper=["ABCD"]
Server b: "Request id = XYZZ" Status keeper=["ABCD", "XYZZ"]
Server c: "Request id = 1234" Status keeper=["ABCD", "XYZZ", "1234"]
Server b: "Request id = FOO" Status keeper=["XYZZ", "1234", "FOO"]
Server a: "Request id = BAR" Status keeper=["1234", "FOO", "BAR"]
At any point in time, the "Status keeper" might be called from a monitoring application that reads these last N requests for an SLA report.
What's the best way to implement this producer-consumer scenario in Java, giving the web servers higher priority than the SLA report?
CircularFifoBuffer seems to be the appropriate data structure to hold the requests, but I'm not sure what's the optimal way to implement efficient concurrency.
Buffer fifo = BufferUtils.synchronizedBuffer(new CircularFifoBuffer());
Here's a lock-free ring buffer implementation. It implements a fixed-size buffer - there is no FIFO functionality. I would suggest you store a Collection of requests for each server instead. That way your report can do the filtering rather than getting your data structure to filter.
/**
* Container
* ---------
*
* A lock-free container that offers a close-to O(1) add/remove performance.
*
*/
public class Container<T> implements Iterable<T> {
// The capacity of the container.
final int capacity;
// The list.
AtomicReference<Node<T>> head = new AtomicReference<Node<T>>();
// TESTING {
AtomicLong totalAdded = new AtomicLong(0);
AtomicLong totalFreed = new AtomicLong(0);
AtomicLong totalSkipped = new AtomicLong(0);
private void resetStats() {
totalAdded.set(0);
totalFreed.set(0);
totalSkipped.set(0);
}
// TESTING }
// Constructor
public Container(int capacity) {
this.capacity = capacity;
// Construct the list.
Node<T> h = new Node<T>();
Node<T> it = h;
// One created, now add (capacity - 1) more
for (int i = 0; i < capacity - 1; i++) {
// Add it.
it.next = new Node<T>();
// Step on to it.
it = it.next;
}
// Make it a ring.
it.next = h;
// Install it.
head.set(h);
}
// Empty ... NOT thread safe.
public void clear() {
Node<T> it = head.get();
for (int i = 0; i < capacity; i++) {
// Trash the element
it.element = null;
// Mark it free.
it.free.set(true);
it = it.next;
}
// Clear stats.
resetStats();
}
// Add a new one.
public Node<T> add(T element) {
// Get a free node and attach the element.
totalAdded.incrementAndGet();
return getFree().attach(element);
}
// Find the next free element and mark it not free.
private Node<T> getFree() {
Node<T> freeNode = head.get();
int skipped = 0;
// Stop when we hit the end of the list
// ... or we successfully transit a node from free to not-free.
while (skipped < capacity && !freeNode.free.compareAndSet(true, false)) {
skipped += 1;
freeNode = freeNode.next;
}
// Keep count of skipped.
totalSkipped.addAndGet(skipped);
if (skipped < capacity) {
// Put the head as next.
// Doesn't matter if it fails. That would just mean someone else was doing the same.
head.set(freeNode.next);
} else {
// We hit the end! No more free nodes.
throw new IllegalStateException("Capacity exhausted.");
}
return freeNode;
}
// Mark it free.
public void remove(Node<T> it, T element) {
totalFreed.incrementAndGet();
// Remove the element first.
it.detach(element);
// Mark it as free.
if (!it.free.compareAndSet(false, true)) {
throw new IllegalStateException("Freeing a freed node.");
}
}
// The Node class. It is static so needs the <T> repeated.
public static class Node<T> {
// The element in the node.
private T element;
// Are we free?
private AtomicBoolean free = new AtomicBoolean(true);
// The next reference in whatever list I am in.
private Node<T> next;
// Construct a node of the list
private Node() {
// Start empty.
element = null;
}
// Attach the element.
public Node<T> attach(T element) {
// Sanity check.
if (this.element == null) {
this.element = element;
} else {
throw new IllegalArgumentException("There is already an element attached.");
}
// Useful for chaining.
return this;
}
// Detach the element.
public Node<T> detach(T element) {
// Sanity check.
if (this.element == element) {
this.element = null;
} else {
throw new IllegalArgumentException("Removal of wrong element.");
}
// Useful for chaining.
return this;
}
public T get () {
return element;
}
#Override
public String toString() {
return element != null ? element.toString() : "null";
}
}
// Provides an iterator across all items in the container.
public Iterator<T> iterator() {
return new UsedNodesIterator<T>(this);
}
// Iterates across used nodes.
private static class UsedNodesIterator<T> implements Iterator<T> {
// Where next to look for the next used node.
Node<T> it;
int limit = 0;
T next = null;
public UsedNodesIterator(Container<T> c) {
// Snapshot the head node at this time.
it = c.head.get();
limit = c.capacity;
}
public boolean hasNext() {
// Made into a `while` loop to fix issue reported by #Nim in code review
while (next == null && limit > 0) {
// Scan to the next non-free node.
while (limit > 0 && it.free.get() == true) {
it = it.next;
// Step down 1.
limit -= 1;
}
if (limit != 0) {
next = it.element;
}
}
return next != null;
}
public T next() {
T n = null;
if ( hasNext () ) {
// Give it to them.
n = next;
next = null;
// Step forward.
it = it.next;
limit -= 1;
} else {
// Not there!!
throw new NoSuchElementException ();
}
return n;
}
public void remove() {
throw new UnsupportedOperationException("Not supported.");
}
}
#Override
public String toString() {
StringBuilder s = new StringBuilder();
Separator comma = new Separator(",");
// Keep counts too.
int usedCount = 0;
int freeCount = 0;
// I will iterate the list myself as I want to count free nodes too.
Node<T> it = head.get();
int count = 0;
s.append("[");
// Scan to the end.
while (count < capacity) {
// Is it in-use?
if (it.free.get() == false) {
// Grab its element.
T e = it.element;
// Is it null?
if (e != null) {
// Good element.
s.append(comma.sep()).append(e.toString());
// Count them.
usedCount += 1;
} else {
// Probably became free while I was traversing.
// Because the element is detached before the entry is marked free.
freeCount += 1;
}
} else {
// Free one.
freeCount += 1;
}
// Next
it = it.next;
count += 1;
}
// Decorate with counts "]used+free".
s.append("]").append(usedCount).append("+").append(freeCount);
if (usedCount + freeCount != capacity) {
// Perhaps something was added/freed while we were iterating.
s.append("?");
}
return s.toString();
}
}
Note that this is close to O1 put and get. A Separator just emits "" first time around and then its parameter from then on.
Edit: Added test methods.
// ***** Following only needed for testing. *****
private static boolean Debug = false;
private final static String logName = "Container.log";
private final static NamedFileOutput log = new NamedFileOutput("C:\\Junk\\");
private static synchronized void log(boolean toStdoutToo, String s) {
if (Debug) {
if (toStdoutToo) {
System.out.println(s);
}
log(s);
}
}
private static synchronized void log(String s) {
if (Debug) {
try {
log.writeLn(logName, s);
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
static volatile boolean testing = true;
// Tester object to exercise the container.
static class Tester<T> implements Runnable {
// My name.
T me;
// The container I am testing.
Container<T> c;
public Tester(Container<T> container, T name) {
c = container;
me = name;
}
private void pause() {
try {
Thread.sleep(0);
} catch (InterruptedException ex) {
testing = false;
}
}
public void run() {
// Spin on add/remove until stopped.
while (testing) {
// Add it.
Node<T> n = c.add(me);
log("Added " + me + ": " + c.toString());
pause();
// Remove it.
c.remove(n, me);
log("Removed " + me + ": " + c.toString());
pause();
}
}
}
static final String[] strings = {
"One", "Two", "Three", "Four", "Five",
"Six", "Seven", "Eight", "Nine", "Ten"
};
static final int TEST_THREADS = Math.min(10, strings.length);
public static void main(String[] args) throws InterruptedException {
Debug = true;
log.delete(logName);
Container<String> c = new Container<String>(10);
// Simple add/remove
log(true, "Simple test");
Node<String> it = c.add(strings[0]);
log("Added " + c.toString());
c.remove(it, strings[0]);
log("Removed " + c.toString());
// Capacity test.
log(true, "Capacity test");
ArrayList<Node<String>> nodes = new ArrayList<Node<String>>(strings.length);
// Fill it.
for (int i = 0; i < strings.length; i++) {
nodes.add(i, c.add(strings[i]));
log("Added " + strings[i] + " " + c.toString());
}
// Add one more.
try {
c.add("Wafer thin mint!");
} catch (IllegalStateException ise) {
log("Full!");
}
c.clear();
log("Empty: " + c.toString());
// Iterate test.
log(true, "Iterator test");
for (int i = 0; i < strings.length; i++) {
nodes.add(i, c.add(strings[i]));
}
StringBuilder all = new StringBuilder ();
Separator sep = new Separator(",");
for (String s : c) {
all.append(sep.sep()).append(s);
}
log("All: "+all);
for (int i = 0; i < strings.length; i++) {
c.remove(nodes.get(i), strings[i]);
}
sep.reset();
all.setLength(0);
for (String s : c) {
all.append(sep.sep()).append(s);
}
log("None: " + all.toString());
// Multiple add/remove
log(true, "Multi test");
for (int i = 0; i < strings.length; i++) {
nodes.add(i, c.add(strings[i]));
log("Added " + strings[i] + " " + c.toString());
}
log("Filled " + c.toString());
for (int i = 0; i < strings.length - 1; i++) {
c.remove(nodes.get(i), strings[i]);
log("Removed " + strings[i] + " " + c.toString());
}
c.remove(nodes.get(strings.length - 1), strings[strings.length - 1]);
log("Empty " + c.toString());
// Multi-threaded add/remove
log(true, "Threads test");
c.clear();
for (int i = 0; i < TEST_THREADS; i++) {
Thread t = new Thread(new Tester<String>(c, strings[i]));
t.setName("Tester " + strings[i]);
log("Starting " + t.getName());
t.start();
}
// Wait for 10 seconds.
long stop = System.currentTimeMillis() + 10 * 1000;
while (System.currentTimeMillis() < stop) {
Thread.sleep(100);
}
// Stop the testers.
testing = false;
// Wait some more.
Thread.sleep(1 * 100);
// Get stats.
double added = c.totalAdded.doubleValue();
double skipped = c.totalSkipped.doubleValue();
//double freed = c.freed.doubleValue();
log(true, "Stats: added=" + c.totalAdded + ",freed=" + c.totalFreed + ",skipped=" + c.totalSkipped + ",O(" + ((added + skipped) / added) + ")");
}
Maybe you want to look at Disruptor - Concurrent Programming Framework.
Find a paper describing the alternatives, design and also a performance comparement to java.util.concurrent.ArrayBlockingQueue here: pdf
Consider to read the first three articles from BlogsAndArticles
If the library is too much, stick to java.util.concurrent.ArrayBlockingQueue
I would have a look at ArrayDeque, or for a more concurrent implementation have a look at the Disruptor library which is one of the most sophisticated/complex ring buffer in Java.
An alternative is to use an unbounded queue which is more concurrent as the producer never needs to wait for the consumer. Java Chronicle
Unless your needs justify the complexity, an ArrayDeque may be all you need.
Also have a look at java.util.concurrent.
Blocking queues will block until there is something to consume or (optionally) space to produce:
http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/BlockingQueue.html
Concurrent linked queue is non-blocking and uses a slick algorithm that allows a producer and consumer to be active concurrently:
http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/ConcurrentLinkedQueue.html
Hazelcast's Queue offers almost everything you ask for, but doesn't support circularity. But from your description I am not sure if you actually need it.
If it were me, I would use the CircularFIFOBuffer as you indicated, and synchronize around the buffer when writing (add). When the monitoring application wants to read the buffer, synchronize on the buffer, and then copy or clone it to use for reporting.
This suggestion is predicated on the assumption that latency is minimal to copy/clone the buffer to a new object. If there are large number of elements, and copying time is slow, then this is not a good idea.
Pseudo-Code example:
public void writeRequest(String requestID) {
synchronized(buffer) {
buffer.add(requestID);
}
}
public Collection<String> getRequests() {
synchronized(buffer) {
return buffer.clone();
}
}
Since you specifically ask to give writers (that is web servers) higher priority than the reader (that is monitoring), I would suggest the following design.
Web servers add request information to a concurrent queue which is read by a dedicated thread, which adds requests to a thread-local (therefore non-synchronized) queue that overwrites the oldest element, like EvictingQueue or CircularFifoQueue.
This same thread checks a flag which indicates if a report has been requested after every request processed, and if positive, produces a report from all elements present in the thread-local queue.