I was tasked to perform the Dijkstra Algorithm on big graphs (25 million nodes). These are represented as a 2D array: -each node as a double[] with latitude, longitude and offset (offset meaning index of the first outgoing edge of that node)
-each edge as a int[] with sourceNodeId,targetNodeId and weight of that edge
Below is the code, I used int[] as a tupel for the comparison in the priority queue.
The algorithm is working and gets the right results HOWEVER it is required to be finished in 15s but takes like 8min on my laptop. Is my algorithm fundamentally slow? Am I using the wrong data structures? Am I missing something? I tried my best optimizing as far as I saw fit.
Any help or any ideas would be greatly appreciated <3
public static int[] oneToAllArray(double[][]nodeList, int[][]edgeList,int sourceNodeId) {
int[] distance = new int[nodeList[0].length]; //the array that will be returned
//the priorityQueue will use arrays with the length 2, representing [index, weight] for each node and order them by their weight
PriorityQueue<int[]> prioQueue = new PriorityQueue<>((a, b) -> ((int[])a)[1] - ((int[])b)[1]);
int offset1; //used for determining the amount of outgoing edges
int offset2;
int newWeight; //declared here so we dont need to declare it a lot of times later (not sure if that makes a difference)
//currentSourceNode here means the node that will be looked at for OUTGOING edges
int[] currentSourceNode= {sourceNodeId,0};
prioQueue.add(currentSourceNode);
//at the start we only add the sourceNode, then we start the actual algorithm
while(!prioQueue.isEmpty()) {
if(prioQueue.size() % 55 == 2) {
System.out.println(prioQueue.size());
}
currentSourceNode=prioQueue.poll();
int sourceIndex = currentSourceNode[0];
if(sourceIndex == nodeList[0].length-1) {
offset1= (int) nodeList[2][sourceIndex];
offset2= edgeList[0].length;
} else {
offset1= (int) nodeList[2][sourceIndex];
offset2= (int) nodeList[2][sourceIndex+1];
}
//checking every outgoing edge for the currentNode
for(int i=offset1;i<offset2;i++) {
int targetIndex = edgeList[1][i];
//if the node hasnt been looked at yet, the weight is just the weight of this edge + distance to sourceNode
if(distance[targetIndex]==0&&targetIndex!=sourceNodeId) {
distance[targetIndex] = distance[sourceIndex] + edgeList[2][i];
int[]targetArray = {targetIndex, distance[targetIndex]};
prioQueue.add(targetArray);
} else if(prioQueue.stream().anyMatch(e -> e[0]==targetIndex)) {
//above else if checks if this index is already in the prioQueue
newWeight=distance[sourceIndex]+edgeList[2][i];
//if new weight is better, we have to update the distance + the prio queue
if(newWeight<distance[targetIndex]) {
distance[targetIndex]=newWeight;
int[] targetArray;
targetArray=prioQueue.stream().filter(e->e[0]==targetIndex).toList().get(0);
prioQueue.remove(targetArray);
targetArray[1]=newWeight;
prioQueue.add(targetArray);
}
}
}
}
return distance;
}
For each node that you process, you are doing a linear scan of the priority queue to see if something is already queued, and a second scan to find all the things that are queued if you have to update the distance. Instead, keep a separate multi-set of things that are in the queue.
This is not a proper Dijkstra's implementation.
One of the key elements of Dijkstra is that you mark nodes as "visited" when they have been evaluated and prevent looking at them again because you can't do any better. You are not doing that, so your algorithm is doing many many more computations than necessary. The only place where a priority queue or sort is required is to pick the next node to visit, from amongst the unvisited. You should re-read the algorithm, implement the "visitation tracking" and re-formulate.
I have an ArrayList of colors and their frequency of appearance. My program should calculate a reordering of those items that maximizes the minimum distance between two equal bricks.
For example, given input consisting of 4*brick 1 (x), 3*brick 2 (y), and 5*brick 3 (z), one correct output would be: z y x z x z y x z x y.
My code does not produce good solutions. In particular, sometimes there are 2 equal bricks at the end, which is the worst case.
import java.util.ArrayList;
import java.util.Collections;
public class Calc {
// private ArrayList<Wimpel> w = new ArrayList<Brick>();
private String bKette = "";
public String bestOrder(ArrayList<Brick> w) {
while (!w.isEmpty()) {
if (w.get(0).getFrequency() > 0) {
bChain += w.get(0).getColor() + "|";
Brick brick = new Wimpel(w.get(0).getVariant(), w.get(0).getFrequency() - 1);
w.remove(0);
w.add(brick);
// bestOrder(w);
} else {
w.remove(0);
}
bestOrder(w);
}
return bOrder;
}
public int Solutions(ArrayList<Wimpel> w) {
ArrayList<Brick> tmp = new ArrayList<Brick>(w);
int l = 1;
int counter = (int) w.stream().filter(c -> Collections.max(tmp).getFrequency() == c.getFrequency()).count();
l = (int) (fakultaet(counter) * fakultaet((tmp.size() - counter)));
return l;
}
public static long fakultaet(int n) {
return n == 0 ? 1 : n * fakultaet(n - 1);
}
}
How can make my code choose an optimal order?
We will not perform your exercise for you, but we will give you some advice.
Consider your current approach: it operates by filling the result string by cycling through the bricks, choosing one item from each brick in turn as long as any items remain in that brick. But this approach is certain to fail when one brick contains at least two items more than any other, because then only that brick remains at the end, and all its remaining items have to be inserted one after the other.
That is, the problem is not that your code is buggy per se, but rather that your whole strategy is incorrect for the problem. You need something different.
Now, consider the problem itself. Which items will appear at the shortest distance apart in a correct ordering? Those having the highest frequency, of course. And you can compute that minimum distance based on the frequency and total number of items.
Suppose you arrange these most-constrained items first, at the known best distance.
What's left to do at this point? Well, you potentially have some more bricks with lesser frequency, and some more slots in which to accommodate their items. If you ignore the occupied slots altogether, you can treat this as a smaller version of the same problem you had before.
I'm a student in computing sciences in Paris. In mathematics this year we have to use the K-means algorithm to solve a problem (the Clustered Capacited Vehicle Routing Problem applied to the resupplying of self-service bicycles' stations). Here is my algorithm :
public void run() {
boolean hasConverged = false;
List<Integer> nearestClusters = null;
//A list used to check if the nearestClusters list has evolved
//If it isn't the case, the algorithm is finish
List<Integer> previousList = new ArrayList<Integer>();
//Random initialization of the clusters' centroids
for (int i = 0; i < clustersNumber; ++i) {
clusters.add(ClusterGenerator.Generate(stationsList,colorList.get(i) ,latMin, latMax, lngMin, lngMax));
}
while (!hasConverged) {
if (nearestClusters != null) {
previousList.clear();
previousList.addAll(nearestClusters);
}
nearestClusters= new ArrayList<Integer>();
//Each point is connected to it nearest cluster
for (int j = 0; j < stationsList.size(); ++j) {
nearestClusters.add(getIndexOfTheNearestCluster(stationsList.get(j)));
}
//We move the clusters centroids to the center of the points they are connected to
for (int k = 0; k < clusters.size(); ++k) {
clusters.get(k).setCentre(stationsCenters(getStationsOfCluster(clusters.get(k), nearestClusters)));
}
if (!nearestClusters.isEmpty() && previousList.equals(nearestClusters))
hasConverged = true;
}
}
Yet, I wanted to show the result of my algorithm with the clusters formed and I found this work on the Internet : https://github.com/ertugrulozcan/K-Means-Simulation
I imported in my project the class ClusterGenerator which creates clusters along with random elements, the class Item, the class Graphic (I didn't touch anything there) and the class MainWindow which initiates all the graphic elements.
I did not manage to display the plots and there are no errors in Eclipse that could give me any clue.
Can someone please explain to me where is the problem ?
Thanks
The problem was that my algorithm was generating clusters for the stations but I did not configure the class Graphic (which I understood later was very important for the display) to render correctly my points. Since, I used latitude and longitude as coordinates for my station, I had to put these coordinates to scale for the window. Here is how I did that (using cross multiplications) : I calculate the "gap" between two units in the graph and added an adjustment because I don't start at zero.
double gapX = (this.getWidth() - 2 * edgeSpace) / (topX-bottomX+1);
int adjustmentX =(int) (-bottomX*gapX);
(getWidth() gives the actual width of the panel where is the graph, edgespace is the padding space between the graph and the edge of the panel, topX is the maximum value of a coordinate and bottomX the minimum value)
I am trying to build a 4 x 4 sudoku solver by using the genetic algorithm. I have some issues with values converging to local minima. I am using a ranked approach and removing the bottom two ranked answer possibilities and replacing them with a crossover between the two highest ranked answer possibilities. For additional help avoiding local mininma, I am also using mutation. If an answer is not determined within a specific amount of generation, my population is filled with completely new and random state values. However, my algorithm seems to get stuck in local minima. As a fitness function, I am using:
(Total Amount of Open Squares * 7 (possible violations at each square; row, column, and box)) - total Violations
population is an ArrayList of integer arrays in which each array is a possible end state for sudoku based on the input. Fitness is determined for each array in the population.
Would someone be able to assist me in determining why my algorithm converges on local minima or perhaps recommend a technique to use to avoid local minima. Any help is greatly appreciated.
Fitness Function:
public int[] fitnessFunction(ArrayList<int[]> population)
{
int emptySpaces = this.blankData.size();
int maxError = emptySpaces*7;
int[] fitness = new int[populationSize];
for(int i=0; i<population.size();i++)
{
int[] temp = population.get(i);
int value = evaluationFunc(temp);
fitness[i] = maxError - value;
System.out.println("Fitness(i)" + fitness[i]);
}
return fitness;
}
Crossover Function:
public void crossover(ArrayList<int[]> population, int indexWeakest, int indexStrong, int indexSecStrong, int indexSecWeak)
{
int[] tempWeak = new int[16];
int[] tempStrong = new int[16];
int[] tempSecStrong = new int[16];
int[] tempSecWeak = new int[16];
tempStrong = population.get(indexStrong);
tempSecStrong = population.get(indexSecStrong);
tempWeak = population.get(indexWeakest);
tempSecWeak = population.get(indexSecWeak);
population.remove(indexWeakest);
population.remove(indexSecWeak);
int crossoverSite = random.nextInt(14)+1;
for(int i=0;i<tempWeak.length;i++)
{
if(i<crossoverSite)
{
tempWeak[i] = tempStrong[i];
tempSecWeak[i] = tempSecStrong[i];
}
else
{
tempWeak[i] = tempSecStrong[i];
tempSecWeak[i] = tempStrong[i];
}
}
mutation(tempWeak);
mutation(tempSecWeak);
population.add(tempWeak);
population.add(tempSecWeak);
for(int j=0; j<tempWeak.length;j++)
{
System.out.print(tempWeak[j] + ", ");
}
for(int j=0; j<tempWeak.length;j++)
{
System.out.print(tempSecWeak[j] + ", ");
}
}
Mutation Function:
public void mutation(int[] mutate)
{
if(this.blankData.size() > 2)
{
Blank blank = this.blankData.get(0);
int x = blank.getPosition();
Blank blank2 = this.blankData.get(1);
int y = blank2.getPosition();
Blank blank3 = this.blankData.get(2);
int z = blank3.getPosition();
int rando = random.nextInt(4) + 1;
if(rando == 2)
{
int rando2 = random.nextInt(4) + 1;
mutate[x] = rando2;
}
if(rando == 3)
{
int rando2 = random.nextInt(4) + 1;
mutate[y] = rando2;
}
if(rando==4)
{
int rando3 = random.nextInt(4) + 1;
mutate[z] = rando3;
}
}
The reason you see rapid convergence is that your methodology for "mating" is not very good. You are always producing two offspring from "mating" of the top two scoring individuals. Imagine what happens when one of the new offspring is the same as your top individual (by chance, no crossover and no mutation, or at least none that have an effect on the fitness). Once this occurs, the top two individuals are identical which eliminates the effectiveness of crossover.
A more typical approach is to replace EVERY individual on every generation. There are lots of possible variations here, but you might do a random choice of two parents weighted fitness.
Regarding population size: I don't know how hard of a problem sudoku is given your genetic representation and fitness function, but I suggest that you think about millions of individuals, not dozens.
If you are working on really hard problems, genetic algorithms are much more effective when you place your population on a 2-D grid and choosing "parents" for each point in the grid from the nearby individuals. You will get local convergence, but each locality will have converged on different solutions; you get a huge amount of variation produced from the borders between the locally-converged areas of the grid.
Another technique you might think about is running to convergence from random populations many times and store the top individual from each run. After you build up a bunch of different local minima genomes, build a new random population from those top individuals.
I think the Sudoku is a permutation problem. therefore i suggest you to use random permutation numbers for initializing population and use the crossover method which Compatible to permutation problems.
I was doing code forces and wanted to implement Dijkstra's Shortest Path Algorithm for a directed graph using Java with an Adjacency Matrix, but I'm having difficulty making it work for other sizes than the one it is coded to handle.
Here is my working code
int max = Integer.MAX_VALUE;//substitute for infinity
int[][] points={//I used -1 to denote non-adjacency/edges
//0, 1, 2, 3, 4, 5, 6, 7
{-1,20,-1,80,-1,-1,90,-1},//0
{-1,-1,-1,-1,-1,10,-1,-1},//1
{-1,-1,-1,10,-1,50,-1,20},//2
{-1,-1,-1,-1,-1,-1,20,-1},//3
{-1,50,-1,-1,-1,-1,30,-1},//4
{-1,-1,10,40,-1,-1,-1,-1},//5
{-1,-1,-1,-1,-1,-1,-1,-1},//6
{-1,-1,-1,-1,-1,-1,-1,-1} //7
};
int [] record = new int [8];//keeps track of the distance from start to each node
Arrays.fill(record,max);
int sum =0;int q1 = 0;int done =0;
ArrayList<Integer> Q1 = new ArrayList<Integer>();//nodes to transverse
ArrayList<Integer> Q2 = new ArrayList<Integer>();//nodes collected while transversing
Q1.add(0);//starting point
q1= Q1.get(0);
while(done<9) {// <<< My Problem
for(int q2 = 1; q2<8;q2++) {//skips over the first/starting node
if(points[q1][q2]!=-1) {//if node is connected by an edge
if(record[q1] == max)//never visited before
sum=0;
else
sum=record[q1];//starts from where it left off
int total = sum+points[q1][q2];//total distance of route
if(total < record[q2])//connected node distance
record[q2]=total;//if smaller
Q2.add(q2);//colleceted node
}
}
done++;
Q1.remove(0);//removes the first node because it has just been used
if(Q1.size()==0) {//if there are no more nodes to transverse
Q1=Q2;//Pours all the collected connecting nodes to Q1
Q2= new ArrayList<Integer>();
q1=Q1.get(0);
}
else//
q1=Q1.get(0);//sets starting point
}![enter image description here][1]
However, my version of the algorithm only works because I set the while loop to the solved answer. So in other words, it only works for this problem/graph because I solved it by hand first.
How could I make it so it works for all groups of all sizes?
Here is the pictorial representation of the example graph my problem was based on:
I think the main answer you are looking for is that you should let the while-loop run until Q1 is empty. What you're doing is essentially a best-first search. There are more changes required though, since your code is a bit unorthodox.
Commonly, Dijkstra's algorithm is used with a priority queue. Q1 is your "todo list" as I understand from your code. The specification of Dijkstra's says that the vertex that is closest to the starting vertex should be explored next, so rather than an ArrayList, you should use a PriorityQueue for Q1 that sorts vertices according to which is closest to the starting vertex. The most common Java implementation uses the PriorityQueue together with a tuple class: An internal class which stores a reference to a vertex and a "distance" to the starting vertex. The specification for Dijkstra's also specifies that if a new edge is discovered that makes a vertex closer to the start, the DecreaseKey operation should then be used on the entry in the priority queue to make the vertex come up earlier (since it is now closer). However, since PriorityQueue doesn't support that operation, a completely new entry is just added to the queue. If you have a good implementation of a heap that supports this operation (I made one myself, here) then decreaseKey can significantly increase efficiency as you won't need to create those tuples any more either then.
So I hope that is a sufficient answer then: Make a proper 'todo' list instead of Q1, and to make the algorithm generic, let that while-loop run until the todo list is empty.
Edit: I made you an implementation based on your format, that seems to work:
public void run() {
final int[][] points = { //I used -1 to denote non-adjacency/edges
//0, 1, 2, 3, 4, 5, 6, 7
{-1,20,-1,80,-1,-1,90,-1}, //0
{-1,-1,-1,-1,-1,10,-1,-1}, //1
{-1,-1,-1,10,-1,50,-1,20}, //2
{-1,-1,-1,-1,-1,-1,20,-1}, //3
{-1,50,-1,-1,-1,-1,30,-1}, //4
{-1,-1,10,40,-1,-1,-1,-1}, //5
{-1,-1,-1,-1,-1,-1,-1,-1}, //6
{-1,-1,-1,-1,-1,-1,-1,-1} //7
};
final int[] result = dijkstra(points,0);
System.out.print("Result:");
for(final int i : result) {
System.out.print(" " + i);
}
}
public int[] dijkstra(final int[][] points,final int startingPoint) {
final int[] record = new int[points.length]; //Keeps track of the distance from start to each vertex.
final boolean[] explored = new boolean[points.length]; //Keeps track of whether we have completely explored every vertex.
Arrays.fill(record,Integer.MAX_VALUE);
final PriorityQueue<VertexAndDistance> todo = new PriorityQueue<>(points.length); //Vertices left to traverse.
todo.add(new VertexAndDistance(startingPoint,0)); //Starting point (and distance 0).
record[startingPoint] = 0; //We already know that the distance to the starting point is 0.
while(!todo.isEmpty()) { //Continue until we have nothing left to do.
final VertexAndDistance next = todo.poll(); //Take the next closest vertex.
final int q1 = next.vertex;
if(explored[q1]) { //We have already done this one, don't do it again.
continue; //...with the next vertex.
}
for(int q2 = 1;q2 < points.length;q2++) { //Find connected vertices.
if(points[q1][q2] != -1) { //If the vertices are connected by an edge.
final int distance = record[q1] + points[q1][q2];
if(distance < record[q2]) { //And it is closer than we've seen so far.
record[q2] = distance;
todo.add(new VertexAndDistance(q2,distance)); //Explore it later.
}
}
}
explored[q1] = true; //We're done with this vertex now.
}
return record;
}
private class VertexAndDistance implements Comparable<VertexAndDistance> {
private final int distance;
private final int vertex;
private VertexAndDistance(final int vertex,final int distance) {
this.vertex = vertex;
this.distance = distance;
}
/**
* Compares two {#code VertexAndDistance} instances by their distance.
* #param other The instance with which to compare this instance.
* #return A positive integer if this distance is more than the distance
* of the specified object, a negative integer if it is less, or
* {#code 0} if they are equal.
*/
#Override
public int compareTo(final VertexAndDistance other) {
return Integer.compare(distance,other.distance);
}
}
Output: 0 20 40 50 2147483647 30 70 60