I'm trying to implement an AI that uses Minimax for the dots and boxs game (http://en.wikipedia.org/wiki/Dots_and_Boxes)
Here is what I have so far:
public Line makeMove(GameState gs) {
if (gs.getRemainingLines().size() == 1) {
return gs.getRemainingLines().get(0);
}
if (gs.getPlayer() == 1) {
int minscore = -1;
GameState g = gs.clone();
Line lnew = null;
List<Line> l = gs.getRemainingLines();
for (Line l2 : l) {
g.addLine(l2);
if (evaluate(g) > minscore) {
minscore = (evaluate(g));
lnew = l2;
}
}
return lnew;
} else {
int maxscore = 999;
GameState g = gs.clone();
Line lnew = null;
List<Line> l = gs.getRemainingLines();
for (Line l2 : l) {
g.addLine(l2);
if (evaluate(g) < maxscore) {
maxscore = (evaluate(g));
lnew = l2;
}
}
return lnew;
}
}
However, it keeps returning null and I don't think I'm impementing minimax correctly. Can anyone give me some pointers.
getRemainingLines() returns a List of moves that are still possible.
evaluate() returns an int for the score.
I would like to suggest that you completely re-factor your code. The problem with looking at your code (and why there haven't been many responses here) is that it's hard to follow and hard to debug. For instance, what is gs.getRemainingLines and what does it do exactly? (Why remaining lines and not all legal lines?)
But, with some simplifications it will be much easier to figure out what is going on and to fix it.
At an abstract level minimax is just this procedure:
float minimax_max(GameState g)
{
if (g is terminal or max depth reached)
return eval(g);
float bestVal = -inf;
bestMove = null;
moves = g->getLegalMoves();
for (m : moves)
{
ApplyMove(m);
if (g->nextPlayer == maxPlayer)
nextVal = minimax_max(g);
else
nextVal = minimax_min(g);
if (nextVal > bestVal)
{
bestVal = nextVal;
bestMove = m;
}
UndoMove(m);
}
return bestVal;
}
I haven't shown exactly how to get/use the last move at the end, but it isn't that hard. You also need another procedure for minimax_min, or you can put an if statement into the code.
If you look at your code, you've written it close to this, but you've left a lot of game specific details in the code. But, you shouldn't have to think about those things to get minimax working correctly.
In particular, most games can be reasoned with abstractly if you provide functions for GetMoves(), ApplyMove(), UndoMove(), and eval(), which evaluates a state. (Further search enhancements would require more functions, but this will get you a long ways.)
Some reasons why you might want to re-factor in this way:
You can now test minimax and your other code separately.
You can test your dots and boxes code by validating that all legal moves are generated and that after applying a move you have a legal state with the correct player moving next. (You can play and undo long sequences of random moves to help validate that you always end up back in the start state afterwards.)
You can test your evaluation function easily on individual states to make sure it works properly. (In practice you can't usually search to the end of the game to determine the winner.)
You can test minimax by using a simple evaluation function and testing to see if the right moves are made. (e.g. if you prefer moves on the edges, a 1-ply search should return a move on the edge)
Other people can read your code more easily. We can look at each piece of code and see if it is correct on its own, instead of having to mix the game-specific implementation details into the minimax-specific details.
If you can apply and undo moves properly, you don't need to make copies of the game states. This will make the code much more efficient.
While you could try to fix your code without refactoring (e.g. just find the first place it returns null, and that will point out where your error is), in the long term your code will be hard to debug and improve without these changes.
The first thing to check is that gs.getRemainingLines() actually has lines remaining.
A separate problem is that you are adding every line to the GameState g to check. You either need to remove each added line after calling evaluate or put the clone inside the loop at the top such as
int minscore = -1;
Line lnew = null;
List<Line> l = gs.getRemainingLines();
for (Line l2 : l) {
GameState g = gs.clone();
g.addLine(l2);
if (evaluate(g) > minscore) {
minscore = (evaluate(g));
lnew = l2;
}
}
or
int minscore = -1;
GameState g = gs.clone();
Line lnew = null;
List<Line> l = gs.getRemainingLines();
for (Line l2 : l) {
g.addLine(l2);
if (evaluate(g) > minscore) {
minscore = (evaluate(g));
lnew = l2;
}
g.removeLine(l2);
}
However if you are trying to use minimax (http://en.wikipedia.org/wiki/Minimax) then you will need to change your code to recursively call makeMove (unless you modify the algorithm to do determine the min-max using loop constructs).
public GameState makeMove(GameState gs) {
if (gs.getRemainingLines().size() == 1) {
GameState g = gs.clone();
g.addLine(gs.getRemainingLines().get(0));
return g;
}
if (gs.getPlayer() == 1) {
GameState g = gs.clone();
g.setPlayer(2);
int bestValue = -1;
Line lbest = null;
List<Line> lines = gs.getRemainingLines();
for (Line line : lines) {
g.addLine(line);
GameState val = makeMove(g);
g.removeLine(line);
if (evaluate(val) > bestValue) {
bestValue = evaluate(g);
lbest = line;
}
}
g.addLine(lbest);
return g;
} else {
GameState g = gs.clone();
g.setPlayer(1);
int bestValue = 999;
Line lbest = null;
List<Line> lines = gs.getRemainingLines();
for (Line line : lines) {
g.addLine(line);
GameState val = makeMove(g);
g.removeLine(line);
if (evaluate(val) < bestValue) {
bestValue = evaluate(g);
lbest = line;
}
}
g.addLine(lbest);
return g;
}
}
Related
I'm having a problem concerning a command pattern with undo/redo function. The simple problem is, when my history is full, I want to remove the least recently used command from the history and add the new one on execute.
I got this code snippet from my professor:
public class CommandHistory implements CommandInterface{
private static final int MAX_COMMANDS = 2;
private Command[] history = new Command[MAX_COMMANDS];
private int current = -1;
#Override
public void execute(Command command) {
current++;
if (current == MAX_COMMANDS){ // if full, then shift
for (int i = 0; i < MAX_COMMANDS - 1; i++){
history[i] = history[i+1];
}
}
history[current] = command;
history[current].execute();
}
In really doubt the if-clause is incorrect, because the current command index remains 2 and only command at index 0 is shifted to 1. But he says this is the way to go. What am I missing?
The loop itself is fine, but two problems:
You're quite correct that when current == MAX_COMMANDS is true and you do the loop, current is incorrect and needs adjusting.
From a maintenance perspective, current == MAX_COMMANDS is the wrong comparison, it should be current == history.length. (Otherwise, it's easy to change the initialization of history to use something other than MAX_COMMANDS but forget to change every check like current == MAX_COMMANDS.)
I would check current before incrementing it, and only increment it if you're not shifting the contents down:
public void execute(Command command) {
if (current == history.length - 1){ // if full, then shift
for (int i = 0; i < history.length - 1; i++) {
history[i] = history[i+1];
}
} else {
current++;
}
history[current] = command;
history[current].execute();
}
So I have written a code that allows a user to find a word in a TextArea. I have nearly succeeded but for one thing. here, I will show you all the code and tell my problem.
if(ta.getText().length() != 0 && t1.getText().length() != 0)
{
char c1[] = ta.getText().trim().toCharArray();
char c2[] = t1.getText().trim().toCharArray();
for(int i=startFlag;i<c1.length;i++)
{
if(c1[i]==c2[0])
{
start = i;
break;
}
}
k=start;
for(int i=0;i<c2.length;i++)
{
if(c2[i] != c1[start++])
{
}
else
countFlag++;
}
if(countFlag==c2.length)
{
ta.select(k,c2.length);
startFlag++;
}
}
For reference, ta is the TextArea and t1 is the TextField where the user enters a word to find. i have a problem in the second for loop. What should I write in the if () block there so that whenever c2[i] != c1[start++] the control is shifted to the first for loop, that would again determine the value of start?
Create a method to get "start" that you can then call whenever you want.
if(ta.getText().length() != 0 && t1.getText().length() != 0)
{
char c1[] = ta.getText().trim().toCharArray();
char c2[] = t1.getText().trim().toCharArray();
k=getStart(startFlag, c1.length);
for(int i=0;i<c2.length;i++)
{
if(c2[i] != c1[start++])
{
start = getStart(startFlag, c1.length);
}
else
countFlag++;
}
if(countFlag==c2.length)
{
ta.select(k,c2.length);
startFlag++;
}
}
And getStart() is:
public int getStart(int startFlag, int length) {
for(int i=startFlag;i<length;i++)
{
if(c1[i]==c2[0])
{
return i;
}
}
}
You may need different inputs to getStart(), but hopefully this gets across the general idea.
The way your code is set up right now, what you're asking for is impossible. To do what you're asking, you'll need to refactor your current code into different methods. More specifically, you'll need to put the for loops into their own methods and then call these methods.
So what you would need to do is make a separate method for the for loop.
public static int startForLoop(int i) {
for(int i=startFlag;i<c1.length;i++)
{
if(c1[i]==c2[0])
{
start = i;
break;
}
}
}
Then you can just call startForLoop(0) initially and in the 2nd for loops if statment:
if(c2[i] != c1[start++])
{
startForLoop(start+1)
}
This will continue the for loop where it left off. If you need to run the 2nd for loop again then you have to make a separate method for it as well and basically place both of them in a while loop that continues till you find the result you want in the 2nd for loop.
May be this code piece help you what you are looking for.
Basically it moves along with the string to be searched in keeping in mind the index of the string to be searched for.
Sorry but i have implemented it in java, but the notion is same and the result returned is the best what i got.you must give it a try!
private static String searchString(String searchIN,String searchFOR){
if (searchFOR != "") {
int index = searchIN.toUpperCase().indexOf(searchFOR.toUpperCase());
String before = "";
String highlighted = "";
String after = "";
while (index >= 0) {
int len = searchFOR.length();
before = searchIN.substring(0, index);
highlighted = "\"" + searchFOR + "\"";//do what ever you want to do with searched string
after = searchIN.substring(index + len);
searchIN = before + highlighted + after;
index = searchIN.toUpperCase().indexOf(searchFOR.toUpperCase(), index + highlighted.length());
}
}
return searchIN;
}
I am currently trying to write a procedure to check whether a directed graph is cyclic or not. I am not sure what i did wrong (it may be well possible that I did everything wrong, so please StackOverflow, show me my stupidity!). I'd be thankful for any kind of help as I've come to the point where I don't know what could be the problem.
The input is an adjacency list such as:
0: 2 4
1: 2 4
2: 3 4
3: 4
4: 0 1 2 3
(0 directs to 2 and 4; 1 directs to 2 and 4 and so on...)
The idea is that I check whether the node I am checking is 'grey' (partially explored) or not. If it is, it must be a back edge and thus a cyclic graph. Black edges are always explored or cross-edges, so this shouldn't trigger a cyclic message. I am aiming to do depth first search
If A-->B and B-->A, this should not trigger a message about cyclic (but A--> B, B-->C, C-->A should).
hasCycle calls hasCycleInSubgraph which calls itself recursively through the Adjency List of the Graph.
class qs {
private ArrayList<Integer>[] adjList;
private Stack<Integer> stack;
private ArrayList<Integer> whiteHat;
private ArrayList<Integer> greyHat;
private ArrayList<Integer> blackHat;
public qs(ArrayList<Integer>[] graph) {
this.adjList = graph;
this.stack = new Stack();
this.whiteHat = new ArrayList<Integer>();
this.greyHat = new ArrayList<Integer>();
this.blackHat = new ArrayList<Integer>();
for (Integer h = 0; h < adjList.length; h++) {
whiteHat.add(h);
}
}
public boolean hasCycle() {
for (Integer i = 0; i < adjList.length; i++) {
// System.out.print("Local root is: ");
// System.out.println(i);
whiteHat.remove(i);
greyHat.add(i);
if (hasCycleInSubgraph(i) == true) {
return true;
}
greyHat.remove(i);
blackHat.add(i);
}
return false;
}
public boolean hasCycleInSubgraph(Integer inp) {
if (blackHat.contains(inp)) {
return false;
}
for (Integer branch : adjList[inp]) {
// System.out.print("Adj is: ");
// System.out.println(branch);
if ( greyHat.contains(branch) && !inp.equals(branch) ) {
return true;
}
whiteHat.remove(branch);
greyHat.add(branch);
if ( hasCycleInSubgraph(branch) == true ) {
return true;
}
greyHat.remove(branch);
blackHat.add(branch);
}
return false;
}
}
You are over-complicating it: a cycle can be detected via a depth-first search: from any given node, walk to each of the connected nodes; if you arrive back at an already-visited node, you've got a cycle.
class qs {
private final ArrayList<Integer>[] graph;
qs(ArrayList<Integer>[] graph) {
this.graph = graph;
}
boolean hasCycle() {
List<Integer> visited = new ArrayList<>();
for (int i = 0; i < graph.length; ++i) {
if (hasCycle(i, visited)) {
return true;
}
}
}
private boolean hasCycle(int node, List<Integer> visited) {
if (visited.contains(node)) {
return true;
}
visited.add(node);
for (Integer nextNode : graph[node]) {
if (hasCycle(nextNode, visited)) {
return true;
}
}
visited.remove(visited.length() - 1);
return false;
}
}
If you want to detect cycles longer than a given length, just check the depth of the recursion:
if (visited.contains(node) && visited.size() > 2) {
Note that this does not require any state to be kept, aside from what is in the stack. Relying upon mutable state makes the code thread-unsafe (e.g. that two threads calling hasCycle at the same time would interfer with each other), and so should be avoided - even if you don't expect the code to be used in a multi-threaded way now, it avoids problems down the line.
I'm creating a program in Java that solves the n-puzzle, without using heuristics, simply just with depth-first and breadth-first searches of the state space. I'm struggling a little bit with my implementation of depth-first search. Sometimes it will solve the given puzzle, but other times it seems to give up early.
Here's my DFS class. DepthFirstSearch() is passed a PuzzleBoard, which is initially generated by shuffling a solved board (to ensure that the board is in a solvable state).
public class DepthFirst {
static HashSet<PuzzleBoard> usedStates = new HashSet<PuzzleBoard>();
public static void DepthFirstSearch(PuzzleBoard currentBoard)
{
// If the current state is the goal, stop.
if (PuzzleSolver.isGoal(currentBoard)) {
System.out.println("Solved!");
System.exit(0);
}
// If we haven't encountered the state before,
// attempt to find a solution from that point.
if (!usedStates.contains(currentBoard)) {
usedStates.add(currentBoard);
PuzzleSolver.print(currentBoard);
if (PuzzleSolver.blankCoordinates(currentBoard)[1] != 0) {
System.out.println("Moving left");
DepthFirstSearch(PuzzleSolver.moveLeft(currentBoard));
}
if (PuzzleSolver.blankCoordinates(currentBoard)[0] != PuzzleSolver.n-1) {
System.out.println("Moving down");
DepthFirstSearch(PuzzleSolver.moveDown(currentBoard));
}
if (PuzzleSolver.blankCoordinates(currentBoard)[1] != PuzzleSolver.n-1) {
System.out.println("Moving right");
DepthFirstSearch(PuzzleSolver.moveRight(currentBoard));
}
if (PuzzleSolver.blankCoordinates(currentBoard)[0] != 0) {
System.out.println("Moving up");
DepthFirstSearch(PuzzleSolver.moveUp(currentBoard));
}
return;
} else {
// Move up a level in the recursive calls
return;
}
}
}
I can assert that my moveUp(), moveLeft(), moveRight(), and moveDown() methods and logic work correctly, so the problem must lie somewhere else.
Here's my PuzzleBoard object class with the hashCode and equals methods:
static class PuzzleBoard {
short[][] state;
/**
* Default constructor for a board of size n
* #param n Size of the board
*/
public PuzzleBoard(short n) {
state = PuzzleSolver.getGoalState(n);
}
public PuzzleBoard(short n, short[][] initialState) {
state = initialState;
}
#Override
public int hashCode() {
final int prime = 31;
int result = 1;
result = prime * result + Arrays.deepHashCode(state);
return result;
}
#Override
public boolean equals(Object obj) {
if (this == obj) {
return true;
}
if (obj == null) {
return false;
}
if (getClass() != obj.getClass()) {
return false;
}
PuzzleBoard other = (PuzzleBoard) obj;
for (int i = 0; i < n; i++) {
for (int j = 0; j < n; j++) {
if (state[i][j] != other.state[i][j])
return false;
}
}
return true;
}
}
As previously stated, sometimes the search works properly and finds a path to the solution, but other times it stops before it finds a solution and before it runs out of memory.
Here is a snippet of the output, beginning a few moves before the search stops searching.
...
Moving down
6 1 3
5 8 2
0 7 4
Moving right
6 1 3
5 8 2
7 0 4
Moving left
Moving right
Moving up
6 1 3
5 0 2
7 8 4
Moving left
Moving down
Moving right
Moving up
Moving up
Moving right
Moving down
Moving up
Moving down
Moving up
Moving down
Moving up
Moving down
Moving up
Moving down
...
I truncated it early for brevity, but it ends up just moving up and down dozens of times and never hits the solved state.
Can anyone shed light on what I'm doing wrong?
Edit: Here is MoveUp(). The rest of the move methods are implemented in the same way.
/**
* Move the blank space up
* #return The new state of the board after the move
*/
static PuzzleBoard moveUp(PuzzleBoard currentState) {
short[][] newState = currentState.state;
short col = blankCoordinates(currentState)[0];
short row = blankCoordinates(currentState)[1];
short targetCol = col;
short targetRow = row;
newState[targetCol][targetRow] = currentState.state[col - 1][row];
newState[targetCol - 1][targetRow] = 0;
return new PuzzleBoard(n, newState);
}
I have had many problems with hashset in the past best thing to try is not to store object in hashset but try to encode your object into string.
Here is a way to do it:-
StringBuffer encode(PuzzleBoard b) {
StringBuffer buff = new StringBuffer();
for(int i=0;i<b.n;i++) {
for(int j=0;j<b.n;j++) {
// "," is used as separator
buff.append(","+b.state[i][j]);
}
}
return buff;
}
Make two changes in the code:-
if(!usedStates.contains(encode(currentBoard))) {
usedStates.add(encode(currentBoard));
......
}
Note:- Here no need to write your own hashcode function & also no need to implement equals function as java has done it for you in StringBuffer.
I got one of the problems in your implementation:-
In th following code:-
static PuzzleBoard moveUp(PuzzleBoard currentState) {
short[][] newState = currentState.state;
short col = blankCoordinates(currentState)[0];
short row = blankCoordinates(currentState)[1];
short targetCol = col;
short targetRow = row;
newState[targetCol][targetRow] = currentState.state[col - 1][row];
newState[targetCol - 1][targetRow] = 0;
return new PuzzleBoard(n, newState);
}
Here you are using the reference of same array as newState from currentState.state so when you make changes to newState your currentState.state will also change which will affect DFS when the call returns. To prevent that you should initialize a new array. Heres what to be done:-
static PuzzleBoard moveUp(PuzzleBoard currentState) {
short[][] newState = new short[n][n];
short col = blankCoordinates(currentState)[0];
short row = blankCoordinates(currentState)[1];
short targetCol = col;
short targetRow = row;
for(int i=0;i<n;i++) {
for(int j=0;j<n;j++) {
newState[i][j] = currentState.state[i][j];
}
}
newState[targetCol][targetRow] = currentState.state[col - 1][row];
newState[targetCol - 1][targetRow] = 0;
return new PuzzleBoard(n, newState);
}
Do this change for all moveup,movedown....
Moreover I donot think your hashset is working properly because if it was then you would always find your new state in hashset and your program would stop. As in equals you comparing the state arrays with same reference hence will always get true. Please try and use my encode function as hash.
I know this kind of question has been asked before, but i was unable to solve my doubts.
I have a simple Othello Engine (it plays very well actually), that uses the class below to get the best move:
import java.util.*;
import java.util.concurrent.*;
public class MinimaxOthello implements Runnable
{
private CountDownLatch doneSignal;
private int maxDepth;
private int calls;
private OthelloMove bestFound;
private OthelloBoard board;
private static float INFINITY = Float.MAX_VALUE/1000;
private boolean solve = false;
private Comparator<OthelloMove> comparator = Collections.reverseOrder(new MoveComparator());
public MinimaxOthello (OthelloBoard board, int maxDepth, CountDownLatch doneSignal, boolean solve)
{
this.board = board;
this.bestFound = new OthelloMove();
bestFound.setPlayer(board.getCurrentPlayer());
this.maxDepth = maxDepth;
this.doneSignal = doneSignal;
this.solve = solve;
}
public OthelloMove getBestFound()
{
return this.bestFound;
}
public void run()
{
float val = minimax(board, bestFound, -INFINITY, INFINITY, 0);
System.out.println("calls: " + calls);
System.out.println("eval: " + val);
System.out.println();
doneSignal.countDown();
}
private float minimax(OthelloBoard board, OthelloMove best, float alpha, float beta, int depth)
{
calls++;
OthelloMove garbage = new OthelloMove();
int currentPlayer = board.getCurrentPlayer();
if (board.checkEnd())
{
int bd = board.countDiscs(OthelloBoard.BLACK);
int wd = board.countDiscs(OthelloBoard.WHITE);
if ((bd > wd) && currentPlayer == OthelloBoard.BLACK)
{
return INFINITY/10;
}
else if ((bd < wd) && currentPlayer == OthelloBoard.BLACK)
{
return -INFINITY/10;
}
else if ((bd > wd) && currentPlayer == OthelloBoard.WHITE)
{
return -INFINITY/10;
}
else if ((bd < wd) && currentPlayer == OthelloBoard.WHITE)
{
return INFINITY/10;
}
else
{
return 0.0f;
}
}
if (!solve)
{
if (depth == maxDepth)
return OthelloHeuristics.eval(currentPlayer, board);
}
ArrayList<OthelloMove> moves = board.getAllMoves(currentPlayer);
if (moves.size() > 1)
{
OthelloHeuristics.scoreMoves(moves);
Collections.sort(moves, comparator);
}
for (OthelloMove mv : moves)
{
board.makeMove(mv);
float score = - minimax(board, garbage, -beta, -alpha, depth + 1);
board.undoMove(mv);
if(score > alpha)
{
alpha = score;
best.setFlipSquares(mv.getFlipSquares());
best.setIdx(mv.getIdx());
best.setPlayer(mv.getPlayer());
}
if (alpha >= beta)
break;
}
return alpha;
}
}
I have a bestFound instance variable and my doubt is, why a have to call
OthelloMove garbage = new OthelloMove();
and pass it along? The code works, but it seems very weird to me!
Is there a 'better' way to get the best move or the Principal Variation?
I really not a recursion expert, and this is very very hard to debug and visualize.
Thanks!
**PS: You can clone it at https://github.com/fernandotenorio/
It looks like you can get rid of the best parameter to minimax, thereby eliminating the need for garbage, and then replace best with this.bestFound. Only set bestFound's attributes if depth = 0.
You can get the principal variation by making this.bestFound an initially empty list. Before the moves loop, create a new move. In the if (score > alpha) part, set its attributes the same as you do now. Push the move to the list right after the loop. The principal variation will then be the reverse of the list.
If it's important, here are some changes you can make to improve the multi-threadability of your class:
Instead of storing the bestFound list as an instance variable, make it a local variable in run and add it as a parameter to minimax
Make Board.makeMove not modify the board, but instead return a new instance of the board with the move applied. You can implement that by cloning the board and applying your move code to the clone instead of mutating this. Then, pass the cloned board to the next invocation of minimax.
The second argument of minimax is used to return the best move.
The business with garbage is used to keep the best move for each turn separate. With the code you've provided, this is not important. But if you wanted to produce a sequence of moves from the current board to the end of the game, you would need to have them be separate move objects.
Using a separate best-move object for each turn allows you to do a number of tricks with threading. First, you might want to limit the thinking time of the Othello AI. Tracking the best move separately at each level means that you always have the best move so far available. It also means that you could cache the best move for a board and look that up in future minimax searches.
Second, you might want to search for the best move in parallel, and this is trivial to implement when each minimax call is independent.