This is my first question here, if I did something wrong, tell me...
I'm currently making a draughts game in Java. In fact everything works except the AI.
The AI is at the moment single threaded, using minimax and alpha-beta pruning. This code works, I think, it's just very slow, I can only go 5 deep into my game tree.
I have a function that recieves my mainboard, a depth (starts at 0) and a maxdepth. At this maxdepth it stops, returns the player's value (-1,1 or 0) with the most pieces on the board and ends the recursive call.
If maxdepth isn't reached yet, I calculate all the possible moves, I execute them one by one, storing my changes to the mainboard in someway.
I also use alpha-beta pruning, e.g. when I found a move that can make the player win I don't bother about the next possible moves.
I calculate the next set of moves from that mainboard state recursively. I undo those changes (from point 2) when coming out of the recursive call. I store the values returned by those recursive calls and use minimax on those.
That's the situation, now I have some questions.
I'd like to go deeper into my game tree, thus I have to diminish the time it takes to calculate moves.
Is it normal that the values of the possible moves of the AI (e.g. the moves that the AI can choose between) are always 0? Or will this change if I can go deeper into the recursion? Since at this moment I can only go 5 deep (maxdepth) into my recursion because otherwise it takes way too long.
I don't know if it's usefull, but how I can convert this recursion into a multithreaded recursion. I think this can divide the working time by some value...
Can someone help me with this please?
1. Is it normal that the values of the possible moves of the AI (e.g. the moves that the AI can choose between) are always 0?
Sounds strange to me. If the number of possible moves is 0, then that player can't play his turn. This shouldn't be very common, or have I misunderstood something?
If the value you're referring to represents the "score" of that move, then obviously "always 0" would indicate that all move are equally good, which obviously doesn't make a very good AI algorithm.
2. I don't know if it's usefull, but how I can convert this recursion into a multithreaded recursion. I think this can divide the working time by some value...
I'm sure it would be very useful, especially considering that most machines have several cores these days.
What makes it complicated is your "try a move, record it, undo it, try next move" approach. This indicates that you're working with a mutable data structure, which makes it extremely complicated to paralellize the algorithm.
If I were you, I would let the bord / game state be represented by an immutable data structure. You could then let each recursive call be treated as a separate task, and use a pool of threads to process them. You would get close to maximum utilization of the CPU(s) and at the same time simplify the code considerably (by removing the whole restore-to-previous-state code).
Assuming you do indeed have several cores on your machine, this could potentially allow you to go deeper in the tree.
I would strongly recommend reading this book:
One Jump Ahead: Computer Perfection At Checkers
It will give you a deep history about computer AI in the game of Checkers and will probably given you some help with your evaluation function.
Instead of having an evaluation function that just gives 1/0/-1 for differing pieces, give a score of 100 for every regular piece and 200 for a king. Then give bonuses for piece structures. For instance, if my pieces form a safe structure that can't be captured, then I get a bonus. If my piece is all alone in the middle of the board, then I get a negative bonus. It is this richness of features for piece configurations that will allow your program to play well. The final score is the difference in the evaluation for both players.
Also, you shouldn't stop your search at a uniform depth. A quiescence search extends search until the board is "quiet". In the case of Checkers, this means that there are no forced captures on the board. If you don't do this, your program will play extremely poorly.
As others have suggested, transposition tables will do a great job of reducing the size of your search tree, although the program will run slightly slower. I would also recommend the history heuristic, which is easy to program and will greatly improve the ordering of moves in the tree. (Google history heuristic for more information on this.)
Finally, the representation of your board can make a big difference. Fast implementations of search do not make copies of the board each time a move is applied, instead they try to quickly modify the board to apply and undo moves.
(I assume by draughts you mean what we would call checkers here in the States.)
I'm not sure if I understand your scoring system inside the game tree. Are you scoring by saying, "Position scores 1 point if player has more pieces than the opponent, -1 point is player has fewer pieces, 0 points if they have the same number of pieces?"
If so, then your algorithm might just be capture averse for the first five moves, or things are working out so that all captures are balanced. I'm not deeply familiar with checkers, but it doesn't seem impossible that this is so for only five moves into the game. And if it's only 5 plies (where a ply is one player's move, rather than a complete set of opposing moves) maybe its not unusual at all.
You might want to test this by feeding in a board position where you know absolutely the right answer, perhaps something with only two checkers on the board with one in a position to capture.
As a matter of general principle, though, the board evaluation function doesn't make a lot of sense-- it ignores the difference between a piece and a crowned piece, and it treats a three piece advantage the same as a one piece advantage.
Related
I'm developing a simple Connect4 game in Android.
Currently I'm using a minimax algorithm with alpha-beta pruning and bit-board state representation so the search is very effective and fast.
The skill is set by setting the maximum depth the algorithm should reach during its DFS search inside the game tree.
I noticed that the time required to choose a move depends on how far we are in the game: at the beginning it takes more time (as there are many possibilities to explore), in mid-game it take a reasonable amount of time and near the end is very fast.
My problem is that if I set a given skill the user has to wait to much on the first/second/third moves. I'd like to speed-up the aperture but I suspect it depends even on the hardware itself how I want to implement the speed-up process.
Can I set a timeout for the thread running the DSF mimimax?
The simplest way to circumvent this issue is to use an opening book for the first few moves. An opening book is a set of predetermined moves for a given scenario. Since there are relatively few possibly board states for the opening moves, you can easily compile a database of all possibly moves for the first three turns, and call upon it instead of actually doing the search. Thus you no longer require a time out and you sped up the search with zero cost to accuracy.
we are working on a little Java game, based on the game Blokus.
Blokus-Manual
I'm a Java-beginner and plan to implement an advanced artificial intelligence. We already have a random AI (picks a random valid move) and an AI with a simple move-rating mechanism. We also want an AI which should be as good as possible (or at least very good ;) ).
The question is: Which AI-concept would be suitable for our purpose?
The minimax-algorithm seems to be a valid choice, but how do you adapt it to a 4-player-game? Are there better concepts for a game like blokus?
Thanks already :)
Min-max is hard to implement in a 4 player game because:
Decision tree grows exponentially, so you're going to be bounded by memory and/or computation time to a log(medMoves)=N steps. For a 4 player game, this is down to N/4. If N is 8 for example, you're only going to be able to see 2 moves ahead for each player.
Player collusion is hard to account for. In a realistic game, some players might help each other out (even if they're not on the same team). This will cause them to deviate from their personal 'maximum'.
If you want Minmax, you're going to have to do a lot of pruning to make it viable. What I would suggest is learning a few patterns so the AI would know how to react. This can be done via neural net, or reinforcement learning with a few tweaks.
These patterns could be be static (you can create the input scenario manually or programmatically), or dynamic (create all valid scenarios and randomly makes moves select the ones with the best score).
Theoretically speaking, an "as good as possible AI" is a perfect AI, which is an AI that has, at any moment in the game, full knowledge of the game state (if the full game state is not known by human players). In case of games that everyone has full game state knowledge (like Blokus), a good as possible AI is an AI that can try to predict the very best move to make (minimax here, as you said). You can also google for genetic algorithms and simulated annealing, as they are valid, depending on what you want. Also, you can use minimax for more than 2 players.
I would recommend minimax algorithm. One thing you can add to make it more efficient (meaning you should be able go more moves deep into the future) is alpha-beta pruning.
The problem with minimax search is that the number of games states it has to examine is exponential in the depth of the tree. Unfortunately, we can't eliminate the exponent, but it turns out we can effectively cut it in half.
The quote is from Chapter 5.3 of Artificial Intelligence: A Modern Approach third edition by Stuart Russel and Peter Norvig. It was holding up my monitor, and I used it in a few of my classes in college. I know people don't often reference books on SO, but it's extremely relevant. I have used it extensively, and I do really recommend it for both being understandable, and covering a wide range of AI content.
It is available on amazon for $104, or * cough cough * I'm sure you can find it online if you don't have that kind of money for a textbook floating around. Looking up the minimax algorithm and alpha beta pruning online should also get you good results.
I think the only circumstance that would make Minimax a poor option for you is if the game state is only partially observable to any given player (they don't know everything about what's going on), or if the game is non-deterministic (it has random elements). Because neither of these are the case for Blokus, I think you made an excellent choice with Minimax.
The area of AI is called Adversarial Search in the textbook (Chapter 5: Adversarial Search), so looking up more info online with that term may get you more helpful information, or help you find an example Java implementation. I do not consider this a beginner's task, but it sounds like you are up to it, if you made the game and can pick random valid moves. Keep up the good work!
In 2011, with many updates since then, a program called Pentobi
was released, and it is a very strong Blokus playing program.
The only one known to date, in fact, which is any good at all, and it
surpasses all the others by a great deal. It will beat many good human players and gives even the best a run for their money.
Its main algorithm is Monte Carlo Search Tree, but it also uses a "book" of openings and some heuristics.
There is documentation and download information at
http://pentobi.sourceforge.net/
I found that using a very simple heuristic provides a fairly intelligent player even using only 1-step look ahead. I implemented what I called "space heuristic" which takes the board state and floods it by coloring all squares adjacent to each placed piece the color of that piece. Then, the total number of colored squares is counted once the flooding terminates. The space heuristic gives a rough estimate of how much a play claims or occupies board space, and way outperforms random play. Could be combined with minimax or MCTS to get way stronger as well.
Introduction
I had made a extremely simple simulation on real time shuffling ball and pick ball on some time interval
https://github.com/gaplo917/Mark6-java/blob/master/src/Mark6.java
What I want to simulate is :
There are total 49 ball in a machine, the machine will keep shuffling the ball. After a period of time, the machine will open the gate pick a ball out.
Here is the video : http://youtu.be/5QHsYA2lcI0?t=2m2s
What I had written ( extremely easy version ):
There are 49 ball and I made it for 49+1 space.
Each shuffle of the ball if the ball is shuffle into the extra space.
After a period of time, I will check the extra space to see the ball is swapped into it and pick it out.
Then mark the space with -1 to represent the spaces is no longer valid.
Instead, I should pick a ball randomly but not using an extra space. Just ignore this scenario...
After this work, I will try to use Android Game engine to simulate the gravity and collision for learning purpose.
Back to my question :
Is it a good way to represent used data by -1 ? Is it a normal approach in real world application?
A more Object-Oriented way to handle this would be to create a BallSlot class with separate members for ball number and for whether it's valid.
It's generally not a good idea to use sentinel values like 0 or -1. It overloads multiple semantic meanings into a single data value, which complicates parsing the value. Practically speaking, it means all the code that uses the value becomes riddled with if() statements.
It would be more Java-like to model the balls as objects, even if for now they only have the one field "id". You may very well want to add other properties (position, momentum, etc, if you will extend this to do physics sim as you indicate), and you will need them to be Objects then!
If you're just using int to represent the balls, yes, it's common practice to do something like "-1 means empty". Please do not become "clever" and start using "-2" and "-3" to indicated other sorts of things, however. =)
I am developing a chess game and at the moment I'm trying to implement a minimax algorithm. I haven't done this before, also the little i known about how to programmatically represent and implement the following evaluation function features(material, mobility, piece square table, centre control, trapped piece, king safety, tempo and pawn structure) is not quite clear to me (I will be grateful if someone can explain to me in detail). I have been able to assign values to each chess pieces, piece action values and a square table for each piece. The problem am having at the moment is how to generate Piece attacked and defended values which will be added or subtracted from the score. The idea here is that i want to reward the AI agent for protecting its pieces and penalize it for having the pieces attacked. thanks in advances.
Each of the evaluation features you mentioned will take up compute time. As you may already be aware, playing strength of a chess engine comes from two sources:
Search
Evaluation
And both contend for the same valuable resource, compute time. Evaluation tends to be heuristics based and hence a bit fuzzy, whereas search tends to yield more concrete and relevant results. If you are starting to build an engine then I would recommend focusing on search while keeping evaluation basic (but not weak!). That way you will be able to tell exactly where something went wrong and hence avoid possible early disappointments. Moreover, popular engines like Stockfish also started out by first building a strong search algorithm.
If you've been patient enough to read this far, let me point you to two useful resources for evaluation:
Chess Programming Wiki's evaluation page: This website is probably the best online resource for chess engine development in general.
Link to a basic but not weak evaluation function: This is C# code. Unfortunately I can't find the original article that I based this evaluation on.
Hope it helps :)
I think that you shouldn't include the computation on attack and defended pieces. That functionality is already taken into account by the minmax algorithm in a more efficient way.
A piece in under attack if at the following move the opponent can take it. If you try to evaluate this possibility in a static evaluation function you will get into troubles if you want to do it correctly. If my protected pawn is taken by the opponent queen that is not an issue. How do you take this into account? If my queen is taken by the opposite pawn but moving the pawn puts the king under attack?
These considerations are better managed by the minmax algorithm, not the evaluator. Consider that to know how many pieces you can eat/can be eaten, you should take into account all possible moves and you probably would spend the same time that would be used to go one level deeper in the minmax algorithm. Moreover that time is wasted if you later decide to indeed proceed one step further in the minmax.
Can anyone shed any light on how a program like that might be structured?
What java classes would they employ to keep track of so many particles and then check against them for things like collision detection? Particles need to know what particles they are next to, or that they're not next to anything so they can fall etc.
Here's an example, incase you're not sure what a sand game is.
Arrays, mainly.
A one-dimensional array of actively moving grains, represented by two coordinates (and possible a velocity if you want gravitational acceleration).
A two-dimensional array of bools (or colours), representing the fixed parts of the world.
The simplest physics model is to remove a grain once it is at rest (e.g. when the world positions below, below to the left and below to the right are filled): instead, the corresponding world coordinate is filled in. This keeps the calculations manageable. Allow a grain to shift down if either the left or right below world coordinate is free. Grain collisions between moving grains can be ignored without loss of much verisimilitude.
(I can't quite get over how much CPU power we've got to spare these days!)
The simple version can be implemented without much trouble on a loose friday night (like I did just now). Simply make a program with a 2D array (of bytes, ints, whatever) representing your field. In each tick, iterate over all the elements and do a check:
If the field below (array[x][y+1]) is empty, move 1 spot down (= set [x][y] to empty, [x][y+1] to occupied)
Else, if [x-1][y+1] is empty, go there.
Else, if [x+1][y+1] is empty, go there.
That's the basics. Next you have to add checks like not 'repeating' the calculation for a grain (set a flag at the new position that's checked at the following iterations).
I followed this tutorial, it's quite good, not too long but still points out common pitfalls and things like that.