Suppose I am building a Neural Net to play tic-tac-toe. As I understand it, a good design for the net would be 9 input neurons [one for each square in the grid] 3 hidden layer neurons and 9 output neurons [one for each potential move]. Now that the Net is built, how do you read the output of the net?
As I understand it, to train the net, I would send the net the game board [via the inputs] everytime I need it to make a decision. But how do I obtain that decision?
The key thing to realize here is that multiple output neural networks change their state to reflect a solution space, but they do not typically give you a hard and fast, final decision.
Ultimately, a neural net doesnt give you "just one answer" , but rather, it modifies its internal state to reflect a probabilistic landscape of solutions.
If you just want a single answer, then you will have to have a nueral net with only one output node.
There is no direct, right answer here
The question you have is actually quite sophisticated - the science of choosing an answer from a nueral net is an entire field of study, in and unto itself :
For some other insights, check out https://mathoverflow.net/questions/10697/methods-for-choosing-a-result-from-a-multiple-output-node-neural-network .\
Also, scan other resources for decision making methods for multiple-output neural nets.
Related
I have a challenge that my teacher gave to beat an army of his soldiers on a 18x24 grid, with random obstacles placed on the board. The game is turn based and I have an army of 50 soldiers, each of which needs to either move or attack on their turn.
My problem is I only have access to creating a class of soldiers to fight in this environment. Currently I have a method that evaluates the board position by looking at how many soldiers there are left from each team and does yourTeam - enemyTeam to get the current score, and I have a method that will produce the legal moves for the soldier.
I want to know how I would create a reinforcement learning agent in Java with what I have access to. If you know any ways to do this or any resources that may help that would be great. Thank you for the help!
Java is not a good language for doing math heavy computation (which is what you will need to do for RL). You could attempt to implement the Q-Learning, value-iteration or policy-iteration algorithms but I would avoid doing anything with neural networks/modern deep RL approaches here as your work load will increase dramatically.
With regard to your problem, if you are to implement one of the old-school algorithms. Think about your state and action space. I have serious concerns about the size of your action space, even with a small number of moves for each solider (say 3 - attack, move up, move down) with 50 soldiers the action space will be very large - 50^3, even this many will be difficult to deal with, any more (even 4 or 5) will send you deep into some complex topics in RL.
Other problems are - defining a good reward signal, efficiently running (potentially millions) of simulated games.
The short answer is, this is not something to be taken lightly, it would be challenging and time consuming even for someone who has experience in the field and using Java is a no-no (Python is better). Given you probably don't have long to find a good solution, I would recommend trying a different approach - planning based maybe, or hard coding a reasonable strategy.
If you still want to go ahead and read up on the topic here are some good resources:
Reinforcement Learning an Introduction (Sutton & Barto) - any edition is fine
Selected chapters in Artificial Intelligence: A Modern Approach (Russel & Norvig)
Hope this helps and sorry it may not have been the answer you we hoping for!
I am creating an evolution simulator in Java. The simulation consists of a map with cold/hot regions and high/low elevation, etc'.
I want the creatures in the world to evolve in two ways- every single creature will evolve it's AI during the course of his lifetime, and when a creature reproduces there is a chance for mutation.
I thought it would be good to make the brain of the creatures a neural network that takes the sensor's data as input (only eyes at the moment), and produces commands to the thrusters (which move the creature around).
However, I only have experience with basic neural networks that recieve desired inputs from the user and calculate the error accordingly. However in this simulator, there is no optimal result. Results can be rated by a fitness function I have created (which takes in count energy changes, amount of offsprings, etc'), but it is unknown which output node is wrong and which is right.
Am I using the correct approach for this problem? Or perhaps neural networks are not the best solution for it?
If it is a viable way to achieve what I desire, how can I make the neural network adjust the correct weights if I do not know them?
Thanks in advance, and sorry for any english mistakes.
You ran into a common problem with neural networks and games. As mentioned in the comments a Genetic algorithm is often used when there is no 'correct' solution.
So your goal is basically to somehow combine neural networks and genetic algorithms. Luckily somebody did this before and described the process in this paper.
Since the paper is relatively complex and it is very time consumeing to implemwnt the algorithm you should consider using a library.
Since I couldn't find any suiting library for me, I decided to write my own one, you can find it here
The library should work good enough for 'smaller' problems like yours. You will find some example code in the Main class.
Combine networks using
Network.breedWith(Network other);
Create networks using
Network net = new Network(int inputs, int outputs);
Mutate networks using
Network.innovate();
As you will see in the example code it is important to always have an initial amount of mutations for each new network. This is because when you create a new network there are no connections, so innovation (fancy word for mutation) is needed to create connections.
If needed you can always create copys of networks (Network.getCopy();). The Network class and all of its attributes implement serializable, so you can save/load a network using an ObjectOutputStream.
If you decide to use my library please let me know what results you got!
I've been using a formula for some time to try to find value in spreads for sports betting and done this by basically creating my own spread and comparing to what bookies offer and would like to automate the process.
I've written some code in java which will do the maths on the data I give it and I'm looking for a way to populate the input data either from a database or from an xml file I create but i'm quite new to programming.
Say for example if I pick two teams to compare. For each team I need a list of the teams they played, how many points each team scored in total, how many points each team conceded in total and how many games each team played so I can run the maths on those figures and I have no idea where to start. Could anyone help me or point me in the right direction?
It sounds like you've defined your problem (how to start), and also listed the information you need to get started (compare two teams, points, previous games, conceded points, etc). Are you sure you don't know how to start?
For a point in the right direction - I recommend creating a test case where you select two teams, give them some sample data for their previous games, points scored and conceded, and start working on the structure of your program.
This question lends itself too much to personal opinion and personal experience, and that makes it difficult to give you definitive answers without looking at any code or a program layout.
Give it your best effort and reply back with what you come up with, that will be much easier to critique and offer suggestions to.
The problem - I have 10 number of cards value 1 to 10. Now I have to arrange the cards in away that adding 5 cards gives me 36 and product of remaining 5 cards give me 360.
I had successfully made a GA to solve cards Problem in java. Now I am thinking to solve same problem with Neural Network. Is it possible to solve this by NN? What approach should I take?
This problem is hard to solve directly with a Neural Network. Neural Networks are not going to have a concept of sum or product, so they won't be able to tell the difference between a valid and invalid solution directly.
If you created enough examples and labelled then then the neural network might be able to learn to tell the "good" and "bad" arrangements apart just by memorising them all. But it would be a very inefficient and inaccurate way of doing this, and it would be somewhat pointless - you'd have to have a separate program that knew how to solve the problem in order to create the data to train the neural network.
P.S. I think you are a bit lucky that you managed to get the GA to work as well - I suspect it only worked because the problem is small enough for the GA to try most of the possible solutions in the vicinity of the answer(s) and hence it stumbles upon a correct answer by chance before too long.
To follow up on #mikera's comments on why Neural Networks (NNs) might not be best for this task, it is useful to consider how NNs are usually used.
A NN is usually used in a supervised learning task. That is, the implementer provides many examples of input and the correct output that goes with that input. The NN then finds a general function which captures the provided input/output pairs and hopefully captures many other previously unseen input/output pairs as well.
In your problem you are solving a particular optimization, so there isn't much training to be done. There is just one (or more) right answers. So, NNs aren't really designed for such problems.
Note that the concept of not having a sum/product doesn't necessarily hurt a NN. You just have to create your own input layer which has sum and product features so that the NN can learn directly from these features. But, in this problem it won't help very much.
Note also that your problem is so small that even a naive enumeration of all combinations (10! = 3,628,800) of numbers should be achievable in a few seconds at most.
I would like to know how practical it would be to create a program which takes handwritten characters in some form, analyzes them, and offers corrections to the user. The inspiration for this idea is to have elementary school students in other countries or University students in America learn how to write in languages such as Japanese or Chinese where there are a lot of characters and even the slightest mistake can make a big difference.
I am unsure how the program will analyze the character. My current idea is to get a single pixel width line to represent the stroke, compare how far each pixel is from the corresponding pixel in the example character loaded from a database, and output which area needs the most work. Endpoints will also be useful to know. I would also like to tell the user if their character could be interpreted as another character similar to the one they wanted to write.
I imagine I will need a library of some sort to complete this project in any sort of timely manner but I have been unable to locate one which meets the standards I will need for the program. I looked into OpenCV but it appears to be meant for vision than image processing. I would also appreciate the library/module to be in python or Java but I can learn a new language if absolutely necessary.
Thank you for any help in this project.
Character Recognition is usually implemented using Artificial Neural Networks (ANNs). It is not a straightforward task to implement seeing that there are usually lots of ways in which different people write the same character.
The good thing about neural networks is that they can be trained. So, to change from one language to another all you need to change are the weights between the neurons, and leave your network intact. Neural networks are also able to generalize to a certain extent, so they are usually able to cope with minor variances of the same letter.
Tesseract is an open source OCR which was developed in the mid 90's. You might want to read about it to gain some pointers.
You can follow company links from this Wikipedia article:
http://en.wikipedia.org/wiki/Intelligent_character_recognition
I would not recommend that you attempt to implement a solution yourself, especially if you want to complete the task in less than a year or two of full-time work. It would be unfortunate if an incomplete solution provided poor guidance for students.
A word of caution: some companies that offer commercial ICR libraries may not wish to support you and/or may not provide a quote. That's their right. However, if you do not feel comfortable working with a particular vendor, either ask for a different sales contact and/or try a different vendor first.
My current idea is to get a single pixel width line to represent the stroke, compare how far each pixel is from the corresponding pixel in the example character loaded from a database, and output which area needs the most work.
The initial step of getting a stroke representation only a single pixel wide is much more difficult than you might guess. Although there are simple algorithms (e.g. Stentiford and Zhang-Suen) to perform thinning, stroke crossings and rough edges present serious problems. This is a classic (and unsolved) problem. Thinning works much of the time, but when it fails, it can fail miserably.
You could work with an open source library, and although that will help you learn algorithms and their uses, to develop a good solution you will almost certainly need to dig into the algorithms themselves and understand how they work. That requires quite a bit of study.
Here are some books that are useful as introduct textbooks:
Digital Image Processing by Gonzalez and Woods
Character Recognition Systems by Cheriet, Kharma, Siu, and Suen
Reading in the Brain by Stanislas Dehaene
Gonzalez and Woods is a standard textbook in image processing. Without some background knowledge of image processing it will be difficult for you to make progress.
The book by Cheriet, et al., touches on the state of the art in optical character recognition (OCR) and also covers handwriting recognition. The sooner you read this book, the sooner you can learn about techniques that have already been attempted.
The Dehaene book is a readable presentation of the mental processes involved in human reading, and could inspire development of interesting new algorithms.
Have you seen http://www.skritter.com? They do this in combination with spaced recognition scheduling.
I guess you want to classify features such as curves in your strokes (http://en.wikipedia.org/wiki/CJK_strokes), then as a next layer identify componenents, then estimate the most likely character. All the while statistically weighting the most likely character. Where there are two likely matches you will want to show them as likely to be confused. You will also need to create a database of probably 3000 to 5000 characters, or up to 10000 for the ambitious.
See also http://www.tegaki.org/ for an open source program to do this.