I am trying to implement a small recognition program, which should take an image with hand-written shapes as an input (shape objects can be nested), and tell me which shapes it consists of, their type, size and at least coordinates of one point in a shape.
Where should I start?
Maybe, some machine learning technics could be used for such a task?
I've tried to look for some examples or tutorials dealing with handwriting recognition of custom shapes, but haven't found something worth looking at.
I think you should take a look at Coursera Machine Learning course taught by Andrew Ng from Stanford.
It will guide you through techniques of ML resulting in writing your own digit recogniting neural network.
It's really deep field and I believe that simply using some tutorial will be not enough - understanding of subject and techniques that you will use is really important.
Related
i was wondering if anyone has knowledge on the recontruction of 3D objects from live video feed. Does any have any java based examples or papers JAVA based that i could be linked to as i have read up on algorithm's used to produce such 3d objects. If possible i would like to construct something such as the program demostrated in the link provided below.
Currently my program logs live video feed.
http://www.youtube.com/watch?v=brkHE517vpo&feature=related
3D reconstruction of an object from a single point of view is not really possible. You have two basic alternatives: a) To have a stereo camera system capturing the object, b) To have only one camera, but rotating the object (so you will have different points of view of the object), like the one in the video. This is a basic concept related with epipolar geometry.
There are other alternatives, but more intrusive. Some time ago I've been working on a 3D scanner based on a single camera and a laser beam.
For this, I used OpenCV which is C++ code, but now I think there are ports for Java. Have in mind that 3D reconstruction is not an easy task, and the resulting app. will have to be largely parametrized to achieve good results.
This isn't a solved problem - certain techniques can do it to a certain degree under the right conditions. For example, the linked video shows a fairly simple flat-faced object being analysed while moving slowly under relatively even lighting conditions.
The effectiveness of such techniques can also be considerably improved if you can get a second (stereo vision) video feed.
But you are unlikely to get it to work for general video feeds. Problem such as uneven lighting, objects moving in front of the camera, fast motion, focus issues etc. make the problem extremely hard to solve. The best you can probably hope for is a partial reconstruction which can then be reviewed and manually edited to correct the inevitable mistakes.
JavaCV and related projects are probably the best resource if you want to explore further. But don't get your hopes too high for a magic out-of-the-box solution!
I would like to know how practical it would be to create a program which takes handwritten characters in some form, analyzes them, and offers corrections to the user. The inspiration for this idea is to have elementary school students in other countries or University students in America learn how to write in languages such as Japanese or Chinese where there are a lot of characters and even the slightest mistake can make a big difference.
I am unsure how the program will analyze the character. My current idea is to get a single pixel width line to represent the stroke, compare how far each pixel is from the corresponding pixel in the example character loaded from a database, and output which area needs the most work. Endpoints will also be useful to know. I would also like to tell the user if their character could be interpreted as another character similar to the one they wanted to write.
I imagine I will need a library of some sort to complete this project in any sort of timely manner but I have been unable to locate one which meets the standards I will need for the program. I looked into OpenCV but it appears to be meant for vision than image processing. I would also appreciate the library/module to be in python or Java but I can learn a new language if absolutely necessary.
Thank you for any help in this project.
Character Recognition is usually implemented using Artificial Neural Networks (ANNs). It is not a straightforward task to implement seeing that there are usually lots of ways in which different people write the same character.
The good thing about neural networks is that they can be trained. So, to change from one language to another all you need to change are the weights between the neurons, and leave your network intact. Neural networks are also able to generalize to a certain extent, so they are usually able to cope with minor variances of the same letter.
Tesseract is an open source OCR which was developed in the mid 90's. You might want to read about it to gain some pointers.
You can follow company links from this Wikipedia article:
http://en.wikipedia.org/wiki/Intelligent_character_recognition
I would not recommend that you attempt to implement a solution yourself, especially if you want to complete the task in less than a year or two of full-time work. It would be unfortunate if an incomplete solution provided poor guidance for students.
A word of caution: some companies that offer commercial ICR libraries may not wish to support you and/or may not provide a quote. That's their right. However, if you do not feel comfortable working with a particular vendor, either ask for a different sales contact and/or try a different vendor first.
My current idea is to get a single pixel width line to represent the stroke, compare how far each pixel is from the corresponding pixel in the example character loaded from a database, and output which area needs the most work.
The initial step of getting a stroke representation only a single pixel wide is much more difficult than you might guess. Although there are simple algorithms (e.g. Stentiford and Zhang-Suen) to perform thinning, stroke crossings and rough edges present serious problems. This is a classic (and unsolved) problem. Thinning works much of the time, but when it fails, it can fail miserably.
You could work with an open source library, and although that will help you learn algorithms and their uses, to develop a good solution you will almost certainly need to dig into the algorithms themselves and understand how they work. That requires quite a bit of study.
Here are some books that are useful as introduct textbooks:
Digital Image Processing by Gonzalez and Woods
Character Recognition Systems by Cheriet, Kharma, Siu, and Suen
Reading in the Brain by Stanislas Dehaene
Gonzalez and Woods is a standard textbook in image processing. Without some background knowledge of image processing it will be difficult for you to make progress.
The book by Cheriet, et al., touches on the state of the art in optical character recognition (OCR) and also covers handwriting recognition. The sooner you read this book, the sooner you can learn about techniques that have already been attempted.
The Dehaene book is a readable presentation of the mental processes involved in human reading, and could inspire development of interesting new algorithms.
Have you seen http://www.skritter.com? They do this in combination with spaced recognition scheduling.
I guess you want to classify features such as curves in your strokes (http://en.wikipedia.org/wiki/CJK_strokes), then as a next layer identify componenents, then estimate the most likely character. All the while statistically weighting the most likely character. Where there are two likely matches you will want to show them as likely to be confused. You will also need to create a database of probably 3000 to 5000 characters, or up to 10000 for the ambitious.
See also http://www.tegaki.org/ for an open source program to do this.
I would like to implement a visualisation of this video in Java as experience to help me understand all of the 'troubles' in creating visualisations. I have some experience in OpenGL, and a good understanding of how to handle the physics involved. However, if anybody knows of any good game engines that may help (or at least do some of the heavy lifting involved in creating a visualisation of the above) I would be grateful.
Also, I noticed that the linked video must use many separate jets in order to operate in the way it does. Is it likely that it was created using something a little lower level such as C? Is it possible to use a higher level language like Java to control such a system?
Honestly, if you want to implement "just that", I think using a game engine is overkill. Just implement a simple particle engine on your own and you are done.
Seriously, that problem is not so difficult, any language can be used for it. The basic principle behind it is the same as behind steam organs or self player pianos. You have an input data that shows what the pattern to play is and you advance it in a given time.
Here is how I would build the basic control system. You take a black and white image. The width is exactly as wide as the number of "emitters" and the length is as long as the pattern needs to be. You read the image and start at the first line. You walk through each pixel in that line and if the pixel is black you emit a drop and if the pixel is white you don't. You then move in a given interval (maybe 25ms) to the next line and set the emitters accordingly.
The cool thing with images is that you can simply paint them in any graphic program. To get the current time to work you render the time into a image buffer in memory, then pass that into the above code. (You even get fonts if you like...)
You can use jMonkeyEngine.
JAVA OPEN GL GAME ENGINE
I'm reading "Beginning Java Game Programming 2/e" and the book heavily emphasizes using AffineTransform.
There is just one problem, it doesn't explain at all what it is, what it does, and the purpose of AffineTransform.
I have done some google searches, but they're just showing me matrix math... Could someone point me in the right direction?
That's because an affine transform is matrix math. It's any kind of mapping from one image to another that you can construct by moving, scaling, rotating, reflecting, and/or shearing the image. The Java AffineTransform class lets you specify these kinds of transformations, then use them to produce modified versions of images.
Answer is years late, but if anyone else is struggling with this then I recommend reading Introduction to 3D Game Programming With DirectX 11 by Frank Luna. It is a very good book. Though book is about DirectX 11 the first three chapters are just about the math concepts needed for graphics. The chapters explains the math well and provided lots of exercises to practice the topics at the end of the chapter. (You may doing 2D games but the math is the same, the vectors and matrices have just one less dimension.)
What should a small team choose for their first game, when they are low on time but have big ambitions? I'm an experienced programmer, but haven't done any game programming before this. My designer is very talented and artistic, he has worked with 3D about an year ago but hasn't done it since so it could take him some time to learn it again, and i'm not sure if he'll be able to do a good job at it even though his graphic design skills are terrific otherwise.
Our primary concern is to get the game finished as quickly as possible, and also to do it easily since this is my first game programming project. At the same time, we don't want to have any limitations that might hinder our progress later on, or otherwise make the game not fun in certain ways.
For example, i learnt that certain animations aren't possible in 2D such as rotation etc. I would like the ability to have the player's character be able to morph into animals and there must be the ability to shoot at the monsters, (like shooting an arrow and seeing it fly through and hit the other person). Are these things possible in 2D?
In the future, if we wanted to go from 3D to 2D, would it be possible without fully rewriting the game?
You don't have to use 3D to allow for a "3D look". Things like rotations, motions or transformations can be prerecorded or prerendered as animations or image sequences and seamlessly integrated into a 2D game.
The only thing that is not possible in 2D is to navigate freely within your "game space" (like walking or flying freely, turning arbitrarily etc.)
The main concern, however, when deciding for 2D or 3D should be the gameplay. There are games that absolutely need 3D (shooters, simulations), while others do perfectly without (adventures, puzzles, ...). So you don't actually have to decide but to choose the better fit for your game idea.
Personally, I'd avoid using 3D in your first game attempt if possible to eliminate all the constraints and hassles that come with it.
When using 3D you typically have to decide for a 3D framework which will heavily influence your software design, game look and feel and overall performance. Java3D for example brings a complicated class structure that you have to adjust to.
And a lot of effort goes into making that 3D stuff work at all. Simple things like rotating a square evolve into matrix operations incorporating quaternions. Every effect has to be done in the complex 3D world, and in such a way that its 2D projected look turns out the way you intended it.
Not to mention that 3D applications often suffer a very stereotypical look that is very hard to overcome.
In 2D, you literally avoid one dimension of complexity. You do everything exactly the way it is supposed to look, you can use standard graphic applications and open file formats to simplify the workflow between the designer and the developer. And a lot of pseudo-3D effects like parallax motion, depth of field and prerendered artwork allow for astonishing looks in a 2D world.
You should go 2D. You have stated several reasons to take this approach:
You are a small team, and 3D takes
more work.
It's your first game, so
you need to keep it as simple as
possible, or the project may fail
(you are exposed to a lot of risk!)
The 3D artist needs to relearn the
crafts!
You have a (seemingly
self-imposed) deadline.
Etc.
3D takes a lot of time, and it will be best invested in improving the rest of the game. Think of all the awesome games that existed previous to the 3D era.
Do not fear to accept restrictions: whilst 3D might give you lots of possibilities, 2D may result in more creative work!
I would considre a couple of factors:
Is it a mobile or applet game? If so, I would go for the 2D version. It's very hard to get 3D right under those circumstances.
What is your designer more comfortable with? Can he more quickly model, texture and animate a characte, rather than creating all the sprites needed for a 2D character?
Tool support. What output formats can your designer produce in both 2D and 3D? What kind of formats is the middle ware, you are going to use, able to handle? Can your 3D engine load, or at least convert, .obj files (just an example, could have used any other model format.)?
Even if you would like to go the 3D way, you can still make a 2D game (sometimes called 2.5D or pseudo 3D). Also there are some nice third party frameworks out there for both 2D and 3D stuff in Java. You don't have to use java2D/java3D.
Given your requirements, I'd choose 2D . You have more tools to choose from . more available talent to contract for artwork .. and the asset pipeline is simpler. In the end it's going to be all about how good the game is on its own merits. 3D is sometimes more difficult for your average user to figure out.
Depending on what you are conceptualizing, rotations and morphing oughta be quite possible in 2D. You might even consider a pseudo-3d approach ala Paper Mario ..
Moving from 2D to 3D should not be a major issue if you design your code structure with that in mind. ..
I would recommend 2D as well. A 3D game will be more graphically demanding on the user's computer, and many things like collision detection are significantly simpler in 2D. You could even make your character models in 3D and then project them down to 2D to make the sprites for the actual game. Puzzle Pirates (http://www.puzzlepirates.com/) takes this approach and it works very well in terms of consistent lighting, etc.
I think writing a 2D or a full 3D game needs a completely different design approach what cannot be modified easily later: SUN provides a Java 2D API and a Java 3D API for the two development decisions.
For the first game with tight deadline I would vote for a 2D version. If the gameplay is interesting and the design is nice then the 3D is not a must.