We Keep Coding

Matching images from file with Sikuli

I just found about Sikuli when I was looking for a library to find matches of a given image within a larger image (both loaded from files).
By default, Sikuli only supports loading the searched image from file, but relies on a proprietary class Screen to take screenshots to use as base for the search... And I'd like to have the ability to use a image file instead.
Looking for a solution has led me to this question, but the answer is a bit vague when you consider that I have no prior experience with Sikuli and the available documentation is not particularly helpful for my needs.
Does anyone have any examples on how to make a customized implementation of Screen, ScreenRegion, ImageScreen and ImageScreenLocation? Even a link to a more detailed documentation on these classes would be a big help.
All I want is to obtain the coordinates of an image match within another image file, so if there's another library that could help with this task I'd more than happy to learn about it!

You can implement it by yourself with something like this:
class MyImage{
private BufferedImage img;
private int imgWidth;
private int imgHeight;
public MyImage(String imagePath){
try{
img = ImageIO.read(getClass().getResource(imagePath));
}catch(IOException ioe){System.out.println("Unable to open file");}
init();
}
public MyImage(BufferedImage img){
this.img = img;
init();
}
private void init(){
imgWidth = img.getWidth;
imgHeight = img.getHeight();
}
public boolean equals(BufferedImage img){
//Your algorithm for image comparison (See below desc for your choices)
}
public boolean contains(BufferedImage subImage){
int subWidth = subImage.getWidth();
int subHeight = subImage.getHeight();
if(subWidth > imgWidth || subHeight > imgHeight)
throw new IllegalArgumentException("SubImage is larger than main image");
for(int x=0; x<(imgHeight-subHeight); x++)
for(int y=0; y<(imgWidth-subWidth); y++){
BufferedImage cmpImage = img.getSumbimage(x, y, subWidth, subHeight);
if(subImage.equals(cmpImage))
return true;
}
return false;
}
}
The contains method will grab a subimage from the main image and compare with the given subimage. If it is not the same, it will move on to the next pixel until it went through the entire image. There might be other more efficient ways than moving pixel by pixel, but this should work.
To compare 2 images for similarity
You have at least 2 options:
Scan pixel by pixel using a pair of nested loop to compare the RGB value of each pixel. (Just like how you compare two int 2D array for similarity)
It should be possible to generate a hash for the 2 images and just compare the hash value.

Aah... Sikuli has an answer for this too... You just didnt look close enough. :)
Answer : The FINDER Class
Pattern searchImage = new Pattern("abc.png").similar((float)0.9);
String ScreenImage = "xyz.png"; //In this case, the image you want to search
Finder objFinder = null;
Match objMatch = null;
objFinder = new Finder(ScreenImage);
objFinder.find(searchImage); //searchImage is the image you want to search within ScreenImage
int counter = 0;
while(objFinder.hasNext())
{
objMatch = objFinder.next(); //objMatch gives you the matching region.
counter++;
}
if(counter!=0)
System.out.println("Match Found!");

In the end I gave up on Sikuli and used pure OpenCV in my Android project: The Imgproc.matchTemplate() method did the trick, giving me a matrix of all pixels with "scores" for the likehood of that being the starting point of my subimage.

With Sikuli, you can check for the presence of an image inside another one.
In this example code, the pictures are loaded from files.
This code tell us if the second picture is a part of the first picture.
public static void main(String[] argv){
String img1Path = "/test/img1.png";
String img2Path = "/test/img2.png";
if ( findPictureRegion(img1Path, img2Path) == null )
System.out.println("Picture 2 was not found in picture 1");
else
System.out.println("Picture 2 is in picture 1");
}
public static ScreenRegion findPictureRegion(String refPictureName, String targetPictureName2){
Target target = new ImageTarget(new File(targetPictureName2));
target.setMinScore(0.5); // Precision of recognization from 0 to 1.
BufferedImage refPicture = loadPicture(refPictureName);
ScreenRegion screenRegion = new StaticImageScreenRegion(refPicture);
return screenRegion.find(target);
}
public static BufferedImage loadPicture(String pictureFullPath){
try {
return ImageIO.read(new File(pictureFullPath));
} catch (IOException e) {
e.printStackTrace();
return null;
}
}
To use Sikuli package, I added this dependency with Maven :
<!-- SIKULI libraries -->
<dependency>
<groupId>org.sikuli</groupId>
<artifactId>sikuli-api</artifactId>
<version>1.1.0</version>
</dependency>

KeyPoints opencv FAST detector android

I'm trying to use Opencv FAST detector setting a threshold in Android. I found a similar solved problem here. I have listed the keypoints list after detect method as suggested, but still doesn't work for me. In my case I want to detect the key points on my camera frame:
public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
MatOfKeyPoint points = new MatOfKeyPoint();
Mat mat=inputFrame.rgba();
FeatureDetector fast = FeatureDetector.create(FeatureDetector.FAST);
fast.detect(mat, points);
// Sort and select 500 best keypoints
List<KeyPoint> listOfKeypoints = points.toList();
Collections.sort(listOfKeypoints, new Comparator<KeyPoint>() {
#Override
public int compare(KeyPoint kp1, KeyPoint kp2) {
// Sort them in descending order, so the best response KPs will come first
return (int) (kp2.response - kp1.response);
}
});
List<KeyPoint> listOfBestKeypoints = listOfKeypoints.subList(0, 500);
points.fromList(listOfBestKeypoints);
Scalar redcolor = new Scalar(255,0,0);
Mat mRgba= mat.clone();
Imgproc.cvtColor(mat, mRgba, Imgproc.COLOR_RGBA2RGB,4);
Features2d.drawKeypoints(mRgba, points, mRgba, redcolor, 3);
return mRgba;
}
Problem is that my listOfKeypoints remains null. If I don't try to set the threshold the code works fine but too slow.
What am I doing wrong here?
Thanks.

I found out that my list is null only the first time onCameraFrame is called. So I can make it works using a class variable and populating my list starting from the second call.
private int count=0;
And then in onCameraFrame;
if (count!=0) {
//Sort and select 500 best keypoints
List<KeyPoint> listOfKeypoints = points.toList();
Collections.sort(listOfKeypoints, new Comparator<KeyPoint>() {
#Override
public int compare(KeyPoint kp1, KeyPoint kp2) {
// Sort them in descending order, so the best response KPs will come first
return (int) (kp2.response - kp1.response);
}
});
List<KeyPoint> listOfBestKeypoints = listOfKeypoints.subList(0, 500);
points.fromList(listOfBestKeypoints);
}
count++;
In this way it works, but I still don't understand why on the first call of onCameraFrame the list is null. Any ideas?

programmatically edit images for puzzle usage

I'm making a sliding puzzle ( 3x3/4x4/5x5 with the lower-right corner cut out ). However I can't figure out where to start with programmatically cutting images ( which will be loaded from own gallery in sdcard or database from app ) in the puzzle pieces.
I've been looking over the internet and nothing really helped me.
What is the best way to cut up this image and store it in a new database (and still will be able to slide them)? Just a push in the right direction would be appreciated.

Check the PhotoGaffe app..
Its available on Google code here.
It allows user to choose between 3x3, 4x4, 5x5, and 6x6 puzzles.
This may help you in doing your task.

Straight from something I'm working on at the moment!
Bitmap main = BitmapFactory.decodeResource(getResources(), R.drawable.puzzle);
if( main.getHeight() > main.getWidth() ){
rescalefactor =((float)screenHeight)/main.getHeight();}
else {
rescalefactor = ( (float)screenWidth)/main.getWidth();
}
main = Bitmap.createScaledBitmap(main,(int)(main.getWidth()*rescalefactor),(int)(main.getHeight()*rescalefactor), false);
Bitmap cropped;
LinearLayout layout[] = new LinearLayout[rows];
int x=0,y=0,i=0,j=0,width=main.getWidth()/column,height=main.getHeight()/rows;
int count = 1;
for(i=0;i<rows;++i)
{
layout[i] = new LinearLayout(this);
for(j=0;j<column;++j)
{
cropped = Bitmap.createBitmap(main,x,y,width,height);
image[i][j] = new Tile(this);
image[i][j].setImageBitmap(cropped);
image[i][j].row =i;image[i][j].column =j;
image[i][j].setPadding(1, 1, 1, 1);
image[i][j].setOnClickListener(this);
image[i][j].setDrawingCacheEnabled(true);
image[i][j].setId(count); count++;
layout[i].addView(image[i][j]);
x += width;
}
x = 0; y += height;
root.addView(layout[i]);
}
This is the line where the work is really done:
cropped = Bitmap.createBitmap(main,x,y,width,height);
The Tile class is super simple. Just an extended ImageView with row and column fields:
public class Tile extends ImageView {
public int row, column;
public Tile(Context context)
{ super(context);}
}

QRCode application with customization

I am working with QRCode api found here
I successfully implemented the QRCode generation through Api but the result is this
(I changed the color from White to yellow in order to ask my question)
So, now if you see you find that the outer boundary is very thick I want it to be thin..like this
The code I used to generate the qrcode and changing the color is this
public boolean writeImage(String qrMessageForGeneratingQRCode,String filename){
boolean result = false;
try{
int length = 200;
int breadth = 200;
BufferedImage originalQRCodeBufferedImage = ImageIO.read(new ByteArrayInputStream(QRCode.from(qrMessageForGeneratingQRCode).withSize(length,breadth).stream().toByteArray()));
BufferedImage changedQRCodeBufferedImage = new ColorChanger().changeColor(originalQRCodeBufferedImage, Color.WHITE, new Color(255,202,0));
ImageIO.write(changedQRCodeBufferedImage,FilesUtil.getProperty("QR_CODE_IMAGE_FORMAT") , new File(filename));
result = true;
}catch(Exception ex){
ex.printStackTrace();
result = false;
}
return result;
}
Please shed some light how can I achieve that using code....Thanks in advance....
Ankur

The quiet zone around the QR code needs to be 4 modules. You don't want to reduce this. Your proposed image will be harder to scan.
You can always edit the BufferedImage after the fact with a simple crop:
BufferedImage crop = original.getSubimage(50, 50, original.getWidth() - 2*50, original.getHeight() - 2*50);
BTW you should update the underlying zxing library used in your solution to at least 2.0, probably 2.1-SNAPSHOT. 1.7 is old.

Obvious answer would be to reduce the length and breadth values. Did you try that?

How to dynamically construct 3d object by adding paths? (Java; OpenGL)

So I have some path generator which now works like this
http://www.openprocessing.org/visuals/?visualID=2615 (There is source; WQRNING - JAVA APPLET)
I want to create some 3D object using paths I generated so it locked in one of perspectives similar to what I get now in 2D.
So how do I dynamically construct 3D object by adding paths?
BTW: actually I ment algorithm like this http://www.derschmale.com/2009/07/20/slice-based-volume-rendering-using-pixel-bender/
So I want to create from such PATH (I do not want to use images and I do not want to use flash I want to use Java + OpenGl)
such 3d image (But note I want openGL Java and Path's))

I'm not sure I understand what you're after.
The example you supplied draws 2d paths, but merely uses z. scaling would have worked
in a similar way.
So How to dinamicly construct 3d
object by adding path's ?
Do you mean extruding/lathing an object, or replicating the scrunch sketch ?
Drawing a path is easy in processing, you just place vertex objects, in a for loop
between beginShape() and endShape() calls.
Here is the bit of code that does that in the example you've sent:
beginShape();
for (int p=0; p<pcount; p++){
vertex(Ring[p].position().x(),Ring[p].position().y());
}
endShape(CLOSE);
you can also call vertex(x,y,z)
I wanted to extrude a path a while back, here is my question in case it helps.
Basic sketch is uploaded here.
EDIT:
If you have an array of 2 polygons, you can just loop through them, and draw
using something similar to beginShape() and endShape(), GL_POLYGON might be handy.
e.g.
import processing.opengl.*;
import javax.media.opengl.*;
int zSpacing = 10;
PVector[][] slices;
void setup() {
size(600, 500, OPENGL);
slices = new PVector[3][3];
//dummy slice 1
slices[0][0] = new PVector(400, 200,-200);
slices[0][1] = new PVector(300, 400,-200);
slices[0][2] = new PVector(500, 400,-200);
//dummy slice 2
slices[1][0] = new PVector(410, 210,-200);
slices[1][1] = new PVector(310, 410,-200);
slices[1][2] = new PVector(510, 410,-200);
//dummy slice 3
slices[2][0] = new PVector(420, 220,-200);
slices[2][1] = new PVector(320, 420,-200);
slices[2][2] = new PVector(520, 420,-200);
}
void draw() {
background(255);
PGraphicsOpenGL pgl = (PGraphicsOpenGL) g; // g may change
GL gl = pgl.beginGL(); // always use the GL object returned by beginGL
for(int i = 0 ; i < slices.length; i ++){
gl.glColor3f(0, .15 * i, 0);
gl.glBegin(GL.GL_POLYGON);
for(int j = 0; j < slices[i].length; j++){
gl.glVertex3f(slices[i][j].x, slices[i][j].y,slices[i][j].z + (zSpacing * i));
}
gl.glEnd();
}
pgl.endGL();
}
The idea is you loop through each slice, and for each slice your loop through all its points. Obviously slices and the number of 3d vectors inside each slice is up to your data. Speaking of which, where does your data come from ?
If slices is not what your after volTron could come in handy:
volTron http://dm.ncl.ac.uk/joescully/voltronlib/images/s2.jpg
HTH,
George