finding image in Sikuli X Java

Question

I am having an issue trying to find an image on the screen, I tried doing it in two different ways and it doesn't seem to work for me. I am trying to do this with Appium running on IOS simulator which shows up on the screen, so I don't see this being a problem of a screenshot being taken.

I am running MAC OSX El Capitan I have imported the Sikuli X java API in my project

Do I need to also import the MAC Sikuli Library jar?

This is what I have tried so far:

1.

Screen s = new Screen();
Pattern test = new Pattern("/Users/ealiaj/Desktop/Automation/workspace/WheelsUp - IOS/screenshot.jpg");
try {
    s.find(test);
} catch (FindFailed e) {

}

2.

Screen s = new Screen();
try {
    s.find("screenshot.jpg");
} catch (FindFailed e) {

}

I keep getting cannot find errors.

error message:

FindFailed: can not find /Users/ealiaj/Desktop/Automation/workspace/WheelsUp - IOS/screenshot1.jpg in S(0)[0,0 1440x900] Line 2189, in file Region.java

This is the image on the screen, The large red rectangle is the image I have created a screenshot for and try to find, but get that error.

The only thing I am able to successfully find is that gray rectangle, or at least it doesn't throw an error for.

You can do this without using Sikuli. Create function in appium which capture screen shot of particular element (what you want to verify ) save it in you system at run time. And match with your base image file using Java code. — Sadik Ali, Jan 06 '16 at 04:52
@SadikAli I am also interested in the way you showed in your comment. Can you please add a sample or link to any (documentation) source ? — Slavo, Jan 14 '16 at 10:34
Are you able to find element of locator of images which you want to verify? If yes please reply, so I can prepare some code which will help you. — Sadik Ali, Jan 15 '16 at 07:10
@SadikAli Yes I am able to find locators for the elements. Thanks. — Elsid, Jan 15 '16 at 14:59

score 2 · Answer 1 · answered Feb 02 '16 at 20:44

2

You can use this method to verify images:

@Test
public void verifyImages() {    

    //WebElement img = driver.findElementByClassName("android.widget.ImageView");

   //take screen shot
    File screen = ((TakesScreenshot) driver)
                        .getScreenshotAs(OutputType.FILE);


    //capture image of searched contact icon
    List<WebElement > imageList = driver.findElementsByXPath("//*[@class='android.widget.ImageView' and @index='0']");
    System.out.println(imageList.size());

    System.out.println(i);
    WebElement image = imageList.get(1);
    Point point = image.getLocation();

    //get element dimension
    int width = image.getSize().getWidth();
    int height = image.getSize().getHeight();

    BufferedImage img = ImageIO.read(screen);
    BufferedImage dest = img.getSubimage(point.getX(), point.getY(), width,
                                                                 height);
    ImageIO.write(dest, "png", screen);
    File file = new File("Menu.png");
    FileUtils.copyFile(screen, file);

    //verify images
    verifyImage("Menu.png", "Menu.png" );
}



public void verifyImage(String image1, String image2) throws IOException{
    File fileInput = new File(image1);
    File fileOutPut = new File(image2);

    BufferedImage bufileInput = ImageIO.read(fileInput);
    DataBuffer dafileInput = bufileInput.getData().getDataBuffer();
    int sizefileInput = dafileInput.getSize();                     
    BufferedImage bufileOutPut = ImageIO.read(fileOutPut);
    DataBuffer dafileOutPut = bufileOutPut.getData().getDataBuffer();
    int sizefileOutPut = dafileOutPut.getSize();
    Boolean matchFlag = true;
    if(sizefileInput == sizefileOutPut) {                         
       for(int j=0; j<sizefileInput; j++) {
             if(dafileInput.getElem(j) != dafileOutPut.getElem(j)) {
                   matchFlag = false;
                   break;
             }
        }
    }
    else                            
       matchFlag = false;
    Assert.assertTrue(matchFlag, "Images are not same");    
 }

answered Feb 02 '16 at 20:44

Gaurav

1,332
11
22

Does this verify that two images are exactly the same? I am trying to find an image within the screen. For example to find a button in the whole image. Thanks – Elsid Feb 02 '16 at 20:56
yes, it does find the two images which are exactly same. I used it to test 2 images on my side and it works. Thanks. – Gaurav Feb 02 '16 at 20:59
Hi Gaurav, to be clear, I have image1 which is the whole screen, and image2 which is a button, this will find image2 inside of image1? – Elsid Feb 02 '16 at 21:00
Yes, just pass the locator like in my code i am trying to match the contact icon in the whole screen. – Gaurav Feb 02 '16 at 21:01
Test it and if you run with problem, i will help you. Thanks – Gaurav Feb 02 '16 at 21:01
Thanks, I use VerifyImage for a single image correct? Or do I need to use both methods? – Elsid Feb 02 '16 at 21:05
Well verifyImage is a method where you pass the two images to verify. And second method "verifyImages" is the test which u pass the locator and get the screenshots to verify the images. – Gaurav Feb 02 '16 at 21:06
Ok will mess around with it, and let you know if i need any help. Thanks – Elsid Feb 02 '16 at 21:08
Hey I see that this takes a screenshot of the element, what If i have the file stored locally already how can I use that instead. Thanks – Elsid Feb 02 '16 at 21:30
First it takes the screenshot at run time and it compares it with the image you have in your project to compare against. Look at the verifyImage method, you pass 2 images. First one, you get from screenshot and second one you have stored in project – Gaurav Feb 02 '16 at 21:48
For verifyImage("Menu.png", "Menu.png" ); can I put the filepath to the file? – Elsid Feb 02 '16 at 21:57
try it...You have to figure out some stuff by yourself..i can help you when you get stuck but first try and test it. Understand the code carefully. – Gaurav Feb 02 '16 at 21:58
Just tried it, but this is not what I want. This is basically creating a screenshot of an element, and then comparing it to make sure both images are the same. I have some text images, that are in 2-3 different elements. I need to take a screenshot of the whole screen, and find an image within. This only compares that 2 screenshots are identical, which they are not, since Screen 1 is the whole screen and screen 2 is an element. – Elsid Feb 02 '16 at 22:41
@Elsid yes just modify it as per your needs. I gave you the code where you can modify it as per your needs. Should be simple for you – Gaurav Feb 02 '16 at 22:47

score 1 · Answer 2 · answered Feb 02 '16 at 15:39

1

The error message says that is the program looking a .PNG file, and in your code your are putting a .JPG file.

answered Feb 02 '16 at 15:39

Orejano

1,824
2
16
16

Sorry that was copied from when I changed the filetype to .PNG it is .PNG is both. – Elsid Feb 02 '16 at 19:29

finding image in Sikuli X Java

2 Answers2