How to compare a picture in Appium, using Python code?

Question

I need to compare template picture in iOS app. What syntax I need to use for it? I found so many information about it, but only with Java syntax.

I use Appium + Python on Mac.

score 1 · Answer 1 · answered Feb 08 '22 at 08:51

I couldn't use openCV library. Because I cannot install opencv4nodejs
You can try to install it like this way, with npm module: npm i -g opencv4nodejs So, I found solution to compare images. You can use this code:

/**
 * Finds the a region in one image that best matches another, smaller, image.
 */
public static int[] findSubImage(BufferedImage im1, BufferedImage im2){
    int w1 = im1.getWidth(); int h1 = im1.getHeight();
    int w2 = im2.getWidth(); int h2 = im2.getHeight();
    assert(w2 <= w1 && h2 <= h1);
    // will keep track of best position found
    int bestX = 0; int bestY = 0; double lowestDiff = Double.POSITIVE_INFINITY;
    // brute-force search through whole image (slow...)
    for(int x = 0;x < w1-w2;x++){
        for(int y = 0;y < h1-h2;y++){
            double comp = compareImages(im1.getSubimage(x,y,w2,h2),im2);
            if(comp < lowestDiff){
                bestX = x; bestY = y; lowestDiff = comp;
            }
        }
    }
    // output similarity measure from 0 to 1, with 0 being identical
    System.out.println(lowestDiff + "output similarity measure from 0 to 1, with 0 being identical");
    Assertions.assertTrue(lowestDiff<0.5);
    // return best location
    return new int[]{bestX,bestY};
}

/**
 * Determines how different two identically sized regions are.
 */
public static double compareImages(BufferedImage im1, BufferedImage im2){
    assert(im1.getHeight() == im2.getHeight() && im1.getWidth() == im2.getWidth());
    double variation = 0.0;
    for(int x = 0;x < im1.getWidth();x++){
        for(int y = 0;y < im1.getHeight();y++){
            variation += compareARGB(im1.getRGB(x,y),im2.getRGB(x,y))/Math.sqrt(3);
        }
    }
    return variation/(im1.getWidth()*im1.getHeight());
}

/**
 * Calculates the difference between two ARGB colours (BufferedImage.TYPE_INT_ARGB).
 */
public static double compareARGB(int rgb1, int rgb2){
    double r1 = ((rgb1 >> 16) & 0xFF)/255.0; double r2 = ((rgb2 >> 16) & 0xFF)/255.0;
    double g1 = ((rgb1 >> 8) & 0xFF)/255.0;  double g2 = ((rgb2 >> 8) & 0xFF)/255.0;
    double b1 = (rgb1 & 0xFF)/255.0;         double b2 = (rgb2 & 0xFF)/255.0;
    double a1 = ((rgb1 >> 24) & 0xFF)/255.0; double a2 = ((rgb2 >> 24) & 0xFF)/255.0;
    // if there is transparency, the alpha values will make difference smaller
    return a1*a2*Math.sqrt((r1-r2)*(r1-r2) + (g1-g2)*(g1-g2) + (b1-b2)*(b1-b2));
}

I found it here. It works good, but too slow. If you also cannot use openCV, you can use this way. Good luck.

score 0 · Accepted Answer · answered Jun 12 '19 at 11:08

I guess that you already retrieved the template image and the real image (to check against the template) by using the get_screenshot_as_base64() (here the documentation).

Then you can try to use the OpenCV library embedded inside Appium to check how similar the two images are by calling the function get_images_similarity() (here the implementation) to get the similarity score of the two images and make reasoning on top of it.

For more detailed information have a look at this useful article!

I learning the article you've mentioned. Thank you so much for the help! — Mikhail Barinov, Jun 13 '19 at 14:12

score 0 · Answer 3 · answered Jun 07 '21 at 08:57

I tried to compare pictures with template by OpenCV library. That's what I do:

Add method to base_page.py.

 def compare_image_with_screenshot(self, image_name: str):
 os.chdir('../src/screenshots/')

 with open(f'{image_name}.png', 'rb') as img:
     first_image = base64.b64encode(img.read()).decode('ascii')
 second_image = base64.b64encode(self._driver.get_screenshot_as_png()).decode('ascii')

 return self._driver.get_images_similarity(first_image, second_image)

Use this method in Page Object file.

 @allure.step('Compare screenshot with template')
 def get_image_comparison_percents(self):
 """
 This method gets screenshot on device with template in repo. Comparison result is percentage of similarity. Test is OK if comparison more than 90%
 """
 result = self.compare_image_with_screenshot(OfflineLocators.offline_stub)
 return result.get('score')

Use step in necessary test.

@allure.link(url='https://jira.myproject.tech/browse/TEST-1', name='TEST-1 - Offline stub')
@allure.title('Offline stub')
def test_offline_stub(appdriver):

TourActions(appdriver).skip_tour()
Navigation(appdriver).open_my_offline_page()

assert Offline(appdriver).get_page_title_text() == 'Offline'
assert Offline(appdriver).get_image_comparison_percents() > 0.9

In result of all of this I get some percentage of pictures similarity. It can be with that percentage what you need. For my tests it is OK.

How to compare a picture in Appium, using Python code?

3 Answers3