Byte array of an image

Question

I am trying to make a browser based remote desktop controller. For this what I am doing is taking the screenshots of the remote desktop and transmitting them periodically to the client. But this method is highly inefficient as the screen doesn't change much. So what I thought of doing was to transmit only the difference of consecutive screenshots. This way I will be able to increase the fps too. Initially I converted the screenshot to pixel array using PIL library of python and then compared the two arrays to find the pixels which differ and send only those. But it was taking a lot of time to find the pixel array of the screenshot. Then I came across byte array which can also be used to represent an image. Converting an image to byte array was way faster than converting it to pixel array(rgba). But how do I interpret the byte array. What is stored in the byte array of an image? I converted the byte array into array of 8 bit integers using this piece of code:

bytes = readimage("./scimg1.png")

i = 0
bytearr = [] 
while i < len(bytes):
    bytearr.append(bytes[i])
    i+=1

How to compare two images using this bytearray and take their difference. I found out that the length of this array comes out to be different for two images of same dimension(length, width).

Basically you are trying to reimplement some kind of video streaming. There are already existing solutions for this involving video codecs which will probably yield MUCH better results — Eric, Nov 02 '14 at 08:55

score 1 · Answer 1 · 2014-11-02T13:17:59.550

Generally byte-by-byte comparison wouldn't be efficient in all the cases. You can use this code from this article to check the difference between images. You can costumize it based on your need to check where the difference is.

from itertools import izip
import Image

i1 = Image.open("image1.jpg")
i2 = Image.open("image2.jpg")
assert i1.mode == i2.mode, "Different kinds of images."
assert i1.size == i2.size, "Different sizes."

pairs = izip(i1.getdata(), i2.getdata())
if len(i1.getbands()) == 1:
    # for gray-scale jpegs
    dif = sum(abs(p1-p2) for p1,p2 in pairs)
else:
    dif = sum(abs(c1-c2) for p1,p2 in pairs for c1,c2 in zip(p1,p2))

ncomponents = i1.size[0] * i1.size[1] * 3
print "Difference (percentage):", (dif / 255.0 * 100) / ncomponents

Check also filecmp.cmp() from filecmp package it might be helpful.

UPDATE

I was trying to check what are the fastest way to compare between two images : Pixel by Pixel, PIL or using statistical techniques, but I couldn't notice a huge difference in performance. The problem might be related to the weight of images.

from PIL import Image, ImageChops

pixelsDifference = ImageChops.difference(Image.open('image1.png'), Image.open('image2.png')).convert('L')
pixelsDifference = pixelsDifference.point(([0] + ([255] * 255)))
Img = pixelsDifference.convert('RGB')

It still takes a lot of time to find the difference between two screenshots. Difference (percentage): 4.38976009701 real 0m1.241s user 0m1.225s sys 0m0.016s — nishantsingh, Nov 02 '14 at 09:13
@user3286661 I was trying to compare between pixels of every image when I stumbled upon this thread. Try the mentioned-Snippit in your side, and let me know if doesn't work out so we can try to find other alternatives. http://stackoverflow.com/a/1927681/4179775 — , Nov 02 '14 at 12:44

score 0 · Answer 2 · answered Nov 02 '14 at 09:28

I would recommend using a video encoding library such as PyMedia or pyffmpeg. You are obviously optimizing for speed here and there probably won't be anything faster than this for transfering images. PyMedia only support mpeg1 and 2 which are rather old.

Your other option would be to drop the custom approach and bind on something already existing such as the RDP protocol for windows or VNC for any platform. (VNC is not THAT responsive)

score 0 · Answer 3 · answered Nov 02 '14 at 10:21

You can take the difference of the images with the ImageChops module. You can use ImageChops.difference(image1, image2) to get the difference between the two screenshots.

You should also use gzip to compress the images before sending them. This way you will need to send less data with every frame. Here's the documentation for the gzip library.

Byte array of an image

3 Answers3