4

Hi I'm writing a client/server remote viewer (desktop sharing) application where screenshots of the desktop are sent across the network over a socket. I'd like to reduce the size of the transfer by getting the difference between the two images and then sending the difference. on the other hand difference will be merge with previous image at other end.

so anyone please guide me how could i accomplish this job. still now i send every time a complete image of the screen over the network programatically and program at other end just show that image. i feel huge data is getting pass over the network and screen update rate at the other end is slow. so please show me good way how to compare between two images and send only difference to other end. also tell me how to merge the difference with actual image at other end.

1) lots of free code and library is available for image comparison but i just do not understand which one i should use and which one would very faster for comparison. so just guide me regarding this.

2) and the most important part is how to send difference only over the network and merge the difference with actual image at other end

i tried lot to get some info regarding my point 2 but got nothing similar. no article i found who can guide me that how to send difference only over the network and merge the difference with actual image at other end

so i am looking for in-depth discussion for my point 2 specially. thanks

Thomas
  • 33,544
  • 126
  • 357
  • 626
  • So if you have an image which the top half is a horse, bottom half has human legs. Another image has top half horse, bottom half zebra. You would want the Zebra legs? Then do what? How would you merge the difference as they are in the same place? – LukeHennerley Nov 30 '12 at 09:39
  • "screen update rate at the other end is slow" Are you trying to send video over the network? There are codecs already that facilitate frame updates data only, what codec are you using? or are you sending Images only - if so, in what format are they? – Kami Nov 30 '12 at 10:12
  • png format. just guide me how to Apply the DIFFERENCE between two image on the present image? this is very vital for me to know. – Thomas Nov 30 '12 at 10:47

3 Answers3

5

You will have to follow three steps:

  1. Create a difference (DIFF) of the two consecutive images. Comparing the two images pixel per pixel will be very time consuming. You should utilize a well-established library like OpenCV; check out the Emgu CV library (http://www.emgu.com/) for .NET: AbsDiff() should be the method you're looking for. The result will be something like DIFF = IMG2 - IMG1.
  2. Send the DIFF through the network. It's still a full image, but JPEG or PNG will utilize its full compression capability assuming it is a mainly black image, i.e. few changes. So these are actually three substeps: Compress - Send - Decompress.
  3. Apply the DIFF on the present image. The recipient can calculate the next image IMG2 = DIFF + IMG1. This can be performed using EmguCV's Add method.
Peopleware
  • 1,399
  • 1
  • 25
  • 42
  • thanks for help but this point is not clear 3) Apply the DIFF on the present image. please tell me in detail how to Apply the DIFF on the present image? can u drive to any code or article. thanks a lot. – Thomas Nov 30 '12 at 10:37
  • i never this lib before. can u plzz give me sample code for this lib which i can use to Create a difference (DIFF) of the two consecutive images and Apply the DIFF on the present image that is IMG2 = DIFF + IMG1. thanks – Thomas Nov 30 '12 at 12:18
  • Emgu actually has an excellent documentation. You can find an [introduction](http://www.emgu.com/wiki/index.php/Setting_up_EMGU_C_Sharp) and some [basic examples](http://www.emgu.com/wiki/index.php/Working_with_Images) on the Wiki. That should be enough to get it started. – Peopleware Nov 30 '12 at 12:53
  • if i use this library then it will be faster to get the diff image and later apply the DIFF on the present image means IMG2 = DIFF + IMG1. – Thomas Nov 30 '12 at 13:00
  • Any clues to My algorithm: http://stackoverflow.com/questions/39690323/efficient-image-sending-by-network-sockets-c-language ? Will it be right to use it and just add compression for image sent? If I know compression will be efficient if image will be black and white but I think such XORing doesn't guarantee that – Michał Ziobro Sep 25 '16 at 19:06
  • 1
    **Absolute** difference will not work, since pixel difference can gives negative values that will be saturated to 0. So the reconstruction will not be possible. – thepirat000 Sep 26 '16 at 02:10
  • 1
    negative values? there is abs() so |a-b| always gives positive |10-250| is 240. But than while adding them there should be check if sum is greater than 255 make the subtraction rather than sum. But I don't know whether jus XOR on both side isn't better I will be checking this today and give some commants on it. – Michał Ziobro Sep 27 '16 at 08:59
1

I know am very late responding but I found this question today

I have done some analysis on Image Differencing but the code was written for java. Kindly look into the below link that may come to help

How to find rectangle of difference between two images

The code finds differences and keeps the rectangles in a Linkedlist. You can use the linkedlist that contains the Rectangles to patch the differences on to the Base Image.

Cheers !

Community
  • 1
  • 1
Nandhan
  • 195
  • 1
  • 1
  • 8
1

What about my approach, I am not sure it will be useful, I would like someone give some hint I am approaching in the right direction with this.

I consider streaming of desktop to mobile phone. I would use server-client model with network TCP sockets. I consider such pattern (protocol).

SERVER:

0)

SCREEN IMAGE - represented as unsigned char *rgba_array IMAGE_SIZE -> width, height

1)

Send image size width, height to Client.

2)

init rgba_array with length <= width x height x 4 bytes (32bits images) and all array zeroed.

0000 0000 0000 0000 0000 0000 0000 0000 (example pixel - 32 bits)

3)

if I have new image of screen (screenshot? or other way), I am converting it to unsigned char *new_rgba_array and than make a XOR operation: 

rgba_array XOR new_rgba_array => diff_rgba_array

4)

than I am only sending to the client this diff_rgba_array COMPRESSED? JPEG?

5) go back to point 3) on server side.

CLIENT:

0)

get width and height -> init rgba_array with length <= width * height * 4 and zeroed it

1)

get diff_rgba_array that represents changes in comparison with previous image and DECOMPRESS it? JPEG?

0011 1010 0110 0111 1110 0000 0100 0000 (example pixel difference)

0 - bit means NO CHANGE

1 - bit means CHANGE to opposed value

2)

apply changes received from server in diff_rgba_array to rgba_array on client side. I think I should make next XORing. Like this 

  10011101   10010101

  00001111   11111111 (XOR) 

= 10010010   01101010

3)

go back to point 1) on client side

Michał Ziobro
  • 10,759
  • 11
  • 88
  • 143