Sending multiple images with socket

Question

In an ambitious attempt making a VNC with Python, I am trying to continuously send screenshots of one user (Server) to another (Client). After hours of trying and hitting Ballmer's peak, I managed to do it. However, now my problem is sending multiple images, a continuous stream of them. I first tried to write all the binary data to one file, which didn't work. When the second image was opened, it crashed. I thought this might be because the binary data somehow got corrupted, so instead I tried making a new file for every image, yet I have the same problem. I know that Tcp is a constant stream of data so that it would be hard to know the end of the first image and start of the next, but by creating another file, I thought I would be all good.

Any help in fixing this and/or increasing the efficiency of this is greatly appreciated :)

Server side:

import socket
from PIL import Image, ImageGrab

PORT = 10007
HOST = '127.0.0.1'

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
    s.bind((HOST, PORT))
    s.listen()
    conn, addr = s.accept()
    with conn:

        counter = 3

        while counter > 0:
            image = ImageGrab.grab(bbox=None)
            image.save('test.png')

            f = open('test.png', 'rb')

            l = f.read(1024)

            while (l):
                conn.send(l)
                l = f.read(1024)

            f.close()
            print('Done sending curr image')
            counter -= 1

conn.close()

Client side:

import socket
from PIL import Image

HOST = '127.0.0.1'
PORT = 10007


with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
    s.connect((HOST, PORT))


    counter = 1

    while counter != 3:
        fname = 'image' + str(counter) + '.png'

        with open(fname, 'wb') as file:

            print('file opened')

            while True:
                data = s.recv(1024)
                if not data:
                    break
                else:
                    file.write(data)

        file.close()
        currImg = Image.open(fname)
        currImg.show()
        counter += 1






s.close()

score 1 · Accepted Answer · answered Dec 01 '19 at 18:56

Your receiver does not know when one file finishes and the next begins. The easiest way to fix that is to send the length of the file (perhaps as a 4-byte unsigned value) to the receiver before sending each file. Then the receiver can read the length, read the file, read the length, read the file, ...

To improve efficiency you can stop saving the file data into an actual file at both ends and instead save it into (and, obviously, read it from) an in-memory buffer. See this answer for explanations of how to do that. In Python 3 it looks like you would use the BytesIO module.

Another improvement would be to only send image data for the parts of the screen that have changed since the previous send. For that you'll need to figure out how to compare the current capture against the previous one. For a first pass you could use PIL.ImageChops.difference followed by PIL.Image.getbbox and then encode and send only that region of the current capture. For that to work, the sender will have to tell the receiver not only the size of the PNG but also the location in the output screen image where the new image patch should be painted. So you'll want to send a position in addition to the size and the encoded image data.

Sending multiple images with socket

1 Answers1