How can I effectively compress a frame in memory?

Question

I'm testing out screen-casting in Python. The code for both the client and server is relatively simple.

In sum, what I was previously doing was:

• On the server:
  • Grabbing screen with mss
  • Serializing with pickle
  • Sending data
  • Repeating for every frame
• On the client:
  • Receiving data and deserializing with pickle
  • Running the received numpy array through CV2 to display it

I tested it locally and it proved to work very well. However, as soon as I started testing within my network (between desktop and laptop) the problems quickly surged. Quality was great but terribly laggy, incredibly low FPS.

I realized that the frames may be too big, so I decided to turn my focus to compression, and I've been searching and trying things for hours to no avail. After a while searching I came across some users on SO making use of zlib, so I decided to do the same, although by compressing (zlib.compress(frame)) before serializing and sending (on the server) and deserializing and decompressing (zlib.decompress(frame)) (on the client) I couldn't turn the data back into a numpy array, and as such CV2 would crash saying something along the lines of TypeError: Expected Ptr<cv::UMat> for argument 'src'.

More than knowing how I can effectively compress the frames, what I'd also really like to know is what's slowing my code down, since I'm not entirely sure the problem is frame size. I'd appreciate any help on the subject and any hints, tips or criticism since I am by no means experienced on the subject.

Code for the server:

import sys, socket as s
import pickle
import struct        
import cv2
import numpy as np
from mss import mss
from PIL import Image
import time

HOST = ''  

if len(sys.argv) > 1:  
    PORT = int(sys.argv[1]) 
else:  
    PORT = 9999  
    
sock = s.socket(s.AF_INET, s.SOCK_STREAM) 
sock.setsockopt(s.SOL_SOCKET, s.SO_REUSEADDR, 1)  

sock.bind((HOST, PORT)) 
sock.listen(1)  

(conn_sock, addr) = sock.accept()


bounding_box = {'top': 0, 'left': 0, 'width': 1920, 'height':1080}

sct = mss()
sct.compression_level = 9

while True:
    try:

        frame = np.array(sct.grab(bounding_box))
        
        msg_bytes = pickle.dumps(frame,-1)
        size_bytes = struct.pack('i',len(msg_bytes))

        # Sending size bytes before actual data
        conn_sock.sendall(size_bytes)
        conn_sock.sendall(msg_bytes)


    except Exception as e:   
        print(e)
        conn_sock.close()

        break

cv2.destroyAllWindows()

Code for the client:

import sys, socket as s
import pickle
import struct
import cv2
import numpy as np
from sock_utils import receive_all


if len(sys.argv) > 1:
    HOST = sys.argv[1]   
    PORT = int(sys.argv[2]) 

else:   
    HOST = '127.0.0.1'   
    PORT = 9999  


conn_sock = s.socket(s.AF_INET, s.SOCK_STREAM)   
conn_sock.connect((HOST, PORT))   
    
while True:   
    
    # Getting size of bytes to receive
    size_bytes = conn_sock.recv(4)
    size = struct.unpack('i',size_bytes)[0]

    # Getting frame bytes
    msg_bytes = receive_all(conn_sock, size)

    # Deserializing frame
    frame = pickle.loads(msg_bytes)


    # Showing output
    cv2.namedWindow("Screencasting", cv2.WINDOW_NORMAL)

    cv2.imshow("Screencasting", frame)

    # Checking for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break



conn_sock.close()