Decode video frames from growing BytesIO

Question

So we have:

• Incoming video segments of one second duration
• Segments are parts of a continuous video
• Each segment is an array of bytes
• Only the first segment has the necessary meta information to decode the entire video

I want to replenish the byte buffer as these videos arrive and immediately decode the received bytes into frames

Server side:

await websocket.accept()
buffer = io.BytesIO()
current_position = 0
container = None
try:
    while True:
        data = await websocket.receive_bytes()
        buffer.write(data)
        buffer.seek(current_position)

        if container is None:
            container = av.open(buffer, mode="r")

        for packet in container.demux():
            if packet.size == 0:
                continue

            current_position += packet.size
            for frame in packet.decode():
                print(frame)
except WebSocketDisconnect:
    container.close()
    buffer.close()

This code is not working at this moment, i've tried to use opencv and pyav but nothing worked for me

Client side:

const socket = new WebSocket("ws://endpoint");

const mediaRecorder = new MediaRecorder(stream);
mediaRecorder.ondataavailable = ev => ev.data.size > 0 && socket.send(ev.data);
mediaRecorder.start(1000);