0

Trying to setup an rtsp stream that receives feed from a camera via mqtt as a "jpg" frame encoded in base64 format.

I'm setting up an mqtt subscriber that receives jpg frames (feed from a camera) encoded as base64 from a topic. These frames are decoded back and are passed on to an rtsp streamer using CV package. I keep receiving an error stating:

OpenCV(4.7.0) D:\a\opencv-python\opencv-python\opencv\modules\videoio\src\cap_images.cpp:253:
error: (-5:Bad argument) CAP_IMAGES: can't find starting number (in the name of file):
rtsp://127.0.0.1:8554/test in function 'cv::icvExtractPattern'

I'm unable to identify if the CV method's arguments are wrong or if I'm to use a different method or a package itself.

Here's the code:

import base64
import cv2 as cv
import numpy as np
import paho.mqtt.client as mqtt


MQTT_BROKER = "0.0.0.0"
MQTT_PORT = 15883
MQTT_RECEIVE = "test/cam"

frame = np.zeros((240, 320, 3), np.uint8)

frame_width = 240
frame_height =  320
   
size = (frame_width, frame_height)

# Set the RTSP stream address
rtsp_url = 'rtsp://127.0.0.1:8554/test'

# Create OpenCV VideoWriter object to write frames to RTSP server
fourcc = cv.VideoWriter_fourcc(*"H264")
out = cv.VideoWriter(rtsp_url, fourcc, 20, (640, 480))


# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, flags, rc):
    print("Connected with result code "+str(rc))

    # Subscribing in on_connect() means that if we lose the connection and
    # reconnect then subscriptions will be renewed.
    client.subscribe(MQTT_RECEIVE)


# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
    global frame
    # Decoding the message
    img = base64.b64decode(msg.payload)
    # converting into numpy array from buffer
    npimg = np.frombuffer(img, dtype=np.uint8)
    # Decode to Original Frame
    frame = cv.imdecode(npimg, 1)


client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message

client.connect(MQTT_BROKER, MQTT_PORT)

# Starting thread which will receive the frames
client.loop_start()

while True:
    cv.imshow("Stream", frame)
    out.write(frame)
    if cv.waitKey(1) & 0xFF == ord('q'):
        break

cv.destroyAllWindows()
# Stop the Thread
client.loop_stop()
out.release()
hardillb
  • 54,545
  • 11
  • 67
  • 105
Nicholas Drogo
  • 3
  • 2
  • Did you check: https://stackoverflow.com/a/67963213/18667225 ? – Markus Mar 17 '23 at 08:09
  • Your `while True:` loop doesn't wait until there has been a frame delivered before trying to write to your output stream.... – hardillb Mar 17 '23 at 08:36
  • Why are you base64-encoding the data anyway? MQTT is perfectly able to send binary data such as JPEGs, so you are needlessly adding latency in order to encode/decode and also an additional 30+% bandwidth demand. – Mark Setchell Mar 17 '23 at 09:06
  • So, If I remove the encoding and directly send and receive jpg, even then I would have to save it as a video and then stream it using RTSP? – Nicholas Drogo Mar 17 '23 at 09:39
  • I can't tell from your question exactly what you are trying to do. All I am saying is that you appear to be base64-encoding JPEGs in order to transmit them via MQTT, and that is unnecessary. You may well have some other reason to do it, I don't know. – Mark Setchell Mar 17 '23 at 09:51
  • Okay, let me check that. If I were to ask you about the RTSP error? Could you help regarding that? I'm unable to start an RTSP server to stream the received frames directly. – Nicholas Drogo Mar 17 '23 at 10:15

0 Answers0