0

I'm trying to read HLS stream (Amazon IVS) using python (So I could process it with openCV for computer vision purposes). I can read simple stream with OpenCV although I notice a huuuuge latency (20+ seconds) comparing to what I see in a Live console of AWS. But putting that aside, I also want to be able to read timestamped metadata, which I need for frames processing. I can't see any reference in the docs for reading metadata with python, I can see events for JS, IOS, Android, Web in general, but nothing for server processing! This is extremely weird. I figured it has to be some kind of WSS, I asked ChatGPT and apparently it was available, but when I try the same approach I get: server rejected WebSocket connection: HTTP 200 error...

Is there anything I'm missing here? How can I read those events with python? The code I'm using is:

    # Establish a WebSocket connection to the IVS stream
    try:
        async with websockets.connect(STREAM_WS) as websocket:
            # Subscribe to the timed metadata event
            await websocket.send(json.dumps({"message": "subscribe", "data": "metadata"}))

            # Continuously receive and print the timed metadata events
            while True:
                metadata_event = await websocket.recv()
                print(metadata_event)
    except Exception as e:
        print(e)

where STREAM_WS ends with .m3u8 (I think that might be important because ChatGPT tells me about something about an Player URL ending with /live)...

I'm literally out of ideas

Christoph Rackwitz
  • 11,317
  • 4
  • 27
  • 36
JFCorleone
  • 623
  • 6
  • 18
  • OpenCV is for computer vision, not for reading video streams off the internet. use gstreamer or ffmpeg. if you decide to use ffmpeg, PyAV is the only proper python binding I know of. – Christoph Rackwitz Dec 10 '22 at 21:34
  • That's fine, it can be ffmpeg, I tried with PyAV, and to process with opencv I still have to transform PIL Image to RGB etc. with something like this: `oimg = cv2.cvtColor(numpy.array(img), cv2.COLOR_RGB2BGR)` and I'm still left with a lot of delay. Anyhow. The main problem is with integrating with IVS so I also receive metadata – JFCorleone Dec 10 '22 at 21:51
  • @ChristophRackwitz I have some idea how to proceed. Apparently, this data is actually embedded as a separate stream, but can I decode two streams (video and data) using av.open(xyz).decode()? – JFCorleone Dec 10 '22 at 22:11
  • 1
    that should be possible, maybe not with that exact expression. I only dabbled in PyAV/ffmpeg enough to read and write video from containers containing a single video stream. PyAV exposes all streams. if you just call [`decode()`](https://pyav.org/docs/develop/api/container.html#av.container.InputContainer.decode) it _should_ give you frames from all streams, that you can then examine and handle/ignore. giving an argument also lets you select a stream of a specific type and index. – Christoph Rackwitz Dec 10 '22 at 22:55
  • I thought so too, but instead it returns only AudioStream... Also, I can't find the ID3 tags, IVS documentation mentions... So annoying – JFCorleone Dec 10 '22 at 23:06
  • just audio stream data? then that's all the "streams" there is. ID3 metadata isn't stream-related but container-related. maybe you can get it from a different place. you should investigate what kinds of metadata ffmpeg on the command line can extract from the stream (or a downloaded copy of it). I found mentions of "id3" (v1/v2) in `libavformat` docs. there is mention of an `MP3Context`. maybe you can get this data easier by inspecting the `.m3u8` file that represents the HLS stream. – Christoph Rackwitz Dec 10 '22 at 23:10
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/250318/discussion-between-jfcorleone-and-christoph-rackwitz). – JFCorleone Dec 10 '22 at 23:16

0 Answers0