Windows media foundation decoding audio streams

Question

I am new to Windows Media Foundation, I am currently following some tutorials on the basics to get started and decoding audio. However, I am running int a few issues. Using: Windows 10 64bit (1809), and am using Python (ctypes and COM's to interface).

1) The IMFSourceReader will not allow me to select or deselect any stream out. I have tried wav and mp3 formats (and multiple different files), but they all error out. According to the docs, to speed up performance you want to deselect other streams and select the stream you want, in this case, audio.

However: source_reader.SetStreamSelection(MF_SOURCE_READER_ANY_STREAM, False)

Produces an error: OSError: [WinError -1072875853] The stream number provided was invalid.

Which should be correct since MF_SOURCE_READER_ANY_STREAM value (DWORD of 4294967294) should be universal? Or am I incorrect in that?

I've tried seeing if I could just select the audio stream: source_reader.SetStreamSelection(MF_SOURCE_READER_FIRST_AUDIO_STREAM, True)

Which produces a different error: OSError: exception: access violation reading 0x0000000000000001

My current code up until that point:

MFStartup(MF_VERSION)  # initialize

source_reader = IMFSourceReader() 

filename = "C:\\test.mp3"

MFCreateSourceReaderFromURL(filename, None, ctypes.byref(source_reader))  # out: source reader.

if source_reader:  # Not null
    source_reader.SetStreamSelection(MF_SOURCE_READER_ANY_STREAM, False)  # invalid stream #?

    source_reader.SetStreamSelection(MF_SOURCE_READER_FIRST_AUDIO_STREAM, True)  # access violation??

IMFSourceReader seems to be functioning just fine for other functions, Such as GetCurrentMediaType, SetCurrentMediaType, etc. Could it still return IMFSourceReader if there are any issues?

2) I am not sure if not being able to select the streams is causing further issues (I suspect it is). If I just skip selecting or deselecting streams, everything actually works up until trying to convert a sample into a single buffer with ConvertToContiguousBuffer, which, according to the docs, outputs into a IMFMediaBuffer. The problem is, after running that, it does return as S_OK, but the buffer is null. I used GetBufferCount to make sure there are some buffers in the sample atleast, and it always returns 1-3 depending on the file used, so it shouldn't be empty.

Here is the relevant code:

while True:
    flags = DWORD()
    sample = IMFSample()

    source_reader.ReadSample(streamIndex, 0, None, ctypes.byref(flags), None, ctypes.byref(sample))  # flags, sample [out]

    if flags.value & MF_SOURCE_READERF_ENDOFSTREAM:
        print("READ ALL OF STREAM")
        break

    if sample:
        buffer_count = DWORD()
        sample.GetBufferCount(ctypes.byref(buffer_count))
        print("BUFFER COUNT IN SAMPLE", buffer_count.value)
    else:
        print("NO SAMPLE")
        continue

    buffer = IMFMediaBuffer()
    hr = sample.ConvertToContiguousBuffer(ctypes.byref(buffer))

    print("Conversion succeeded", hr == 0)  # true

    if buffer:
        print("CREATED BUFFER")
    else:
        print("BUFFER IS NULL")
        break

I am unsure where to go from here, I couldn't find much explanations on the internet regarding these specific issues. Is WMF still the goto for Windows 10? Should I be using something else? I really am stumped and any help is greatly appreciated.

Answer 1

Try using MF_SOURCE_READER_ALL_STREAMS instead of MF_SOURCE_READER_ANY_STREAM.

source_reader.SetStreamSelection(MF_SOURCE_READER_ALL_STREAMS, False) 

When reading the samples you also need to specify a valid stream index which in your case I suspect 0 isn't. Try:

source_reader.ReadSample((DWORD)MF_SOURCE_READER_FIRST_AUDIO_STREAM, 0, None, ctypes.byref(flags), None, ctypes.byref(sample))

Also does your Python wrapper return a result when you make your Media Foundation calls? Nearly all Media Foundation methods return a HRESULT and it's important to check it equals S_OK before proceeding. If you don't it's very hard to work out where a wrong call occurred.

Is WMF still the goto for Windows 10?

Many people have asked that question. The answer depends on what you need to do (in your case audio codec support is very limited so perhaps it isn't the best option). But for things like rendering audio/video, reading/writing to media files, audio/video device capture etc. it is still the case that Media Foundation is the most up to date Microsoft supported option.

Answer 2

Usually, with MediaFoundation, you need to call CoInitializeEx before MFStartup :

Media Foundation and COM

Best Practices for Applications

In Media Foundation, asynchronous processing and callbacks are handled by work queues. Work queues always have multithreaded apartment (MTA) threads, so an application will have a simpler implementation if it runs on an MTA thread as well. Therefore, it is recommended to call CoInitializeEx with the COINIT_MULTITHREADED flag.

MFCreateSourceReaderFromURL function

Remarks

Call CoInitialize(Ex) and MFStartup before calling this function.

In your code, I don't see the call to CoInitializeEx.

EDIT

Because you are using audio file, normally you should have only one audio stream, and index should be 0. Try :

source_reader.SetStreamSelection(0, True)

and tell us the result.