1

I am streaming audio using FFMPeg and need to mix two audio sources using FFMpeg and set the volume level dynamically. I.e. once the stream starts, I need to be able update the ratio of the two volumes.

Currently, I have the volume mixing and streaming working using the CLI version of FFMPeg but the volume mix ratio is static.

Is there a way to dynamically set the volume ratio using the CLI tool? Perhaps something with an FFMpeg expression?

Or is using the API the only option? If so, can anyone point me towards an example of dynamically mixing audio? I haven't been able to find one.

Edit: here are the params I currently pass to mix audio. Again, this works fine, so not including the log as there is no error to fix. The question is how can I adjust the ratio of the amix mix after the process has launched. Willing to use the API if need be.

                "-f rawvideo" + // container
                " -vcodec rawvideo" + // codec
                " -s " + width + "x" + height + // input video size, must be correct
                " -pix_fmt rgba" + // pixel format
                " -framerate " + frameRate +
                " -i pipe:0" + // from stdin in via pipe
               " -f dshow" +
                " -i audio=\"Stereo Mix (Realtek(R) Audio)\"" + //
                " -f dshow" +
                " -i audio=\"Microphone Array (Xbox NUI Sensor)\"" + // 
                " -filter_complex \"amix\"" +  // mix the two inputs, can added ratio if needed
                " -c:v libx264" + // x264 software encoder
                " -g " + frameRate *2 +
                " -keyint_min " + frameRate +
                " -c:a aac" + // audio format
                " -b:v 6M -maxrate 2M -bufsize 1M" + // constrain bitrate per twitch
                " -f flv" +
                " " + address 
            );
jvhang
  • 747
  • 6
  • 19
  • 1
    Show your command and the log. – llogan May 26 '21 at 16:17
  • The volume filter supports commands. – Gyan May 26 '21 at 16:32
  • My existing cli call works fine using a volume but it's not mixing the audio based on a dynamic ratio. For example, at minute 2 of the stream I am not changing the value of the mic to be louder, which is what I want to do. – jvhang May 26 '21 at 19:36
  • 1
    @gyan can you point me to an example of using commands to affect ffmpeg once it's running? I found this doc but there are no examples: https://ffmpeg.org/ffmpeg-filters.html#Changing-options-at-runtime-with-a-command – jvhang May 26 '21 at 20:47
  • Add your existing command + log. – Gyan May 27 '21 at 04:04
  • 1
    I am also keen to find a solution for this. I don't believe there is a way to dynamically adjust the relative volume of the inputs _once the filters have been created_. Given the following `ffmpeg -i speech.mp4 -i backgroundmusic.mp4 -filter_complex "[0:a]volume=0.9[a1];[1:a]volume=0.1[a2];[a1][a2]amix[aout]" -vn -dn -map [aout] -c:a aac mixed.mp4`, how can i vary the volumes in real time? I don't believe this is possible using the API either, since I think the filters are static once created, but I'd love to be corrected. I think this can be done with gstreamer, but I always prefer ffmpeg! – Anonymous Coward May 27 '21 at 08:30
  • I've added the command params I am currently using, but again, this works fine, so not including a log. – jvhang May 27 '21 at 12:54

1 Answers1

2

Commands

  • Some filters support commands which allow temporal control of some filter options. Not all filters have commands. Sometimes commands get added so always use a recent ffmpeg to take advantage of new features.
  • Refer to ffmpeg -filters and look for C next to the filter name, or view man ffmpeg-filters or FFmpeg Filters documentation and search for commands under each filter.
  • Not all options have a command equivalent.

(a)sendcmd and (a)zmq

  • asendcmd / sendcmd - does scheduled commands or interactive commands via interactive mode.
  • azmq / zmq - does interactive, on demand, live commands. To enable (a)zmq you need to install the libzmq library and headers and configure ffmpeg with --enable-libzmq.

asendcmd example

Halve the volume at timestamp 10:

ffmpeg -i input -filter_complex "asendcmd=c='10.0 volume volume 0.5',volume" output

If you want to make many adjustments then it is easier to manage in a separate text file and use the f option. See (a)sendcmd documentation. See more examples.

Interactive mode

You can use c (Send command to first matching filter supporting it) or C (Send/Queue command to all matching filters) during the encoding to send a command. Use ? when encoding for more info, see example, and refer to (a)sendcmd documentation for syntax.

azmq example

Run ffmpeg, then execute zmqsend whenever you want to adjust volume.

ffmpeg -i input -filter_complex "volume@foo,azmq" output
echo "volume@foo volume 0.5" | tools/zmqsend

zmqsend is available in the tools directory of the FFmpeg source (install zmq, ./configure --enable-libzmq && make -j && make tools/zmqsend). If you're on Windows just download one of Gyan's FFmpeg builds and also get the Tools package.

llogan
  • 121,796
  • 28
  • 232
  • 243
  • thank you for such an awesome answer. I haven't tried this yet but it filled the documentation gap I had so marking as the accepted answer. – jvhang May 28 '21 at 17:11
  • small correction for anyone else in this position, the correct volume command is `echo 'volume@foo volume 0.5' | tools/zmqsend` – jvhang May 31 '21 at 23:22
  • 1
    @jvhang Yeah, the `echo` definitely needs to be quoted due to the spaces. I remember quoting it in my test, but I forgot to do the same in my answer. Updated. – llogan Jun 01 '21 at 15:47