0

Streaming via ffmpeg. The filter_complex option allows us to synthesize two sound sources, but is it possible to turn on/off the synthesized sound source at an arbitrary timing? One of the sound sources is audio from the microphone input, which will be on throughout the streaming. Other than that, we would like to turn on/off several sound sources at any given time.

For example, to turn on the microphone input and one sound source at all times, the following command is executed

ffmpeg -rtbufsize 100M  -f dshow -video_device_number 0 -i video="my camera" -i my_audio.mp3 -filter_complex "[1:a]volume=0.05,apad[0];[0][1]amerge[out]" -map 0:v -map [out]:a -f hls -hls_segment_type fmp4 -hls_time 5 -hls_fmp4_init_filename init.mp4 hls.m3u8 -f mpegts -flush_packets 0 udp://XXX.XXX.XXX.XXX:XXX?pkt_size=1316
its-ogawa
  • 69
  • 6
  • [This post and its answer](https://stackoverflow.com/questions/56058909) should point you in a right direction. My approach to your goal would be to use pairs of `astreamselect`-`anullsrc` and mix them with `amix` (and command each `astreamselect` to on/off its feed) – kesh Mar 28 '22 at 14:36
  • 1
    I [posted an example](https://superuser.com/a/1705970/635712) of muting an unmuting an audio stream. It should work if you use `[1:a]azmq,volume@my=volume=0.05` and execute `echo volume@my volume 0 | zmqsend` for muting the audio (it is not difficult to extend the solution to two audio sources). I am not sure how long is the latency between the command and the muting. – Rotem Mar 28 '22 at 20:48
  • kesh, Rotem Thanks for the great suggestion. I am interested in using ZeroMQ. However, my environment does not seem to have ZeroMQ. I think it is because I got FFMPEG as EXE. (https://ffmpeg.org/download.html#build-windows) How can I use ZeroMQ? I would appreciate your advice. – its-ogawa Mar 29 '22 at 09:58
  • Does ZeroMQ not support Windows? Every installation that can be transitioned from the URL below has broken links. http://wiki.zeromq.org/docs:windows-installations Do you have any good ideas? – its-ogawa Mar 30 '22 at 06:04
  • I have given up on windows and I am trying on centos using virtualbox. But FFMPEG does not recognize ZeroMQ. Do you know what caused this? ``` ... snip ... [AVFilterGraph @ 0x25f6100] No such filter: 'azmq' Error reinitializing filters! Failed to inject frame into filter network: Invalid argument Error while processing the decoded data for stream #0:0 Conversion failed! ``` – its-ogawa Mar 31 '22 at 09:26
  • @Rotem I was able to use ZeroMQ by taking the ffpeg source(git.ffmpeg.org/ffmpeg.git) and rebuilding it to include libzmq. However, when I do zmqsend, I get 0 Success. Do you know why? Here are the commands. ``` ffmpeg -rtbufsize 100M -i /dev/video0 -i Normal_day.mp3 -af "[1:a]azmq,volume@my=volume=0.05" -f hls -hls_segment_type fmp4 -hls_time 5 -hls_fmp4_init_filename init.mp4 hls.m3u8 -f mpegts -flush_packets 0 udp://XXX.XXX.XXX.XXX:XXXX?pkt_size=1316 ``` ``` echo 'volume@my volume 0' | zmqsend ``` – its-ogawa Apr 04 '22 at 06:32
  • I don't know... I never had to build FFmpeg from sources. Try to follow [my answer](https://superuser.com/a/1705970/635712) first (for testing). In case you are using Windows, you don't have to build FFmpeg from sources. – Rotem Apr 04 '22 at 07:36
  • @Rotem Thanks for replying so quickly. Sorry, I followed your advice in the beginning. Specifically, I downloaded ffmpeg-tools.zip from [gyan.dev](https://www.gyan.dev/ffmpeg/builds/), which you told me to do. tools.zip) and placed the extracted file in the same hierarchy as ffmpeg.exe. Then I added `-af "azmq,volume@my=volume=1"` to the FFmpeg command and ran it, the error that occurs is as follows. `No such filter: 'azmq'` `Error reinitializing filters!` I assume this is due to the fact that ffmpeg does not include libzmq. Is there a Windows version of ffmpeg that includes libzmq? – its-ogawa Apr 04 '22 at 10:01
  • I don't have time to test it right now. Try downloading FFmpeg static build from Gyan dev – Rotem Apr 04 '22 at 10:29
  • @Rotem Sorry to bother you when you don't have time. I downloaded ffmpeg-release-essentials and ffmpeg-release-full from Gyandev and tried them, and I am happy to report that the results were the same. I hope you can give me some ideas when you have some time. I will report back if I find out anything too. – its-ogawa Apr 04 '22 at 10:41
  • I suppose ffmpeg-release-essentials is not the full version. I have tested with [ffmpeg-4.4.1-full_build.7z](https://www.gyan.dev/ffmpeg/builds/packages/ffmpeg-4.4.1-full_build.7z), and [ffmpeg-tools.zip](https://www.gyan.dev/ffmpeg/builds/ffmpeg-tools.zip) (version version: 2022-01-01-git-d6b2357edd), and it's working. – Rotem Apr 04 '22 at 12:36
  • @Rotem Thank you very much. It works for me! However, when I run `zmqsend` I get the message `0 Error number 0 occurred` and the audio is not muted. Do you know why? I ran the following command `ffmpeg -rtbufsize 100M -f dshow -video_device_number 0 -i video="Logitech Qcam Orbit/Sphere" -i Normal_day.mp3 -af "[1:a]azmq,volume@my=volume=0.05" -f hls -hls_segment_type fmp4 -hls_time 5 -hls_fmp4_init_filename init.mp4 hls.m3u8 -f mpegts -flush_packets 0 udp://XXX.XXX.XXX.XXX:XXX?pkt_size=1316`. Next, the following command was executed at another prompt. `echo volume@my volume 0 | zmqsend`. – its-ogawa Apr 05 '22 at 04:05
  • My suggestion is to begin with simpler command. First, make sure that `-af "[1:a]volume=volume=0"` mutes the audio. – Rotem Apr 05 '22 at 07:29
  • @Rotem Thanks for your suggestion. The ffmpeg command itself seems to be working fine. When I run the following simple command, no audio is heard. `ffmpeg -i Normal_day.mp3 -af "[0:a]volume=volume=0" -f mpegts -flush_packets 0 udp://XXX.XXX.XXX.XXX:XXX?pkt_size=1316` Also, with `-af "[0:a]volume=volume=1"`, you will hear audio. This indicates that the syntax of the options specified for `af` is correct. Thanks. On the other hand, after `-af "[0:a]volume@my=volume=1"`, `echo volume@my volume 0 | zmqsend` will not set the audio to 0. – its-ogawa Apr 05 '22 at 08:57
  • Sorry. The above is an error in description. I included `azmq`. Like `-af "[0:a]azmq,volume@my=volume=1"`. – its-ogawa Apr 05 '22 at 09:22
  • Hmmm. I'll report I've succeeded with the microphone! `ffmpeg" -rtbufsize 100M -f dshow -video_device_number 0 -i video="USB Video Device":audio="microphone (USB Audio Device)" -af "[0:a]azmq,volume@my=volume=1" -f mpegts -flush_packets 0 udp://XXX.XXX.XXX.XXX:XXX?pkt_size=1316` will mute with `echo volume@my volume 0 | zmqsend`. I want to turn on/off the audio that is played at the same time as the microphone, but this one does not work. I don't know why. – its-ogawa Apr 05 '22 at 11:09
  • @Rotem I have successfully adjusted the volume of both audio and digital sound sources in the manner you indicate. I ran the following commands, and then ran `ffmpeg -f dshow -i video=:audio= -re -stream_loop -1 -i -filter_complex "[0]aformat=sample_fmts=fltp:sample_rates=44100:channel_layouts=stereo,volume@voice=volume=1[voice],[1]aformat=sample_fmts=fltp:sample_rates=44100:channel_layouts=stereo,azmq,volume@bgm=volume=0.2[bgm],[voice][bgm]amerge=inputs=2[out]" -map 0:v -map [out]:a -f mpegts -flush_packets 0 udp://XXX?pkt_size=1316 -f hls -hls_time 5 hls.m3u8` – its-ogawa Apr 07 '22 at 01:26
  • Then, on another console, `echo volume@voice volume 0 | zmqsend` will mute the volume of the voice and digital sound sources. Streaming and created sound files were checked with the vlc player. @Rotem Thanks for patiently listening to me. – its-ogawa Apr 07 '22 at 01:27
  • One note of caution. I have played an m3u8 created this way and I cannot hear the supposedly synthesized digital sound source. I thought this was another problem, so I am creating a new topic. https://stackoverflow.com/questions/71775653/how-to-save-synthesized-audio-during-streaming-distribution It would be great if you could check with us as well. – its-ogawa Apr 07 '22 at 02:30

0 Answers0