How to join one audio and video file -- Xuggler

Question

I want to combine a video file (flv) with no audio with an audio file (mp3) using Xuggler. At the moment I have taken two streams and combined the video and audio parts of those streams separately like picture in picture. Now i want to combine the audio and video files with each other.. Any suggestion or hints will be appreciated. I am using red5 server. Thanks.

Answer 1

I developed a class that will take two inputs, one audio file and second video file and will merge them to a single audio video file.

 public static void main(String[] args)
  {

    String filenamevideo = "f:/testvidfol/video.mp4"; //video file on your disk
    String filenameaudio = "f:/testvidfol/audio.wav"; //audio file on your disk


    IMediaWriter mWriter = ToolFactory.makeWriter("f:/testvidfol/videowriter.flv"); //output file

    IContainer containerVideo = IContainer.make();
    IContainer containerAudio = IContainer.make();

    if (containerVideo.open(filenamevideo, IContainer.Type.READ, null) < 0)
        throw new IllegalArgumentException("Cant find " + filenamevideo);

    if (containerAudio.open(filenameaudio, IContainer.Type.READ, null) < 0)
        throw new IllegalArgumentException("Cant find " + filenameaudio);

    int numStreamVideo = containerVideo.getNumStreams();
    int numStreamAudio = containerAudio.getNumStreams();

    System.out.println("Number of video streams: "+numStreamVideo + "\n" + "Number of audio streams: "+numStreamAudio );

int videostreamt = -1; //this is the video stream id
int audiostreamt = -1;

IStreamCoder  videocoder = null;

    for(int i=0; i<numStreamVideo; i++){
        IStream stream = containerVideo.getStream(i);
        IStreamCoder code = stream.getStreamCoder();

        if(code.getCodecType() == ICodec.Type.CODEC_TYPE_VIDEO)
        {
            videostreamt = i;
            videocoder = code;
            break;
        }

    }

    for(int i=0; i<numStreamAudio; i++){
        IStream stream = containerAudio.getStream(i);
        IStreamCoder code = stream.getStreamCoder();

        if(code.getCodecType() == ICodec.Type.CODEC_TYPE_AUDIO)
        {
            audiostreamt = i;
            break;
        }

    }

    if (videostreamt == -1) throw new RuntimeException("No video steam found");
    if (audiostreamt == -1) throw new RuntimeException("No audio steam found");

    if(videocoder.open()<0 ) throw new RuntimeException("Cant open video coder");
    IPacket packetvideo = IPacket.make();

    IStreamCoder audioCoder = containerAudio.getStream(audiostreamt).getStreamCoder();

    if(audioCoder.open()<0 ) throw new RuntimeException("Cant open audio coder");
    mWriter.addAudioStream(0, 0, audioCoder.getChannels(), audioCoder.getSampleRate());

    mWriter.addVideoStream(1, 1, videocoder.getWidth(), videocoder.getHeight());

    IPacket packetaudio = IPacket.make();

    while(containerVideo.readNextPacket(packetvideo) >= 0 ||
            containerAudio.readNextPacket(packetaudio) >= 0){

        if(packetvideo.getStreamIndex() == videostreamt){

            //video packet
            IVideoPicture picture = IVideoPicture.make(videocoder.getPixelType(),
                    videocoder.getWidth(),
                    videocoder.getHeight());
            int offset = 0;
            while (offset < packetvideo.getSize()){
                int bytesDecoded = videocoder.decodeVideo(picture, 
                        packetvideo, 
                        offset);
                if(bytesDecoded < 0) throw new RuntimeException("bytesDecoded not working");
                offset += bytesDecoded;

                if(picture.isComplete()){
                    System.out.println(picture.getPixelType());
                    mWriter.encodeVideo(1, picture);

                }
            }
        } 

        if(packetaudio.getStreamIndex() == audiostreamt){   
        //audio packet

            IAudioSamples samples = IAudioSamples.make(512, 
                    audioCoder.getChannels(),
                    IAudioSamples.Format.FMT_S32);  
            int offset = 0;
            while(offset<packetaudio.getSize())
            {
                int bytesDecodedaudio = audioCoder.decodeAudio(samples, 
                        packetaudio,
                        offset);
                if (bytesDecodedaudio < 0)
                    throw new RuntimeException("could not detect audio");
                offset += bytesDecodedaudio;

                if (samples.isComplete()){
                     mWriter.encodeAudio(0, samples);

        }
            }

    }

  }
}

Hope it will help u.

Answer 2

I know this thread is very old but the provided solution did not work for me and I figured I'd share my solution here...

I added the following dependencies to my pom.xml

<!-- https://mvnrepository.com/artifact/org.bytedeco/ffmpeg -->
   <dependency>
       <groupId>org.bytedeco</groupId>
       <artifactId>ffmpeg</artifactId>
       <version>RELEASE</version>
   </dependency>

   <!-- https://mvnrepository.com/artifact/org.bytedeco/javacv-platform -->
   <dependency>
       <groupId>org.bytedeco</groupId>
       <artifactId>javacv-platform</artifactId>
       <version>RELEASE</version>
   </dependency>

   <!-- https://mvnrepository.com/artifact/org.bytedeco/javacpp -->
   <dependency>
       <groupId>org.bytedeco</groupId>
       <artifactId>javacpp</artifactId>
       <version>RELEASE</version>
   </dependency>

Then I use ProcessBuilder to execute FFMPEG commands. To merge audio and video, I use the following command:

String ffmpeg = Loader.load(org.bytedeco.ffmpeg.ffmpeg.class);
    try {
        ProcessBuilder pb = new ProcessBuilder(
                ffmpeg,
                "-i",
                [PATH TO AUDIO FILE],
                "-i",
                [PATH TO VIDEO FILE],
                "-acodec",
                "copy",
                "-vcodec",
                "copy",
                [PATH TO OUTPUT FILE]
        );
        
        pb.redirectErrorStream(true);
        Process process = pb.start();
        process.waitFor();

    } catch (IOException | InterruptedException e) {
        e.printStackTrace();
    } 

Answer 3

Use a MediaConcatenator. See example code "Concatenate Audio And Video"