Find an android library which similar to web audio API

Question

I have a html/javascript project that play streaming video, using web audio API to separate video's sound to multiple channel (left-right volume, only vocal sound, only guitar sound etc..) and allow to change or mute volume of each. Now I wonder if it possible in android, I have never used android before, so does it has any library that can do the same things?

Aside from that, I also wanna know how to split this video to 4 angles, and display it like 4 small videos. In javascript I'm using canvas & setInterval like:

setInterval(function () {
        Object.keys(Angles).forEach(function (angle_id) {
            var top = Angles[angle_id]['position'][0];
            var left = Angles[angle_id]['position'][1];
            var canvas_id = 'canvas' + angle_id;
            var canvas = document.getElementById(canvas_id);
            canvas.getContext("2d").drawImage(_Self.Video, left, top, 480, 270);
        });
    }, 1000 / 30);

So how to do the same thing in android? Thanks very much.

Answer 1

< /Hey >

Audio Playback

If you wanted to transfer your web development skills to mobile then you could use something like Cordova or PhoneGap & continue to develop your mobile app in JS/CSS/HTML. Since this type of app uses a browser engine to run your code, it's up to the browser which device it supports. Here's just one example on Ionic using the web-Audio API in an iOS / Android app!

Otherwise, from what I can see there is no Web-Audio based API natively for Android, however there are many audio DSP libraries that should be able to carry out any processing like you intend (I'm sure you'll find plenty more from looking but here's to get started):

• Android's built in Soundpool (Java)
• TarsosDSP (Java)
• Native processing (NDK) Pure C/C++ (Just if you were curious)

I think Soundpool is your best bet for playing multiple audio files simultaneously with low latency. It also has left-right channel volume control and the other things you mentioned; see the docs. Here are also some starting examples.

---

Video Playback

After looking around substantially, I can say that splitting the video like you said seems rather non-trivial on native android at least.

However, it looks like FFmpeg's crop feature allows you to do what you were looking for. You will need to be using the Android version of course, and I've found one example of someone carrying it out here:

 String[] cmd = {"-i", in, "-filter:v", "crop=" + 240 + ":" + 120 + ":" + 100 + ":" + 100, "-c:a", "copy", out};

 execFFmpegCommand(cmd);

This was the only technique I found for splitting a video like this so please comment if you know any others ( I will update post ).

Otherwise maybe your current technique would work on the Web-Mobile development technologies I have mentioned at the start.