3

I am trying to play an audio file with binary string format that Amazon Polly returns. For that, I am using 'react-native-fetch-blob' and reading a stream, but just keep getting errors from the bridge saying 'Invalid data message - all must be length: 8'. It happens when I try to open the stream: ifstream.open()

This is the code:

//polly config
const params = {
    LexiconNames: [], 
    OutputFormat: "mp3", 
    SampleRate: "8000", 
    Text: "All Gaul is divided into three parts", 
    TextType: "text", 
    VoiceId: "Joanna"
};

Polly.synthesizeSpeech(params, function(err, data) {
    let _data = "";
    RNFetchBlob.fs.readStream(
        // file path
        data.AudioStream,
        // encoding, should be one of `base64`, `utf8`, `ascii`
        'ascii'
    )
    .then((ifstream) => {
        ifstream.open()
        ifstream.onData((chunk) => {
            _data += chunk
        })
        ifstream.onError((err) => {
            console.log('oops', err.toString())
        })
        ifstream.onEnd(() => {  
            //pasing _data to streaming player or normal audio player
            ReactNativeAudioStreaming.play(_data, {showIniOSMediaCenter: true, showInAndroidNotifications: true});
        })  
    })
});

Another solution I have also tried is to save the stream into a file to load it later on, but I got similars bugs. RNFetchBlob.fs.createFile("myfile.mp3", dataG.AudioStream, 'ascii');

Huge thanks in advance

rubelux
  • 61
  • 1
  • 6
  • Where is `_data` defined? Can you include full path to audio file at Question? – guest271314 Aug 04 '17 at 17:24
  • Why do you request a `Blob` if you are expecting to read a stream of `Uint8Array`s in chunks? – guest271314 Aug 04 '17 at 17:29
  • @guest271314 thanks for helping on this. – rubelux Aug 04 '17 at 21:21
  • No worries. Have no experience using `react-native`, `amazon-polly` or `react-native-fetch-blob`, though have tried `fetch()` and `ReadableStream` and various methods of trying text-to-speech at browser [How to use Web Speech API at chromium?](https://stackoverflow.com/questions/44013933/how-to-use-web-speech-api-at-chromium), [How to create or convert text to audio at chromium browser?](https://stackoverflow.com/q/44346410/), [How to capture generated audio from window.speechSynthesis.speak() call?](https://stackoverflow.com/q/45003548/) – guest271314 Aug 04 '17 at 21:23
  • I have just added all the missing code include the AWS Polly function. About requesting a `Blob`, I am not really sure, I would like just to be able to make `Uint8Array` a format compatible with the react player. Maybe push all the chunks into one array and save into a file. ideas? – rubelux Aug 04 '17 at 21:28
  • You can use `Response.arrayBuffer()` at `fetch()` call then pass the array buffer to `new Uint8Array()`, though again, have no experience using react – guest271314 Aug 04 '17 at 21:44
  • Has anyone soved this yet? I am using react-native-video and having the same problem..... – Shawn Apr 08 '20 at 02:41

2 Answers2

1

You could use the getSynthesizeSpeechUrl method from AWS.Polly.Presigner. I’m doing this and using react-native-sound to play the mp3. I ran into an issue where the mp3 wouldn’t play because my presigned URL contained special characters, but there’s a fix here.

Zach Gibson
  • 82
  • 1
  • 6
0

You can use fetch() to request one or more media resources, return Response.body.getReader() from .then() to get a ReadableStream of the response. Read the Uint8Array values returned as the stream as read with .read() method of the ReadableStream, append to value to SourceBuffer of MediaSource to stream the media at an HTMLMediaElement.

For example, to output the audio of two requested audio resources, in sequence

window.addEventListener("load", () => {

  const audio = document.createElement("audio");

  audio.controls = "controls";

  document.body.appendChild(audio);

  audio.addEventListener("canplay", e => {
    audio.play();
  });

  const words = ["hello", "world"];

  const mediaSource = new MediaSource();

  const mimeCodec = "audio/mpeg";

  const mediaType = ".mp3";

  const url = "https://ssl.gstatic.com/dictionary/static/sounds/de/0/";

  Promise.all(
      words.map(word =>
        fetch(`https://query.yahooapis.com/v1/public/yql?q=select * from data.uri where url="${url}${word}${mediaType}"&format=json&callback=`)
        .then(response => response.json())
        .then(({
            query: {
              results: {
                url
              }
            }
          }) =>
          fetch(url).then(response => response.body.getReader())
          .then(readers => readers)
        )
      )
    )
    .then(readers => {

      audio.src = URL.createObjectURL(mediaSource);
      mediaSource.addEventListener("sourceopen", sourceOpen);

      async function sourceOpen() {
        var sourceBuffer = mediaSource.addSourceBuffer(mimeCodec);
        // set `sourceBuffer` `.mode` to `"sequence"`
        sourceBuffer.mode = "segments";

        const processStream = ({
          done,
          value
        }) => {
          if (done) {
            return;
          }
          // append chunk of stream to `sourceBuffer`
          sourceBuffer.appendBuffer(value);
        }
        // at `sourceBuffer` `updateend` call `reader.read()`,
        // to read next chunk of stream, append chunk to 
        // `sourceBuffer`
        for (let [index, reader] of Object.entries(readers)) {
          sourceBuffer.addEventListener("updateend", function() {
            reader.read().then(processStream);
          });

          let stream = await reader.read().then(processStream)
            .then(() => reader.closed)
            .then(() => "done reading stream " + index);

          console.log(stream);
        }

      }
    })
})

plnkr http://plnkr.co/edit/9zHwmcdG3UKYMghD0w3q?p=preview

guest271314
  • 1
  • 15
  • 104
  • 177