2

Has anyone had any luck attempting to use client-transcribe-streaming in React native?

I'm trying to stream audio from the device microphone to get a transcription from AWS. When I attempt to begin a transcription I get the error TypeError: Object is not async iterable even though I have an async generator handling the audio to the TranscribeStreamingClient.

Any examples I've found involve web/browser solutions using the Node.js microphone-stream package, but I can't find any equivalent package for React Native. The closest I've found is the react-native-live-audio-stream, but I'm getting an error and the AWS documentation is no help.

I do get a response returned from the .send() function:

{"$metadata": {"attempts": 1, "cfId": undefined, "extendedRequestId": undefined, "httpStatusCode": 200, "requestId": undefined, "totalRetryDelay": 0}, "SessionId": "633592f0-8ce5-487d-a67e-6c05001f7b98", "TranscriptResultStream": {"options": {"deserializer": [Function anonymous], "messageStream": [MessageDecoderStream]}}} 

but when I dig into the the messageDecoderStream it contains an empty array. I've double checked to ensure audio is being captured and pushed into the audioPayloadStream.

I'm thoroughly confused as to why I'm getting a TypeError.

I'm stuck, I can't find any successful React Native attempts at this.

Here is the code I'm using

import 'react-native-get-random-values'; 
import 'react-native-url-polyfill/auto'; 
import { TranscribeStreamingClient, StartStreamTranscriptionCommand } from '@aws-sdk/client-transcribe-streaming'; 
import LiveAudioStream from 'react-native-live-audio-stream'; 
import { Buffer } from 'buffer'; 

const useTranscribeController = () => { 
  let isStarted = false; 
  let audioPayloadStream: any = []; 
  const audioStream = LiveAudioStream; 
  const client = new TranscribeStreamingClient({ 
    region: 'us-west-2', 
    credentials: { 
      accessKeyId: process.env.AWS_ACCESS_KEY_ID, 
      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY, 
      sessionToken: process.env.AWS_SESSION_TOKEN, 
    }, 
  }); 
  
  const audioGenerator = async function* () {
    for await (const chunk of audioPayloadStream) {
      yield { AudioEvent: { AudioChunk: chunk } };
    } 
  }; 

  const stop = () => {
    audioPayloadStream = [];
    audioStream.stop();
    client.destroy();
    isStarted = false;
  };

  const start = async () => {
    try {
      isStarted = true;
      const command = new StartStreamTranscriptionCommand({
        LanguageCode: 'en-US',
        MediaEncoding: 'pcm',
        MediaSampleRateHertz: 44100,
        AudioStream: audioGenerator(),
      });
      const response = await client.send(command);
      console.log(response);
      if (response.TranscriptResultStream) {
        for await (const event of response.TranscriptResultStream) {
          const message = event.TranscriptEvent;
          const results = event.TranscriptEvent?.Transcript?.Results;
          results?.map(result => {
            (result.Alternatives || []).map(alternative => {
              const transcript = alternative.Items?.map(item => item.Content).join(' '); 
              console.log({ transcript });
            });
          });
        }
      } else {
        console.log('the response is undefined');
      }
    } catch (e) {
      console.log(e);
      if (isStarted) {
        stop();
      }
    } 
  }; 

  const initialize = async () => {
    const options = {
      sampleRate: 44100, 
      channels: 1, 
      bitsPerSample: 16, 
      audioSource: 6, 
      bufferSize: 1024, 
      wavFile: 'pcm.wav', 
    }; 
    
    audioStream.init(options); 
    audioStream.start(); 
    audioStream.on('data', data => { 
      audioPayloadStream.push(Buffer.from(data, 'base64')); 
      if (!isStarted && audioPayloadStream.length !== 0) {
        start(); 
      }  
    }); 
  }; 
  return { initialize }; 
}; 

export default useTranscribeController; 
spaceman30
  • 31
  • 3

0 Answers0