Has anyone had any luck attempting to use client-transcribe-streaming in React native?
I'm trying to stream audio from the device microphone to get a transcription from AWS. When I attempt to begin a transcription I get the error TypeError: Object is not async iterable
even though I have an async generator handling the audio to the TranscribeStreamingClient.
Any examples I've found involve web/browser solutions using the Node.js microphone-stream
package, but I can't find any equivalent package for React Native. The closest I've found is the react-native-live-audio-stream
, but I'm getting an error and the AWS documentation is no help.
I do get a response returned from the .send()
function:
{"$metadata": {"attempts": 1, "cfId": undefined, "extendedRequestId": undefined, "httpStatusCode": 200, "requestId": undefined, "totalRetryDelay": 0}, "SessionId": "633592f0-8ce5-487d-a67e-6c05001f7b98", "TranscriptResultStream": {"options": {"deserializer": [Function anonymous], "messageStream": [MessageDecoderStream]}}}
but when I dig into the the messageDecoderStream
it contains an empty array. I've double checked to ensure audio is being captured and pushed into the audioPayloadStream
.
I'm thoroughly confused as to why I'm getting a TypeError.
I'm stuck, I can't find any successful React Native attempts at this.
Here is the code I'm using
import 'react-native-get-random-values';
import 'react-native-url-polyfill/auto';
import { TranscribeStreamingClient, StartStreamTranscriptionCommand } from '@aws-sdk/client-transcribe-streaming';
import LiveAudioStream from 'react-native-live-audio-stream';
import { Buffer } from 'buffer';
const useTranscribeController = () => {
let isStarted = false;
let audioPayloadStream: any = [];
const audioStream = LiveAudioStream;
const client = new TranscribeStreamingClient({
region: 'us-west-2',
credentials: {
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
sessionToken: process.env.AWS_SESSION_TOKEN,
},
});
const audioGenerator = async function* () {
for await (const chunk of audioPayloadStream) {
yield { AudioEvent: { AudioChunk: chunk } };
}
};
const stop = () => {
audioPayloadStream = [];
audioStream.stop();
client.destroy();
isStarted = false;
};
const start = async () => {
try {
isStarted = true;
const command = new StartStreamTranscriptionCommand({
LanguageCode: 'en-US',
MediaEncoding: 'pcm',
MediaSampleRateHertz: 44100,
AudioStream: audioGenerator(),
});
const response = await client.send(command);
console.log(response);
if (response.TranscriptResultStream) {
for await (const event of response.TranscriptResultStream) {
const message = event.TranscriptEvent;
const results = event.TranscriptEvent?.Transcript?.Results;
results?.map(result => {
(result.Alternatives || []).map(alternative => {
const transcript = alternative.Items?.map(item => item.Content).join(' ');
console.log({ transcript });
});
});
}
} else {
console.log('the response is undefined');
}
} catch (e) {
console.log(e);
if (isStarted) {
stop();
}
}
};
const initialize = async () => {
const options = {
sampleRate: 44100,
channels: 1,
bitsPerSample: 16,
audioSource: 6,
bufferSize: 1024,
wavFile: 'pcm.wav',
};
audioStream.init(options);
audioStream.start();
audioStream.on('data', data => {
audioPayloadStream.push(Buffer.from(data, 'base64'));
if (!isStarted && audioPayloadStream.length !== 0) {
start();
}
});
};
return { initialize };
};
export default useTranscribeController;