How to enable text to speech service on introduction message in MS BOT FRAMEWORK sdk4

Question

I have implemented a chat (MS Botframework sdk4 version) using MS Speech service API for text to speech and speech to text conversion.

Here is the link to the demo code which I have referenced to implement this. https://github.com/Microsoft/ailab/tree/master/BuildAnIntelligentBot

The Speech service only activates when I click on the microphone button in my client page. Hence because of this the Introductory message of my bot is unable to enable text to speech service. Hence only the following subsequent dialogs are speech enabled when I click on the microphone icon. How do I enable Text to speech on the introductory message as well ?

How are your users interacting with the bot? Through Web Chat? — mdrichardson, Mar 13 '19 at 18:30

mdrichardson · Answer 1 · 2019-03-19T20:25:40.663

This is barely possible. Web Chat is designed to not speak unless the user has spoken first (see the first paragraph after the first image).

You can get around this by tricking Web Chat into thinking the user has already spoken by sending a fake speech message with the following code (in your .html file hosting the Web Chat):

<script>
    // Must create store so we can send command
    const store = window.WebChat.createStore();
    window.WebChat.renderWebChat({
    directLine: window.WebChat.createDirectLine({ secret: 'YourBotSecret' }),
        // Add speech capability
        webSpeechPonyfillFactory: window.WebChat.createBrowserWebSpeechPonyfillFactory(),
        store
    }, document.getElementById('webchat'));

    document.querySelector('#webchat > *').focus();

    // When mic button is clicked, send message as though user has spoken
    // Note: 'test' will not be displayed at all to the user. You can replace it with anything.
    document.querySelector('[title="Speak"]').addEventListener('click', () => {
    store.dispatch({
        type: 'WEB_CHAT/START_SPEAKING',
        payload: { text: 'test' }
    });
    });

    // Programatically click the mic button
    document.querySelector('[title="Speak"]').click();

    // Programatically click the mic button back off
    window.setTimeout(() => document.querySelector('[title="Speak"]').click(), 500);

    // Stop listening
    document.querySelector('[title="Speak"]').removeEventListener('click', () => {
    store.dispatch({
        type: 'WEB_CHAT/START_SPEAKING',
        payload: { text: 'test' }
    });
    });
</script>

Important Note: Chrome doesn't like this:

The only way around this is to force the user to interact with the website first. Any of these interactions work. A key point is that the event must have isTrusted, which is only available for actual user interactions. How you do that is up to you. Maybe you force the user to click a "run bot" button prior to connecting the chat.

How to enable text to speech service on introduction message in MS BOT FRAMEWORK sdk4

1 Answers1

References