2

I am currently using Microsoft Azure to get transcribed text from live speech recognition. With that transcribed text, I put that into a TextRank to extract keywords from that speech stream. However, when I run this I lose a lot of speech recognition while running the TextRank code. Is there a way to run speech recognition continuously while passing the transcribed results to the next process, at the same time processing the TextRank keyword extraction, so that I don't lose any speech and extract keywords?

def from_mic():
    speech_config = speechsdk.SpeechConfig(subscription="", region="")
    speech_recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config)

    # print("Speak into your microphone.")
    result = speech_recognizer.recognize_once_async().get()
    print(result.text)
    return result.text

for i in range(1,10):
    transcript = from_mic()
    summa_keywords = summa_keyword_extractor.keywords(transcript, ratio=1.0)
    print(summa_keywords)
rustyhu
  • 1,912
  • 19
  • 28
Brian Lee
  • 31
  • 2
  • It seems multi-thread or multi-process are needed, to make the 2 procedures `speech_recognizer` and `keyword_extractor` work parallelly. – rustyhu Nov 12 '20 at 11:59

1 Answers1

1

You need to setup two parallel processes but interlinked with a task-queue.

This is because you have a dependency of extraction on recorder-process.

Here is an attempt of one way to achieve this (obviously its not polished and can be improved further):

def recorder_process(recorder_queue, extractor_queue):
  speech_config = speechsdk.SpeechConfig(subscription="", region="")
  speech_recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config)

  while True:
    request = recorder_queue.get()
    result = speech_recognizer.recognize_once_async().get()
    extractor_queue.put(result.text)

def extractor_process(extractor_queue, results_queue):
  while True:
    transcript = extractor_queue.get()
    summa_keywords = summa_keyword_extractor.keywords(transcript, ratio=1.0)
    results_queue.put({'transcript': transcript, 'keywords': summa_keywords})

if __name__ == "__main__":
    # Connect to remote host over TCP
    client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    client.connect((HOST,PORT))

    # Set up a Queue to pass data to the update process, and another one
    # for the two children to communicate
    recorder_queue = Queue()
    extractor_queue = Queue()
    results_queue = Queue()

    # Create two child processes, pass a reference to the Queue to each
    recorder = Process(target=recorder_process, args=(recorder_queue, extractor_queue))
    extractor = Process(target=extractor_process, args=(extractor_queue, results_queue))

    recorder.start()
    extractor.start()

    index = 0
    while True:
      recorder_queue.put(index)
      index += 1
      sleep(1)

    recorder.join()
    extractor.join()
Serial Lazer
  • 1,667
  • 1
  • 7
  • 15
  • Thank you for your comment. Do you have any suggestions on a website or a book that I could learn this method step by step? – Brian Lee Nov 18 '20 at 09:55
  • What is the reason that you used the socket? what was the reason for connecting to a remote host over TCP? – Brian Lee Dec 12 '20 at 06:34