8

I have a lot of questions about the VoiceInteractionService.

https://developer.android.com/reference/android/service/voice/VoiceInteractionService

What seems clear to me is that it is the mechanism by which the Google assistant has hooks into the OS in order to actually work as a default assistant. However, there’s still a lot about it that requires more explanation.

In VoiceInteractionSession:

  1. The documentation says that “The user interface is initially shown by default, and can be created be overriding onCreateContentView() (https://developer.android.com/reference/android/service/voice/VoiceInteractionSession.html#onCreateContentView()) in which the UI can be built.” How should the UI for a voice service be built? Currently our UI exists within an Activity. But the existence of getLayoutInflater(), getWindow(), onShow(), and onCreateContentView() in the session makes me unsure how the UI is meant to be shown. Should I be sending an intent in order to show our UI activity in onShow()? Or should I be creating the UI in Java and returning it as a view in onCreateContentView()? Should I be using the Window to draw? Do I need to call setContentView() at any point?
  2. It is also unclear to me the relationship between the startAssistantActivity() method, startVoiceActivity() method, and the above methods. When would we want to call startAssistantActivity? When would we want to call startVoiceActivity? In the documentation, it says “By default, the system will create a window for the UI for this session. If you are using an assistant activity instead, then you can disable the window creation by calling setUiEnabled in onPrepareShow” Is this the view returned by onCreateContentView()? Is startAssistantActivity() meant to be an alternative way of showing assistant UI as compared to onCreateContentView?
  3. How does onLockscreenShown() get called? Is it called when the assistant is launched from the lockscreen? If the user locks their phone while the assistant is open?
  4. Is there anything special needed to be done to show the assistant UI over the lockscreen?
  5. Why wouldn’t you call closeSystemDialogs()? It seems as though you almost never want system windows visible over the assistant app.
  6. If the user invoked the assistant through holding the home button, or even some other way, if there a way to reliably get the foreground component name? I see that getActivityComponent() is an accessor in AssistStructure, but it is not clear to me in which situations the session would have access to that object.
  7. What are the lifecycles of the various services and sessions? If there’s a separate service actually handling the processing of audio data, when should that service be bound/unbound? In onShow() and onHide()?

VoiceInteractionService:

  1. createAlwaysOnHotwordDetector - What if I wanted multiple hotwords active at once? For instance, “Hey Google” and “OK Google”?
  2. About the hotword detector callback: when onDetected is called, there’s a method getTriggerAudio() which contains the raw audio that triggered the request. How do you ensure that you don’t miss any of the user’s speech? For instance, it’s a much better experience for the user to say “Hey Google time” as opposed to “Hey Google …. time”.
  3. Is the correct “response” in the callback to call showSession?
  4. Assuming that the assistant wanted to run secondary wake word verification (aka make sure that the user actually said the hotword) before actually showing any UI, is the best way to do that:
    1. Run the verification in the callback from the hotword detector
    2. If it is indeed the wake word, call show session
    3. Otherwise, ignore silently
  5. The documentation for onLaunchVoiceAssistFromKeyguard says that you need to start an activity with the show on lockscreen flag. This is a little confusing because I’m still not sure if I should be starting an activity to show UI or somehow using onCreateContentView()?
  6. What does setUiHints do? What is the context in which the hints will be appearing in? Are they Toasts? Are they embedded into other UI?

General questions:

  1. What's the relationship between the following manifest attributes and a VoiceInteractionService? Based on some other documentation on the web, what is needed to show as a potential default assistant is the following code snipped in AndroidManifest.xml, like so: https://gist.github.com/pedrovgs/c424fe754a74f326e997. But it also seems that an app that contains a VoiceInteractionService is automatically in the list. Is there a recommended way moving forward for having a different assistant?
Eric
  • 81
  • 1
  • 2

1 Answers1

0

For the UI in VoiceInteractionSession , you are supposed to inflate your layout (with getLayoutInflater().inflate(layout) in onCreateContentView() , find your views by id and return the view.

If you need to set a theme , it should be set with setTheme(theme) and called in onCreate() before calling super.onCreate()

mate00
  • 2,727
  • 5
  • 26
  • 34
Louis Moreau
  • 16
  • 2