Get user information | google assistant | actions-on-google | Firebase

Question

Update: Having my own OAuth server:

Thanks for sharing the step-by-step instructions link. Also, I don't have firebase hosting.

Following is my understanding, please correct me wherever I am wrong:

Approach 1

1. I will need to activate firebase hosting and build 2 endpoints. One for authorization exchange and another for token exchange.

2. I will need to use an OAuth server. Let say: ory-hydra and configure it with the endpoints I created in firebase. Or AWS Lambda.

3. Host the OAuth (ory-hydra) server somewhere on the internet.

4. Use these endpoints on actions on google and make a simple webpage where users will be redirected to authenticate.

Approach 2

1. I will need to activate firebase hosting and build 2 endpoints. One for authorization exchange and another for token exchange.

2. Use the firebase functions to implement OAuth and token endpoints. (I am not sure how to do this and if its possible)

3. Use these endpoints on actions on google and make a simple webpage where users will be redirected to authenticate.

Please correct me if I am wrong.

Update: After making the following changes:

1. Changed the authorization URL to:

https://accounts.google.com/o/oauth2/v2/auth

2. Changed the authorization type to Implicit.

Now, I am getting the authorization URL in the debug section, and I am able to authorize by pasting that URL in another tab.

However, I am still facing issues in getting user information. I have following code in the input.welcome intent:

'input.welcome': () => {
  // Use the Actions on Google lib to respond to Google requests; for other requests use JSON
  if (requestSource === googleAssistantRequest) {
    sendGoogleResponse('Hello, Welcome to my First Fulfillment agent!'); // Send simple response to user
      app.askForSignIn();
      let displayName = app.getUserName().displayName;
      console.log(displayName)
  }  

All I am getting is Null in the debug logs.

Previous Question I am trying to get user's information in the google action intent. Following is what I did:

1.) Created an OAUTH key for my project from google developer console. https://console.developers.google.com/apis/dashboard?project=.

2.) Logged into console.actions.google.com and clicked on Account Linking.

3.) Entered the information. Please refer the screenshot to see the information I entered.

4.) In my 'input.welcome' intent added following code:

app.askForSignIn(); 

Now in the simulator, I am getting: "It looks like your my test app account is not linked yet." I have also checked sign-in required for my intent from the dialog flow UI.

Previous question I am trying to send an email from my google assistant conversational bot. I am able to send emails to using nodemailer. However, I am not able to get the user email address.

Following is my code to send email:

var transporter = nodemailer.createTransport({
    service: 'gmail',
    auth: {
        user: 'My-Email',
        pass: 'My-Pass',
    }
});

var mailOptions = {
    from: 'Sender-Email',
    to: 'Receiver-Email',
    subject: 'Requested Information',
    text: 'Your information is here' 
};

function sendEmail(){
        transporter.sendMail(mailOptions, function(error, info){
            if (error) {
                console.log(error);
            } else {
                console.log('Email sent: ' + info.response);
            }
        });
}

And finally, I am calling it in the action intent:

'input.sendmail': () => {
    sendEmail();
}, 

Till this point I am able to send emails when someone says, send email to my google assistant action.

After this, I tried to get the user's email address using the following methods:

const app = new DialogflowApp({request: request, response: response});
console.log(app.getUserName())
console.log(app.getUser().userName)
console.log(app.getUser().userId)

But none of them gave me user's information. Instead, I am getting following information in the dialogflow console log:

{ 
userStorage: '{"data":{}}',
lastSeen: '2018-03-05T10:18:17Z',
locale: 'en-US',
userId: 'ABadfdfrffsdffNa0H4hlCy_eyZmVNa8LweMJMCyirUg-
qAx8FHwvSI49QurUhxhgLsT6IUU4nGfF1',
user_id: 'ABerysteui4hlCy_eyZmVNa8LweMJMCyirUg-
qAx8FHwvSI49QurUhxhgLsT6IUU4nGfF1',
access_token: undefined,
userName: null 
}

I tried to google this issue and it seems I will need to follow [1], not sure though.

I will appreciate if someone can tell me if [1] is the correct guide to follow, or I will need to something else?

Thanks!

[1] https://developers.google.com/actions/identity/account-linking

Answer 1

That is correct. For security and privacy purposes, there is no way to get the email address associated with the account used to setup the Assistant account.

The correct way to go about this would be to implement account linking. With this, you would create an account on your system and, as part of that account, get the user's email address (typically via an app or webapp). The account linking would then connect your account to the Assistant's account and, when they use that account to access your Action, you'd be sent an access token which you can use to identify which of your accounts this is. You can then get the user's information from your account info.

When the user is prompted to link their account for the first time, Google Home users will get an activity card on the Google Home app on their phone which will direct them to your auth page. From a mobile device, it should open in the Assistant directly.

If you do not already have accounts or a login page, you should be able to build this with Firebase Authentication and the Google auth provider.

Update for clarity: To be clear - just linking your account to their Assistant account won't automatically give you the information about their Assistant account. You can get their email address (your original request) when they setup the account with you by requesting the profile scope as part of the OAuth. You can then use the information you've collected about them when you know they have connected to you via the Assistant.

In your updated question, you're trying to get their name after they have logged in through the Assistant. If all you wanted was their name, you could have asked for permission to get this without requiring Account Linking or login. (Or, as noted above, you could have asked for this when they created the account with you.)

Update (Based on your question about OAuth)

Two things to your update.

1. Being prompted "It looks like your account isn't linked yet" is normal. You'll need to use the URL provided in the response tab to continue the account linking.

2. More significantly, however, it doesn't sound like you've setup an OAuth server - just that you're trying to configure things. The screen shot make it look like you're just using the URL that is supposed to be to respond with auth tokens - not where the system will go to request them. Make sure you have read the documentation at https://developers.google.com/actions/identity/account-linking to see what values should be in the configuration and what other tasks you need to do.

Update based on your comment that you need an OAuth server.

Yes, you need an OAuth server. You cannot just use Google's, even if you just want your users to log into their Google account.

There are quite a few OAuth servers available, however a google search for "open source openid connect server" or "open source oauth server" shows some promising results.

Additionally, Google gives you step-by-step instructions on what it is expecting for an OAuth server it connects to. So you would need to

1. Implement a way for a user to create and log into an account on your service and
2. Implement the OAuth exchange protocols as Google has described (I suggest the Auth Code Flow method).

You do not need your own domain - you can implement both of these through Firebase Functions and Firebase Hosting which includes a SSL certificate for a hostname for your project.

Update addressing your possible approaches to implementing an OAuth2 server.

First of all - you don't need to use Firebase Hosting and/or Firebase Functions for anything. They're just an option that provide you a valid HTTPS endpoint.

Approach 1 - use an external package such as ory-hydra

I'm not sure what the point of the Firebase Hosting would be in your example. The auth exchange endpoint and token exchange endpoint are exactly what the OAuth2 server is there to do.

I don't know much about ory-hydra, but it certainly seems a reasonable solution. You would need to host it somewhere (AWS, Google Compute Engine, or other hosting provider that would work with it), but it should provide the endpoints you need. From a quick reading of ora-hydra, you will need to provide an account backend of some sort and a way for your users to login to that account.

Approach 2 - implement using Firebase

You have this completely correct. It is fairly straightforward (not necessarily easy - but straightforward) to do a simple OAuth2 implementation with Firebase Cloud Functions combined with a login page hosted on Firebase Hosting that uses Firebase Authentication for the login.

Which approach you take is up to you. Using an existing solution is certainly easier, hopefully more reliable and secure, and will let you focus on the Action more itself, but may still require a lot of integration work. Implementing an OAuth2 server will give you a better understanding of OAuth2, but runs a higher risk of problems.

Answer 2

I am able to make it work after a long time. We have to enable the webhook first and we can see how to enable the webhook in the dialog flow fulfillment docs If we are going to use Google Assistant, then we have to enable the Google Assistant Integration in the integrations first. Then follow the steps mentioned below for the Account Linking in actions on google:-

Go to google cloud console -> APIsand Services -> Credentials -> OAuth 2.0 client IDs -> Web client -> Note the client ID, client secret from there -> Download JSON - from json note down the project id, auth_uri, token_uri -> Authorised Redirect URIs -> White list our app's URL -> in this URL fixed part is https://oauth-redirect.googleusercontent.com/r/ and append the project id in the URL -> Save the changes

Actions on Google -> Account linking setup 1. Grant type = Authorisation code 2. Client info 1. Fill up client id,client secrtet, auth_uri, token_uri 2. Enter the auth uri as https://www.googleapis.com/auth and token_uri as https://www.googleapis.com/token 3. Save and run 4. It will show an error while running on the google assistant, but dont worry 5. Come back to the account linking section in the assistant settings and enter auth_uri as https://accounts.google.com/o/oauth2/auth and token_uri as https://accounts.google.com/o/oauth2/token 6. Put the scopes as https://www.googleapis.com/auth/userinfo.profile and https://www.googleapis.com/auth/userinfo.email and weare good to go. 7. Save the changes.

In the hosting server(heroku)logs, we can see the access token value and through access token, we can get the details regarding the email address.

Append the access token to this link "https://www.googleapis.com/oauth2/v1/userinfo?access_token=" and we can get the required details in the resulting json page.

`accessToken = req.get("originalRequest").get("data").get("user").get("accessToken")
r = requests.get(link)
print("Email Id= " + r.json()["email"])
print("Name= " + r.json()["name"])`