LUIS App - Get popular intents with endpoint hits count

Question

Lets assume LUIS app with 100 intents. How can I retrieve popular top ten intent names(predicted as the top scoring intent) along with endpoint hit count?

I want to show something like this on UI.

Answer 1

Microsoft has a tutorial to add LUIS data to Application Insights (nodejs, but I'm sure you can find/do something similar if you are using C#). From there, you can use Logs (Analytics) to identify the intents for each utterance and count/graph them. In my case, I removed None and NULL intents as I wanted a breakdown of recognized intents only, but you could leave out those lines if you want to see everything. The "take" operator gives you the first n rows. Make sure you sort first or it won't necessarily be the top n.

requests
| where url endswith "messages"
| where timestamp > ago(30d)
| project timestamp, duration, performanceBucket, resultCode, url, id
| parse kind = regex url with *"(?i)http://"botName".azurewebsites.net/api/messages"
| join kind= inner (
traces | extend id = operation_ParentId
) on id
| where message == "LUIS"
| extend topIntent = tostring(customDimensions.LUIS_luisResponse_luisResult_topScoringIntent_intent)
| where topIntent != "None"
| where topIntent != ""
| summarize count() by topIntent
| order by count_ desc
| take 10

Here is the resulting table. I actually like to render piechart for mine, but I made this one a table of top 10 to fit your requirement.

If this isn't what you're after, let me know and I'll try to assist further.

Answer 2

It turns out LUIS actually does have a way of recording intent predictions for you. As long as you call your prediction endpoint with the log parameter set to true, the query results will be logged. The v2 API says log is true by default, though my testing indicates that you have to explicitly set log to true for the v3 prediction endpoint.

You can retrieve those logs using this v2 API or this v3 API and then you can query the data from there. Here's a C# 8.0 example with some code borrowed from this answer:

// Get LUIS logs
using var luis = new LUISAuthoringClient(new Microsoft.Azure.CognitiveServices
    .Language.LUIS.Authoring.ApiKeyServiceClientCredentials(YOUR_AUTHORING_KEY))
{
    Endpoint = $"https://{YOUR_AUTHORING_REGION}.api.cognitive.microsoft.com"
};

using var stream = await luis.Apps.DownloadQueryLogsAsync(new Guid(YOUR_LUIS_APP_ID));
using var parser = new TextFieldParser(stream);

parser.SetDelimiters(",");

// The first row should be something like:
// Query,UTC DateTime,Response,SecondaryId,Region
var csvHeaders = parser.ReadFields();
var responseColumn = Array.IndexOf(csvHeaders, "Response");
var intentCount = new Dictionary<string, int>();

while (!parser.EndOfData)
{
    // Process row
    var fields = parser.ReadFields();
    var responseObject = JObject.Parse(fields[responseColumn]);
    var topIntent = responseObject["prediction"]["topIntent"].ToString();

    if (!intentCount.TryGetValue(topIntent, out var count))
    {
        count = 0;
    }

    intentCount[topIntent] = ++count;
}

// Query processed logs
var top10 = intentCount.OrderByDescending(kvp => kvp.Value).Take(10);

// Example output: [["Greeting",5],["None",3],["Cancel",1],["Help",1]]
Console.WriteLine(JsonConvert.SerializeObject(top10
    .Select(kvp => new object[] { kvp.Key, kvp.Value })));