Azure Monitor Agent Custom Logs not working

Question

I added my laptop to Azure ARC and installed the Azure Monitoring Agent. It all works great and I get the heartbeats.

I tried 2 methods for adding a custom log, both of which does not work

Method 1

When adding a custom log though is where things fall apart for me, not sure if I am missing something. I followed this tutorial: https://learn.microsoft.com/en-us/azure/azure-monitor/agents/data-collection-text-log?tabs=portal

I created the Data Collection Rule and added a resource (my laptop) Screenshot of DCR Resource

I then created a custom text logs data sources and the table is the one that I created with Powershell according to the tutorial. Data Source

The destination of this data source is the log analytics workspace that contains this table Data Source Destination

The table does not seem to contain any data collection rules No data collection rules associated with table

I left everything for about 30 minutes and checked for data, but no data was imported. The log files contain only 2 columns to match with the table format and keep it simple for now. Here is an example: 2023-03-29T12:19:52.983Z myclass.py

Method 2

I created a new Custom Log (DRC based) table in the same Log Analytics Workspace. I then create a new Data Collection Rule (I also tried pointing this new table to an existing Data Collection Rule).

In both cases a weird entry like this one gets created. When I then add a Custom Text Log Data source so that I can tell the DCR where to find the logs, this weird entry gets removed Weird Entry

When I check the table, there are still no DCR associated with the table.

Logic tells me when you right click the table and click on "Manage Table" a DCR should be associated with the table, otherwise where will the DCR know to which table in the Log Analytics Workspace to send the custom text log data to?

Or I am missing something probably very obvious and that is why I cannot get it to work.

Any help will be appreciated.

Answer 1

I followed the below steps and able to create custom logs.

Create a new workspace from log analytics by choosing your subscription and resource group.

Choose the same region of the resource group to workspace.

Choose custom logs.

Add a sample log file.

After this you need to create a Data Collection Endpoint from Monitor.

Make sure the region is same as for the resource group.

After that you need to create a Data Collection Rule with a Resource and Data source.

Query the logs from the log analytics using KQL.

Check if any records have been collected for your custom log table by running the below query in Log Analytics with difference in time range.

<YourCustomLog>_CL
| where TimeGenerated > ago(48h)
| order by TimeGenerated desc

And also verify that Azure Monitor agent is communicating properly by running the below query in Log Analytics to check if there are any records in the Heartbeat table.

Heartbeat
| where TimeGenerated > ago(24h)
| where Computer has "<computer name>"
| project TimeGenerated, Category, Version
| order by TimeGenerated desc

Also verify that the filePatterns element specifies the path to the log file to collect from the agent computer.

For further information refer to this Link.