I tried to reproduce the same in my environment and got the results like below:
I generated access token using below parameters:
https://login.microsoftonline.com/TenantID/oauth2/v2.0/token
client_id:ClientID
client_secret:ClientSecret
scope:https://graph.microsoft.com/.default
grant_type:client_credentials

However, the access token does not contain any claim with the App Registration name.
Note that: By default, the access token will contain App Name and App ID.
When I decoded the access token, the App Name and App ID is present like below:

Hence there is no need to create custom claims for App Name and App ID.
It is not possible to add custom claims while using client credential flow.
I created a custom claim like below:
New-AzureADPolicy -Definition @('
{
"ClaimsMappingPolicy":
{
"Version":1,"IncludeBasicClaimSet":"true",
"ClaimsSchema": [{"Source":"user","ID":"extensionattribute1","SamlClaimType":"http://schemas.xmlsoap.org/ws/2005/05/identity/claims/appname","JwtClaimType":"appname"}]
}
}') -DisplayName "appname" -Type "ClaimsMappingPolicy"
Add-AzureADServicePrincipalPolicy -Id d53b9689-3c0d-4c39-ae6d-efd733e10776 -RefObjectId fd49e655-b790-4368-b9d9-1cf09fda1ba2
Get-AzureADServicePrincipalPolicy -Id d53b9689-3c0d-4c39-ae6d-efd733e10776

I assigned the value to the claim like below:
PATCH https://graph.microsoft.com/v1.0/me
{
"onPremisesExtensionAttributes":
{
"extensionAttribute1": "testclaim"
}
}

Now when I generated access token using Authorization Code Flow I got the claim successfully like below:
