Since the System.Web.Security isn't build with .NETStandard, we couldn't use it in the asp.net core application.
According to your description, if you want to decrypt the machine key encrypt value. I suggest you could try to use some third party library which implement the asp.net MachineKey.upprotect method. For example: AspNetTicketBridge. You could directly install it from Nuget Package.
More details, you could refer to below test demo:
Protect the message in an asp.net application.
protected void Page_Load(object sender, EventArgs e)
{
//Label1.Text = HttpContext.Current.User.Identity.Name;
var message = "My secret message";
var encodedMessage = Encoding.ASCII.GetBytes(message);
var protectedMessage = MachineKey.Protect(encodedMessage);
var protectedMessageAsBase64 = Convert.ToBase64String(protectedMessage);
Label2.Text = protectedMessageAsBase64;
}
Machine key:
<machineKey compatibilityMode="Framework45" validationKey="xxx" decryptionKey="xxx" validation="SHA1" decryption="Auto" />
Unprotect the message in an asp.net core application:
string validationKey = "machinekey value";
string decryptionKey = "machinekey value";
var message = "above base64 message";
var convertFromBase64 = Convert.FromBase64String(message);
var ticket = MachineKeyTicketUnprotector.Unprotect(convertFromBase64, decryptionKey, validationKey, "AES", "HMACSHA1", "User.MachineKey.Protect");
var decodedMessage = Encoding.ASCII.GetString(ticket);
Result:

If you want to achieve using machinekey in asp.net core application, you could try to implement the source code from the .net Framework machinekey.cs. Details, you could refer to this answer.