Exception handling in asp net core

Question

I'm looking for some guidance on how to best handle exceptions in asp net core. Based on these docs from microsoft I've setup the UseStatusCodePagesWithRedirects middleware. This works for things such as 404's. However for API requests within my code that return exceptions this doesn't work. So following this doc I've setup an exception filter. This kinda works and this is where I'm looking for help.

I have the following code in a class library so it can be re used elsewhere. A mini wrapper for my api. I'm using Flurl to construct the request. My viewModelBuilder calls GetAll.

public async Task<List<TableDto>> GetAll(int branchId)
{

    var result = await _baseUrl.AppendPathSegment("/v1/Table/GetAll").WithOAuthBearerToken(await _authenticationManager.GetToken()).GetJsonAsync<List<TableDto>>();
    return result;
}

If GetAll throws an exception then the following exception filter comes into play.

public class CustomExceptionFilterAttribute : ExceptionFilterAttribute
{
    public override void OnException(ExceptionContext context)
    {

        // ex.Call.Response.StatusCode // This works with the original exception but not here
        if (context.HttpContext.Response.StatusCode == (int)HttpStatusCode.Forbidden)
        {
            // Always 200
        }
    }
}

I would like to do different things based on the exception status code. So for a 401 or 404 I would like to show an access denied screen or a login screen and then for other errors maybe just a general error screen. However context.HttpContext.Response.StatusCode is always 200.

1. Is this the right way to go about doing this? The error handing documentation suggests that middleware is better but I'm not sure if its referring to this use case as I could only get it working for 404s.
2. How do I get the correct status code in a exception filter?

I know that if I put a try catch around the original GetAll as below

try
{
    var result = await _baseUrl.AppendPathSegment("/v1/Table/GetAll").WithOAuthBearerToken(await _authenticationManager.GetToken()).GetJsonAsync<List<TableDto>>();
    return result;

}
catch (FlurlHttpException ex)
{
    if (ex.Call.Response != null)
    {
        if (ex.Call.Response.StatusCode == HttpStatusCode.Forbidden)
        {
            throw new ForbiddenException();
        }
    }
}

then in the exception filter I can do the following

public class CustomExceptionFilterAttribute : ExceptionFilterAttribute
{
    public override void OnException(ExceptionContext context)
    {
        if (context.Exception is ForbiddenException)
        {

        }
    }
}

and then I can do specific things for specific exceptions however thing means I have to try catch every call and voids the point of the global exception filter. Any advice would be appreciated.

Thanks M

EDIT

Based on this answer here I could write a custom middleware and I quickly did that but I'm still stuck with the same problem of not being able to check for a 401 or 403 and only being able to get access to custom exceptions that have been throw. If I can access the status code then I think both solutions will work