Cookie is not set although Set-Cookie Header is present when requested from different port

Question

I have an ASP.NET core backend and the frontend is written in Blazor (Blazor WebAssembly). When I start my backend the frontend is started as well and both run under https://localhost:5000. When I log in the user with the SignInManager the framework correctly sets the Set-Cookie header in the response and the browser correctly sets the cookie and it is an authenticated session from now on.
However, I sometimes start the frontend with the command dotnet watch run separately so I have a better "Edit and continue" experience. Started like this, the frontend runs under https://localhost:7026 and requests against the backend running under https://localhost:5000. This works. Requests work, CORS is set up and the backend is reachable. With that setup, the frontend running under :7026, when I login the user, the framework again correctly sets the Set-Cookie Header in the Response but the browser does not set the application cookie.
What I did notice, is that when the user is signed in from the frontend running under :5000 the cookie is also set for the frontend running under :7026 but the browser does not send that cookie with the requests, thus resulting in an unauthenticated session for the frontend running under :7026.

I did read that cookies should be port agnostic so I am wondering why the browser does not set the cookie when the frontend runs under :7026.
Cookie config:

builder.Services.ConfigureApplicationCookie(options =>
{
    options.Cookie.HttpOnly = true;
    options.Cookie.IsEssential = true;
    options.ExpireTimeSpan = TimeSpan.FromDays(5);
    options.ReturnUrlParameter = CookieAuthenticationDefaults.ReturnUrlParameter;
    options.SlidingExpiration = true;

    options.Events = new CookieAuthenticationEvents
    {
        OnRedirectToLogin = context =>              {
            if (context.Request.Path.StartsWithSegments("/api"))
            {
                context.Response.StatusCode = (int)HttpStatusCode.Unauthorized;
            }
            return Task.CompletedTask;
        }
    };
});

Cors:

builder.Services.AddCors(
    options => options.AddDefaultPolicy(
        policy => policy.WithOrigins("https://localhost:7026").AllowAnyMethod().AllowAnyHeader().AllowCredentials()));

//request pipeline
app.UseRouting();

app.UseAuthentication();
app.UseAuthorization();

app.UseCors();
//shortened for brevity...

I tried:

1. Different Cookie.SameSite policies
2. Different Cookie.SecurePolicy policies
3. Setting Cookie.Domain to specific domain
4. Reinstalling the self signed asp.net core certificate to run localhost under https

None of those resulted in having the cookie correctly set when the frontend runs under :0726 and the backend under :5000