Could not deserialize key data (RSA). The data may be in an incorrect format

Question

I have this flask endpoint. But openssl is raising a weird error.

import jwt

@app.route('/api/secure')
def secure_endpoint():
    options = {'verify_aud': False, 'require_sub': True}
    access_token = request.headers.get('Authorization').split('Bearer ')[1]
    jwks_uri = 'https://login.microsoftonline.com/common/discovery/v2.0/keys'
    jwkeys = requests.get(jwks_uri).json()['keys']
    token_key_id = jwt.get_unverified_header(access_token)['kid']
    jwk = [key for key in jwkeys if key['kid'] == token_key_id][0]
    der_cert = b64decode(jwk['x5c'][0])
    cert = x509.load_der_x509_certificate(der_cert, default_backend())
    public_key = cert.public_key()
    pem_key = public_key.public_bytes(encoding=serialization.Encoding.PEM,
                                      format=serialization.PublicFormat.SubjectPublicKeyInfo)
    token_claims = jwt.decode(jwt=access_token, verify=False, key=pem_key,
                              audience=app_config.CLIENT_ID, algorithms=['RS256'], options=options)
    print(token_claims)

The error happens at the jwt.decode( line.

I tried to add padding pem_key = pem_key.decode().split('\n-----END PUBLIC KEY-----\n')[0] + '==' + '\n-----END PUBLIC KEY-----\n' then got below error

ValueError: ('Could not deserialize key data. The data may be in an incorrect format, it may be encrypted with an unsupported algorithm, or it may be an unsupported key type (e.g. EC curves with explicit parameters).', [_OpenSSLErrorWithText(code=75497572, lib=9, reason=100, reason_text=b'error:0480

context: I am tying to verify a token generated by azure AD (MSAL) from angular client. I am not able to decode the token. Please help me. Thanks in advance.

This error also happened with RSA512. The jwt package used is PyJwt.