How to enable TLS verification for C# Kubernetes client without a kubeconfig file?

Question

I want C# code to connect to a K8s cluster and for now just list the namespaces. The following C# code I tried works, and connects to a Kubernetes cluster, but it only works when "SkipTlsVerify = true". When I set SkipTlsVerify to false I get the following error:

Unhandled exception: k8s.Exceptions.KubeConfigException: A CA must be set when SkipTlsVerify === false

I don't want to skip TLS verification. How can I provide the necessary CA information to the C# Kubernetes Client to enable TLS verification?

I don't want to use BuildConfigFromConfigFile(). None of this can reach into a file system to read or write a file. For now all I can use is local variables

I tried this at first, and it works, but it's skipping TLS verification:

        var contextName = "mycontext, i copied this from my kubeconfig file"
        var server = "https://...*** copied from 'server' in kubeconfig file *** "

        var config = new KubernetesClientConfiguration()
        {
            Host = server,
            AccessToken = accessToken,
            SkipTlsVerify = true,
        };
        var client = new Kubernetes(config);
        var namespaces = client.CoreV1.ListNamespace();
        foreach (var ns in namespaces)
        {
            Console.WriteLine(ns.Name());
        }

I also tried the following code, and got the same error message result. The following code works, unless config.SkipTlsVerify is set to false. The inner SkipTlsVerify has no effect, but the outer SkipTlsVerify does affect the result. I assume that the "ClientCertificateKeyData" C# field matches to the "client-key-data" in my kubeconfig file.

        var clientCertificateData = "*** copied and pasted from client-certificate-data in kube-config ****";
        var clientKeyData = "*** copied from client-key-data in kube-config ****";
        var certificateAuthorityData = "...copied from certificate-authority-data in kube-config"
        var config = KubernetesClientConfiguration.BuildConfigFromConfigObject(new K8SConfiguration
        {
            ApiVersion = "v1",
            Clusters = new List<Cluster>
            {
                new()
                {
                    ClusterEndpoint = new ClusterEndpoint
                    {
                        CertificateAuthorityData = certificateAuthorityData,
                        Server = server,
                        //SkipTlsVerify = true // This one has no effect. I still get the same 
                                               //error even when setting this to true
                    },
                    Name = contextName
                }
            }
        }, masterUrl: server); // I think it's a little strange that I need to put in server here  
                               // If I omit masterUrl, i get the error 
                               //"k8s.Exceptions.KubeConfigException: 
                               // Cannot infer server host url either from context or masterUrl"
        //config.SkipTlsVerify = true;  // uncommenting this makes it work
        config.Host = server;
        config.AccessToken = accessToken;
        config.ClientCertificateData = clientCertificateData;
        config.ClientCertificateKeyData = clientKeyData; // I assume this line is supposed to be client-key-data from the kubeconfig?
        var client2 = new Kubernetes(config);
        var namespaces2 = client2.CoreV1.ListNamespace();
        foreach (var ns in namespaces2)
        {
            Console.WriteLine(ns.Name());
        }

Andrew · Answer 1 · 2023-03-30T17:30:42.640

Excuse me my rusty dotnet, i'm mainly a python programmer. This answer is mostly based on code from https://github.com/kubernetes-client/csharp/issues/621#issuecomment-843584577

So the algorithm boils to this:

Create KubernetesClientConfiguration named config, as you already did
Extract CA cert from base64-encoded certificate-authority-data, if you want to use that string
Create X509Certificate2 from that string
Create X509Certificate2Collection, containing former X509Certificate2
Add this X509Certificate2Collection as config.SslCaCerts

Here is my test, my cluster uses cert-based auth instead of token, so config object a little bit different, you shold just use accessToken as you did. Built with latest mcr.microsoft.com/dotnet/sdk image.

InvestigateX509.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net7.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>


  <ItemGroup>
    <PackageReference Include="KubernetesClient" Version="5.0.5" />
  </ItemGroup>

</Project>

InvestigateX509.cs

namespace InvestigateX509
{
    using System;
    using System.Security.Cryptography.X509Certificates;
    using k8s;

    class Program
    {

        static void Main(string[] args)
        {
            var server = "https://YOUR_HOST:6443";
            var certificateAuthorityData = "YOUR_BASE64_ENCODED_certificate-authority-data_FROM_KUBECONFIG";
            var clientCertificateData = "YOUR_BASE64_ENCODED_client-certificate-data_FROM_KUBECONFIG";
            var clientCertificateKeyData = "YOUR_BASE64_ENCODED_client-key-data_FROM_KUBECONFIG";
            var myNamespace = "ingress-nginx";

            var config = new KubernetesClientConfiguration()
            {
                Host = server,
                ClientCertificateKeyData = clientCertificateKeyData,
                ClientCertificateData = clientCertificateData,
            };
            byte[] decodedCertificateAuthorityData = Convert.FromBase64String(certificateAuthorityData);
            X509Certificate2 caCert = new X509Certificate2(decodedCertificateAuthorityData);
            X509Certificate2Collection caCollection = new X509Certificate2Collection();
            caCollection.Add(caCert);
            config.SslCaCerts = caCollection;
            var client = new Kubernetes(config);
            var pods = client.ListNamespacedPod(myNamespace);
            foreach (var pod in pods.Items)
            {
                Console.WriteLine($"Pod: {pod.Metadata.Name}");
            }
        }
    }
}

Example output from my dotnet container

root@84fc24bd18c2:/src/InvestigateX509# ./bin/Debug/net7.0/InvestigateX509 
Pod: ingress-nginx-1656961013-controller-97k82
Pod: ingress-nginx-1656961013-controller-h7dhm
Pod: ingress-nginx-1656961013-controller-lndjw

It prints all 3 pods from ingress-nginx ingress in my 3-node cluster

How to enable TLS verification for C# Kubernetes client without a kubeconfig file?

1 Answers1