How to deserialize Kubernetes YAML file

Question

How can I deserialize a Kubernetes YAML file into an Go struct? I took a look into the kubectl code, but somehow I get an error for every YAML file:

no kind "Deployment" is registered for version "apps/v1beta1"

This is an MWE:

package main

import (
    "fmt"

    "k8s.io/client-go/pkg/api"
)

var service = `
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: my-nginx
spec:
  replicas: 2
  template:
    metadata:
      labels:
        run: my-nginx
    spec:
      containers:
      - name: my-nginx
        image: nginx
        ports:
        - containerPort: 80
`

func main() {
    decode := api.Codecs.UniversalDecoder().Decode
    //decode := api.Codecs.UniversalDeserializer().Decode

    obj, _, err := decode([]byte(service), nil, nil)
    if err != nil {
        panic(err)
    }

    fmt.Printf("%#v\n", obj)
}

I am using client version 2.0.0. The glide.yaml looks like this:

package: test/stackoverflow
import:
- package: k8s.io/client-go
  version: ^2.0.0

These are the references to kubectl:

Unfortunately, the docs are very confusing to me, so I have no idea how to tackle this problem.

Edit:

This problem also exists with other resource types:

no kind "Service" is registered for version "v1"

Wanna give Go dep a try? https://hackernoon.com/using-go-dep-as-a-project-maintainer-641d1f3006d7 — Michael Hausenblas, Jun 01 '17 at 12:25
Yup, I know (hence my answer below) but trying to encourage people to move to a more future-proof alternative ;) — Michael Hausenblas, Jun 01 '17 at 12:35

score 5 · Accepted Answer · answered Jun 01 '17 at 14:28

5

You need to import _ "k8s.io/client-go/pkg/apis/extensions/install" otherwise the schema is empty, see also docs.

The complete working example is:

$ go get -u github.com/golang/dep/cmd/dep
$ dep init
$ go run main.go

With the following main.go:

package main

import (
    "fmt"

    "k8s.io/client-go/pkg/api"
    _ "k8s.io/client-go/pkg/api/install"
    _ "k8s.io/client-go/pkg/apis/extensions/install"
)

var deployment = `
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: my-nginx
spec:
replicas: 2
template:
  metadata:
    labels:
      run: my-nginx
  spec:
    containers:
    - name: my-nginx
      image: nginx
      ports:
      - containerPort: 80
`

func main() {
    // decode := api.Codecs.UniversalDecoder().Decode
    decode := api.Codecs.UniversalDeserializer().Decode

    obj, _, err := decode([]byte(deployment), nil, nil)
    if err != nil {
        fmt.Printf("%#v", err)
    }

    fmt.Printf("%#v\n", obj)
}

Note that I also imported _ "k8s.io/client-go/pkg/api/install" for you so that you can use objects in v1 such as pods or services.

EDIT: Kudos to my colleague Stefan Schimanski who proposed the initial solution.

answered Jun 01 '17 at 14:28

Michael Hausenblas

13,162
4
52
66

Whoa. That's very magic ;-) On the other hand it's understandable, if one wants a extendable system. I guess the docs are improvable. Thanks! – svenwltr Jun 01 '17 at 14:33
Pretty close to magic indeed and I was noodling around for a while myself a bit before Stefan came to the rescue. The docs need tons of improvement alright, agreed! – Michael Hausenblas Jun 01 '17 at 14:35
Hi @MichaelHausenblas, I'm trying to make this work (parse a k8s yaml to use Go structs) but cannot make it work with dep (or vndr). Copying you exact same example, and running "dep init" gets: No versions of k8s.io/apimachinery met constraints: master: unable to update checked out version: : command failed: [git checkout 1168e538ea3ccf444854d1fdd5681d2d876680a7]: exit status 128 (next comment) – Alberto Megía Sep 20 '17 at 17:18
release-1.6: unable to update checked out version: : command failed: [git checkout 35be0db31bd6e5635ef6366708dc2b137392f6a2]: exit status 128 release-1.7: unable to update checked out version: : command failed: [git checkout 8ab5f3d8a330c2e9baaf84e39042db8d49034ae2]: exit status 128 – Alberto Megía Sep 20 '17 at 17:19
Ok I've been able to fix it importing "k8s.io/client-go/pkg/api", "k8s.io/client-go/pkg/api/install", "k8s.io/client-go/pkg/apis/apps/install" and "k8s.io/client-go/pkg/apis/extensions/install". But if I run it, I get a bunch of # github.com/amegianeg/example/vendor/k8s.io/client-go/pkg/api vendor/k8s.io/client-go/pkg/api/types.generated.go:633: too many arguments in call to r.DecodeBytes have ([]byte, bool, bool) want ([]byte, bool) – Alberto Megía Sep 20 '17 at 17:47
What if there is multiple object types in the same file? I am having the exact scenario. – Sujai Sivasamy Dec 06 '17 at 05:41
1

@MichaelHausenblas Looks like the above example is deprecated, as that `import _ "k8s.io/client-go/pkg/api/install"` is not working. Also clicking on docs shows `Not Found`. Can you please update the same. – Abhishek Kashyap May 21 '18 at 15:54
9

Accepted answer is outdated - there is no more `k8s.io/client-go/api/install` package. Please see github discussion with corrected version: https://github.com/kubernetes/client-go/issues/193#issuecomment-377140518 – yname Jun 18 '18 at 23:52

score 1 · Answer 2 · edited Feb 12 '21 at 11:15

1

I've been using api machinery'sk8s.io/apimachinery/pkg/util/yaml to decode kubernete's deployment and service manifests.

import (
   "fmt"
   "bytes"
   appsv1 "k8s.io/api/apps/v1"
    k8Yaml "k8s.io/apimachinery/pkg/util/yaml"
)
...

d := &appsv1.Deployment{}
dec := k8Yaml.NewYAMLOrJSONDecoder(bytes.NewReader([]byte(deploymentManifest)), 1000)

if err := dec.Decode(&d); err != nil {
        return nil, err
}

fmt.Printf("%+v", d)

edited Feb 12 '21 at 11:15

Alex Punnen

5,287
3
59
71

answered Jan 31 '19 at 09:39

Ramiro Berrelleza

2,254
1
15
27

How do you fetch deployment.yaml/service.yaml of a particular pod – Harmeet Kaur Mar 24 '21 at 19:50
if you have the pod object, you can check the `ownerReferences` to get the replicaSet that owns the pod. In the replicaset object, check the value of `ownerReferences` to get the deployment. – Ramiro Berrelleza Apr 01 '21 at 22:37
The service is trickier, you can use the labels on the pod to find a match in the services, but it's not very efficient. – Ramiro Berrelleza Apr 01 '21 at 22:38

Alex Punnen · Answer 3 · 2021-02-12T12:01:11.397

import (
    "fmt"
    "gopkg.in/yaml.v2"
    "log"
    //corev1 "k8s.io/api/core/v1"
    "k8s.io/apimachinery/pkg/runtime/serializer/json"
    "k8s.io/client-go/kubernetes/scheme"
    v1alpha1 "github.com/argoproj/argo-cd/pkg/apis/application/v1alpha1"
)

....

func ParseYaml2(yaml []byte) (v1alpha1.Application, error) {
    // Create a YAML serializer.  JSON is a subset of YAML, so is supported too.
    s := json.NewYAMLSerializer(json.DefaultMetaFactory, scheme.Scheme,
        scheme.Scheme)

    // Decode the YAML to an object.
    var app v1alpha1.Application
    _, _, err := s.Decode(yaml, nil, &app)
    if err != nil {
        panic(err)
    }
    //fmt.Printf("%#v\n", app)
    return app, err
}
---
go.mod
// https://github.com/argoproj/argo-cd/issues/4055
replace github.com/argoproj/argo-cd => github.com/argoproj/argo-cd v1.5.5

var yaml2 = []byte(`
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
...

var app v1alpha1.Application
app,err := ParseYaml2(yaml2)
// Types from https://github.com/argoproj/argo-cd/blob/master/pkg/apis/application/v1alpha1/types.go
    fmt.Printf("--- t:\n%s\n\n", app.Spec.Source.Path)
    fmt.Printf("--- t:\n%s\n\n", app.Spec.Source.Helm.ValueFiles)
----

How to deserialize Kubernetes YAML file

3 Answers3

Linked