2

I am trying to implement a basic Tekton CI pipeline. All the pipeline does is 1) get the source code 2) build an image with a new version and push it to an image registry.

The image version is generated by a Tekton Step. Images are built by another Tekton step that uses Kaniko as described here.

I am aware of using workspaces to pass variables between Tekton steps. Meaning I can write the version to a file in the workspace. But cant figure out a way to read this version from the file in the Kaniko build step below:

  steps:
    - name: build-and-push
      image: gcr.io/kaniko-project/executor:latest
      # specifying DOCKER_CONFIG is required to allow kaniko to detect docker credential
      env:
        - name: "DOCKER_CONFIG"
          value: "/tekton/home/.docker/"
      command:
        - /kaniko/executor
      args:
        - --dockerfile=$(params.pathToDockerFile)
        - --destination=$(resources.outputs.builtImage.url):<IMAGE-VERSION-NEEDED-HERE>
        - --context=$(params.pathToContext)
        - --build-arg=BASE=alpine:3

There should be a common pattern to resolve this but I am not sure if I am looking at the right places in Tekton documentation for this.

Can anyone offer some pointers?

Cyac
  • 447
  • 3
  • 15
  • I don't think the Step can read it. You probably need to do it in another Task and pass the value as a parameter to the Task. – Jonas Oct 17 '21 at 14:52
  • Well the tasks are sequenced in a pipeline. So then the pipeline has to know the parameter. The problems is I want one task to generate the input parameter that needs to go to the second task. I have not seen this possible with pipeline. Hope you understand what I mean. – Cyac Oct 17 '21 at 15:02
  • See https://stackoverflow.com/questions/69533163/is-there-any-way-to-pass-params-between-task-in-tekton on how to do that. – Jonas Oct 17 '21 at 15:04
  • Thanks Jonas. I traced the documentation you shared there and found that this link for an end to end example is broken - https://github.com/tektoncd/pipeline/tree/origin/release-v0.27.x/examples/v1beta1/pipelineruns/task_results_example.yaml. Is there a working link? – Cyac Oct 17 '21 at 15:50
  • Many examples here https://github.com/tektoncd/pipeline/tree/main/examples/v1beta1/pipelineruns – Jonas Oct 17 '21 at 15:58
  • Thank you @Jonas. Looks like the link you shared has the examples that will help solve what I am trying to do. I will try this out and post the results here. On a different note if it is possible to emit results from a Task to another why is it not possible to do the same between steps. Is there a design reason behind this? – Cyac Oct 17 '21 at 17:07
  • You _can_ pass things to Steps - but Steps are typically _static_ commands_ and this command need to be pre-designed to read that value. – Jonas Oct 17 '21 at 17:46

3 Answers3

1

This is to confirm that I managed to resolve the issue by redesigning the steps to tasks as suggested by @Jonas.

Tekton Tasks can have outputs which can be referred in other tasks. At the time of writing this Tekton steps don't seem to have this feature.

For more details refer the links in @Jonas comments above.

Cyac
  • 447
  • 3
  • 15
  • tasks allow for this: as they are running in separate pods. As suggested in another answer below: if you need to exchange data in between steps of the same task .... your steps are containers within the same pod: do you need a tekton-specific feature here? Or can you figure out a way to share those values yourself, with kubernetes native configurations, maybe, ilke a volume? – SYN Oct 02 '22 at 05:51
0

All steps in a Task share the same Pod and thus as access to a shared workspace implemented as an emptyDir volume:

Volumes:
  tekton-internal-workspace:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>

A common way to share data between steps is to a file in the /workspace and read it in the next step.

Alternatively, as suggested by @Jonas, if you use different Tasks you can write a result in the first Task and feed it into a parameter of the second Task in the Pipeline definition.

Using results this way implicitly creates a dependency between the two Tasks, so the Tekton controller won't schedule the second Task until the first one has successfully completed and results are available.

andreaf
  • 51
  • 2
  • Thank you. The volume based approach works in some cases. The issue here is the Kaniko build step (at the time of writing this) does not provide a way to read a file from a volume and use it as an image tag. For this reason, I had to redesign steps as tasks as described in my answer. And the issue is now resolved. – Cyac Nov 08 '21 at 10:12
0

You can use the gcr.io/kaniko-project/executor:debug image that has shell at /busybox/sh.

And create something like this (pass kaniko commands via script):

  steps:
    - name: write-to-workspace
      image: ubuntu
      script: |
        #!/usr/bin/env bash
        echo "IMAGE_VERSION" > /workspace/FOO
    - name: read-from-workspace
      image: gcr.io/kaniko-project/executor:debug
      script: |
        #!/busybox/sh
        export IMAGE_VERSION=$(cat /workspace/FOO)
        echo "$IMAGE_VERSION"
        /kaniko/executor \
          --dockerfile=$(params.pathToDockerFile) \
          --destination=$(resources.outputs.builtImage.url):"${IMAGE_VERSION}" \
          --context=$(params.pathToContext) \
          --build-arg=BASE=alpine:3

You can refer to this discussion: https://github.com/tektoncd/pipeline/issues/1476

t7e
  • 322
  • 1
  • 3
  • 9