I'm using dynamically provisioned Podman agents as Kubernetes pods in Jenkins helm chart.
Podman agent Dockerfile is below:
FROM jenkins/jnlp-agent-alpine
RUN apk add podman
RUN apk add openrc
RUN apk --no-cache add curl
RUN sed -i 's/#mount_program/mount_program/' /etc/containers/storage.conf
RUN echo 'jenkins:100000:65536' > /etc/subuid
RUN echo 'jenkins:100000:65536' > /etc/subgid
USER jenkins
Sometimes, random below error happens when running podman build command inside the agent during builds using the agent.
19:35:44 + podman build -t [registry_url]/docker-test/jenkins-docker .
19:35:44 STEP 1: FROM docker.io/jenkins/jenkins:lts-jdk17
19:35:44 Trying to pull docker.io/jenkins/jenkins:lts-jdk17...
19:35:52 Getting image source signatures
19:35:52 Copying blob sha256:0de24a2d2be853abdcd06832a09e57d7678175268d93f94feee70a7d0b3977a0
19:35:52 Copying blob sha256:ee7fe1de523477eca2eb90fe79bed0deebd6d31c705127e1d9cdd4cc4c1397b4
19:35:52 Copying blob sha256:635f2040c45de6fbf42620356288b5d627a377fa8bd6ff9fe35187657ec6051d
19:35:53 Copying blob sha256:a167e5ee9d3d3ffd6cde558495dba214573df628c6ca12eec6a60f3b6ca5c527
19:35:53 Copying blob sha256:1339eaac5b67d16d6d9f41fb7a7b96f7cebf3ba4beab36cbb60935aa772af583
19:35:53 Copying blob sha256:713854f8e14e0ccbb9e8f06502dc918c1fbef28adfc90203e5d3a6e759f84966
19:35:57 Copying blob sha256:e7c234de7c34b27bf9a11bf00d1aa790d67069f5e92b18c8301cdca1f1deaf61
19:35:57 Copying blob sha256:8e275b05ce882a8d2497a36a7f967c7ae92c2eb128aeb352b4fde50b28316fd9
19:35:57 Copying blob sha256:e7c993dd256dd0fa1e61f8179729e91fca4eb7e0bce77100a6849d16f1ecebd8
19:35:57 Copying blob sha256:47adde5316a6cb00d1bcc0a0bbbf5eb88fcb8f6101e62e60c64d12031f0ac1df
19:35:57 Copying blob sha256:d2bb529ca8e2e57815ace9733921afd453815c67255f2c7a98c758ca344eaa3a
19:35:57 Copying blob sha256:8821f791e8ec2560d716446b1242d816818b0e8934676a1e614a3316f49dc581
19:35:58 Copying blob sha256:3e37a8ce17a8091dd0c40ec08039576d294648cc8151bae851bb53daaa84b420
19:35:58 Copying blob sha256:0df22705f7759ee6d3011782bb226ba34bb6c768722272ac43aa8a2da22bf014
19:35:58 Error: error creating build container: Error writing blob: error storing blob to file "/var/tmp/storage947537847/5": error happened during read: local error: tls: bad record MAC
Jenkins pipeline runs podman build in sh step:
rc = sh(returnStatus: true, script: "podman build -t ${REGISTRY}/${REPO_NAME}/${NAME} .")
Sometimes, the same image build succeeds. Some images builds almost always succeed. Below image build fails with above error frequently:
FROM docker.io/jenkins/jenkins:lts-jdk17
ENV JAVA_OPTS="-Djenkins.install.runSetupWizard=false"
# Add Jenkins init files
COPY src/ /usr/share/jenkins/ref/
COPY --chown=jenkins:jenkins plugins.txt /usr/share/jenkins/ref/plugins.txt
RUN jenkins-plugin-cli --verbose --plugin-file /usr/share/jenkins/ref/plugins.txt
USER jenkins
Sometimes, the error looks differently Error writing blob: error storing blob to file \"/var/tmp/storage959144339/2\": error happened during read: read tcp ... read: connection timed out
Sometimes, the build succeeds, but takes a lot of time comparing to successful run.
I'm not sure if the issue with podman agent, Kubernetes cluster, agent's images or somewhere else.
I tried to build the same image using docker and it always succeeds.
Also, tried to restart jenkins and podman build sometimes succeeds afterwards.
