Why helm upgrade --install failed when previous install is failure?

Question

This is the helm and tiller version:

> helm version --tiller-namespace data-devops
Client: &version.Version{SemVer:"v2.9.1", GitCommit:"20adb27c7c5868466912eebdf6664e7390ebe710", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.9.1", GitCommit:"20adb27c7c5868466912eebdf6664e7390ebe710", GitTreeState:"clean"}

The previous helm installation failed:

helm ls --tiller-namespace data-devops
NAME            REVISION        UPDATED                         STATUS  CHART                   NAMESPACE
java-maven-app  1               Thu Aug  9 13:51:44 2018        FAILED  java-maven-app-1.0.0    data-devops

When I tried to install it again using this command, it failed:

helm --tiller-namespace data-devops upgrade java-maven-app helm-chart --install \
        --namespace data-devops \
        --values helm-chart/values/stg-stable.yaml
Error: UPGRADE FAILED: "java-maven-app" has no deployed releases

Is the helm upgrade --install command going to fail, if the previous installation failed? I am expecting it to force install. Any idea?

Are you ok to delete the release? Maybe you could do `helm delete java-maven-app --purge --no-hooks` and then do a helm install — Ryan Dawson, Aug 09 '18 at 08:40
Unfortunately, this is not okay on my case. I am using ci/cd to deploy our application to gke, so idempotent execution is what I want to achieve. — Agung Pratama, Aug 09 '18 at 09:36
If it helps at all I've been using jenkins-x and it runs that same command. I've occasionally hit that same error and had to intervene manually by deleting the release, correcting the issue and running the pipeline again. I think it only happens under particular conditions where the fist install of a chart goes wrong but I'm not sure exactly what conditions. — Ryan Dawson, Aug 09 '18 at 09:44
I see, so it is only happened on the first install of a chart? I think i can tolerate that by delete the release. I was afraid that the error also happened on existing running release, so delete the running release will affect my running application. — Agung Pratama, Aug 09 '18 at 23:24
[rollback](https://stackoverflow.com/a/65135726/1518100) is one option. — Lei Yang, Apr 08 '22 at 06:12

score 27 · Answer 1 · answered Aug 10 '18 at 06:47

27

This is or has been a helm issue for a while. It only affects the situation where the first install of a chart fails and has up to helm 2.7 required a manual delete of the failed release before correcting the issue and installing again. However there is now a --force flag available to address this case - https://github.com/helm/helm/issues/4004

answered Aug 10 '18 at 06:47

Ryan Dawson

11,832
5
38
61

1

For some reason `--force` do not help in this situation. And I'd prefer that failed release be owervritten by fixed one without forcing. So it seems more as a bug in Helm – esboych Oct 06 '20 at 14:24

score 20 · Answer 2 · answered Jan 25 '21 at 09:40

It happens when a deployment fails as unexpected.

First, check the status of the helm release deployment;

❯ helm ls  -n $namespace

NAME    NAMESPACE   REVISION    UPDATED STATUS  CHART   APP VERSION

Most probably you will see nothing about the problematic helm deployment. So, check the status of the deployment with the -a option;

❯ helm list -n $namespace -a
NAME            NAMESPACE       REVISION    UPDATED          STATUS             CHART         APP VERSION
$release_name   $namespace      7           $update_date     pending-upgrade    $chart_name   $app_version

As you can see the deployment stuck with the pending-upgrade status.

Check the helm deployment secrets;

❯ kubectl get secret -n $namespace                                                                                                                                             42s ⎈ eks_non-prod/monitoring
NAME                                 TYPE                  DATA   AGE
sh.helm.release.v1.$namespace.v1     helm.sh/release.v1    1      2d21h
sh.helm.release.v1.$namespace.v2     helm.sh/release.v1    1      21h
sh.helm.release.v1.$namespace.v3     helm.sh/release.v1    1      20h
sh.helm.release.v1.$namespace.v4     helm.sh/release.v1    1      19h
sh.helm.release.v1.$namespace.v5     helm.sh/release.v1    1      18h
sh.helm.release.v1.$namespace.v6     helm.sh/release.v1    1      17h
sh.helm.release.v1.$namespace.v7     helm.sh/release.v1    1      16h

and describe the last one;

❯ kubectl describe secret sh.helm.release.v1.$namespace.v7
Name:         sh.helm.release.v1.$namespace.v7
Namespace:    $namespace
Labels:       modifiedAt=1611503377
              name=$namespace
              owner=helm
              status=pending-upgrade
              version=7
Annotations:  <none>

Type:  helm.sh/release.v1

Data
====
release:  792744 bytes

You will see the secret has the same status with the failed deployment. So delete the secret;

❯ kubectl delete secret sh.helm.release.v1.$namespace.v7

Now, you should be able to upgrade the helm release. You can check the status of the helm release after the upgrade;

❯ helm list -n $namespace -a
NAME            NAMESPACE       REVISION    UPDATED          STATUS       CHART         APP VERSION
$release_name   $namespace      7           $update_date     deployed     $chart_name   $app_version

For me this is the better solution, cause it will not delete the deployment and keep your services up and running til the next deployment succeeds. — scasei, Sep 23 '21 at 14:50

score 14 · Accepted Answer · edited Sep 23 '20 at 00:21

14

Try:

helm delete --purge <deployment>

This will do the trick

and for helm3 onwards you need to uninstall eg.

helm uninstall <deployment>  -n <namespace>

edited Sep 23 '20 at 00:21

Don

3,876
10
47
76

answered Apr 02 '20 at 15:56

Dhiraj Surve

310
2
5

This only works for manual deployments. For CI/CD, hard to automate. – Oliver Feb 23 '22 at 19:58
works flawlessly! *note - it kills the pods and needs redeployment – Gaurav May 09 '22 at 12:38
1

I don't think it is a good solution, it will delete all resources. What if someone does not want to remove pods or ingresses? – swietyy Aug 19 '22 at 09:56

JDTLH9 · Answer 4 · 2020-03-11T16:40:15.097

Just to add...

I have often seen the Error: UPGRADE FAILED: "my-app" has no deployed releases error in Helm 3. Almost every time, the error was in either kubectl, aws-cli or aws-iam-authenticator not Helm. Seems that a lot of problems seem to bubble-up to this exception, which is not ideal.

To diagnose the true issue you can run simple commands in one or more of these tools if you are using them and you should be able to quickly diagnose your problem.

For example:

aws-cli - aws --version to ensure you have the cli installed.

aws-iam-authenticator - aws-iam-authenticator version to check that this is correctly installed.

kubectl - kubectl version will show if the tool is installed.

kubectl - kubectl config current-context will show if you have provided a valid config that can connect to Kubernetes.

@SomeGuyWhoCodes I normally use: `kubectl version` or `kubectl config current-context` — JDTLH9, Mar 05 '20 at 14:36
These will allow you to see if `kubectl` is available and that you are correctly configured to connect to Kubernetes. — JDTLH9, Mar 05 '20 at 14:40

Why helm upgrade --install failed when previous install is failure?

4 Answers4