kubernetes: Failed to update endpoints warning

Question

Here is something I noticed in my kubectl get events output

Warning   FailedToUpdateEndpoint   Endpoints                 Failed to update endpoint mynamespace/myservice: Operation cannot be fulfilled on endpoints "myservice": the object has been modified; please apply your changes to the latest version and try again

I am aware of this discussion, but I do not think is applicable, given I am not explicitly creating an Endpoint resource via yaml.

I am noticing some minor service unavailability during image updates so I am trying to check if this is related.

Using GKE with version v1.12.7-gke.25 on both masters and nodes, on top of istio.

So K8s api server works on eventual consistency , if more than 1 controllers are trying to modify the same resources. one of them will get this error, as endpoint is managed by few controllers such as endpoint controllers, kube-proxy, service controllers — Suresh Vishnoi, Sep 16 '19 at 14:51
yea, it's perfectly fine. I have provided bunch of statement with links. — Suresh Vishnoi, Sep 16 '19 at 18:51

Suresh Vishnoi · Accepted Answer · 2019-09-17T07:38:45.830

It's common behaviour of k8s to let the k8s clients (Controllers) know to try again.

Kubernetes leverages the concept of resource versions to achieve optimistic concurrency. concurrency-control-and-consistency

It's populated by the system.

To enable clients to build a model of the current state of a cluster, all Kubernetes object resource types are required to support consistent lists and an incremental change notification feed called a watch. Every Kubernetes object has a resourceVersion field representing the version of that resource as stored in the underlying database. When retrieving a collection of resources (either namespace or cluster scoped), the response from the server will contain a resourceVersion value that can be used to initiate a watch against the server. The server will return all changes (creates, deletes, and updates) that occur after the supplied resourceVersion. This allows a client to fetch the current state and then watch for changes without missing any updates. If the client watch is disconnected they can restart a new watch from the last returned resourceVersion, or perform a new collection request and begin again efficient-detection-of-changes

Best answer I have seen yet to this perplexing problem. Thanks! — Randy L, Oct 11 '21 at 20:02

kubernetes: Failed to update endpoints warning

1 Answers1

Linked