Can anyone explain in simple terms what is the difference between deploying Kafka through Kubernetes operator (for example Strimzi ) and Kafka helm chart or manifest file?
Previously in my project we used helm chart for kafka but for now we've received requirement to move to kafka strimzi-operator. I can't access people who invented it but my colleguess also don't know the reason.
So please explain why kafka strimzi operator better (or maybe worse) than kafka helm chart ?
Helm is like a package manager. It can install applications on your cluster, but it has only some basic logic for updates to its configuration or for version upgrades. You control it through the helm commands and call it when you need it. So it helps you with some tasks, but it is still up to you to run your Kafka cluster day-to-day.
Operators on the other hand are (usually) more sophisticated. They don't handle only the installation but also day-2 operations. They essentially try to encode the knowledge and the tasks a human operator running someting like a Kafka cluster would need and do into an application (= the operator). The operator runs all the time in your cluster, and constantly monitors the Kafka cluster to see what is happening in it, if some actions should be taken, and so on. For something like Kafka, the Strimzi operator for example incorporates the rolling update knowledge such as that the controller broker should be rolled last and partition replicas kept in-sync, it deals with upgrades which in Kafka usually consist of multiple rolling updates, handles certificate renewals, and much more.
So an operator will normally do a lot more things for you than a Helm Chart as it operates the Kafka cluster for you. For stateful applications such as Kafka or for example databases, this can often make a huge difference. But it is usually also more opinionated as it does things the way it was programmed to which might be different from what you were used to. Helm Charts normally give you a lot of freedom to do things any way you want.
Note: Different operators have different features and levels of maturity. So they might or might not support different tasks.
If you google for it, you will find many different articles, videos, or conference talks about the Kubernetes operator pattern and compare it with Helm Charts which will explain the differences.
(Disclaimer: I'm one of the Strimzi project maintatainers)
Some generic thoughts on Helm charts vs. operators:
A Helm chart directly includes Kubernetes YAML files using the Go text/template language. An operator has actual code, frequently written in Go using the Kubernetes SDK, that creates the same objects.
If you're going to install some application, using a Helm chart it's possible to inspect the chart or use a tool like helm template to see what it's going to do. You do not have that level of control over an operator: you can assign it some very broad permissions to create and edit StatesulSets and Secrets and it will do...something. A Helm chart will visibly fail quickly if some configuration is wrong, but an operator can only report its state via the status: in its custom resources, so you can have limited visibility into what's going wrong if an operator isn't working.
As an implementer, if you're familiar with the Kubernetes YAML syntax already, it's a straightforward transition to turn it into a Helm chart. The template language is Turing-complete, though, and it's possible to write arbitrarily complex logic. Testing the templated logic becomes tricky. You also need to carefully manage whitespace and YAML layout concerns in the output of your templates. Once you've gotten up to this level of complexity, the Go native testing package with the support tools in packages like Kubebuilder make testing an operator much easier.
Operators and controllers do have some additional capabilities. They run arbitrary code, can edit objects in the cluster (given the right RBAC permissions), can inspect external state, and keep running after the initial installation. It is straightforward to layer operators by having one operator create the resource that triggers another (as in standard Kubernetes where a Deployment creates ReplicaSets which create Pods). Helm's dependency system is a little more robustly defined, but runs into trouble when you do try to have nested dependencies.
If most of your environment is in Helm anyways, it might make sense to prefer Helm charts for everything. Tools like Helmfile can make installing multiple Helm charts more straightfoward. If you're not already invested in Helm and are using other tools, and you don't mind not being able to see what the operator is doing, then a controller will be likely be simpler to use.
(In my day job, I maintain both Helm charts and custom operators. My application uses Kafka, but I do not maintain the Kafka installation. Our Helmfile-oriented developer setup installs Kafka using a Helm chart.)