Unable to connect KEDA to AWS MSK with scaled object

Question

I have created an AWS MSK cluster with 3 brokers. The authentication to the cluster is with IAM role-based authentication and SASL/SCRAM authentication. I already use MSK within my microservices and I send and receive events in the brokers. Now I want to scale my Kubernetes pods based on the Kafka topics' event count.

I have an EKS cluster with multiple microservices, the namespace of those microservices is called default.

I have installed KEDA in a new namespace called keda. I also added a TriggerAuthentication to KEDA and installed it in the default namespace. I used the SASL/SCRAM authentication to authenticate KEDA with MSK:

apiVersion: v1
kind: Secret
metadata:
  name: keda-kafka-secrets
  namespace: {{ .Release.Namespace }}
data:
  username: "{{ $.Values.kafka.username | b64enc }}"
  password: "{{ $.Values.kafka.password | b64enc }}"
---
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: keda-trigger-auth-kafka-conn
  namespace: {{ .Release.Namespace }}
spec:
  secretTargetRef:
  - parameter: username
    name: keda-kafka-secrets
    key: username
  - parameter: password
    name: keda-kafka-secrets
    key: password

I added ScaledObject to the Helm Charts of my microservice:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: user-service-scaledobject
spec:
  scaleTargetRef:
    kind: Deployment
    name: {{ $.Chart.Name }}
  pollingInterval: 30 # Optional. Default: 30 seconds
  cooldownPeriod:  300 # Optional. Default: 300 seconds
  idleReplicaCount: 0 # Optional. Default: 0
  minReplicaCount: 0 # Optional. Default: 0
  maxReplicaCount: 100 # Optional. Default: 100
  triggers:
  - type: kafka
    authenticationRef:
        name: keda-trigger-auth-kafka-conn
    metadata:
        bootstrapServers: {{ .Values.kafka.bootstrapServers }}
        consumerGroup: {{ .Values.kafka.consumerGroup }}
        topic: {{ .Values.kafka.topic }}
        lagThreshold: "500"
        version: 2.8.1
        sasl: scram_sha512

This is the Deployment of the microservice:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ $.Chart.Name }}
  labels:
    app: {{ $.Chart.Name }}
    component: {{ $.Chart.Name }}
    release: {{ $.Release.Name }}
spec:
  replicas: {{ .Values.deployment.replicas }}
  strategy:
    type: RollingUpdate
    rollingUpdate:
      {{- if .Values.deployment.rollingUpdate.enabled }}
      maxSurge: {{ $.Values.deployment.rollingUpdate.maxSurge }}
      maxUnavailable: {{ $.Values.deployment.rollingUpdate.maxUnavailable }}
      {{- end }}
  selector:
    matchLabels:
      app: {{ $.Chart.Name }}
      release: {{ $.Release.Name }}
  template:
    metadata:
      labels:
        app: {{ $.Chart.Name }}
        version: {{ $.Chart.Version }}
        release: {{ $.Release.Name }}
    spec:
      imagePullSecrets:
      - name: {{ $.Values.deployment.image.imagepullsecret }}
      containers:
      - name: {{ $.Chart.Name }}
        image: "{{ .Values.deployment.image.repository }}:{{ .Values.deployment.image.tag }}"
        ports:
        - name: {{ $.Chart.Name }}
          containerPort: {{ $.Values.deployment.ports.http }}
          protocol: {{ $.Values.service.protocol }}
        imagePullPolicy: {{ $.Values.deployment.image.pullPolicy | quote }}
        resources:
          requests:
            memory: "{{ .Values.deployment.resources.requests.memory }}"
            cpu: "{{ .Values.deployment.resources.requests.cpu }}"
          limits:
            memory: "{{ .Values.deployment.resources.limits.memory }}"
            cpu: "{{ .Values.deployment.resources.limits.cpu }}"
        {{- if $.Values.deployment.volumes.enabled }}
        volumeMounts:
        - name: volume
          mountPath: /var/volume/myapp
        {{- end }}
        {{- if $.Values.deployment.env }}
        env:
          {{- range .Values.deployment.env }}
          - name: {{ .name }}
            value: "{{ .value }}"
          {{- end }}
        {{- end }}
      {{- if $.Values.deployment.volumes.enabled }}
      volumes:
      - name: volume
        emptyDir: {}
      {{- end }}
      {{- if $.Values.deployment.nodeSelector }}
      nodeSelector:
        {{- toYaml $.Values.deployment.nodeSelector | nindent 8 }}
      {{- end }}

When I try to add the ScaledObject to the microservice namespace (default), I get this errors in the KEDA operator pod:

ERROR   scale_handler   error resolving auth params
ERROR   Error getting scalers
ERROR   Failed to create new HPA resource
Warning  KEDAScalerFailed         13m (x82 over 18h)  keda-operator  error creating kafka client: kafka: client has run out of available brokers to talk to
Warning  ScaledObjectCheckFailed  13m (x82 over 18h)  keda-operator  failed to ensure HPA is correctly created for ScaledObject

What might be the problem?

Thanks!