2

I need some assistance to know what I'm missing here.. I'm trying to deploy streamsets application from customized values.yml file for my Lab (localhost-master).I'm trying to deploy the pod on "streamset-ns" namespace. I'm facing the below error. stderr: No resources found in streamset-ns namespace.

Main Script under Task

- name: Listing all Namespaces
  command: "kubectl get namespaces"
  register: namespace_list

- name: Checking if streamsets is installed in  "{{streamsets_namespace}}"
  command: "kubectl get pods -n {{streamsets_namespace}}"
  register: if_streamsets

- set_fact:
    message: "{{ ((task_type == 'install') and ('deployed' in if_streamsets.stdout)) or ((task_type == 'uninstall') and ('deployed' in if_streamsets.stdout)) | ternary('streamsets is installed', 'streamsets is not installed') }}"

- debug:
    msg: "{{message}}"

- name: Checking streamsets running status
  block:
    - debug:
        msg: "streamsets is already deployed in {{streamsets_namespace}}"

    - name: Getting deployed pod status
      command: "kubectl get pods -n {{streamsets_namespace}}"
      register: streamsets_pod_status

    - debug:
        var: streamsets_pod_status.stdout_lines
  when: "'streamsets' in if_streamsets.stdout and 'deployed' in if_streamsets.stdout"


- name: Installing streamsets
  block:
    - name: Create a Kubernetes namespace for streamsets
      k8s:
        name: "{{streamsets_namespace}}"
        api_version: v1
        kind: Namespace
        state: present
      when: "streamsets_namespace not in namespace_list.stdout_lines"

#    - name: Adding Helm Repository for Streamsets
#      command: "{{helm_location}}/helm repo add streamsets {{streamsets_helm_charts}}"
#      register: helm_repo_results
#      changed_when: False
#      failed_when: "'Error' in helm_repo_results.stderr"
#
    - name: Applying Template Module
      template:
        src: "roles/streamsets/templates/values.yml.j2"
        dest: "/home/{{ansible_user}}/values.yml"
        mode: '0644'

    - name: Install streamsets using Command Module
      command: "kubectl create -f /home/{{ansible_user}}/values.yml -n {{streamsets_namespace}}"
      register: streamsets_result
      failed_when: "'Error' in streamsets_result.stderr"

    - debug:
        var: streamsets_result.stdout_lines

#    - name: Checking Streamsets Deployment Status
#      action:
#        shell kubectl get pods -n "{{streamsets_namespace}}"| grep "{{streamsets_release_name}}" | grep '1/1' |wc -l
#      register: streamsets_deployment_status
#      until: streamsets_deployment_status.stdout|int == streamsets_replicas | int
#      retries: 5
#      delay: 60
#      
#    - debug:
#        var: streamsets_deployment_status.stdout_lines


    - name: Checking Streamsets Deployment Status
      command: kubectl -n "{{streamsets_namespace}}" wait --for=condition=Ready pods --all --timeout=180s
      register: Streamsets_pod_status
      failed_when: "'Error' in Streamsets_pod_status.stderr"


    - name: Removing deployed configuration files for Streamsets
      file:
       path: "/home/{{ansible_user}}/values.yml"
       state: absent
  when: "'install' == task_type and 'streamsets' not in if_streamsets.stdout and 'deployed' not in if_streamsets.stdout"


- name: Unistalling streamsets from K8s
  block:
    - name: Removing Statefulsets & Service from  "{{streamsets_namespace}}"
      action:
        shell kubectl -n "{{streamsets_namespace}}" delete statefulsets "{{streamsets_release_name}}" && kubectl -n "{{streamsets_namespace}}" delete service "{{streamsets_release_name}}"-service
      register: streamsets_removal_status
    - debug:
        var: streamsets_removal_status.stdout_lines

#    - name: Checking PVC status in "{{streamsets_namespace}}"
#      shell: kubectl get pvc -n "{{streamsets_namespace}}" | grep -v NAME | cut -d ' ' -f1
#      register: streamsets_pvc_status
#    - debug:
#        var: streamsets_pvc_status.stdout_lines

#    - name: Delete occupied pvc for streamsets
#      command: "kubectl delete pvc -n {{streamsets_namespace}} {{streamsets_pvc_status.stdout}}"
#      register: pvc_delete_status
#      when: streamsets_pvc_status.stdout_lines != ''
#    - debug:
#        var: pvc_delete_status.stdout_lines

  when: "'uninstall' == task_type and 'streamsets' in if_streamsets.stdout and 'deployed' in if_streamsets.stdout"


- name: Playbook Signature
  block:
    - debug:
        msg: "No 'task_type' supplied. Playbook signature: ansible-playbook -i <hosts file> <playbook> --extra-vars 'task_type=<install/uninstall>'"
  when: "task_type == '' or ('install' or 'uninstall') not in task_type"

yml created under templates

---
apiVersion: v1
kind: Service
metadata:
  name: streamsets-service
  labels:
    name: streamsets
spec:
  type: NodePort
  ports:
  - port: {{streamsets_port}}
    targetPort: 18630
    nodePort: {{streamsets_nodePort}}
  selector:
    role: streamsets
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: streamsets
spec:
  serviceName: streamsets-service
  replicas: {{streamsets_replicas}}
  selector: 
    matchLabels:
      name: streamsets
  template:
    metadata:
      labels:
        role: streamsets
        environment: test
        replicaset: streamsetsRepSet
    spec:
      terminationGracePeriodSeconds: 10
      containers:
        - name: {{streamsets_image_container}}
          image: {{streamsets_image_name}}:{{streamsets_image_version}}
          imagePullPolicy: Always
          ports:
            - containerPort: 18630
          volumeMounts:
          - name: data
            mountPath: /data
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: {{streamsets_storageClass}}
      resources:
        requests:
          storage: {{streamsets_storage_volume}}

Can you please help me?

  • Hi, You start to check if the namespace already has some pods: `...Checking if streamsets is installed in ..` But if there is no pod you will get the error like below: `kubectl -n foo get pods No resources found in foo namespace.` Also for your Ansible-Playbook improvement, you can use the template directly in the k8s Ansible-Module: `- name: Read definition file from the Ansible controller file system after Jinja templating k8s: state: present definition: "{{ lookup('template', '/testing/deployment.yml') }}"` Maybee you can also tell us, where your Playbook fails? – CLNRMN Apr 25 '20 at 08:34
  • @CLNRMN - The error I'm getting while Installing Streamset is - `The StatefulSet \"streamsets\" is invalid: spec.template.metadata.labels: Invalid value: map[string]string{\"environment\":\"test\", \"replicaset\":\"streamsetsRepSet\", \"role\":\"streamsets\"}: `selector` does not match template `labels`` – Shuvodeep Ghosh Apr 25 '20 at 10:29

1 Answers1

3

When i have a look at your error which you have posted as comment, it looks like you have a mismatch in the selector.

selector: 
  matchLabels:
    name: streamsets
template:
  metadata:
    labels:
      role: streamsets
      environment: test
      replicaset: streamsetsRepSet

"...You must set the .spec.selector field of a StatefulSet to match the labels of its .spec.template.metadata.labels..."[1]

Can you adjust your labels and that the selector can also match a template label and try it again?

Example:

selector: 
  matchLabels:
    name: streamsets
template:
  metadata:
    labels:
      name: streamsets

[1]https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#pod-selector

CLNRMN
  • 1,409
  • 9
  • 23
  • 2
    Yes, I found where I was doing mistake, Thanks a lot @CLNRMN for your help. The pod is deployed. `NAME READY STATUS RESTARTS AGE` `pod/streamsets-0 1/1 Running 0 5h8m` ` – Shuvodeep Ghosh Apr 25 '20 at 19:24
  • @ShuvodeepGhosh Could you hit the checkmark to show that this answer solved the problem? Thanks! – metadaddy Apr 27 '20 at 15:44
  • Another issue I'm facing now with this play. I've made some conditional check 'set_fact' to decide 'Installation' or 'Un-installation'. I've modified the code on the upper part. The conditional checks are not getting satisfied in any of the conditions. I'm not able to find out where I'm missing out. Can you please help me? Kindly check the modifed code on the above. – Shuvodeep Ghosh Apr 28 '20 at 10:05
  • I didn't get any error on the output, but when I check the pod and service status , I get this - `[root@master-node ~]# kc get all -n data-collector` `NAME READY STATUS RESTARTS AGE` `pod/streamsets-0 0/1 Pending 0 4m39s` `NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE` `service/streamsets-service NodePort 10.96.119.252 8630:30029/TCP 4m39s` `NAME READY AGE` `statefulset.apps/streamsets 0/1 4m39s` – Shuvodeep Ghosh Apr 28 '20 at 13:34
  • Can someone help me? – Shuvodeep Ghosh Apr 28 '20 at 13:36
  • `Warning FailedScheduling storageos-scheduler Failed filter with extender at URL http://storageos.storageos.svc.cluster.local:5705/v1/scheduler/filter, code 500 Warning FailedScheduling 4m40s (x2 over 14m) storageos-scheduler failed to filter nodes: id or name not specified Warning FailedScheduling (x7 over 9m1s) storageos-scheduler failed to filter nodes: id or name not specified` – Shuvodeep Ghosh Apr 28 '20 at 14:09
  • Can anyone help me on this ? – Shuvodeep Ghosh Apr 29 '20 at 06:44