Iceberg on kubernetes, rest container problem

Question

I'm trying to run iceberg on kubernetes.

Here are the files that I'm using:

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    kompose.cmd: kompose convert
    kompose.version: 1.27.0 (b0ed6a2c9)
  creationTimestamp: null
  labels:
    io.kompose.service: mc
  name: mc
spec:
  replicas: 1
  selector:
    matchLabels:
      io.kompose.service: mc
  strategy: {}
  template:
    metadata:
      annotations:
        kompose.cmd: kompose convert
        kompose.version: 1.27.0 (b0ed6a2c9)
      creationTimestamp: null
      labels:
        io.kompose.service: mc
    spec:
      containers:
        - command:
            - /bin/sh
            - -c
            - ' until (/usr/bin/mc config host add minio http://minio:9000 admin password) do echo ''...waiting...'' && sleep 1; done; /usr/bin/mc rm -r --force minio/warehouse; /usr/bin/mc mb minio/warehouse; /usr/bin/mc policy set public minio/warehouse; exit 0; '
          env:
            - name: AWS_ACCESS_KEY_ID
              value: admin
            - name: AWS_REGION
              value: us-east-1
            - name: AWS_SECRET_ACCESS_KEY
              value: password
          image: minio/mc
          name: mc
          resources: {}
      restartPolicy: Always
status: {}
---
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    kompose.cmd: kompose convert
    kompose.version: 1.27.0 (b0ed6a2c9)
  creationTimestamp: null
  labels:
    io.kompose.service: minio
  name: minio
spec:
  replicas: 1
  selector:
    matchLabels:
      io.kompose.service: minio
  strategy: {}
  template:
    metadata:
      annotations:
        kompose.cmd: kompose convert
        kompose.version: 1.27.0 (b0ed6a2c9)
      creationTimestamp: null
      labels:
        io.kompose.service: minio
    spec:
      containers:
        - args:
            - server
            - /data
            - --console-address
            - :9001
          env:
            - name: MINIO_ROOT_PASSWORD
              value: password
            - name: MINIO_ROOT_USER
              value: admin
          image: minio/minio
          name: minio
          ports:
            - containerPort: 9001
            - containerPort: 9000
          resources: {}
      restartPolicy: Always
status: {}
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    kompose.cmd: kompose convert
    kompose.version: 1.27.0 (b0ed6a2c9)
  creationTimestamp: null
  labels:
    io.kompose.service: minio
  name: minio
spec:
  type: LoadBalancer
  ports:
    - name: "9001"
      port: 9001
      targetPort: 9001
    - name: "9000"
      port: 9000
      targetPort: 9000
  selector:
    io.kompose.service: minio
status:
  loadBalancer: {}
---
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    kompose.cmd: kompose convert
    kompose.version: 1.27.0 (b0ed6a2c9)
  creationTimestamp: null
  labels:
    io.kompose.service: rest
  name: rest
spec:
  replicas: 1
  selector:
    matchLabels:
      io.kompose.service: rest
  strategy: {}
  template:
    metadata:
      annotations:
        kompose.cmd: kompose convert
        kompose.version: 1.27.0 (b0ed6a2c9)
      creationTimestamp: null
      labels:
        io.kompose.service: rest
    spec:
      containers:
        - env:
            - name: AWS_ACCESS_KEY_ID
              value: admin
            - name: AWS_REGION
              value: us-east-1
            - name: AWS_SECRET_ACCESS_KEY
              value: password
            - name: CATALOG_IO__IMPL
              value: org.apache.iceberg.aws.s3.S3FileIO
            - name: CATALOG_S3_ENDPOINT
              value: http://minio:9000
            - name: CATALOG_WAREHOUSE
              value: s3a://warehouse/wh/
          image: tabulario/iceberg-rest:0.2.0
          name: rest
          ports:
            - containerPort: 8181
          resources: {}
      restartPolicy: Always
status: {}
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    kompose.cmd: kompose convert
    kompose.version: 1.27.0 (b0ed6a2c9)
  creationTimestamp: null
  labels:
    io.kompose.service: rest
  name: rest
spec:
  type: LoadBalancer
  ports:
    - name: "8181"
      port: 8181
      targetPort: 8181
  selector:
    io.kompose.service: rest
status:
  loadBalancer: {}
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  creationTimestamp: null
  labels:
    io.kompose.service: spark-iceberg-claim0
  name: spark-iceberg-claim0
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 100Mi
status: {}
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  creationTimestamp: null
  labels:
    io.kompose.service: spark-iceberg-claim1
  name: spark-iceberg-claim1
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 100Mi
status: {}
---
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    kompose.cmd: kompose convert
    kompose.version: 1.27.0 (b0ed6a2c9)
  creationTimestamp: null
  labels:
    io.kompose.service: spark-iceberg
  name: spark-iceberg
spec:
  replicas: 1
  selector:
    matchLabels:
      io.kompose.service: spark-iceberg
  strategy:
    type: Recreate
  template:
    metadata:
      annotations:
        kompose.cmd: kompose convert
        kompose.version: 1.27.0 (b0ed6a2c9)
      creationTimestamp: null
      labels:
        io.kompose.service: spark-iceberg
    spec:
      containers:
        - env:
            - name: AWS_ACCESS_KEY_ID
              value: admin
            - name: AWS_REGION
              value: us-east-1
            - name: AWS_SECRET_ACCESS_KEY
              value: password
          image: tabulario/spark-iceberg
          name: spark-iceberg
          ports:
            - containerPort: 8888
            - containerPort: 8080
          resources: {}
          volumeMounts:
            - mountPath: /home/iceberg/warehouse
              name: spark-iceberg-claim0
            - mountPath: /home/iceberg/notebooks/notebooks
              name: spark-iceberg-claim1
      restartPolicy: Always
      volumes:
        - name: spark-iceberg-claim0
          persistentVolumeClaim:
            claimName: spark-iceberg-claim0
        - name: spark-iceberg-claim1
          persistentVolumeClaim:
            claimName: spark-iceberg-claim1
status: {}
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    kompose.cmd: kompose convert
    kompose.version: 1.27.0 (b0ed6a2c9)
  creationTimestamp: null
  labels:
    io.kompose.service: spark-iceberg
  name: spark-iceberg
spec:
  type: LoadBalancer
  ports:
    - name: "8882"
      port: 8882
      targetPort: 8888
    - name: "8081"
      port: 8081
      targetPort: 8080
  selector:
    io.kompose.service: spark-iceberg
status:
  loadBalancer: {}

after deploy all pods are working fine except the rest pod

kubectl get pod

when I see the log I get this message

Exception in thread "main" java.lang.NumberFormatException: For input string: "tcp://10.101.184.144:8181"

kubectl logs res-pod

How can I get the rest pod work properly?

I try to change the image of the rest deployment and change some ports of the service

score 0 · Answer 1 · answered Apr 04 '23 at 20:00

Encountered the same issue. The reason is because the wrong value for the REST_PORT env variable is being set in the rest container.

If you add a printenv to the rest Deployment you'll see the following value for REST_PORT:

REST_PORT=tcp://10.101.184.144:8181

I haven't dug deeper to understand why this is happening, but a way to fix this is to explicitly set the value of that env variable.

TLDR

Add these lines to the rest Deployment env variables to fix it:

 spec:
      containers:
        - env:
            - name: AWS_ACCESS_KEY_ID
              value: admin
            - name: AWS_REGION
              value: us-east-1
            - name: AWS_SECRET_ACCESS_KEY
              value: password
            - name: CATALOG_IO__IMPL
              value: org.apache.iceberg.aws.s3.S3FileIO
            - name: CATALOG_S3_ENDPOINT
              value: http://minio:9000
            - name: CATALOG_WAREHOUSE
              value: s3://warehouse/
            - name: REST_PORT  # <<<<<<<<<< Add this line
              value: "8181"    # <<<<<<<<<< Add this line

Iceberg on kubernetes, rest container problem

1 Answers1