0

So, for non-kafka deployment, we specify offset storage file with debezium.source.offset.storage.file.filename as per documentation, how does the storage is reclaimed or cleanup happens?

I am planning to have k8s deployment with mounted volume as,

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: debezium-gke
  labels:
    name: debezium
spec:
  serviceName: debezium
  selector:
    matchLabels:
      name: debezium
  template:
    spec:
      containers:
        - name: debezium
          image: debezium_server:1.10
          imagePullPolicy: Always
          volumeMounts:
            - name: debezium-config-volume
              mountPath: /debezium/conf
            - name: debezium-data-volume
              mountPath: /debezium/data
      volumes:
        - name: debezium-config-volume
          configMap:
            name: debezium
  volumeClaimTemplates:
    - metadata:
        name: debezium-data-volume
      spec:
        accessModes: [ "ReadWriteOnce" ]
        resources:
          requests:
            storage: 10G

But wondering if the disk spaces gets filled in if storage reclaim doesn't happen!

Thanks

Yogen Rai
  • 2,961
  • 3
  • 25
  • 37
  • What do you mean by "if storage get's filled"? Debezium will keep on that mounted volume a single file named `offsets.dat`. That's it. That file should not be removed if you want Debezium to not lose data. – Tudor Plugaru Dec 05 '22 at 08:09
  • @TudorPlugaru ya, i hear you that it writes into that file in volume mounted, but over the time, file gets bigger and bigger that is larger than size of volume! how does it handle that? – Yogen Rai Dec 13 '22 at 14:27
  • I assume you can just trim the file. From my knowledge, Debezium only needs the last commited offset when it's starting. Old offsets are not used. – Tudor Plugaru Dec 21 '22 at 10:56
  • @TudorPlugaru oh okay! have you done yourself or see someone doing? i can give a try, but need to figure out how to trim dat file, like cron or something else – Yogen Rai Dec 21 '22 at 19:36

0 Answers0