2

I was using confluent kafka package for my data streaming server in golang using ubuntu 20.04 and now I changed my os to ubuntu 22.04.

Now im getting these errors:

- kafka.producer/producer.go:18:26: undefined: kafka.Producer
- kafka.producer/producer.go:35:17: undefined: kafka.ConfigMap
- kafka.producer/producer.go:40:30: undefined: kafka.NewProducer
- kafka.producer/producer.go:133:50: undefined: kafka.Message
- kafka.producer/producer.go:134:27: undefined: kafka.TopicPartition
- kafka.producer/producer.go:136:23: undefined: kafka.PartitionAny

But the segmentio package is working fine in the same code. The thing is I want to use confluent package because it has subscribe and unsubscribe features, which I felt advantage over the segemntio package.

Does anyone knows how to fix this error?

Zeke Lu
  • 6,349
  • 1
  • 17
  • 23
Suriya Varman N
  • 79
  • 5
  • Go packages are not tied to OS version. Something else has changed, maybe the version of the kafka package you're importing? Compare everything from the old to the new to see what's different, that's likely the source of the errors. – Adrian May 12 '23 at 19:28
  • @Adrian The `github.com/confluentinc/confluent-kafka-go/v2/kafka` package uses bindings on-top of the librdkafka C library, so it's kind of tied to OS version. – Zeke Lu May 12 '23 at 23:04
  • 1
    @Suriya Varman N, by default the bundled platform-specific static build of librdkafka will be used. So it should work out of the box. But if you build with `-tags dynamic`, a shared librdkafka library must be installed manually through other means. Please provide more details about how your app is built. Thank you! – Zeke Lu May 12 '23 at 23:10
  • @Zeke Lu im not using any tags to build jst using "go build -o kafka-server", while running this comment i got error also same error is showing in dvs code error lens also. – Suriya Varman N May 13 '23 at 04:28
  • Can you edit your question to include the output from `go env`? Make sure the C compiler specified by `go env CC` is available. – Zeke Lu May 13 '23 at 04:34
  • in "go env" CC is there it is mentioned as CC="gcc". Should i need to change anything there – Suriya Varman N May 13 '23 at 04:52
  • Is `gcc` available in the system? Is `CGO_ENABLED` set to `1`? – Zeke Lu May 13 '23 at 04:55
  • CGO_ENABLED set to 0 in my pc – Suriya Varman N May 13 '23 at 04:57
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/253638/discussion-between-zeke-lu-and-suriya-varman-n). – Zeke Lu May 13 '23 at 04:59

1 Answers1

2

Since the github.com/confluentinc/confluent-kafka-go/v2/kafka package uses bindings on-top of the librdkafka C library, you have to make sure cgo is configured correctly first.

The common things to check are:

  1. make sure cgo is enabled (CGO_ENABLED="1"):

    $ go env CGO_ENABLED
0
$ go env -w CGO_ENABLED="1"

  2. make sure the C compiler is available. run go env CC to see which C compiler is used.

Besides checking cgo, it's also necessary to make sure librdkafka is available. See build tags:

  • By default the bundled platform-specific static build of librdkafka will be used. This works out of the box on Mac OSX and glibc-based Linux distros, such as Ubuntu and CentOS.
  • -tags musl - must be specified when building on/for musl-based Linux distros, such as Alpine. Will use the bundled static musl build of librdkafka.
  • -tags dynamic - link librdkafka dynamically. A shared librdkafka library must be installed manually through other means (apt-get, yum, build from source, etc).
Zeke Lu
  • 6,349
  • 1
  • 17
  • 23