Stack Overflow
Stack Overflow

Questions tagged [aiven]

Aiven is a cloud data hub, offering managed databases and other data solutions in the cloud. Use this tag for questions relating to the Aiven platform or any of the Aiven-maintained open source tools.

Aiven is a platform offering hosted and managed open source database solutions. It offers Kafka, PostgreSQL (and extensions), time series databases including M3, and Grafana - deployed to a choice of cloud platforms.

Links

14 questions
3
votes
2 answers

Moving from ElasticSearch to OpenSearch in PHP Symfony

We are currently using ElasticSearch with PHP Symfony which is hosted by Aiven but Aiven currently no longer offers Elasticsearch. This offer is replaced by OpenSearch 1.0. So, Aiven plans to migrate all existing ElasticSearch projects to OpenSearch…
Techs
  • 169
  • 1
  • 1
  • 10
3
votes
0 answers

Sometimes node-rdkafka consumer not reading any messages from topic

The below code snippet is working fine. But sometimes it's not reading messages from Kafka's topic. I am not getting any errors. At Kafka's side(we are using Aiven Managed Kafka), the consumer group has associated with the topic and consumer script…
Priyabrata
  • 629
  • 1
  • 8
  • 23
2
votes
3 answers

How to access a confluent schema registry server secured with a password using Spring cloud stream?

I'm using spring cloud stream alongside Aiven's schema registry which uses confluent's schema registry. Aiven's schema registry is secured with a password. Based on these instructions, these two config parameters need to be set to successfully…
Milad
  • 608
  • 1
  • 6
  • 14
1
vote
0 answers

Failed to compile react webapp after using javascript prisma for mysql

Hello beautiful humans! I'm working on building a project that uses React Webapp for the frontend, using Prisma to go from javascript to MySql with Aiven to then Twitter API to create auto posts. GitHub:…
Jenn Junod
  • 11
  • 1
1
vote
1 answer

How to enable RedisJson module on Aiven Redis instances

I do not see any option to enable modules in aiven Redis. I tried MODULE LOAD /usr/lib/redis/modules/rejson.so MODULE LIST gives (empty array)
Shashikumar KL
  • 1,007
  • 1
  • 10
  • 25
0
votes
3 answers

How to use Spark -> Kafka -> JDBC sink connector with Avro correctly?

I have a simple Spark app generating Kafka messages by import org.apache.spark.sql.SparkSession import org.apache.spark.sql.functions.{col, struct} import org.apache.spark.sql.avro.functions.to_avro import org.apache.spark.sql.types.{DoubleType,…
Hongbo Miao
  • 45,290
  • 60
  • 174
  • 267
0
votes
1 answer

How to prevent producing to kafka topic?

We are debugging a bug and we need to prevent all producers to produce messages to a specific kafka-topic. There are alot of producers and I can't control them. How can I do that on the kafka-side?
Stav Alfi
  • 13,139
  • 23
  • 99
  • 171
0
votes
1 answer

Are 3k kafka topics decrease performance?

I have a Kafka Cluster (Using Aivan on AWS): Kafka Hardware Startup-2 (2 CPU, 2 GB RAM, 90 GB storage, no backups) 3-node high availability set Ping between my consumers and the Kafka Broker is 0.7ms. Backgroup I have a topic such that: It…
Stav Alfi
  • 13,139
  • 23
  • 99
  • 171
0
votes
2 answers

kafka connect not loading manually built connectors

I downloaded aiven-kafka-connect-jdbc and built the jar manually.Placed the same jar in kafka-connect plugin directory and mentioned the plugin path in connect-standalone.properties. But when i run the connect it throws error saying "Failed to find…
vishnupriya
  • 55
  • 9
0
votes
2 answers

Get kafka message processing time

I have 2 services: producer and consumer. As far as I understand, message.ts is the time the producer produced the message (not the time the kafka-broker received the message). Questions When the consumer consume the message, how can I know how…
Stav Alfi
  • 13,139
  • 23
  • 99
  • 171
0
votes
1 answer

cloud not resolve CNAME in GCP Cloud DNS

We have peered two VPCs. GCP & third party VPC (Aiven). We have few services in aiven which require to talk with resources in gcp vpc such as elasticsearch, redis etc. Let's say, DNS-CNAME: test-elasticsearch.example.io ES-URL:…
Arvin
  • 315
  • 1
  • 3
  • 15
0
votes
1 answer

How to use wildcard field in Aiven's ElasticSearch?

I wanted to test the new wildcard field type in my ElasticSearch instance (Aiven). I've tried this: PUT /wildcard_test { "mappings" : { "properties" : { "wildcard_field" : { "type" : "wildcard" } } …
mathiasfk
  • 1,278
  • 1
  • 19
  • 38
0
votes
1 answer

Spring Kafka: Credential Rotation

We are using Aiven as a Kafka as a Service provider, and they support credential rotation. E.g. every 24 hours. So far, we have yet to find a way to create new consumers / producers in Spring Kafka when we need to rotate. Basically, a…
Kenneth Gunnerud
  • 208
  • 4
  • 13
0
votes
1 answer

How to connect Kafka (Aiven) to on-premise SQL Server using the Debezium connector

I have a managed kafka cluster running in the cloud on Aiven (GCP). I'm trying to use the Debezium SQL Server connector to stream CDC records from our on-premise database. Does anybody know how to connect from kafka in the cloud to an on-premise…
ThinkBig
  • 11