Confluent Maven repository not working?

Question

I need to use the Confluent kafka-avro-serializer Maven artifact. From the official guide I should add this repository to my Maven pom

<repository>
  <id>confluent</id>
  <url>http://packages.confluent.io/maven/</url>
</repository>

The problem is that the URL http://packages.confluent.io/maven/ seems to not work at the moment as I get the response below

<Error>
  <Code>NoSuchKey</Code>
  <Message>The specified key does not exist.</Message>
  <Key>maven/</Key>
  <RequestId>15E287D11E5D4DFA</RequestId>
  <HostId>
    QVr9lCF0y3SrQoa1Z0jDWtmxD3eJz1gAEdivauojVJ+Bexb2gB6JsMpnXc+JjF95i082hgSLJSM=
  </HostId>
</Error>

In fact Maven does not find the artifact

<dependency>
  <groupId>io.confluent</groupId>
  <artifactId>kafka-avro-serializer</artifactId>
  <version>3.1.1</version>
</dependency>

Do you know what the problem could be? Thank you

The URL needs to be in https instead of the http – Arefe Sep 15 '22 at 08:30 — Arefe, Sep 15 '22 at 08:30

sujithramanathan · Answer 1 · 2022-06-03T11:33:40.767

40

Needs to add confluent repositories in pom.xml
Please add below lines in the pom.xml

<repositories>
    <repository>
        <id>confluent</id>
        <url>https://packages.confluent.io/maven/</url>
    </repository>
</repositories>

edited Jun 03 '22 at 11:33

answered Mar 28 '20 at 21:51

sujithramanathan

915
9
8

1

this helped my url was http://packages.confluent.io/ – ronit May 26 '21 at 06:21
It worked for me! Thanks! – Onur Tokat Jan 04 '22 at 22:01
1

Worked for me with a slight change of `http` to `https`. – Lily Apr 27 '22 at 20:16

score 25 · Accepted Answer · answered Apr 19 '17 at 07:50

25

The file is available, since you can download it if you go to it directly: http://packages.confluent.io/maven/io/confluent/kafka-avro-serializer/3.1.1/kafka-avro-serializer-3.1.1.jar

You could try adding the -U flag to your maven command to force download of cached files.

The root of the repo isn't browsable which is why you are getting the message when browsing to http://packages.confluent.io/maven/

answered Apr 19 '17 at 07:50

Kevin

1,516
1
16
31

Thanks @Kevin. I'm using a central Artifactory mirror so I should add the Confluent repository to the mirrored ones in Artifactory. But when I try to add it, I get an error `Input may not be null`. How can I let Artifactory resolve the Confluent artifacts if the repository is not browsable? – gvdm Apr 19 '17 at 07:58
With not browsable i mean through a browser by us, which makes us guess about the contents instead of being able to verify it visually. When adding to Artifactory: You should enter "maven" as repository key and "http:// packages.confluent.io/" as url (ignore the space, otherwise it didnt show the http) – Kevin Apr 19 '17 at 08:17
I tried adding only `http://packages.confluent.io/` and the error does not appear, but no artifact is fetched from it. – gvdm Apr 19 '17 at 08:34
1

I think the artifacts are only fetched when you call for them, so you should now be able to add your artifactory as a repository (instead of the coonfluent one) and your dependency should be loaded through your artifactory (and will then be visible there) – Kevin Apr 19 '17 at 08:46
2

Note that this repo is availaible in https as well just as `https://repo.maven.apache.org/maven2/` – bric3 Jan 30 '18 at 10:45
This command works fine: `spark-shell --packages org.apache.spark:spark-avro_2.11:2.4.4 --repositories http://packages.confluent.io/maven/ --packages io.confluent:kafka-avro-serializer:5.2.2` – Amir Dadkhah Feb 02 '20 at 07:22

score 17 · Answer 3 · answered Apr 24 '18 at 15:28

Just like you I use a company repository (Sonatype Nexus) and was not able to proxy the confluent's repository.

Then I changed my maven settings.xml to exclude confluent form the mirrored repository:

    <mirrors>
        <mirror>
            <id>nexus</id>
            <mirrorOf>*,!confluent</mirrorOf> <!-- mirror anything but confluent as Nexus cannot proxy it -->
            <url>repository.company.local/nexus/content/groups/public</url>
        </mirror>
    </mirrors>
    ...
        <repositories>
            ...
            <repository>
                <id>confluent</id>
                <url>http://packages.confluent.io/maven/</url>
            </repository>
        </repositories>

This way, artifacts resolution works for confluents' artifacts as well.

Not as neat as proxying the repo but at least less cumbersome than downloading and registering each dependency manually.

Just a additional help that the mirror should be adjusted in conf/settings.xml of m2. And the confluent dependency should be adjusted in the module where it is listed. My additional requirement was to point it to local .m2 so confluent url I put as: file:///c/Users//.m2/repository/. Hope it will help someone searching to point to a local folder. — Kabeer Ahmed, Dec 13 '18 at 17:45

score 13 · Answer 4 · answered Aug 30 '21 at 05:25

13

seams jar file removed from http url or http url not working. https url worked for me.

<repositories>
<repository>
    <id>confluent</id>
    <url>https://packages.confluent.io/maven/</url>
</repository>

answered Aug 30 '21 at 05:25

Nalin Kularathna

306
3
8

score 7 · Answer 5 · answered Nov 14 '17 at 19:44

You can add a mirror in you maven settings file to fetch the jars from confluent repo along with repository config . Changes needed are Add a mirror in settings.xml

   <mirror>
      <id>confluent</id>
      <mirrorOf>confluent</mirrorOf>
      <name>Nexus public mirror</name>
      <url>http://packages.confluent.io/maven/</url>
</mirror>

In repository section of maven settings add this

<repository>
          <id>confluent</id>
          <url>http://packages.confluent.io/maven/</url>
          <releases>
            <enabled>true</enabled>
          </releases>
          <snapshots>
            <enabled>true</enabled>
          </snapshots>
</repository>

just the mirror worked fine for me, so have an up-vote. Thank you — bigbadmouse, Apr 02 '19 at 10:58

score 4 · Answer 6 · answered Sep 21 '17 at 20:27

4

When trying to connect Artifactory to the Confluent Maven repository you have to set the repo URL in Artifactory to be either http://packages.confluent.io/maven or https://packages.confluent.io/maven (both schemes seem to work fine). The confusing part is that when you ask Artifactory to test that URL it will fail with the message "Input may not be null". You are also unable to browse the repository in Artifactory. However, regardless of these problems, artifacts will be downloaded and cached when clients request them.

answered Sep 21 '17 at 20:27

rndgstn

321
2
7

Hi @rndgstn, my problem is that the Kafka repository is not "proxable" by the company's Artifactory repository, so in our company we cannot use the Artifacotry to access also the Kafka repo. At the moment we manually download the artifacts and upload them to the company repo. This is not the ideal solution, so I hope that in the future it will be proxable – gvdm Sep 25 '17 at 08:28
It is working fine for us. What error are you seeing when a client makes a request? – rndgstn Sep 25 '17 at 11:32
FWIW, the http scheme worked for me as a proxied repo with Sonatype Nexus. https scheme did not work. – user944849 Jun 11 '18 at 21:28

score 2 · Answer 7 · edited Apr 02 '23 at 00:52

Had issue with:

<repositories>
        <repository>
            <id>confluent</id>
            <url>http://packages.confluent.io/maven/</url>
        </repository>
</repositories>

working:

<repositories>
        <repository>
            <id>confluent</id>
            <url>https://packages.confluent.io/maven/</url>
        </repository>
</repositories>

Changed "http" with "https" worked for me as maven has blocked the HTTP due to security reasons.

score 1 · Answer 8 · answered Mar 11 '21 at 21:07

1

Adding confluent repository worked in POM file for Maven project.

Use below in build.sbt for SBT projects

resolvers += "confluent" at "https://packages.confluent.io/maven/"

answered Mar 11 '21 at 21:07

Venkat

357
4
9

Confluent Maven repository not working?

8 Answers8

Linked