-1

I am working with Kafka Streams and I have run the example stream found here, and this works fine. I want to know if there is a way to replace the following command, 

./bin/kafka-run-class org.apache.kafka.streams.examples.wordcount.WordCountDemo

with one that uses a local file instead of this source on Github. I tried copying the file from Github and putting it in a local file, streams like so:

./bin/kafka-run-class ./streams/WordCountDemo.java

but it gives me the following error message:

Error: Could not find or load main class ..streams.WordCountDemo.java
OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Ryan
  • 105
  • 4
  • 14
  • Copy the file to your local drive using the SAME package name. Then place the location in the classpath before the kafka classes. It has to have the same package name, but it will allow the other classes to load along side it. – Dakoda May 11 '17 at 14:56
  • @Dakoda I am not sure what you mean. What do I put as the command to access the file? – Ryan May 11 '17 at 15:37
  • You don't need a special command. You just use or declare it normally. The classloader will use the classpath to find the class, so if you put you local directory in front of the kafka location, it will load yours first. When it has another usage to an associated class, it will try to find it in your directory first (cannot find it), then search the kafka location. That's why you need to keep the package names the same. – Dakoda May 11 '17 at 16:12
  • Also, dont use "./bin/kafka-run-class ./streams/WordCountDemo.java". Use "./bin/kafka-run-class org.apache.kafka.streams.examples.wordcount.WordCountDemo" – Dakoda May 11 '17 at 16:17
  • I tried doing it that way, but it seems to always go to the one on Github. – Ryan May 11 '17 at 16:36

3 Answers3

0

org.apache.kafka.streams.examples.wordcount is the name of the package containing the class WordCountDemo. Putting it in another (local) path will not change the package.

If you want to use streams/WordcountDemo, you have to change the package name to streams.

Pastafari
  • 171
  • 9
  • Changing the package name didn't change anything. – Ryan May 11 '17 at 15:35
  • _./bin/kafka-run-class ./streams/WordCountDemo.java_ will not work. It has to look like the original _./bin/kafka-run-class streams/WordCountDemo_. Btw why do you want to change the package name? If you have downloaded the files you can use _org.apache.kafka.streams.examples.wordcount_ local. – Pastafari May 11 '17 at 15:45
  • _./bin/kafka-run-class streams/WordCountDemo_ did not work either. Also, I don't want to change the name of the package. I just want to use it locally, but how does it know where the file is if I don't link to it directly? – Ryan May 11 '17 at 15:58
0

OK. @Pastafari and I are saying the same thing.

  1. Copy github's word count and all associated jar/classes/etc to your local directory. Put them into a, say, source directory and compile/run them. This is the most important step - get them running using steps 1-1B.

A. Add the source's target directory to your classpath. This is where your .class files are located.

B. run "./bin/kafka-run-class org.apache.kafka.streams.examples.wordcount.WordCountDemo" which uses their wordCount class.

When everything is running ...

  1. Copy the WordCount to another directory (say, source2). Make your changes to it. Your WordCount will be in source2 (target directory)\org\apache\kafka\streams\examples\wordcount directory. Your WordCount will have the same package name. Now there are 2 copies of WordCount.class, but in different directories.
  2. Change the classpath to be "source2;source" This tells the classloader to look in the source2 directory first. Then search for the rest of the stuff in source.
  3. Run "./bin/kafka-run-class org.apache.kafka.streams.examples.wordcount.WordCountDemo" Your class will load first.
Dakoda
  • 175
  • 7
0

What you think is happening:

Kafka is using HTTP to fetch the file WordCountDemo from GitHub, and using that as a class.

That is not what is happening.

What is actually happening:

Kafka is asking Java for a class called org.apache.kafka.streams.examples.wordcount.WordCountDemo.

Exactly the same class you would get if you did:

import org.apache.kafka.streams.examples.wordcount.WordCountDemo;

...

WordCountDemo demo = new WordCountDemo(...);

On this command line, you have no import statement or similar, so you use the fully qualified class name, which includes the package.

Java searches its classloaders for a WordCountDemo.class file in the right part of a filesystem. It's very likely to be inside a .jar file somewhere on your classpath.

You can make your own class available by compiling it (and optionally putting it into a JAR).

This is no different from normal Java class loading -- read about it in any introductory Java book or tutorial.

slim
  • 40,215
  • 13
  • 94
  • 127
  • I think I understand a bit better now, but I'm not entirely sure where to put the class, and what to put in my command. If I were to create a class from scratch, where should I put it and what should I put in my command? – Ryan May 11 '17 at 20:05
  • Learn Java from one of the many books and guides around. You are not doing anything unusual here. – slim May 11 '17 at 20:21