run Scala code in package (LinkedIn Norbert)

Question

I am trying to run a Scala object called NorbertClusterClientMain which is in package com.linkedin.norbert.cluster. The source code for it is in folder examples/src/main/scala of rhavyn's open-source branch of LinkedIn Norbert, and I am working on a Linux command line.

Although I've been told that running Scala code in a package is like running Java in a package, I am in examples/src/main/scala but cannot use this command:

 $ scala com.linkedin.norbert.cluster.NorbertClusterClientMain

I am getting "No such file or class on classpath", even though the file exists.

I was successfully able to compile Norbert with

$ mvn clean -DskipTests install

How can I run the NorbertClusterClientMain? Please let me know. I appreciate your help.

I have explained below how I managed to execute the class NorbertClusterClientMain. Does it work for you? — tuxdna, Dec 23 '13 at 09:39
Thanks. I ended up building using sbt and creating a long and complex classpath, which I ran in Java. None of the answers quite nailed what I ended up doing. — La-comadreja, Dec 23 '13 at 21:31
I actually wanted a simple way to get classpath via sbt, but its not yet easy to do so. I had filed a ticket here - https://github.com/sbt/sbt/issues/1044 . So I also used Maven. — tuxdna, Dec 23 '13 at 23:23

Daniel C. Sobral · Answer 1 · 2013-12-05T22:23:33.040

1

It is the same. So, in this case, it is looking for this file:

./com/linkedin/norbert/cluster/NorbertClusterClientMain.class

This is how Java works, and since "running" a Scala program is just running java passing the Scala library in the classpath, it has to be the same.

How did you compile it, by the way? Nevermind, saw your comment. At the directory you ran mvn, you should probably be able to run it like this:

scala -cp target com.linkedin.norbert.cluster.NorbertClusterClientMain

Failing that, find the class file, and pass the directory where com/ is to the classpath.

edited Dec 05 '13 at 22:23

answered Dec 05 '13 at 22:18

Daniel C. Sobral

295,120
86
501
681

I compiled it using Maven, so there are JAR and .class files in the target/ directories. The .class files don't seem to be on that path. – La-comadreja Dec 05 '13 at 22:23
tried to run it with -cp target in several directories and it still didn't work. – La-comadreja Dec 05 '13 at 22:33

Glen Best · Answer 2 · 2013-12-17T00:11:54.703

Your mvn script produced JAR and class files within target directory:

./target/com/.../<someClassName1>.class
./target/com/.../<someClassName2>.class
... etc
./target/<someJarName1>.jar
./target/<someJarName2>.jar
... etc

Great! Now do the same thing that you must do for java; include in your classpath:

the target base directory (this "picks up" all class files in directory hierarchy beneath target)

each jar file (this "picks up" all class files in directory hierarchy within each JAR)

scala -cp target:target/<someJarName1>.jar:target/<someJarName2>.jar:... etc ./com.linkedin.norbert.cluster.NorbertClusterClientMain

Here -cp (or equivalently, CLASSPATH environment variable) is the java classpath and so has the same syntax and rules as java.

BTW: "sbt" is a standard, powerful, usable way to build scala projects. It uses Ivy to "pull" dependencies from code repositories (i.e. mvn++). The best way to get started with it is to download sbt example projects, search for "sbt tutorial" blogs and read the sbt docs. :)

The OP is on Linux, re: semis in class paths. – som-snytt Dec 16 '13 at 23:58 — som-snytt, Dec 16 '13 at 23:58

score 1 · Answer 3 · edited May 23 '17 at 11:50

I don't anything about the code base, but that class is in the examples subproject.

This shows that it loads normally. (I haven't configured anything, because I don't know anything about the code base.)

apm@mara:~/clones/norbert$ cd examples
/home/apm/clones/norbert/examples
apm@mara:~/clones/norbert/examples$ ls
pom.xml  src  target
apm@mara:~/clones/norbert/examples$ mvn exec:java -Dexec.mainClass=com.linkedin.norbert.cluster.NorbertClusterClientMain
[INFO] Scanning for projects...
<snip...>
[INFO] --- exec-maven-plugin:1.2.1:java (default-cli) @ norbert-examples ---
[WARNING] 
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:297)
    at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 0
    at com.linkedin.norbert.cluster.NorbertClusterClientMain$.main(NorbertClusterClientMain.scala:22)
    at com.linkedin.norbert.cluster.NorbertClusterClientMain.main(NorbertClusterClientMain.scala)
    ... 6 more

Here's how to add args:

https://stackoverflow.com/a/9846103/1296806

I don't use maven much anymore.

Edit: I don't use Scala 2.7.7 much anymore either.

score 0 · Answer 4 · answered Dec 05 '13 at 20:46

0

try $ scala ./com.linkedin.norbert.cluster.NorbertClusterClientMain or -cp .

you "current directory" might not be in classpath

answered Dec 05 '13 at 20:46

Konstantin

3,254
15
20

oh, i think you're in the wrong folder altogether, your compiled files will be in `bin`, you mention you're in `src` – Konstantin Dec 05 '13 at 20:55
There is no bin/ folder in the source code. There are several target/ folders though... – La-comadreja Dec 05 '13 at 20:57
umm, silly question, but are you sure you've compiled it? similar to javac/java you need to do scalac/scala – Konstantin Dec 05 '13 at 21:02
I compiled it using Maven $ mvn -DskipTests install according to the online instructions – La-comadreja Dec 05 '13 at 21:44

score 0 · Answer 5 · answered Dec 20 '13 at 08:41

Norbert uses a Scala 2.7 so directly using Scala from CLI may not work for you. Therefore find all the dependency jar using Maven and use it.

This is how I did it.

First, check out the code:

$ git clone https://github.com/rhavyn/norbert
$ cd norbert/

Build and install the dependencies in local repository first:

$ mvn clean install

Setup a classpath variable which we will use later for examples/ folder:

$ cd examples/
$ export CP=$(mvn dependency:build-classpath | grep -A1 'Dependencies classpath:' | tail -1)

Run server from examples/ folder:

$ java -cp $CP:target/classes com.linkedin.norbert.network.javaapi.NorbertJavaNetworkServerMain arg0 arg1
log4j:ERROR Could not find value for key log4j.appender.R
log4j:ERROR Could not instantiate appender named "R".
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 2
    at com.linkedin.norbert.network.javaapi.NorbertJavaNetworkServerMain.main(NorbertJavaNetworkServerMain.java:33)

Run client from examples/ folder:

$ java -cp $CP:target/classes com.linkedin.norbert.cluster.NorbertClusterClientMain localhost 1011
log4j:ERROR Could not find value for key log4j.appender.R
log4j:ERROR Could not instantiate appender named "R".
> h2013-12-20 13:59:44,323 - WARN  [pool-1-thread-2-SendThread(0.0.3.243:2181):ClientCnxn$SendThread@1120] - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.SocketException: Invalid argument
    at sun.nio.ch.Net.connect0(Native Method)
    at sun.nio.ch.Net.connect(Net.java:364)
    at sun.nio.ch.Net.connect(Net.java:356)
    at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:623)
    at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1009)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1036)
2013-12-20 13:59:46,192 - WARN  [pool-1-thread-2-SendThread(0.0.3.243:2181):ClientCnxn$SendThread@1120] - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.SocketException: Invalid argument
    at sun.nio.ch.Net.connect0(Native Method)
    at sun.nio.ch.Net.connect(Net.java:364)
    at sun.nio.ch.Net.connect(Net.java:356)
    at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:623)
    at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1009)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1036)

Thats it. Now all you have to do is setup the required services like Zookeper etc.

run Scala code in package (LinkedIn Norbert)

5 Answers5