How do i get Gremlin-Python with gremlin-server 3.4.3 to work?

Question

My environment is:

java -version
java version "1.8.0_172"
Java(TM) SE Runtime Environment (build 1.8.0_172-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.172-b11, mixed mode)

python3 --version
Python 3.7.4

Which should be the only relevant environment parts. Unfortunately it might make a difference that I am on MacOS and using macports which should not be an issue.

I just tested things on Ubuntu 18.04 LTS see http://wiki.bitplan.com/index.php/Gremlin_python#Installlation_automation and the behavior is the same.

So i wanted to tryout gremlin-python. I found https://pypi.org/project/gremlinpython/ and happily added

gremlinpython

to my requirements.txt

sudo -H pip install -r requirements.txt

ran successfully- great. Let's find some python code to play with .

https://gist.githubusercontent.com/okram/f193d5616563a69ad5714a42c504276f/raw/b8075410e400e18f18360015945f3760d99d044a/gremlin-python-play.py

has it.

Oh - I need a server ...

Let's download it

http://ftp-stud.hs-esslingen.de/pub/Mirrors/ftp.apache.org/dist/tinkerpop/3.4.3/apache-tinkerpop-gremlin-server-3.4.3-bin.zip

unzip it and create a starter file for it as outlined in the example above:

gspath=$HOME/Downloads/apache-tinkerpop-gremlin-server-3.4.3
#$gspath/bin/gremlin-server.sh install org.apache.tinkerpop gremlin-python 3.4.3
$gspath/bin/gremlin-server.sh $gspath/conf/gremlin-server-modern-py.yaml

It doesn't work - get a 599 timeout from tornado.

Look at:

Why can't I connect to Gremlin-Server?

Try hints e.g. using 0.0.0.0 as host

Check the startup.

Oops - there is an error:

Could not initialize gremlin-jython GremlinScriptEngine as init script could not be evaluated
java.util.concurrent.CompletionException: java.lang.IllegalArgumentException: gremlin-jython is not an available GremlinScriptEngine
    at java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:375)
    at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1934)
    at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.lambda$new$4(ServerGremlinExecutor.java:141)
    at java.util.LinkedHashMap$LinkedKeySet.forEach(LinkedHashMap.java:559)
    at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.<init>(ServerGremlinExecutor.java:136)
    at org.apache.tinkerpop.gremlin.server.GremlinServer.<init>(GremlinServer.java:122)
    at org.apache.tinkerpop.gremlin.server.GremlinServer.<init>(GremlinServer.java:86)
    at org.apache.tinkerpop.gremlin.server.GremlinServer.main(GremlinServer.java:345)
Caused by: java.lang.IllegalArgumentException: gremlin-jython is not an available GremlinScriptEngine
    at org.apache.tinkerpop.gremlin.jsr223.CachedGremlinScriptEngineManager.registerLookUpInfo(CachedGremlinScriptEngineManager.java:95)
    at org.apache.tinkerpop.gremlin.jsr223.CachedGremlinScriptEngineManager.getEngineByName(CachedGremlinScriptEngineManager.java:58)
    at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:266)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
[WARN] ServerGremlinExecutor - Could not initialize gremlin-python GremlinScriptEngine as init script could not be evaluated
java.util.concurrent.CompletionException: java.lang.IllegalArgumentException: gremlin-python is not an available GremlinScriptEngine
    at java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:375)
    at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1934)
    at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.lambda$new$4(ServerGremlinExecutor.java:141)
    at java.util.LinkedHashMap$LinkedKeySet.forEach(LinkedHashMap.java:559)
    at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.<init>(ServerGremlinExecutor.java:136)
    at org.apache.tinkerpop.gremlin.server.GremlinServer.<init>(GremlinServer.java:122)
    at org.apache.tinkerpop.gremlin.server.GremlinServer.<init>(GremlinServer.java:86)
    at org.apache.tinkerpop.gremlin.server.GremlinServer.main(GremlinServer.java:345)
Caused by: java.lang.IllegalArgumentException: gremlin-python is not an available GremlinScriptEngine
    at org.apache.tinkerpop.gremlin.jsr223.CachedGremlinScriptEngineManager.registerLookUpInfo(CachedGremlinScriptEngineManager.java:95)
    at org.apache.tinkerpop.gremlin.jsr223.CachedGremlinScriptEngineManager.getEngineByName(CachedGremlinScriptEngineManager.java:58)
    at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:266)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

gremlin-python is not an available GremlinScriptEngine

mentions the plugin for the server should be installed. I had assumed that

$gspath/bin/gremlin-server.sh install org.apache.tinkerpop gremlin-python 3.4.3

would do that and when i run it again I get:

Installing dependency org.apache.tinkerpop gremlin-python 3.4.3
Could not install the dependency: a module with the name gremlin-python is already installed

How to get this working from here?

I cut down the python test code to:

# https://pypi.org/project/gremlinpython/

# start GremlinServer
# bin/gremlin-server.sh -i org.apache.tinkerpop gremlin-python 3.2.2-SNAPSHOT
# bin/gremlin-server.sh conf/gremlin-server-modern-py.yaml

# in practice, you really only need the 3 imports below

from gremlin_python import statics
from gremlin_python.structure.graph import Graph
from gremlin_python.driver.driver_remote_connection import DriverRemoteConnection
# this allows us to do g.V().repeat(out()) instead of g.V().repeat(__.out())-type traversals

#statics.load_statics(globals())

# create a remote connection using RemoteStrategy

graph = Graph()
g = graph.traversal().withRemote(DriverRemoteConnection('ws://localhost:8182','g'))

print (g.V().count().value())

https://issues.apache.org/jira/browse/TINKERPOP-2293 is now a bug report related to this question and https://issues.apache.org/jira/projects/TINKERPOP/issues/TINKERPOP-2294 a request for improvement — Wolfgang Fahl, Sep 15 '19 at 06:45

score 2 · Answer 1 · answered Dec 01 '19 at 17:27

I suspect you have a workaround for this already - but the 599 is caused I think because you can't get to the ws port on anything other than a local machine setup. I faced a similar problem with a segregated install.

If you set the host property in your gremlin yaml to 0.0.0.0 - it will listen on all interfaces. Testing everything on localhost gave me a false sense of security - worked when I listened on all network interfaces - which in turn allowed me to have separate clients to the gremlin server.

score 0 · Answer 2 · answered Sep 16 '19 at 10:44

0

I think you've just grabbed bits and pieces of code, scripts and answers and they are mismatching in terms of the various versions over multiple years of evolution of TinkerPop.

I think you just need to start over and simplify.

Settle on the must recently released version at 3.4.3 and download Gremlin Server.
Unzip and start the server with: bin/gremlin-server.sh conf/gremlin-server-modern.yaml and initially try to connect from as "localhost" so that no configuration changes are required and to keep things simple.
Validate that you can connect to it with Gremlin Console.
Now install gremlinpython with pip being sure to use 3.4.3.

At which point I would expect that the following code would work out of the box:

g=traversal().withRemote(DriverRemoteConnection('ws://localhost:8182/gremlin','g'))
print (g.V().count().toList())

Note that I terminated the traversal with toList() (I noticed that your code had no such terminating step) - without that step the traversal will not be executed. I'm not sure if that is part of the problem you're facing.

As for this command:

bin/gremlin-server.sh install org.apache.tinkerpop gremlin-python 3.4.3

you only need to run that if you intend to send native python lambdas to the server. While that may sound neat, keep in mind that you should avoid lambdas if at all possible and that the python ScriptEngine isn't nearly as robust as the Groovy one in terms of caching and execution. If you absolutely had to use a lambda I'd prefer Groovy even if you were coding in Python. That said, I'm not sure why you are getting an error in server startup (fwiw, i actually tested the simple startup with the conf/gremlin-server-modern-py.yaml and it started without error). Perhaps if you retry with a fresh setup you will get a clean start.

Once you have the basic configuration establish, then you can make adjustments, like altering the host configuration in the Gremlin Server yaml file so that you can connect to it from an external IP address.

answered Sep 16 '19 at 10:44

stephen mallette

45,298
5
67
135

My question describes the procedure you recommend as a remedy as not working. I believe that things work in your enviroment. They do not in mine and we need to find out the reason. The error message appears in step 2 – Wolfgang Fahl Sep 17 '19 at 08:26
Step 3 is not possible without downloading the gremlin console first. – Wolfgang Fahl Sep 17 '19 at 08:50
I can't imagine how step 2 in my instructions (not a remedy - just trying to figure out what's wrong by simplifying things) can possibly produce that "Could not initialize gremlin-jython" error as `conf/gremlin-server-modern.yaml` does not contain a configuration for that scriptengine. You must still be starting it with `conf/gremlin-server-modern-py.yaml` somehow or an otherwise modified version `conf/gremlin-server-modern.yaml`. – stephen mallette Sep 17 '19 at 10:37
and yes i'm asking you to please download Gremlin Console if you don't have it. i like using that for determining out-of-the-box connectivity. it is not a step python users should be required to when setting up Gremlin Server....i'm doing it for debugging purposes with you since verifying basic connectivity there will rule out other major problems. that said, everyone should download Gremlin Console. there is no better tool for debugging Gremlin traversals in my mind :) – stephen mallette Sep 17 '19 at 10:42
I see - you'd like me to get a workaround working and not answer the original question about the error message and bug report. That's why the small "detail" of using another yaml file is important. I never used the gremlin console before and tried to avoid it up to now. That's why i created http://wiki.bitplan.com/index.php/Gremlin which is Java based. I'd know like to try out python and avoid groovy again :-) – Wolfgang Fahl Sep 17 '19 at 11:43
I just want to incrementally determine where the failure is. What I've described is the exact debugging process I would take to solve this problem myself if I had it. I'm sure that all of it can be working all the way to your original question/error/bug report (no workarounds in the end), but I have to get a baseline understanding of what's happening in your environment that I'm not getting from your description. If you want to skip the Gremlin Console part you can - i just want to validate that you can connect to a running server (without the python ScriptEngine as you don't really need it). – stephen mallette Sep 17 '19 at 11:55
See http://wiki.bitplan.com/index.php/Gremlin_python#Trial_script for the error message i get. – Wolfgang Fahl Sep 17 '19 at 12:11
I think that you are pasting Python code into Gremlin Console. The link I provided for Gremlin Console expresses what I was hoping you would test: `:remote connect tinkerpop.server conf/remote.yaml` followed by a simple query like `:> g.V().values('name')` sorry if that wasn't clear. Does that work for you? – stephen mallette Sep 17 '19 at 12:19
https://stackoverflow.com/a/52998299/1497139 works with the peculiar syntax needed. – Wolfgang Fahl Sep 17 '19 at 12:20
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/199574/discussion-between-wolfgang-fahl-and-stephen-mallette). – Wolfgang Fahl Sep 17 '19 at 12:23

Wolfgang Fahl · Accepted Answer · 2019-09-17T16:28:25.627

Based on @stephen mallette's efforts I have created:

https://github.com/WolfgangFahl/gremlin-python-tutorial

which is explained in detail at

http://wiki.bitplan.com/index.php/Gremlin_python

There is a script based installation available:

./run -h
usage: ./run  [-h|-i|-s|-c]
  -h|--help: show this usage
  -i|--install: install prerequisites
  -s|--server: start server
  -c|--console: start console
  -p|--python: start python trial code

It tries to automate the necessary steps

Installation
Gremlin-Server start
Gremlin-Console start (for debugging)
Python script start

1. Installation

 run -i

installs

gremlin server
gremlin console
gremlin python module

2. Gremlin-Server start

 ./run -s

starts the gremlin server with a default yaml-file in foreground

3. Gremlin-Console start (for debugging)

 ./run -c

starts the gremlin console

4. Python script start

./run -p

starts the python test script.

# minimal imports
from gremlin_python.process.anonymous_traversal import traversal
from gremlin_python.driver.driver_remote_connection import DriverRemoteConnection
# get the remote graph traversal
g = traversal().withRemote(DriverRemoteConnection('ws://localhost:8182/gremlin','g'))
vCount=g.V().count().next()
print("The modern graph has %d vertices" % (vCount))

with the expected result:

starting python test code
The modern graph has 6 vertices

The above script has been tested with:

Ubuntu 18.04 LTS
Travis
MacOS and MacPorts

How do i get Gremlin-Python with gremlin-server 3.4.3 to work?

3 Answers3