3

I'm seeing .project().by() traversals returning {} under Gremlin JS 3.4.0. When I downgrade to 3.2.10 they work correctly.

gremlin> g.addV("trip").property(single, "trackName", "Ohio")

==>v[1]

In Gremlin JS `3.4.0`:

const result = await g.V("1").project("trackName").by("trackName").next();

result:

{
  "value": {},
  "done": false
}

but when I downgrade to Gremlin 3.2.10 the result is correct:

{
  "value": {
    "trackName": "Ohio"
  },
  "done": false
}

Do I need to change how I use project in 3.4.0?

EDIT: Results from testing against different versions. I ran each test for a gremlin version, captured results, then bumped up the version and ran the tests again. I am only running a single Neptune instance so we can be sure this is the same data.

enter image description here

A failing test means it returned data in the form of:

"results": {
 "value": {},
 "done": false
}

For the console testing I removed the final .next().

The environment I am testing in is:

AWS Lambda Node 8.10

AWS Neptune 1.0.1.0

EDIT 2: Adding JS files used during Neptune test.

index.js

const gremlin = require("gremlin");

const { DriverRemoteConnection } = gremlin.driver;
const { Graph } = gremlin.structure;

const initGremlinClient = () => {
  try {
    const dc = new DriverRemoteConnection(
      `ws://my-cluster.XXXXXXX.us-east-1.neptune.amazonaws.com:8182/gremlin`,
      {}
    );
    const graph = new Graph();
    return {
      g: graph.traversal().withRemote(dc),
      closeGremlinConnection: () => dc.close()
    };
  } catch (error) {
    console.log("[GREMLIN INIT ERROR]", error);
    throw new Error(error);
  }
};

exports.handler = async event => {
  const { g, closeGremlinConnection } = initGremlinClient();

  const result = await g
    .addV("test")
    .property("myProp", "myValue")
    .project("myProp")
    .by("myProp")
    .next();
  closeGremlinConnection();
  return result;
};

package.json

{
  "name": "gremlinTest",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "dependencies": {
    "gremlin": "3.4.0"
  }
}
Fook
  • 5,320
  • 7
  • 35
  • 57
  • In your example you create a property called "trackName" but search for one called "trackLength" - that would cause an empty result if "trackLength" does not exist I think. – Kelvin Lawrence Jan 16 '19 at 22:54
  • Shoot, that was a typo in the example. I'll correct the question but the issue still exists. I've replicated it across several vertexes and properties. Thanks for catching it. – Fook Jan 16 '19 at 23:31
  • as i suggested in a different question of yours related to 3.4.0 usage i would consider using 3.3.5 with neptune at least until they announce official support for 3.4.0. while 3.4.0 should generally work with 3.3.x there are differences that might cause you hassles in both big ways (like using steps only available in 3.4.0 mistakenly) and in subtle ways (perhaps like what you're experiencing now). if you try 3.3.5 and still have a problem that would be more unexpected in my mind. please let us know if you've tried that version and ended up with the same problem. – stephen mallette Jan 17 '19 at 01:43
  • @stephenmallette It appears to have the same issue in 3.3.5. – Fook Jan 17 '19 at 14:01
  • 3.2.11 is the last release that doesn't have this issue. I've tested with 3.3.5 and 3.4.0. – Fook Jan 17 '19 at 14:12
  • this is a pretty simple query to be failing like this and there are definitely tests that validate this kind of behavior. what happens with `g.V("1").project("trackName").by(values("trackName"))`? what about `g.V('1').valueMap('trackName')`? does that `Map` return? Did you try against a raw Gremlin Server instance with TinkerGraph to validate that it works there? did you try this query with Java (i'd just use the Gremlin Console and a small script) against Neptune? Did you try to upgrade to 3.3.4 (which released with 3.2.10)? In short, can you try to isolate the problem further? – stephen mallette Jan 21 '19 at 12:30
  • I tried your addV and then used your query using Gremlin Javascript 3.3.5 and Neptune and I got what I think is the expected result. `{ value: Map { 'trackName' => 'Ohio' }, done: false }` Is there any chance that somehow your 3.3.5 test is connecting to a different cluster or that the vertex is somehow no longer there? You will see what you see if the vertex is not found. You do not show a property(id,'1') step in your addV(). Did you use one? If not you will get a Neptune generated ID. – Kelvin Lawrence Jan 22 '19 at 15:42
  • @stephenmallette I have added some testing results in the original question. I tested with the versions mentioned but only in my Lambda / Neptune stack. If someone can provide a Java sample I can try to get that running too. – Fook Jan 22 '19 at 16:13
  • @KelvinLawrence I did specify the vertex ID in `addV` in my original script. The tests I added above all use the same traversals, the only difference is the version of gremlin. – Fook Jan 22 '19 at 16:14
  • If you just do a `g.V("1").valueMap().toList(true)` do you get some results? – Kelvin Lawrence Jan 22 '19 at 23:28
  • It returned results in 3.2.10, returned nothing (`results: [ {} ]`) in 3.3.5 and 3.4.0. It worked in the console after I removed `true` from `toList`, otherwise it would error: `Query parsing failed at line 1, character position at 62, error message : extraneous input 'true' expecting ')'` – Fook Jan 23 '19 at 03:58
  • Also I am using Webpack during the deployment to AWS if that makes any difference. – Fook Jan 23 '19 at 03:59
  • Apologies the true was meant to be inside `valueMap(true)` – Kelvin Lawrence Jan 23 '19 at 15:58
  • Re Webpack - I suppose it is possible it is causing something to break. Could you try without using Webpack to rule that out? I have tried your query using multiple versions of Node and multiple versions of the Gremlin client and they all worked fine. – Kelvin Lawrence Jan 23 '19 at 16:16
  • I created a basic Lambda function with `gremlin@3.2.10`. It contains this traversal: `g.addV("test").property("myProp", "myValue").project("myProp").by("myProp").next()`. I zipped this up and uploaded to Lambda. There is no Webpack or Babel involved in this test, only plain JS. I get the same results as before. With `3.2.10` it works correctly. I then updated to `3.3.5` and `3.4.0`, re-upload, and these return nothing. I do note that all attempts do actually create the vertex in Neptune. However only `3.2.10` handles `.project().by()` correctly. Perhaps this issue is specific to Neptune? – Fook Jan 23 '19 at 17:56
  • I should have added, I did all my testing using Neptune also and it worked fine. Trying to think of any other things that could be different between your setup and mine. It might be worth applying the latest Neptune maintenance update (if you have not already) just to make sure there is nothing there. I am using the latest. I'll continue to look into this as well. – Kelvin Lawrence Jan 23 '19 at 18:31
  • I've added the script I used to test with. The only thing that stands out is the `{}` added as the second arg to `DriverRemoteConnection`. Without it I get `TypeError: Cannot read property 'reader' of undefined`. Not sure if that is related. I have minor version updates enabled in Neptune and it seems to be at the latest. What results do you get if you run the script I included above? – Fook Jan 23 '19 at 19:48
  • The need to add {} was something that TinkerPop 3.4/3.3.5 introduced. I believe there is a TinkerPop Jira open for that issue. Using Node.js connected to Neptune here is what I get for your example. { value: Map { 'myProp' => 'myValue' }, done: false } Feel free to reach out to me by e-mail if it would help. – Kelvin Lawrence Jan 24 '19 at 22:08
  • I was finally able to test this without webpack and still see the same problem. – Fook Mar 13 '19 at 21:56
  • This is quite strange. In my tests using your samples everything seems to work. @stephenmallette can you think of anything that might give a clue here? I'll do some more thinking but have as yet only been able to get the expected results. – Kelvin Lawrence Mar 14 '19 at 16:16
  • no - i've pointed others at this as well and no one has come up with any reason why there would be problems. – stephen mallette Mar 14 '19 at 16:37

1 Answers1

4

I spoke with someone on the AWS team. There is a bug affecting interoperability between Gremlin ^3.3.5 and Lambda. Specifically, the issue is with the underlying GraphSON v3 engine and how Lambda parses JSON.

The temporary workaround is to fall back to GraphSON v2 when instantiating DriverRemoteConnection:

const dc = new DriverRemoteConnection(
  `ws://my-neptune-cluster.us-east-1.neptune.amazonaws.com:8182/gremlin`,
  { mimeType: "application/vnd.gremlin-v2.0+json" } // Fall back to GraphSON v2
);

Edit: This issue still exists as of gremlin@3.4.6.

Fook
  • 5,320
  • 7
  • 35
  • 57
  • 1
    I just ran into this issue - quite painful and counter-intuitive. It causes issues with (at least): - valueMap(), - propertyMap(), - project(). So if your queries are working just fine in a straight Gremlin console/Sagemaker notebook, but gremlin-javascript just returns empty sets (like '[{}, {}]'), this may be your problem. Also, this issue still exists as of gremlin@3.5.0 – KingJackaL Jul 21 '21 at 22:42