2

Continuing discussion from here.

I've a query that provides list of paths along-with elementMap(). How do I convert this data to JSON to be consumed by rest of the code.

I did look here that talks about using GraphSONMapper. However, I am using gremlinpython (v 3.6.1) and I couldn't find GraphSONMapper with the library. There is graphsonV2d0.py and graphsonV3d0.py , but not clear on using that.

Sample Graph:

g.addV('color-group').property(id,1).property(single, 'Color', 'primary').next()
g.addV('primary-color').property(id,2).property(single, 'Color', 'red').next()
g.addV('primary-color').property(id,3).property(single, 'Color', 'blue').next()
g.addV('primary-color').property(id,4).property(single, 'Color', 'yellow').next()
g.addV('secondary-color').property(id,5).property(single, 'Color', 'red-10').next()
g.V(1).addE('links').to(V(2)).property(single, 'Key', 'value').next()
g.V(1).addE('links').to(V(3)).property(single, 'Key', 'value').next()
g.V(1).addE('links').to(V(4)).property(single, 'Key', 'value').next()
g.V(4).addE('links').to(V(5)).property(single, 'Key', 'value').next()

Query:

 g.V().hasLabel('color-group').has('Color', 'primary').outE().inV().optional(outE().inV()).path().by(elementMap()).store('data').cap('data').next()

Current Result:

 [path[{<T.id: 1>: 1, <T.label: 4>: 'color-group', 'Color': 'primary'}, {<T.id: 1>: 7, <T.label: 4>: 'links', <Direction.IN: 2>: {<T.id: 1>: 2, <T.label: 4>: 'primary-color'}, <Direction.OUT: 3>: {<T.id: 1>: 1, <T.label: 4>: 'color-group'}, 'Key': 'value'}, {<T.id: 1>: 2, <T.label: 4>: 'primary-color', 'Color': 'red'}], path[{<T.id: 1>: 1, <T.label: 4>: 'color-group', 'Color': 'primary'}, {<T.id: 1>: 8, <T.label: 4>: 'links', <Direction.IN: 2>: {<T.id: 1>: 3, <T.label: 4>: 'primary-color'}, <Direction.OUT: 3>: {<T.id: 1>: 1, <T.label: 4>: 'color-group'}, 'Key': 'value'}, {<T.id: 1>: 3, <T.label: 4>: 'primary-color', 'Color': 'blue'}], path[{<T.id: 1>: 1, <T.label: 4>: 'color-group', 'Color': 'primary'}, {<T.id: 1>: 9, <T.label: 4>: 'links', <Direction.IN: 2>: {<T.id: 1>: 4, <T.label: 4>: 'primary-color'}, <Direction.OUT: 3>: {<T.id: 1>: 1, <T.label: 4>: 'color-group'}, 'Key': 'value'}, {<T.id: 1>: 4, <T.label: 4>: 'primary-color', 'Color': 'yellow'}, {<T.id: 1>: 10, <T.label: 4>: 'links', <Direction.IN: 2>: {<T.id: 1>: 5, <T.label: 4>: 'secondary-color'}, <Direction.OUT: 3>: {<T.id: 1>: 4, <T.label: 4>: 'primary-color'}, 'Key': 'value'}, {<T.id: 1>: 5, <T.label: 4>: 'secondary-color', 'Color': 'red-10'}]]

How to convert above to get following Output in JSON?
Truncated rest as patter is same.

[[{
    "< T.id: 1 >": 1,
    "< T.label: 4 >": "color-group",
    "Color": "primary"
}, {
    "< T.id: 1 >": 7,
    "< T.label: 4 >": "links",
    "< Direction.IN: 2 >": {
        "< T.id: 1 >": 2,
        "< T.label: 4 >": "primary-color"
    },
    "< Direction.OUT: 3 >": {
        "< T.id: 1 >": 1,
        "< T.label: 4 >": "color-group"
    },
    "Key": "value"
}, {
    "< T.id: 1 >": 2,
    "< T.label: 4 >": "primary-color",
    "Color": "red"
}], ..... ]

EDIT 1.

I was able to make some progress using graphsonV3d0

from gremlin_python.structure.io import graphsonV3d0
obj = graphsonV3d0.GraphSONWriter()
query = g.V().hasLabel('color-group').has('Color', 'primary').outE().inV().optional(outE().inV()).path().by(elementMap()).store('data').cap('data').next()
obj.to_dict(query)

Result:

{'@type': 'g:List', '@value': [path[{<T.id: 1>: 1, <T.label: 4>: 'color-group', 'Color': 'primary'}, {<T.id: 1>: 7, <T.label: 4>: 'links', <Direction.IN: 2>: {<T.id: 1>: 2, <T.label: 4>: 'primary-color'}, <Direction.OUT: 3>: {<T.id: 1>: 1, <T.label: 4>: 'color-group'}, 'Key': 'value'}, {<T.id: 1>: 2, <T.label: 4>: 'primary-color', 'Color': 'red'}], path[{<T.id: 1>: 1, <T.label: 4>: 'color-group', 'Color': 'primary'}, {<T.id: 1>: 8, <T.label: 4>: 'links', <Direction.IN: 2>: {<T.id: 1>: 3, <T.label: 4>: 'primary-color'}, <Direction.OUT: 3>: {<T.id: 1>: 1, <T.label: 4>: 'color-group'}, 'Key': 'value'}, {<T.id: 1>: 3, <T.label: 4>: 'primary-color', 'Color': 'blue'}], path[{<T.id: 1>: 1, <T.label: 4>: 'color-group', 'Color': 'primary'}, {<T.id: 1>: 9, <T.label: 4>: 'links', <Direction.IN: 2>: {<T.id: 1>: 4, <T.label: 4>: 'primary-color'}, <Direction.OUT: 3>: {<T.id: 1>: 1, <T.label: 4>: 'color-group'}, 'Key': 'value'}, {<T.id: 1>: 4, <T.label: 4>: 'primary-color', 'Color': 'yellow'}, {<T.id: 1>: 10, <T.label: 4>: 'links', <Direction.IN: 2>: {<T.id: 1>: 5, <T.label: 4>: 'secondary-color'}, <Direction.OUT: 3>: {<T.id: 1>: 4, <T.label: 4>: 'primary-color'}, 'Key': 'value'}, {<T.id: 1>: 5, <T.label: 4>: 'secondary-color', 'Color': 'red-10'}]]}

This is little better, but yet not entirely in JSON format.

Kelvin Lawrence
  • 14,674
  • 2
  • 16
  • 38
Anil_M
  • 10,893
  • 6
  • 47
  • 74
  • The Gremlin Python client will try to de-serialize the results into a native Python structure that best matches the result type. Behind the scenes (over the wire) that result flows using a TinkerPop representation that is known as GraphSON (or a binary flavor of that JSON). Can you say a bit more about why you need the JSON - Is it some sort of export case where you need to feed the JSON to another application? In general you can manipulate query results to be very close to what you might need using steps like `project`. – Kelvin Lawrence Apr 07 '23 at 15:11
  • When you use `obj.to_dict(query)` you are actually seeing the GraphSON form of the result. GraphSON is JSON but sometimes optimized for streaming (one doc per row kind of thing). When Python prints it of course you tend to get single quotes rather than doubles which can mess up some JSON tools. – Kelvin Lawrence Apr 07 '23 at 15:17
  • Hi, thanks for the response. I need pure JSON with source node , its properties, target nodes, its properties and path data to form edges so that this data can be used by other application to form frontend visualization. – Anil_M Apr 07 '23 at 16:10

1 Answers1

0

I edited the creation steps slightly, to work with the database I am using to test this (Amazon Neptune) and ran queries using a graph-notebook.

g.addV('color-group').property(id,'I1').property(single, 'Color', 'primary').as('i1').
  addV('primary-color').property(id,'I2').property(single, 'Color', 'red').as('i2').
  addV('primary-color').property(id,'I3').property(single, 'Color', 'blue').as('i3').
  addV('primary-color').property(id,'I4').property(single, 'Color', 'yellow').as('i4').
  addV('secondary-color').property(id,'I5').property(single, 'Color', 'red-10').as('i5').
  addE('links').from('i1').to('i2').
  addE('links').from('i1').to('i3').
  addE('links').from('i1').to('i4').
  addE('links').from('i4').to('i5')

You can get a lot closer to the form you are looking for if we change the by modulation of the path to use a project step. For example:

g.V().hasLabel('color-group').
      has('Color', 'primary').
      outE().inV().
      optional(outE().inV()).
      path().
        by(project('id','label','properties').
             by(id).
             by(label).
             by(valueMap().by(unfold()))).
      toList()

Which yields

1   path[{'id': 'I1', 'label': 'color-group', 'properties': {'Color': 'primary'}}, {'id': 'aec3ae0e-c3d3-5c55-1ee1-fb3996158e1e', 'label': 'links', 'properties': {}}, {'id': 'I2', 'label': 'primary-color', 'properties': {'Color': 'red'}}]
2   path[{'id': 'I1', 'label': 'color-group', 'properties': {'Color': 'primary'}}, {'id': '12c3ae0e-c3d6-c04c-b386-5273adc8b640', 'label': 'links', 'properties': {}}, {'id': 'I3', 'label': 'primary-color', 'properties': {'Color': 'blue'}}]
3   path[{'id': 'I1', 'label': 'color-group', 'properties': {'Color': 'primary'}}, {'id': 'bec3ae0e-c3d7-5cb5-1616-e53d4bba45fd', 'label': 'links', 'properties': {}}, {'id': 'I4', 'label': 'primary-color', 'properties': {'Color': 'yellow'}}, {'id': '06c3ae0e-c3da-604a-d98d-65c93547209a', 'label': 'links', 'properties': {}}, {'id': 'I5', 'label': 'secondary-color', 'properties': {'Color': 'red-10'}}]

If we now go one step further, using Python, and the query result is in the variable called res - you can do:

for r in res[0]:
    print(r,'\n')

Which gives us

{'id': 'I1', 'label': 'color-group', 'properties': {'Color': 'primary'}} 

{'id': 'aec3ae0e-c3d3-5c55-1ee1-fb3996158e1e', 'label': 'links', 'properties': {}} 

{'id': 'I2', 'label': 'primary-color', 'properties': {'Color': 'red'}}

Using those building blocks, I think you can stich together whatever JSON-like result you may need.

Once the result has been captured, we can turn one of the "rows" into JSON as follows:

import json
s = '['
for r in res[0]:
    s += str(r)[:-1] + '},'
j = s.replace("'","\"")[:-1] + ']'

print(j)
print(json.loads(j))

Which gives us (now valid JSON):

[{"id": "I1", "label": "color-group", "properties": {"Color": "primary"}},{"id": "aec3ae0e-c3d3-5c55-1ee1-fb3996158e1e", "label": "links", "properties": {}},{"id": "I2", "label": "primary-color", "properties": {"Color": "red"}}]


[{'id': 'I1', 'label': 'color-group', 'properties': {'Color': 'primary'}}, {'id': 'aec3ae0e-c3d3-5c55-1ee1-fb3996158e1e', 'label': 'links', 'properties': {}}, {'id': 'I2', 'label': 'primary-color', 'properties': {'Color': 'red'}}]
Kelvin Lawrence
  • 14,674
  • 2
  • 16
  • 38
  • thx. for me (on gremlin-server + gremlinpython its giving error for the updated query: `TypeError: 'GraphTraversal' object is not callable` – Anil_M Apr 07 '23 at 16:24
  • I'm adding one more step to the answer to turn it fully into JSON. My query is basic Gremlin, you'll need to add the basic Python style idioms/names back in. – Kelvin Lawrence Apr 07 '23 at 16:27
  • Its working with aws neptune + Jupyter Notebook though. May be cause I am using gremlin line magic. I'll dig deeper on why local instance is not working which is pure python. – Anil_M Apr 07 '23 at 16:33
  • Also with this approach (using project) while we are converting native gremlin types (T.id etc) to strings, we are also loosing mapping info (e.g. 'Direction.IN' , 'Direction.OUT') that helps with edge formation. – Anil_M Apr 07 '23 at 16:44
  • You can definitely add that back in - you have full control over how you build that result set. Hopefully though you can now see how to get usable JSON back. – Kelvin Lawrence Apr 07 '23 at 16:51
  • It's hard to select things like `Direction.OUT` in the actual Gremlin query, so I would probably do that in your Python code along the lines of my example and just convert it to something meaningful for your application. – Kelvin Lawrence Apr 07 '23 at 17:08
  • You'll need this `import` to access the `Direction` parts of the dict from code: `from gremlin_python.process.traversal import Direction` – Kelvin Lawrence Apr 07 '23 at 17:16
  • Curious: `.path().by(project('OUT').by(Direction.OUT).by(unfold())).toList()` this is not working even after import. error: `"Could not locate method: DefaultGraphTraversal.by(Direction)",` I do see there's a static method in `process/traversal.py` => `statics.add_static('OUT', Direction.OUT)` ... Unless I am using it incorrectly. – Anil_M Apr 07 '23 at 18:51
  • In that case, `by` will try to look for a key called `Direction.OUT` - but there is not going to be one there as it's the `elementMap()` that builds that structure. It's not available straight from the edge itself. For an edge the direction can be inferred looking at the `outV` and `inV` IDs. If you save the path result, more of that information can be pulled from the serialized path by your Python code. – Kelvin Lawrence Apr 07 '23 at 18:58