0

I'm trying to upsert a vertex representing a directory in a directory tree in accordance with this pattern (the old coalesce trick). The properties WORK_ENVIRONMENT_ID and PATH are combined in a unique index.

If the vertex does not exist, it is also connected to a given environmentVertex via an Edge labelled CONTAINS. The vertex's DISCOVERED_AT property is updated nonetheless.

Simplified query:

            final String addedVertexStep = "addedVertex";

            Vertex fileVertex = (Vertex) graph.traversal().V()
                /* ... querying for unique properties */
                .fold()
                .coalesce(__.unfold(),
                        __.addV(VertexLabels.FILE)
                                /* setting unique properties */
                                .as(addedVertexStep)
                                .V(environmentVertex.id())
                                .addE(EdgeLabels.CONTAINS)
                                .to(addedVertexStep)
                                .property(PropertyKeys.WORK_ENVIRONMENT_ID, workEnvironmentId.asUUID())
                                .select(addedVertexStep)
                )
                /* updating properties */
                .next();

Full query:

            final String addedVertexStep = "addedVertex";

            Vertex fileVertex = (Vertex) graph.traversal().V()
                .hasLabel(VertexLabels.FILE)
                .has(PropertyKeys.WORK_ENVIRONMENT_ID, workEnvironmentId.asUUID())
                .has(PropertyKeys.PATH, fileMetaInformation.fullPath())
                .fold()
                .coalesce(__.unfold(),
                        __.addV(VertexLabels.FILE)
                                .property(PropertyKeys.PATH, fileMetaInformation.fullPath())
                                .property(PropertyKeys.NAME, fileMetaInformation.simpleName())
                                .property(PropertyKeys.WORK_ENVIRONMENT_ID, workEnvironmentId.asUUID())
                                .property(PropertyKeys.PARENT_PATH, fileMetaInformation.directory().path())
                                .property(PropertyKeys.ID, fileMetaInformation.getId().asUUID())
                                .as(addedVertexStep)
                                .V(environmentVertex.id())
                                .addE(EdgeLabels.CONTAINS)
                                .to(addedVertexStep)
                                .property(PropertyKeys.WORK_ENVIRONMENT_ID, workEnvironmentId.asUUID())
                                .select(addedVertexStep)
                )
                .property(PropertyKeys.DISCOVERED_AT, new Date())
                .next();

This gets executed hundreds of times in a loop, in a cached thread pool, but using the same threaded transaction:

final Graph threadedGraph = graph.tx().createThreadedTx();

Is there something wrong with the design of the query or the way it is executed that causes a deadlock? Can I redesign it to prevent this? Here's the exception I'm getting.

org.janusgraph.core.JanusGraphException: Possible dead lock detected. Waited for transaction lock without success
    at org.janusgraph.graphdb.transaction.lock.ReentrantTransactionLock.lock(ReentrantTransactionLock.java:46) ~[janusgraph-core-0.4.0.jar:?]
    at org.janusgraph.graphdb.transaction.lock.CombinerLock.lock(CombinerLock.java:45) ~[janusgraph-core-0.4.0.jar:?]
    at org.janusgraph.graphdb.transaction.lock.CombinerLock.lock(CombinerLock.java:42) ~[janusgraph-core-0.4.0.jar:?]
    at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:769) ~[janusgraph-core-0.4.0.jar:?]
    at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:745) ~[janusgraph-core-0.4.0.jar:?]
    at org.janusgraph.graphdb.vertices.AbstractVertex.property(AbstractVertex.java:152) ~[janusgraph-core-0.4.0.jar:?]
    at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:72) ~[janusgraph-core-0.4.0.jar:?]
    at org.janusgraph.graphdb.util.ElementHelper.attachProperties(ElementHelper.java:80) ~[janusgraph-core-0.4.0.jar:?]
    at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsTransaction.addVertex(JanusGraphBlueprintsTransaction.java:122) ~[janusgraph-core-0.4.0.jar:?]
    at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsTransaction.addVertex(JanusGraphBlueprintsTransaction.java:44) ~[janusgraph-core-0.4.0.jar:?]
    at org.apache.tinkerpop.gremlin.process.traversal.step.map.AddVertexStep.map(AddVertexStep.java:81) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.step.map.AddVertexStep.map(AddVertexStep.java:43) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.step.map.MapStep.processNextStart(MapStep.java:37) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.step.map.GraphStep.processNextStart(GraphStep.java:158) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.step.map.MapStep.processNextStart(MapStep.java:36) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.step.map.MapStep.processNextStart(MapStep.java:36) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.step.map.SelectOneStep.processNextStart(SelectOneStep.java:131) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.hasNext(DefaultTraversal.java:192) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.step.map.CoalesceStep.flatMap(CoalesceStep.java:58) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.step.map.FlatMapStep.processNextStart(FlatMapStep.java:49) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.SideEffectStep.processNextStart(SideEffectStep.java:38) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:128) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:38) ~[gremlin-core-3.4.1.jar:3.4.1]
    at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.next(DefaultTraversal.java:200) ~[gremlin-core-3.4.1.jar:3.4.1]
Double M
  • 1,449
  • 1
  • 12
  • 29

1 Answers1

0

Turns out that it was not a deadlock, but rather lock congestion because I was bulk loading thousands of vertices with uniqueness constraints within the same transaction.

To fix it, I opened a new transaction for each vertex (or vertex chain as in the directory example above) and committed the transaction before upserting the next vertex (chain).

Double M
  • 1,449
  • 1
  • 12
  • 29