write lock behavior in neo4j cypher over transational REST api

Question

I am using CYPHER like below (with Neo4j 2.2.3, over the transactional cypher REST endpoint) to atomically read and increment a counter property on a node:

MATCH node WHERE id(node)={nodeId}
SET node.counter=COALESCE(node.counter,0) + 1
RETURN node.counter

I have multiple threads making REST posts like this, and I have now seen a case where two threads got the same counter value returned.

This behavior looks similar to what is described as a possible regression from 2.1 to 2.2 here: Neo4j 2.2 Cypher locking regression?

However, earlier it was suggested that this isn't possible in CYPHER (see comments here: https://stackoverflow.com/a/18486645) ... even earlier it seems to have been possible (see: https://stackoverflow.com/a/18450171)

Is there a CYPHER query, similar to above, that will reliably produce unique (monotonically increasing) counter values when there are multiple concurrent REST POSTs of the query?

Sean Timm · Accepted Answer · 2015-08-06T13:27:47.587

2

In Neo4j, you need to obtain a write lock before you read and update the value. We do this as two statements in a single transaction (one REST call). We use MERGE/ON CREATE/ON MATCH, but I do not think that is necessary or would even work in your case. I believe this should work for you.

statement 1:

MATCH node WHERE id(node)={nodeId}
SET node.__lock=true
RETURN node.__lock

statement 2:

MATCH node WHERE id(node)={nodeId}
SET node.counter=COALESCE(node.counter,0) + 1
SET node.__lock=false
RETURN node.counter

edited Aug 06 '15 at 13:27

answered Aug 04 '15 at 21:06

Sean Timm

80
6

Thanks @sean-timm. I will try this. Now I understand why the 2 statement structure was used in question 31755509 (see link in Q above). But I also wonder if you are seeing any bad behavior like described in 31755509 with 2.2.x releases? – Chris Aug 04 '15 at 22:04
please see this gist that shows the cypher json input and output from when I tried this: https://gist.github.com/cjdaly/aa5334b58c34eeff85fa . Do you get errors like this and then retry the operation? Or is this not what you would expect to see? – Chris Aug 05 '15 at 05:20
I wouldn't expect to get that error. That error sounds like you are creating a constraint in Neo4j concurrently with the write. Do you consistently get the same error? – Sean Timm Aug 06 '15 at 00:48
We are using this pattern in Neo4j 2.2.0 Enterprise without issue. – Sean Timm Aug 06 '15 at 00:55
I'm using the 'community' edition download, v. 2.2.3. I am using 'CREATE CONSTRAINT' which I was doing in parallel with the counter increments, so it seems like I was out of bounds in terms of what is currently supported. I'm going to mark this answer correct, because I think you explained how splitting into 2 statements is the right general approach (nitpick: in statement 1, RETURN node.__lock). For now, in my application I have actually serialized all the neo4j REST traffic and things are running smoothly. My next plan is to try to serialize the writes and let reads go parallel... – Chris Aug 06 '15 at 02:33
I fixed the nitpick. :-) – Sean Timm Aug 06 '15 at 13:28
You should only have to CREATE CONSTRAINT once. Do that at initialization time before you do any real work and you should be good. The CREATE CONSTRAINT call is idempotent, so it will not hurt to call it on a DB that already has constraints set. – Sean Timm Aug 06 '15 at 13:30

Daniel Garmoshka · Answer 2 · 2016-09-11T08:07:16.567

2

Short solution is:

MATCH node WHERE id(node)={nodeId}
SET 
  node.__lock=true, 
  node.counter=COALESCE(node.counter,0) + 1,
  node.__lock=false
RETURN node.counter

It is improved solution of Sean Timm. This notation is more efficient (faster), than two transactions

edited Sep 11 '16 at 08:07

answered Aug 21 '16 at 09:42

Daniel Garmoshka

5,849
39
40

score 0 · Answer 3 · answered Jul 27 '17 at 06:55

Adding to @SeanTimm's answer, you can explicitly lock nodes via Cypher if you use the popular (and open source) APOC plugin for Neo4j. APOC has a selection of explicit Locking procedures that can be called via Cypher such as call apoc.lock.nodes([nodes])

If this option is available to you, it is probably preferable to some of the other, hacky, solutions.

Learn more at neo4j-contrib.github.io/neo4j-apoc-procedures/#_locking

write lock behavior in neo4j cypher over transational REST api

3 Answers3