8

In DynamoDB an Atomic Counter is a number that avoids race conditions

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/WorkingWithItems.html#WorkingWithItems.AtomicCounters

What makes a number atomic, and can I add/subtract from a float in non-unit values?

Currently I am doing: "SET balance = balance + :change"

(long version) I'm trying to use DynamoDB for user balances, so accuracy is paramount. The balance can be updated from multiple sources simultaneously. There is no need to pre-fetch the balance, we will never deny a transaction, I just care that when all the operations are finished we are left with the right balance. The operations can also be applied in any order, as long as the final result is correct.

From what I understand, this should be fine, but I haven't seen any atomic increment examples that do changes of values other than "1"

My hesitation arises because questions like Amazon DynamoDB Conditional Writes and Atomic Counters suggest using conditional writes for similar situation, which sounds like a terrible idea. If I fetch balance, change and do a conditional write, the write could fail if the value has changed in the meantime. However, balance is the definition of business critical, and I'm always nervous when ignoring documentation

-Additional Info-

All writes will originate from a Lambda function, and I expect pretty much 100% success rates in writes. However, I also maintain a history of all changes, and in the event the balance is in an "unknown" state (eg network timeout), could lock the table and recalculate the correct balance from history.

This I think gives the best "normal" operation. 99.999% of the time, all updates will work with a single write. Failure could be very costly, as we would need to scan a clients entire history to recreate the balance, but in terms of trade-off that seems a pretty safe bet.

FrozenKiwi
  • 1,362
  • 13
  • 26

2 Answers2

3

It appears that this concept is workable, from a AWS staff reply

Often application writers will use a combination of both approaches, where you can have an atomic counter for real-time counting, and an audit table for perfect accounting later on.

https://forums.aws.amazon.com/thread.jspa?messageID=470243&#470243

There is also confirmation that the update will be atomic and any update operation will be consistent

All non batch requests you send to DynamoDB gets processed atomically - there is no interleaving involved of any sort between requests. Write requests are also consistent, so any write request will update the latest version of the item at the time the request is received.

https://forums.aws.amazon.com/thread.jspa?messageID=621994&#621994

In fact, every write to a given item is strongly consistent

in DynamoDB, all operations against a given item are serialized.

https://forums.aws.amazon.com/thread.jspa?messageID=324353&#324353

FrozenKiwi
  • 1,362
  • 13
  • 26
  • "operations against given item are serialised"... but your code doesn't know what serialisation order was chosen. Consider that `UPDATE(+1), DELETE, CREATE(10) = 10` may become `DELETE, CREATE(10), UPDATE(+1) = 11`. – Dima Tisnek Sep 14 '21 at 06:19
  • Yes, it breaks as soon as you have absolute actions. As long as the operations are relative, your ok – FrozenKiwi Sep 14 '21 at 16:12
  • Actually it's also "broken" with relative ops in global tables. The reconciliation is image-based, meaning that in case of cross-regional conflict, one version of entire item wins. – Dima Tisnek Sep 15 '21 at 01:43
2

The documentation for atomic counter is pretty clear and in my opinion it will be not safe for your use case.

The problem you are solving is pretty common, AWS recommends using optimistic locking in such scenarios. Please refer to the following AWS documentation, http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBMapper.OptimisticLocking.html

Shibashis
  • 8,023
  • 3
  • 27
  • 38
  • 1
    All these documents are ensuring that changes are made based on the correct client-side version of a number. I never retrieve 'balance', so there is no risk of a stale version. – FrozenKiwi May 19 '16 at 12:53
  • If the update operation does not ensure consistency, then I simply see no way to guarantee a correct number regardless of which method I use. For a conditional write, we write IFF the stored number matches our expected initial condition, however if the update is not guaranteed to be consistent, then it sounds possible for the IFF test to pass (init value matches), an external process changes the value, then we write over the change with our value – FrozenKiwi May 19 '16 at 12:58
  • To put it another way, if any number is an "Atomic Counter" when doing "Update" operations, then either the use case works, or Atomic Counters couldn't fulfill the expectations. If an Atomic Counter is a special number type, or only functions works when incrementing by "1", then my operations don't count as atomic and I'm pooched :) – FrozenKiwi May 19 '16 at 13:07
  • I did not follow your comment about client side version. The way optimistic locking works is, it expects you retrieve the balance and the subsequent write ensures that the database value has not changed since the last retrieval. I am not sure why it does not work for your usecase. – Shibashis May 19 '16 at 13:17
  • I don't never actually get the current balance. Instead of sending a "SET" on the db value, I send an offset instead. This means I actually don't care what the stored value is, just that the offset is recorded appropriately, and if 5 operations simultaneously offset the value, the final value is correct. – FrozenKiwi May 19 '16 at 13:32
  • I do understand you don't retrieve offset. But with dynamodb you will not get ACID support. I.e there is no guarantee, which offset gets recorded last. So its handled programatically. The recommendation is for such cases, you should retrieve the entry and then submit an update command. If the update is unsuccessful, the dynamodb api will throw an exception which you can handle programmatically. – Shibashis May 19 '16 at 13:37
  • Full ACID support isn't a requirement, just that the properties of "Atomic Counters" are present on any number add/subtract (which is neither confirmed nor denied by the docs). I don't care which offset is recorded last, as long as the correct balance is maintained – FrozenKiwi May 19 '16 at 13:42
  • If sequence does not matter, may be you can use atomic counters – Shibashis May 19 '16 at 14:11