Quick point of clarification before I begin. I am assuming a distributed, service-oriented architecture, and I assume that all reads and writes to this DynamoDB table occurs only through one service. I will use "application" to refer to the software that you are building that accesses the table and "client" to refer to anything at all that is a client of your application.
Are there any built-in options like conditional update, but for inserting items?
The short answer is "yes", using optimistic locking based on version numbers.
You need to start by changing your sort key to an sequential event number. This will be the attribute for versioning your items, which uses conditional updates, and incurs the least amount of additional overhead of any solution to your problem.
Let's start by looking at some sample data for the proposed schema. I've take the liberty to add some more status types.
invoiceId | eventNo | eventStatus | datetime
----------|---------|-------------|---------------------
111 | 0 | created | 2019-07-11T00:01:00
111 | 1 | approved | 2019-07-11T00:02:00
111 | 2 | modified | 2019-07-12T00:03:00
111 | 3 | approved | 2019-07-12T00:04:00
111 | 4 | settled | 2019-07-13T00:05:00
The general idea of optimistic locking is that you read the current state, and then update the state by inserting a new record with an incremented eventNo (equivalent to version in the AWS docs) with the condition that the eventNo is not already present for that invoiceId. The reason this works is because when you read the existing state, you always know what the next eventNo should be (unlike using timestamps for the sort key).
To make this more concrete, on 2019-07-13, when the client sends a request to settle the invoice, your application reads the latest state, sees that the eventNo is 3 and that status is "approved", and so it submits an UpdateItem request to DynamoDB that (translated to plain english) says
insert a status update with invoiceId=111 and eventNo=4 only if no other status update already exists with invoiceId=111 and eventNo=4
If two clients were to try to update the state at the same time, only one of the UpdateItem requests would succeed, and the other one would return a ConditionalCheckFailedException.
Okay, so how do I code this?
I have not used C# in more than a decade, so please forgive any syntax or formatting errors that may be present.
AmazonDynamoDBClient client = new AmazonDynamoDBClient();
// These should be method parameters/args, but I'm directly assigning them to
// keep this code sample simple.
var invoiceToUpdate = 123;
var invoiceNewState = "settled";
// Here's the useful part of the sample code
// First we make a query to get the current state
var queryRequest = new QueryRequest
{
TableName = "Invoices",
KeyConditionExpression = "invoiceId = :inv",
ExpressionAttributeValues = new Dictionary<string, AttributeValue> {
{":inv", new AttributeValue {N = invoiceIdToUpdate.toString() }}
},
// This assumes we only need to check the current state and not any of the historical
// state, so we'll limit the query to return only one result.
Limit = 1,
// If we're limiting it to only one result, change the sort order to make sure we get
// the result with the largest eventNo (and therefore the most recent state).
ScanIndexForward = false,
// This is not strictly necessary for correctness because of the condition expression
// in the PutItem request, but including it will help reduce the likelihood of getting
// a ConditionalCheckFailedException later on.
ConsistentRead = true
};
var queryResponse = client.Query(queryRequest);
// Check to see if there is any previous record for this invoice
// Setup the default values if the query returned no results
int newEventNo = 0;
string invoiceCurrentState = null;
if (queryResponse.Items.Count > 0) {{
// If there is any existing record, then increment the eventNo for the new record
var latestRecord = queryResponse.QueryResult().Items[0];
newEventNo = Convert.ToInt32(latestRecord["eventNo"]) + 1;
invoiceCurrentState = latestRecord["eventStatus"];
}
var isValidChange = MyBusinessLogic.isValidChange(invoiceCurrentState, invoiceNewState);
if (isValidChange) {
var putItemRequest = new PutItemRequest
{
TableName = "Invoices",
Item = new Dictionary<string,AttributeValue>() {
{ "invoiceId", new AttributeValue {N = invoiceIdToUpdate.toString() }},
{ "eventNo", new AttributeValue {N = newEventNo.toString()}},
{ "eventStatus", new AttributeValue {S = invoiceNewState}},
{ "datetime", new AttributeValue {S = DateTime.UtcNow.ToString("yyyy-MM-ddTHH:mm:ssZ") }}
},
// Every item must have the key attributes, so using 'attribute_not_exists'
// on a key attribute is functionally equivalent to an "item_not_exists"
// condition, causing the PUT to fail if it would overwrite anything at all.
ConditionExpression = "attribute_not_exists('invoiceId')"
};
try {
var putItemResponse = client.PutItem(putItemRequest);
} catch (ConditionalCheckFailedException ex) {
// How you handle this is up to you. I recommend choosing one of these options:
// (1) Throw an exception with a more useful message explaining that the state changed while the
// request was being processed
// (2) Automatically try again, starting with the query and including the business validations,
// and if the state change is still valid, submit a new PutItem request with the new eventNo.
}
// Return an acknowledgement to the client
} else {
throw new System.InvalidOperationException("Update is not valid for the current status of the invoice.");
}
Here's some of the relevant documentation for the code sample I've given.
Is it possible to lock entire partition to prevent insert new items from other client?
Yes, but it's not a lock that's implemented in the database. The locking has to occur in your application using a separate locking library that incurs additional overhead, so you should not use this approach unless you have no other alternative. For anyone reading the question who cannot change the schema of their table, you can set up your application(s) to use DynamoDB lock client to lock on the partition key, read the current state, perform the write (if allowed), and then release the lock.
