3

I am working on AWS lambda (Stateless service where you can run your code) Link - What is AWS lambda

In my case, ,lambda is running 1 service, for which every request has to perform write operation. Every request binds with one Id.

If, requests are binds with same id. Then, that should have to process sequentially.

So, to do this. we have to take lock on "id". That way, we can avoid write operation clashes.

To do this(My research)-

  1. Create MySQL table with 1 column of Id, and it is primary key. Means, you cannot persist 1 id more than 1 time.

Pseudocode-

acquirelock perform write operation release lock

Here, acquire lock means -

If, id is already present, then wait, else insert id into Mysql table

release means-

Delete id from MySQL

Code-

public void run() {

    String host = "AWS-RDS_HOST";
    String userName = "USERNAME";
    String password = "PASSWORD";
    String dbName = "DBNAME";
    Connection connection = createConnection(host, userName, password, dbName);
    Statement statement = null;

    try {
        statement = connection.createStatement();
    } catch(SQLException e) {
        throw new RuntimeException("Failed at taking statement");
    }

    // Aquire lock.
    while(true) {
        try {
            statement.executeUpdate("insert into locktable values('"+id+"');");
        } catch(SQLException e) {
            System.out.println("Failed in taking lock. Waiting for others to release the lock.");
            continue;
        }

        System.out.println("Yes...I took lock");
        break;
    }

    // Execute write operation
    System.out.println("Doing write operation");
    System.out.println("Wrtie operation completed");

    // Release lock.
    while(true) {
        try {
            statement.executeUpdate("DELETE from locktable where id='"+id+"'");
        } catch(SQLException e) {
            System.out.println("Failed to remove lock - " + id);
            continue;
        }
        System.out.println("Lock removed. Id - " + id);
        break;
    }
}`

Questions-

  1. Is this idea correct to maintain sequential behaviour for same id ?
  2. Is there any better solution/idea for this problem ?
  3. If you have some blogs for such problem, please share me.
matsev
  • 32,104
  • 16
  • 121
  • 156
devsda
  • 4,112
  • 9
  • 50
  • 87

1 Answers1

0

To partially answer your question:

  1. Is there any better solution/idea for this problem ?

I suggest implementing a solution based on Kinesis Streams. Copied from Kinesis Streams FAQ:

Q: When should I use Amazon Kinesis Streams, and when should I use Amazon SQS?

We recommend Amazon Kinesis Streams for use cases with requirements that are similar to the following:

  • Routing related records to the same record processor [...]. For example, counting and aggregation are simpler when all records for a given key are routed to the same record processor.

  • Ordering of records.

The idea is to have one Lambda act as a Kinesis Producer, i.e. handle incoming requests and put records to Kinesis Streams. A second Lambda will be implemented as a Kinesis Consumer (aka Kinesis Streams Application), it will get data from Kinesis Streams and perform your write operation. You can base the Kinesis Streams Partition Keys on your id.

See also this SO answer.

Community
  • 1
  • 1
matsev
  • 32,104
  • 16
  • 121
  • 156
  • I totally agree with your solution. But, when requests count increases, then lambda horizontally scales, and perform tasks. In that case, some request comes under 1 lambda instance, and some request of the same id goes under another lambda instance. I am seeing this problem, in this solution. Correct me, if I am wrong. – devsda Aug 07 '16 at 12:12
  • As I understand it, there will be one Lambda instance per shard. Copied from [Lambda Function Concurrent Executions](http://docs.aws.amazon.com/lambda/latest/dg/concurrent-executions.html): `the number of shards per stream is the unit of concurrency. If your stream has 100 active shards, there will be 100 Lambda functions running concurrently. Then, each Lambda function processes events on a shard in the order that they arrive.` – matsev Aug 07 '16 at 13:40