1

We're building a ticketing system for our customers (enterprise clients) and we're trying to come up with a way to have a sequential, unique ticket ID generation system that is also user-friendly.

Unique ID generation is a fairly well-known problem and we've gone through some of the solutions used by other companies, such as Flickr, or Twitter, but they don't serve our purpose of being "user-friendly". They're far too long for our customers to remember or use them in communication via email or telephone.

Run-of-the-mill GUIDs are also not usable by us since we want sequential ordering.

We initially thought of generating the ID as a combination of the customer ID and the ticket ID itself. For example, customer 1's first ticket will be C1-T1, and then C1-T2. Customer 2's first ticket will be C2-T1, and then C2-T2.

This is certainly doable. We can simply look at the last customer_id-ticket_id combination and just add 1 to it, but it involves a DB query, and we also need to take a lock, so that concurrent transactions don't re-use the same incremented ticket ID. This essentially means moving this to an asynchronous flow as we can't hamper any synchronous flow. Keeping it in sync means that customers who have high volume, may end up waiting a long time for an API call to finish because a bunch of concurrent transactions (from previous tickets of the same customer) are still waiting.

But the business requirement is such that the ticket ID needs to be generated immediately for consumption by the user.

So, while we have some solutions to work with, every one of them has some downsides or the other. Either they hamper the latency of a customer-facing API, or they are not immediately available, or they're far too long to be user-friendly.

We're stuck right now and have no good leads to work with.

So, we wanted to know from the community if there is any way we can generate a sequential, unique ID that is also short enough (maximum 8-9 characters maybe).

Sidharth Samant
  • 714
  • 8
  • 28
  • This issue normally depends on the specific database server you are using. You may tag the question with the server. A very different answer exists for SQL Server than DynamoDB than… – Mitch May 09 '23 at 13:11
  • Thanks @Mitch, we use MySQL. I've tagged the server with the version. – Sidharth Samant May 09 '23 at 13:32
  • Are you using a single DB instance? Also, have you considered using a cache of sorts, like Redis, to handle the ticket generation alone and then saving them to MySQL? This complicates the architecture but could offer other benefits to other parts of your service in the future (I don't have an answer, just trying to squeeze more details). – Jetto Martínez May 09 '23 at 15:11
  • Hi @JettoMartínez, yes, we just have 1 DB instance for now. Can you please expand a bit on ticket generation using Redis? Are you saying caching the most recent ID generated, so that we don't have to do a lookup the next time? Yes, we have that in mind, but it won't solve the issue with high traffic. During high traffic, caching would mean we save on the expensive DB lookup call, but the lock wait time still remains. – Sidharth Samant May 09 '23 at 15:46

2 Answers2

1

This is similar to Mitch's answer, but without the detail for SP, and using the LAST_INSERT_ID() example from the MySQL docs.

A simplified table for tickets -

CREATE TABLE tickets (
    customer_id INT UNSIGNED NOT NULL,
    ticket_id INT UNSIGNED NOT NULL,
    PRIMARY KEY (customer_id, ticket_id)
);

A table for ticket sequences per customer -

CREATE TABLE customer_ticket_sequence (
    customer_id INT UNSIGNED NOT NULL PRIMARY KEY,
    seq INT UNSIGNED NOT NULL
);

-- initialise sequences for customers 1, 2 & 3
INSERT INTO customer_ticket_sequence VALUES (1, 0), (2, 0), (3, 0);

Now, instead of doing a SELECT followed by an UPDATE, we just use a single update statement, storing the new sequence value in LAST_INSERT_ID(), ready for the INSERT into tickets -

UPDATE customer_ticket_sequence SET seq = LAST_INSERT_ID(seq + 1) WHERE customer_id = 1;
INSERT INTO tickets VALUES (1, LAST_INSERT_ID());

db<>fiddle

user1191247
  • 10,808
  • 2
  • 22
  • 32
  • I think this neglects the primary point to minimize contention: don't lock the `customer_ticket_sequence` table when you are doing a different transaction. Acquire an ID, then separately deal with other transactions. Unrelated genuine question: are you sure that the `update` is equivalent to the `SELECT FOR UPDATE` / `UPDATE`? It doesn't seem to be on `READ COMMITTED`, but I suppose it may be in `REPEATABLE READ`. – Mitch May 10 '23 at 00:09
  • @Mitch I am reasonably confident that this gives the shortest possible lock time, as there is no separate read and then update, and thus minimizes contention. The `UPDATE` acquires an exclusive lock. – user1191247 May 10 '23 at 00:43
  • Thanks @user1191247, this looks promising and I'll try this out. Although, I wanted to get your opinion on using `MAX(ticket_id)` instead of using `LAST_INSERT_ID()`. `MAX(ticket_id)` from `tickets` table would run in constant time, right? So it'll be as fast as `LAST_INSERT_ID()`? – Sidharth Samant May 10 '23 at 09:26
  • Hi @user1191247, while this does work, it's failing when I run multiple parallel processes for the same customer. For example, C1 is creating 2 tickets at the same, so when these sequence of queries get executed by 2 parallel running processes, I get an `IntegrityError` since 2 separate processes are trying to `INSERT` into `tickets` with the same primary key (`1-1`). – Sidharth Samant May 10 '23 at 12:19
  • @SidharthSamant I am not sure what you mean by _"in constant time"_. Presumably there will be lots of other read and write traffic against the `tickets` table, so using `SELECT MAX(ticket_id) FROM tickets WHERE customer_id = ?` is just going to add to that. Try it and see how it behaves. If using `SELECT MAX...` as a subquery in `VALUES` you will need to nest it in another subquery, otherwise you will get a [1093 error](https://dev.mysql.com/doc/refman/8.0/en/subquery-errors.html#sa16292290:~:text=Incorrectly%20used%20table%20in%20subquery). – user1191247 May 10 '23 at 12:39
  • Hi @user1191247, what I meant by constant time is that when I run `explain select max(ticket_id) from tickets where customer_id = 1;`, the `EXPLAIN` plan shows `Select tables optimized away`, which according to [this](https://stackoverflow.com/a/7604853/4305436), means it runs in constant time. – Sidharth Samant May 10 '23 at 14:04
  • 1
    @user1191247, yes, I was running them in the same transaction. I'll separate them out. Thank you tremendously for your help! This has really helped us out! – Sidharth Samant May 10 '23 at 14:06
0

Unless there is a high volume of writes (>1000/s), the temporary lock on a dedicated "IDs" table is not likely to be costly.

An example of this, while maintaining high throughput, would be:

DELIMITER ;
-- Schema
CREATE TABLE Customers (
    CustomerID INT NOT NULL AUTO_INCREMENT,
    Name VARCHAR(50) NOT NULL,
    PRIMARY KEY (CustomerID)
);

CREATE TABLE CustomerSequences (
    CustomerID INT NOT NULL,
    Sequence varchar(50) NOT NULL,
    NextValue INT NOT NULL,
    PRIMARY KEY (CustomerID, Sequence),
    FOREIGN KEY (CustomerID) REFERENCES Customers (CustomerID)
);

-- Sample Data
INSERT INTO Customers (Name) VALUES ('Customer 1'), ('Customer 2');
INSERT INTO CustomerSequences (CustomerID, Sequence, NextValue)
SELECT CustomerID, 'Ticket', 1
FROM Customers;
INSERT INTO CustomerSequences (CustomerID, Sequence, NextValue)
SELECT CustomerID, 'Asset', 1
FROM Customers;

DELIMITER $$
CREATE PROCEDURE GetCustomerSequenceNo (IN cid int, IN seq varchar(50), OUT val int)
BEGIN
    START TRANSACTION;

    SELECT NextValue into val 
    FROM CustomerSequences 
    WHERE CustomerID = cid and Sequence = seq 
    FOR UPDATE;
    
    UPDATE CustomerSequences 
    SET NextValue = val + 1
    WHERE CustomerID = cid and Sequence = seq;
    
    COMMIT;
END$$
DELIMITER ;

-- Example use
select * from CustomerSequences;

call GetCustomerSequenceNo(1, 'Ticket', @Ticket11ID);
select @Ticket11ID;
call GetCustomerSequenceNo(1, 'Ticket', @Ticket12ID);
select @Ticket12ID;
call GetCustomerSequenceNo(2, 'Ticket', @Ticket21ID);
select @Ticket21ID;

select * from CustomerSequences;

You acquire an ID as a separate transaction from the longer-lived transaction you may start for business purposes. This matches the behavior of AUTO_INCREMENT and may cause gaps - but in most cases that is not a problem.

If you have a small, fixed number of items to maintain separate sequences, you may use the SEQUENCE schema object instead. Doing so for hundreds or thousands of customers would be a good way to piss off some DBA's, though.

Mitch
  • 21,223
  • 6
  • 63
  • 86