1

Hello I have a table created by the following query MariaDB version 10.5.9

CREATE TABLE `test` (
  `id` int unsigned NOT NULL AUTO_INCREMENT,
  `status` varchar(60) DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `test_status_IDX` (`status`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8mb4 

I always thought that the primary key is by default the clustered index which also defines the order of the rows in the table but here it seems that the index on the status is picked as the clustered. Why is this happening and how can I change it?

MariaDB [test]> select * from test;
+----+--------+
| id | status |
+----+--------+
|  2 | cfrc   |
|  5 | hjr    |
|  1 | or     |
|  3 | test   |
|  6 | verve  |
|  4 | yes    |
+----+--------+
6 rows in set (0.001 sec)
TomDim
  • 55
  • 10

4 Answers4

3

It is not safe to assume that the results of SELECT will be ordered by any column across dB engines. You should always use ORDER BY col [ASC|DESC] if you expect sorting to happen. I see records being displayed in the order they were added, but that can change after deletions/insertions etc, and should not be relied on. See here for more details.

Andrej Prsa
  • 551
  • 3
  • 14
3

(I am going to cite MySQL docs in my answer but in the context of this question, the information applies to MariaDB as well.)

First of all, let's talk about index extensions. The InnoDB engine automatically creates an additional (composite) index behind the scenes whenever you define a secondary index (i.e. any index that is not the clustered index). That is called an index extension. This extra index contains the columns you defined in your original secondary index (in the same order) with the columns of the primary key added after them. So, in your example, InnoDB creates an index extension for test_status_IDX (let's call it X), with columns (status, id).

Now let's look at the query select * from test;. There is no WHERE clause here, so all the optimizer needs to do to satisfy this query is fetch all columns for all rows of the table. This boils down to fetching status & id since there are no other columns in the table. These exact fields happen to be stored within the extended index X. This makes index X a covering index for this query. A covering index is an index that, given a query, can fully produce the results of the query without having to read any actual data rows. Therefore, the optimizer reads & returns the values needed for the result of the query from index X, in the order that they appear there, which is by status, hence the order you observed.

To further demonstrate and extend (pun intended) this point, let's reproduce the example (tested with MariaDB 10.4):

1. First create the table & add the rows

CREATE TABLE foo (
  id int(10) unsigned NOT NULL AUTO_INCREMENT,
  status varchar(60) DEFAULT NULL,
  PRIMARY KEY (id)
) ENGINE=InnoDB;

INSERT INTO foo VALUES
(1, 'or'),
(2, 'cfrc'),
(3, 'test'),
(4, 'yes'),
(5, 'hjr'),
(6, 'verve');

SELECT * FROM foo;
+----+--------+
| id | status |
+----+--------+
|  1 | or     |
|  2 | cfrc   |
|  3 | test   |
|  4 | yes    |
|  5 | hjr    |
|  6 | verve  |
+----+--------+`

2. Now let's add the secondary index and check the order again

CREATE INDEX secondary_idx ON foo (status);

SELECT * FROM foo;
+----+--------+
| id | status |
+----+--------+
|  2 | cfrc   |
|  5 | hjr    |
|  1 | or     |
|  3 | test   |
|  6 | verve  |
|  4 | yes    |
+----+--------+

As described above, the rows are returned in the order they appear in the (extended) secondary_idx

3. Now let's drop the index and re-add it with a prefix length of 2 bytes. This means that the index will not store the full value of the column but only its first two bytes, which means the extended index is no longer a covering index because it cannot fully produce the results of the query. Thus the clustered index will be used

ALTER TABLE foo DROP INDEX secondary_idx;

CREATE INDEX secondary_idx ON foo (status(2));

SELECT * FROM foo;
+----+--------+
| id | status |
+----+--------+
|  1 | or     |
|  2 | cfrc   |
|  3 | test   |
|  4 | yes    |
|  5 | hjr    |
|  6 | verve  |
+----+--------+

4. Let's showcase this behaviour in another way. Here we will retain the original secondary index (without a prefix length) but we will add a 3rd column to the table. This will once again render the secondary index a non covering index (because it does not contain the 3rd column), therefore, the clustered index will be used here as well.

ALTER TABLE foo DROP INDEX secondary_idx;

CREATE INDEX secondary_idx ON foo (status);

ALTER TABLE foo ADD bar integer NOT NULL;

SELECT * FROM foo;
+----+--------+-----+
| id | status | bar |
+----+--------+-----+
|  1 | or     |   0 |
|  2 | cfrc   |   0 |
|  3 | test   |   0 |
|  4 | yes    |   0 |
|  5 | hjr    |   0 |
|  6 | verve  |   0 |
+----+--------+-----+

Adding bar to the index (or dropping it from the table) will again make the query use the secondary index.

ALTER TABLE foo DROP INDEX secondary_idx;

CREATE INDEX secondary_idx ON foo (status, bar);

SELECT * FROM foo;
+----+--------+-----+
| id | status | bar |
+----+--------+-----+
|  2 | cfrc   |   0 |
|  5 | hjr    |   0 |
|  1 | or     |   0 |
|  3 | test   |   0 |
|  6 | verve  |   0 |
|  4 | yes    |   0 |
+----+--------+-----+

You can also use EXPLAIN on all of the SELECT statements above to see which index is used at each stage.

O8eiosSam
  • 78
  • 1
  • 9
1

@aprsa is right I falsely assumed that the results will be in the same order as the clustered index but in this case(using INNODB) the status index is used for the query's evaluation so that's why it appears to be 'sorted' by the status. If I select the id then the primary index is used and the results appear to be 'sorted' by the id. In another engine this might not be true.

TomDim
  • 55
  • 10
1

That particular table is composed of 2 BTrees:

  • The data, sorted by the PRIMARY KEY. Yes, it is clustered and is ordered 1,2,3,...

  • The secondary index, sorted by status. Each secondary index contains a copy of the PK so that it can reach into the other BTree to get the rest of the columns (not that there are any more!). That is, the is BTree is equivalent to a 2-column table with PRIMARY KEY(status) plus an id.

Note how the output is in status order. I have to assume it decided to simply read the secondary index in its order to provide the results.

Yes, you must specify an ORDER BY if you want a particular ordering. You must not assume the details I just discussed. Who knows, tomorrow there may be something else going, such as an in-memory "hash" that has the information scrambled in some other way!

(This Answer applies to both MySQL and MariaDB. However, MariaDB is already playing a game with hashing that MySQL has not yet picked up. Be warned! Or simply add an ORDER BY.)

Rick James
  • 135,179
  • 13
  • 127
  • 222