MariaDB with MyRocks engine: Many huge files after insertions

Question

I am using MariaDB with MyRocks engine. After short time, I try to insert bunch of data into some tables (tables could be InnoDB and MyRocks engine), my disk grows very fast. I check MariaDB data directory and I see this:

drwxr-x--x 2 mysql mysql  12K Jun 26 12:07 #rocksdb
drwxr-xr-x 7 mysql mysql 4.0K Jun 26 12:07 .
drwxr-xr-x 6 root  root  4.0K Jun 21 10:22 ..
-rw-rw---- 1 mysql mysql 1.1G Jun 25 21:39 1.000001
-rw-rw---- 1 mysql mysql 1.1G Jun 25 21:40 1.000002
-rw-rw---- 1 mysql mysql 1.6G Jun 25 21:44 1.000003
-rw-rw---- 1 mysql mysql 1.1G Jun 25 21:48 1.000004
-rw-rw---- 1 mysql mysql 1.1G Jun 25 21:50 1.000005
-rw-rw---- 1 mysql mysql 218M Jun 26 06:53 1.000006
-rw-rw---- 1 mysql mysql  357 Jun 26 06:56 1.000007
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:06 1.000008
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:09 1.000009
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:17 1.000010
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:19 1.000011
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:22 1.000012
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:24 1.000013
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:27 1.000014
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:29 1.000015
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:32 1.000016
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:34 1.000017
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:37 1.000018
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:40 1.000019
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:42 1.000020
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:45 1.000021
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:51 1.000022
-rw-rw---- 1 mysql mysql 1.1G Jun 26 08:00 1.000023
-rw-rw---- 1 mysql mysql 1.1G Jun 26 08:07 1.000024
-rw-rw---- 1 mysql mysql 1.1G Jun 26 08:15 1.000025
...

I see many files whose name contains only numbers. I don't know those files were created by MyRocks or MySQL, and can I delete them?

score 1 · Accepted Answer · answered Jun 27 '19 at 01:06

A guess: You configured the following, assuming it turned on binlogging:

log_bin = 1

and those are binlogs starting with the '1'.

A new file every few minutes implies a huge amount of writes (such as the inserts you mention).

If you don't need binlogging and change to this and restart, then the files should stop being created:

log_bin = OFF

The files (1.*) can be removed.

If they don't stop, then my guess is wrong.

If you do need binlogging (for replication and/or point-in-time backups, then we need to investigate the huge rapidity of them.

I'll bet you did a restart about Jun 26 06:53.

Rocksdb is probably not relevant to the question. (But I could be wrong.)

Thanks so much. I have tested and as you said, this is because binary log. I have another question related to mysql, about custom data directory. Can you help me? Thanks. https://stackoverflow.com/questions/56797063/mariadb-custom-data-directory-always-need-to-keep-var-lib-mysql-directory — Trần Kim Dự, Jun 27 '19 at 18:45

MariaDB with MyRocks engine: Many huge files after insertions

1 Answers1