0

I have the same configuration with the following link - Wordpress running on AWS EC2 micro instance with Linux AMI, so implement the answers recommended by adding swapfile. It seems not working properly still. It keeps crashing every 3 - 5 days, so I have to restart it by "sudo service mysqld restart".

Amazon EC2, mysql aborting start because InnoDB: mmap (x bytes) failed; errno 12

my configuration /etc/my.cnf looks like this:

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
# Disabling symbolic-links is recommended to prevent assorted security risks 
symbolic-links=0
# Settings user and group are ignored when systemd is used.
# If you need to run mysqld under a different user or group,
# customize your systemd unit file for mysqld according to the
# instructions in http://fedoraproject.org/wiki/Systemd

innodb_buffer_pool_size = 128M

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

And free -m looks like this.

             total       used       free     shared    buffers     cached
Mem:           996        134        861          0          2         16
-/+ buffers/cache:        116        880
Swap:         1023         40        983

I attach /var/log/mysqld.log of past few days. Appreciated to receive recommendations.

160103 04:26:52 mysqld_safe Number of processes running now: 0
160103 04:26:52 mysqld_safe mysqld restarted
160103  4:26:54 [Note] Plugin 'FEDERATED' is disabled.
160103  4:26:54 InnoDB: The InnoDB memory heap is disabled
160103  4:26:54 InnoDB: Mutexes and rw_locks use GCC atomic builtins
160103  4:26:54 InnoDB: Compressed tables use zlib 1.2.7
160103  4:26:54 InnoDB: Using Linux native AIO
160103  4:26:54 InnoDB: Initializing buffer pool, size = 128.0M 
InnoDB: mmap(137363456 bytes) failed; errno 12
160103  4:26:54 InnoDB: Completed initialization of buffer pool
160103  4:26:54 InnoDB: Fatal error: cannot allocate memory for the buffer pool
160103  4:26:54 [ERROR] Plugin 'InnoDB' init function returned error.
160103  4:26:54 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
160103  4:26:54 [ERROR] Unknown/unsupported storage engine: InnoDB
160103  4:26:54 [ERROR] Aborting

160103  4:26:54 [Note] /usr/libexec/mysqld: Shutdown complete

160103 04:26:54 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
160103 04:38:51 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
160103  4:38:51 [Note] Plugin 'FEDERATED' is disabled.
160103  4:38:51 InnoDB: The InnoDB memory heap is disabled
160103  4:38:51 InnoDB: Mutexes and rw_locks use GCC atomic builtins
160103  4:38:51 InnoDB: Compressed tables use zlib 1.2.7
160103  4:38:51 InnoDB: Using Linux native AIO
160103  4:38:51 InnoDB: Initializing buffer pool, size = 128.0M
160103  4:38:51 InnoDB: Completed initialization of buffer pool
160103  4:38:51 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
160103  4:38:51  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
160103  4:38:52  InnoDB: Waiting for the background threads to start
160103  4:38:53 InnoDB: 5.5.42 started; log sequence number 189468094
160103  4:38:53 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
160103  4:38:53 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
160103  4:38:53 [Note] Server socket created on IP: '0.0.0.0'.
160103  4:38:53 [Note] Event Scheduler: Loaded 0 events
160103  4:38:53 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.42'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL)
160103 04:40:41 mysqld_safe Number of processes running now: 0
160103 04:40:41 mysqld_safe mysqld restarted
160103  4:40:43 [Note] Plugin 'FEDERATED' is disabled.
160103  4:40:44 InnoDB: The InnoDB memory heap is disabled
160103  4:40:44 InnoDB: Mutexes and rw_locks use GCC atomic builtins
160103  4:40:44 InnoDB: Compressed tables use zlib 1.2.7
160103  4:40:44 InnoDB: Using Linux native AIO
160103  4:40:44 InnoDB: Initializing buffer pool, size = 128.0M
InnoDB: mmap(137363456 bytes) failed; errno 12
160103  4:40:44 InnoDB: Completed initialization of buffer pool
160103  4:40:44 InnoDB: Fatal error: cannot allocate memory for the buffer pool
160103  4:40:44 [ERROR] Plugin 'InnoDB' init function returned error.
160103  4:40:44 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
160103  4:40:44 [ERROR] Unknown/unsupported storage engine: InnoDB
160103  4:40:44 [ERROR] Aborting

160103  4:40:45 [Note] /usr/libexec/mysqld: Shutdown complete

160103 04:40:46 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
160103 04:42:22 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
160103  4:42:22 [Note] Plugin 'FEDERATED' is disabled.
160103  4:42:22 InnoDB: The InnoDB memory heap is disabled
160103  4:42:22 InnoDB: Mutexes and rw_locks use GCC atomic builtins
160103  4:42:22 InnoDB: Compressed tables use zlib 1.2.7
160103  4:42:22 InnoDB: Using Linux native AIO
160103  4:42:22 InnoDB: Initializing buffer pool, size = 128.0M
160103  4:42:22 InnoDB: Completed initialization of buffer pool
160103  4:42:22 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
160103  4:42:22  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
160103  4:42:22  InnoDB: Waiting for the background threads to start
160103  4:42:23 InnoDB: 5.5.42 started; log sequence number 189494130
160103  4:42:23 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
160103  4:42:23 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
160103  4:42:23 [Note] Server socket created on IP: '0.0.0.0'.
160103  4:42:23 [Note] Event Scheduler: Loaded 0 events
160103  4:42:23 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.42'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL)
160105  3:28:14 [Note] /usr/libexec/mysqld: Normal shutdown

160105  3:28:14 [Note] Event Scheduler: Purging the queue. 0 events
160105  3:28:14  InnoDB: Starting shutdown...
160105  3:28:17  InnoDB: Shutdown completed; log sequence number 204674622
160105  3:28:17 [Note] /usr/libexec/mysqld: Shutdown complete

160105 03:28:17 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
160105 03:28:19 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
160105  3:28:19 [Note] Plugin 'FEDERATED' is disabled.
160105  3:28:19 InnoDB: The InnoDB memory heap is disabled
160105  3:28:19 InnoDB: Mutexes and rw_locks use GCC atomic builtins
160105  3:28:19 InnoDB: Compressed tables use zlib 1.2.7
160105  3:28:19 InnoDB: Using Linux native AIO
160105  3:28:19 InnoDB: Initializing buffer pool, size = 128.0M
160105  3:28:19 InnoDB: Completed initialization of buffer pool
160105  3:28:19 InnoDB: highest supported file format is Barracuda.
160105  3:28:19  InnoDB: Waiting for the background threads to start
160105  3:28:20 InnoDB: 5.5.42 started; log sequence number 204674622
160105  3:28:20 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
160105  3:28:20 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
160105  3:28:20 [Note] Server socket created on IP: '0.0.0.0'.
160105  3:28:20 [Note] Event Scheduler: Loaded 0 events
160105  3:28:20 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.42'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL)
160109 09:28:55 mysqld_safe Number of processes running now: 0
160109 09:28:55 mysqld_safe mysqld restarted
160109  9:29:02 [Note] Plugin 'FEDERATED' is disabled.
160109  9:29:03 InnoDB: The InnoDB memory heap is disabled
160109  9:29:03 InnoDB: Mutexes and rw_locks use GCC atomic builtins
160109  9:29:03 InnoDB: Compressed tables use zlib 1.2.7
160109  9:29:03 InnoDB: Using Linux native AIO
160109  9:29:03 InnoDB: Initializing buffer pool, size = 128.0M
InnoDB: mmap(137363456 bytes) failed; errno 12
160109  9:29:03 InnoDB: Completed initialization of buffer pool
160109  9:29:03 InnoDB: Fatal error: cannot allocate memory for the buffer pool
160109  9:29:03 [ERROR] Plugin 'InnoDB' init function returned error.
160109  9:29:03 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
160109  9:29:03 [ERROR] Unknown/unsupported storage engine: InnoDB
160109  9:29:03 [ERROR] Aborting

160109  9:29:03 [Note] /usr/libexec/mysqld: Shutdown complete

160109 09:29:04 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
160109 12:11:23 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
160109 12:11:23 [Note] Plugin 'FEDERATED' is disabled.
160109 12:11:23 InnoDB: The InnoDB memory heap is disabled
160109 12:11:23 InnoDB: Mutexes and rw_locks use GCC atomic builtins
160109 12:11:23 InnoDB: Compressed tables use zlib 1.2.7
160109 12:11:23 InnoDB: Using Linux native AIO
160109 12:11:23 InnoDB: Initializing buffer pool, size = 128.0M
160109 12:11:23 InnoDB: Completed initialization of buffer pool
160109 12:11:23 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
160109 12:11:23  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
160109 12:11:23  InnoDB: Waiting for the background threads to start
160109 12:11:24 InnoDB: 5.5.42 started; log sequence number 209589920
160109 12:11:24 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
160109 12:11:24 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
160109 12:11:24 [Note] Server socket created on IP: '0.0.0.0'.
160109 12:11:24 [Note] Event Scheduler: Loaded 0 events
160109 12:11:24 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.42'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL)
160109 13:22:23 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
160109 13:22:24 [Note] Plugin 'FEDERATED' is disabled.
160109 13:22:24 InnoDB: The InnoDB memory heap is disabled
160109 13:22:24 InnoDB: Mutexes and rw_locks use GCC atomic builtins
160109 13:22:24 InnoDB: Compressed tables use zlib 1.2.7
160109 13:22:24 InnoDB: Using Linux native AIO
160109 13:22:24 InnoDB: Initializing buffer pool, size = 128.0M
160109 13:22:24 InnoDB: Completed initialization of buffer pool
160109 13:22:24 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
160109 13:22:24  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
160109 13:22:24  InnoDB: Waiting for the background threads to start
160109 13:22:25 InnoDB: 5.5.42 started; log sequence number 209621895
160109 13:22:25 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
160109 13:22:25 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
160109 13:22:25 [Note] Server socket created on IP: '0.0.0.0'.
160109 13:22:25 [Note] Event Scheduler: Loaded 0 events
160109 13:22:25 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.42'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL)
160109 13:22:31 [Note] /usr/libexec/mysqld: Normal shutdown

160109 13:22:31 [Note] Event Scheduler: Purging the queue. 0 events
160109 13:22:33  InnoDB: Starting shutdown...
160109 13:22:35  InnoDB: Shutdown completed; log sequence number 209634722
160109 13:22:35 [Note] /usr/libexec/mysqld: Shutdown complete

160109 13:22:36 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
160109 13:22:37 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
160109 13:22:37 [Note] Plugin 'FEDERATED' is disabled.
160109 13:22:37 InnoDB: The InnoDB memory heap is disabled
160109 13:22:37 InnoDB: Mutexes and rw_locks use GCC atomic builtins
160109 13:22:37 InnoDB: Compressed tables use zlib 1.2.7
160109 13:22:37 InnoDB: Using Linux native AIO
160109 13:22:37 InnoDB: Initializing buffer pool, size = 128.0M
160109 13:22:37 InnoDB: Completed initialization of buffer pool
160109 13:22:37 InnoDB: highest supported file format is Barracuda.
160109 13:22:37  InnoDB: Waiting for the background threads to start
160109 13:22:38 InnoDB: 5.5.42 started; log sequence number 209634722
160109 13:22:38 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
160109 13:22:38 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
160109 13:22:38 [Note] Server socket created on IP: '0.0.0.0'.
160109 13:22:38 [Note] Event Scheduler: Loaded 0 events
160109 13:22:38 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.42'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL)
160109 13:32:02 mysqld_safe Number of processes running now: 0
160109 13:32:02 mysqld_safe mysqld restarted
160109 13:32:05 [Note] Plugin 'FEDERATED' is disabled.
160109 13:32:06 InnoDB: The InnoDB memory heap is disabled
160109 13:32:06 InnoDB: Mutexes and rw_locks use GCC atomic builtins
160109 13:32:06 InnoDB: Compressed tables use zlib 1.2.7
160109 13:32:06 InnoDB: Using Linux native AIO
160109 13:32:06 InnoDB: Initializing buffer pool, size = 128.0M
InnoDB: mmap(137363456 bytes) failed; errno 12
160109 13:32:06 InnoDB: Completed initialization of buffer pool
160109 13:32:06 InnoDB: Fatal error: cannot allocate memory for the buffer pool
160109 13:32:06 [ERROR] Plugin 'InnoDB' init function returned error.
160109 13:32:06 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
160109 13:32:06 [ERROR] Unknown/unsupported storage engine: InnoDB
160109 13:32:06 [ERROR] Aborting

160109 13:32:07 [Note] /usr/libexec/mysqld: Shutdown complete

160109 13:32:08 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
160109 13:40:18 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
160109 13:40:18 [Note] Plugin 'FEDERATED' is disabled.
160109 13:40:18 InnoDB: The InnoDB memory heap is disabled
160109 13:40:18 InnoDB: Mutexes and rw_locks use GCC atomic builtins
160109 13:40:18 InnoDB: Compressed tables use zlib 1.2.7
160109 13:40:18 InnoDB: Using Linux native AIO
160109 13:40:18 InnoDB: Initializing buffer pool, size = 128.0M
160109 13:40:18 InnoDB: Completed initialization of buffer pool
160109 13:40:18 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
160109 13:40:18  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
160109 13:40:18  InnoDB: Waiting for the background threads to start
160109 13:40:19 InnoDB: 5.5.42 started; log sequence number 209690481
160109 13:40:19 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
160109 13:40:19 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
160109 13:40:19 [Note] Server socket created on IP: '0.0.0.0'.
160109 13:40:19 [Note] Event Scheduler: Loaded 0 events
160109 13:40:19 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.42'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL)
160109 13:43:42 mysqld_safe Number of processes running now: 0
160109 13:43:42 mysqld_safe mysqld restarted
160109 13:43:46 [Note] Plugin 'FEDERATED' is disabled.
160109 13:43:47 InnoDB: The InnoDB memory heap is disabled
160109 13:43:47 InnoDB: Mutexes and rw_locks use GCC atomic builtins
160109 13:43:47 InnoDB: Compressed tables use zlib 1.2.7
160109 13:43:47 InnoDB: Using Linux native AIO
160109 13:43:47 InnoDB: Initializing buffer pool, size = 128.0M
InnoDB: mmap(137363456 bytes) failed; errno 12
160109 13:43:47 InnoDB: Completed initialization of buffer pool
160109 13:43:47 InnoDB: Fatal error: cannot allocate memory for the buffer pool
160109 13:43:47 [ERROR] Plugin 'InnoDB' init function returned error.
160109 13:43:47 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
160109 13:43:47 [ERROR] Unknown/unsupported storage engine: InnoDB
160109 13:43:47 [ERROR] Aborting

160109 13:43:47 [Note] /usr/libexec/mysqld: Shutdown complete

160109 13:43:49 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
Community
  • 1
  • 1
Marconi
  • 3
  • 4
  • 1
    This is a very common issue, caused not by MySQL, but actually by *another process* -- usually the web server -- hogging memory on a machine that does not have sufficient memory for both, under load. http://dba.stackexchange.com/questions/25077/mysql-innodb-crash-post-mortem – Michael - sqlbot Jan 10 '16 at 15:01
  • Thanks, I will look into that and check the other process by checking syslog. However, Amazon EC2 is very unstable these days, even cannot log on (based in Taiwan, AWS server in Tokyo.). Take time and patience to get it connected to check. – Marconi Jan 11 '16 at 04:54
  • I'm sorry, but if you are claiming that EC2 is inherently unstable, that is a rather extraordinary claim. If *your* EC2 instance is unstable... well, yes, you have a very small machine with insufficient resources for the workload you are offering it. – Michael - sqlbot Jan 11 '16 at 09:44
  • 1
    Yes, it is EC2 micro instance. I use it only for personal web hosting, and not big traffic, so should not be a big issue. Further search and found a good article "Making WordPress Stable on EC2-Micro" It seems working fine after tuning Apache and limiting MySQL memory. Thanks a lot !!! http://www.frameloss.org/2011/11/04/making-wordpress-stable-on-ec2-micro/ – Marconi Jan 11 '16 at 10:10

0 Answers0