1

We are running docker 1.7.1, build 786b29d on RHEL 6.7. Recently we have had multiple times when the docker daemon locked up and we had to reboot the machine to get it back.

A typical scenario is that a container that has been running fine for weeks suddenly starts throwing errors. Sometime we can restart the container and all is well. But other times all docker commands will hang, and restarting the daemon fails, and I see this in a ps:

4 Z root 4895 1 0 80 0 - 0 exit Aug23 ? 00:01:24 [docker]

Looking in the system log I've seen this:

device-mapper: ioctl: unable to remove open device docker-253:6-1048578-317bb6ad40cded3fbfd752d95551861c2e4ef08dffc1186853fea0e85da6b12b
INFO: task docker:16676 blocked for more than 120 seconds.
Not tainted 2.6.32-573.12.1.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
docker D 000000000000000b 0 16676 1 0x00000080
ffff88035ef13ea8 0000000000000082 ffff88035ef13e70 ffff88035ef13e6c
ffff88035ef13e28 ffff88062fc29a00 0000376c85170937 ffff8800283759c0
0000000000000400 00000001039d40c7 ffff8803000445f8 ffff88035ef13fd8
Call Trace:
[] _mutexlock_slowpath+0x96/0x210
[] ? wake_up_process+0x15/0x20
[] mutex_lock+0x2b/0x50
[] sync_filesystems+0x26/0x150
[] sys_sync+0x17/0x40
[] system_call_fastpath+0x16/0x1b

The latest docker version is 1.12.1 and we are on 1.7.1. Can or should I install a new version? 1.7.1 is the version yum installs. If I did want a new version how would I install that (sorry if that is a dumb question, I am not a sys admin).

Googling, I found on this on a Red Hat site "Red Hat does not recommend running any version of Docker on any RHEL 6 releases." We have been running docker on RHEL 6 for a few years, so this confuses me. Upgrading to RHEL 7 is not really an option for us right now.

Can anyone shed any light on these issue? We need docker to work reliably without having to reboot often.

Larry Martell
  • 3,526
  • 6
  • 40
  • 76

2 Answers2

1

Docker 1.7.1 is really old by today's standards. There have been hundreds of bugs fixed, enhancements to driver stacks, security patches, and valuable features added in the versions since. It looks like you're having a issue with your storage stack, and there is a good chance this is fixed in a newer version.

Docker has stated that default versions in package management systems like yum and apt can be way out of date, and that you should use their repo. The best way to do this is add their Yum repo information to your system so you can install it like other packages. The instructions are here: Installation on Red Hat Enterprise Linux.

Note: This will allow you to install Docker, and the service will be called docker, but the package is docker-engine. This has confused some people in the past.

yum install docker-engine

Docker has also provided a script that does this to make things easier (run as admin/root):

curl -fsSL https://get.docker.com/ | sh
DevOps Dan
  • 1,724
  • 11
  • 11
  • Thanks so much for the info. Can I install the latest and greatest on RHEL 6, or will only up to a certain version run there? – Larry Martell Aug 29 '16 at 05:03
  • 1
    [RHEL6 support was dropped from Docker 1.8](http://stackoverflow.com/a/37450607/1318694) – Matt Aug 29 '16 at 05:35
1

Don't use a RHEL6 based system.

RHEL6 uses a 2.6 kernel with backported fixes to keep Docker working. Docker would normally require a 3.10+ kernel. Docker dropped support for RHEL6 from v1.8 on so it's unlikely there will be any more packages for it.

If you must use RHEL6, don't use the default loopback devicemapper for storage. Setup an LVM thin pool for Docker to use.

Community
  • 1
  • 1
Matt
  • 68,711
  • 7
  • 155
  • 158