1

I have noticed a pattern that is easily reproducible but I can not answer why for the life of me its happening. For some reason if I use the command docker-compose kill (to kill all the containers), and then start it back up again with docker-compose -d my web container gets stuck into an infinite loop of restarting. I have looked all over the internet for answers on this and could not find anything.

Looks something like below which shows the steps to see what is happening.

enter image description here

Please refer to this post I made (Docker-Compose won't volume my php.ini file) for my complete docker setup that is used here in the exact same way, but I'll attach my docker-compose file here for quick reference.

docker-compose.yml

version: '2'
services:
    dblive:
        image: mysql:5.5.52
        volumes:
            - ./db_data_live:/var/lib/mysql
        restart: always
        environment:
            MYSQL_ROOT_PASSWORD: ****
            MYSQL_DATABASE: ****
            MYSQL_USER: ****
            MYSQL_PASSWORD: ****

    dbdev:
        image: mysql:5.5.52
        volumes:
            - ./db_data_dev:/var/lib/mysql
        restart: always
        environment:
            MYSQL_ROOT_PASSWORD:****
            MYSQL_DATABASE: ****
            MYSQL_USER: ****
            MYSQL_PASSWORD: ****

    phpmyadmin:
        depends_on:
            - dblive
            - dbdev
        image: phpmyadmin/phpmyadmin
        environment:
            PMA_ARBITRARY : 1
        restart: always
        ports:
            - "8081:80"

    web:
        build: ./
        depends_on:
            - dblive
            - dbdev
        volumes:
            - ./web:/var/www
            - ./config/custom.php.ini:/etc/php5/apache2/conf.d/custom.php.ini
            - ./logs/apache_error.log:/var/log/apache2/error.log
            - ./logs/apache_access.log:/var/log/apache2/access.log
            - ./config/apache_default.conf:/etc/apache2/sites-enabled/000-default.conf
        restart: always
        ports: 
            - "80:80"
            - "443:443"

Steps to Reproduce

  1. Refer to my linked post here to get my complete setup to follow along.
  2. Run docker-compose up -d. This will start the containers.
  3. Run docker ps to verify they are all up and have been for 15 seconds or more.
  4. Run docker-compose kill to stop the containers. Run docker-compose up -d to start the containers again, and follow up with a docker ps every 5 seconds to notice the consistent restarting on the web container. Feel free to kill and restart many times and the restarting will keep happening.
  5. Interestingly enough, comment out a line in the docker-compose.yml file. In my case I commented out the voluming of the php.ini file like this, #- ./config/custom.php.ini:/etc/php5/apache2/conf.d/custom.php.ini
  6. Run docker-compose kill to stop the containers again. Then run docker-compose up -d and this time the container will magically work without restarting??

My question is why does changing the docker-compose.yml file slightly and then starting up cause it to work now? If I kill the container for any reason after this and try to start up again without making a change in the docker-compose.yml file then the restart loop occurs on the web container. Why is it doing this?

Attached below is an image showing it working after the previous restarts from the first image I attached. In the image please ignore and excuse the ERROR: yaml.scanner.ScannerError: I forgot I had XX'ed out my MySQL passwords so make sure you just put something there to make it happy. The content after the error shows that it suddenly starts working again.

enter image description here

Community
  • 1
  • 1
Joseph Astrahan
  • 8,659
  • 12
  • 83
  • 154
  • 1
    What does `docker logs` and `docker inspect` show for the restarting container? – BMitch Jan 18 '17 at 01:58
  • Can you give me an example how to use those commands? At what point would you like me to use the commands also? (For example when its restarting, after its restarting etc...) – Joseph Astrahan Jan 18 '17 at 02:14
  • I also want to note that just recently I discovered if I do docker-compose down I don't seem to have the restart issue that occurs with docker-compose kill. Maybe I just should not be using kill?? – Joseph Astrahan Jan 18 '17 at 02:15
  • The commands take the container name as an argument (if you run without, it will print a usage). Output when the container is currently down and waiting for it's timeout to restart is fine. – BMitch Jan 18 '17 at 02:16
  • 1
    Do you have any Dockerfile involved for web? I know every step in a Dockerfile is a layer itself and is cached. So maybe docker-compose.yml instructions are also cached (I don't know) and as soon as you touch something that layer recreates and the following layers also do. Not sure if I can satisfy your questions, but just to try thing you may run "docker-compose up --force-recreate" – mayid Jan 18 '17 at 02:17
  • Yes the Dockerfile is the other question, please refer to the link, I just didn't have enough space to add everything in the question. Regarding BMitch I will run the command now to see what it shows. – Joseph Astrahan Jan 18 '17 at 03:40
  • I just got what you were saying mayid, you might be right about the cache of results since dockerfiles do that, I will try your command also. I recently discovered that docker-compose down seems to work also, maybe that gracefully brings them down removing cache? – Joseph Astrahan Jan 18 '17 at 03:49
  • You were correct Mayid that was the error, basically I need to use docker-compose down instead to get rid of the cached older copies. This was causing the weird random restarts. You can post that as answer. – Joseph Astrahan Jan 18 '17 at 04:17
  • I think it's because you set restart policy to `always`. When container got killed, docker attempt to bring it up. – jsxqf Jan 19 '17 at 08:32

1 Answers1

3

The answer is to not use run docker-compose kill but use docker-compose down instead because docker-compose kill only kills the containers but does not clear the cache that docker-compose makes for you.

Joseph Astrahan
  • 8,659
  • 12
  • 83
  • 154
  • 2
    Thank you sir!, spent a lot of time, even tried using different versions of mysql/mariadb and it just kept restarting...then I tried using docker run separately and it worked properly, but docker-compose was not working at all...in case someone else is going through the same, this worked! – Benjamin Vison Sep 30 '17 at 23:39