1

Using docker to deploy a uvicorn server to serve some tensorflow model. The end of the dockerfile looks like this.

# Start ASGI server
CMD ['./runserver.sh']

The runserver.sh looks like this

#!/usr/bin/env bash
# encoding:utf-8
# This is a blocking call
uvicorn gateway:app --host=0.0.0.0 --workers 20 # Default port 8000

This is the command I am using to start the container

docker run --detach --publish 8000:8000 tensor_image

My expectation and goal: The container will stay alive until killed with a docker stop command and a client can send request to the uvicorn server.

What is happening: The docker run command just echoes a long id of the container on terminal and then the container dies.

How to keep it running? Also, how to view the server log itself if I make uvicorn log its content to a local file inside the container?

Using Linux mint ulyana as my operating system if that is important. Some additional clari

Della
  • 1,264
  • 2
  • 15
  • 32
  • Your Dockerfile is incomplete. Please see [mcve]. – BMitch Nov 18 '21 at 11:40
  • 2
    What does it say if you remove the `--detach` option? If your `CMD` is exactly what you show, I suspect things are going wrong because you don't have correct JSON-array syntax; does changing to double quotes `CMD ["./runserver.sh"]` fix it? – David Maze Nov 18 '21 at 11:44

2 Answers2

1

Your problem has to do with the way docker runs your containers.

The main process your container will run depends on the combination of two Dockerfile instructions, ENTRYPOINT and CMD.

In a nutshell - please, consider review for instance this related SO question for a deeper insight - ENTRYPOINT defines the command that will be run when your container starts. The default ENTRYPOINT command is /bin/sh -c and it cannot be overwritten by the docker command line.

On the other hand, CMD could be consider as a complement, arguments, that will be appended to the provided ENTRYPOINT. It will be overwritten in case you provide some arguments when running your container.

With this background, I think you could run your application in several ways.

For instance, by making your Dockerfile ENTRYPOINT the actual shell script you want to run:

ENTRYPOINT ["./runserver.sh"]

Other, by using CMD but modifying your runserver.sh script and run uvicorn with exec:

#!/usr/bin/env bash
# encoding:utf-8
# To run in production with multiple slaves
exec uvicorn gateway:app --host=0.0.0.0 --workers 20 # Default port 8000

exec triggers the process running the script to be replaced with the command given, instead of starting it as a new process, which is the default behavior when running a shell command.

You can use CMD and provide the uvicorn command directly as well - it should be visible for your PATH variable:

CMD ["uvicorn", "gateway:app", "--host", "0.0.0.0", "--workers", "20"]

Performing any of these changes will ensure that the container is not terminated early.

jccampanero
  • 50,989
  • 3
  • 20
  • 49
  • Thanks for the answer, and I will try this. I admit my conceptual gaps in terms of a container's lifecycle, especially why the container is stopping when the ```runserver.sh``` is a blocking script (because of its last and only command). Also, what do you mean by the _main process_ of a container? Is it the process with pid=0, i.e. systemd? – Della Nov 19 '21 at 01:52
  • 1
    Sorry for the late reply @Della. You are welcome. Were you able to try it? No please, do not get confused with pids, systemd, etcetera. I am sorry if I didn't explain myself very well. I updated the answer trying explaining what I wanted to mean and trying providing another option. Please, could you review it? I hope it helps. – jccampanero Nov 19 '21 at 19:55
  • Thanks. Actually, making it exec did not help, the problem was the single quote as another comment to the original question said. In python, I barely notice whether a string literal is single or double quoted, so I am carrying over the bad habit to other languages as well. Gotta be careful. – Della Nov 20 '21 at 04:58
  • Thank you very much for the feedback @Della. I am glad to see that the problem was solved. I am sorry, I saw the comment, but I thought it was something related to the way you wrote the question, and not the actual code. In any way, I hope the answer provides you some useful advice regarding the use of docker. – jccampanero Nov 20 '21 at 18:38
0

In order to detach a container you have to run for CMD statement in blocking mode.

In your case, which I cannot reproduce, you should run uvicorn not as a daemon, but as a blocking application.

Antonio Petricca
  • 8,891
  • 5
  • 36
  • 74
  • Uvicorn is a blocking application, i.e. when I run it from terminal with the command I shared, it fires up the workers, starts listening to port 8000 and blocks the terminal waiting for requests. Any reason why it would not be blocking inside the container, to keep the container alive? – Della Nov 19 '21 at 01:39