create temporary postgres server in docker

Question

I want to unit-test my app which uses a postgres database inside a docker.

EDIT: based on the suggested answer I modified the Dockerfile:

FROM postgres
USER postgres
RUN sleep 2 # remark 1
RUN initdb  # remark 2
RUN sleep 3 # remark 1
RUN psql --host=localhost -l

The remarks are:

from this reference

Try putting a sleep in there and see if it's still a problem

from the docs:

The default postgres user and database are created in the entrypoint with initdb.

Here is the Dockerfile from the original question:

FROM postgres
COPY input.json .
RUN createdb -h localhost -p 7654 -U moish myLovelyAndTemporaryDb
#
# [ 1 ] run application on input.json
# [ 2 ] check db content after run
#

When I use the above Dockerfile I seem to be missing something: (The errors from the edited version are the same)

$ docker build --tag host --file Dockerfile .
[+] Building 0.3s (7/7) FINISHED                                                                                                       
 => [internal] load build definition from Dockerfile                                                                              0.0s
 => => transferring dockerfile: 125B                                                                                              0.0s
 => [internal] load .dockerignore                                                                                                 0.0s
 => => transferring context: 2B                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/postgres:latest                                                                0.0s
 => [internal] load build context                                                                                                 0.0s
 => => transferring context: 40B                                                                                                  0.0s
 => CACHED [1/3] FROM docker.io/library/postgres                                                                                  0.0s
 => [2/3] COPY input.json .                                                                                                       0.0s
 => ERROR [3/3] RUN createdb -h localhost -p 7654 -U moish myLovelyAndTemporaryDb                                                                0.2s
------
 > [3/3] RUN createdb -h localhost -p 7654 -U moish myLovelyAndTemporaryDb:
#7 0.188 createdb: error: connection to server at "localhost" (127.0.0.1), port 7654 failed: Connection refused
#7 0.188    Is the server running on that host and accepting TCP/IP connections?
#7 0.188 connection to server at "localhost" (::1), port 7654 failed: Cannot assign requested address
#7 0.188    Is the server running on that host and accepting TCP/IP connections?
------

A Dockerfile normally constructs a reusable image that you can run containers from; it's not intended as a general-purpose script runner. In the context of a Dockerfile `RUN` statement, the normal main container process won't be running, and you can't connect to other services. Do you want to run this _integration_ (not unit) test from somewhere else, maybe a test framework entirely outside of Docker? — David Maze, Jan 05 '23 at 15:31
@DavidMaze how would you run a CI/CD pipeline that needs to test such a thing? a docker image isn't *always* reusable ... — OrenIshShalom, Jan 05 '23 at 16:18
I'd write that logic in, say, a `Jenkinsfile` that plugs into the CI system. — David Maze, Jan 05 '23 at 17:34
@OrenIshShalom It either looks like you are running behind a firewall which block the connection, or the server is actually not running .. — Jib, Jan 08 '23 at 13:13
@Jib I am trying to access the postgres server *from within the docker itself* — OrenIshShalom, Jan 08 '23 at 13:55
@OrenIshShalom could you try stating the postgres service? Is it said to be running? If so, try connect to it using telnet or some simple tool. How does it goes? — Jib, Jan 08 '23 at 14:12
@Jib you mean just do `initdb` and try to connect to it from the host? — OrenIshShalom, Jan 08 '23 at 16:05

mello · Answer 1 · 2023-01-05T16:39:13.230

1

During the build step of the postgres Docker you cannot run postgres commands. Postgres database will only be available after you run the Docker. As specified in the postgres Docker documentation you can add customization to your postgres instance through scripts placed in docker-entrypoint-initdb.d directory.

Alternatively you could use a RUN directive to start the postgres database and after that run the postgres commands you want (making sure to wait for the DB to accept connections), as mentioned here.

Another side note, I personally avoid using real databases for unit testing applications. To me, it's always better to mock the database for unit tests, in python you can do this with unittest.mock.

edited Jan 05 '23 at 16:39

answered Jan 05 '23 at 15:14

mello

114
4

I edited the question and tried multiple ways - none worked. I keep getting the same error – OrenIshShalom Jan 08 '23 at 07:17
This is the proper answer to the question, use docker-entrypoint-initdb.d scripts or switch to proper in memory DB for testing. – hakamairi Jan 10 '23 at 11:01

Slava Kuravsky · Accepted Answer · 2023-01-11T09:03:49.267

Postgres database starts only after you create a container based on postgres image. docker build process doesn't start entrypoint script. You might need a bash script or CI pipeline where you firstly start postgres container and then use it in your unit tests

$ docker run --name mypg -p 5432:5432 -e POSTGRES_PASSWORD=mypgpass -d postgres:9
# copy a script to the mypg container
$ docker cp run.sh mypg:/root/run.sh
# run the script
$ docker exec mypg bash /root/run.sh
...
# use postgres client on your host to connect to mypg container
$ PGPASSWORD="mypgpass" psql -U postgres -p 5432 -h localhost -c "select version()"
                                                               version
--------------------------------------------------------------------------------------------------------------------------------------
 PostgreSQL 9.6.24 on x86_64-pc-linux-gnu (Debian 9.6.24-1.pgdg90+1), compiled by gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516, 64-bit
(1 row)

Postgres container docs

Postgres client authentication

EDIT:

By trying to run initdb, psql etc directly in Dockerfile, you are reinventing the docker-entrypoint.sh

I want to connect to the dockerized postgres from within the docker image, *not* from the host. — OrenIshShalom, Jan 10 '23 at 11:34
You can connect to the dockerized postgres (container A) either from a none-dockerized environment (your host system, remote host) or from within another docker container B with you unit test runner. `docker build` command creates an image, `docker run` command starts containers based on an image. — Slava Kuravsky, Jan 10 '23 at 11:48
what about using `docker build` up to `initdb`, then to the rest inside some script that runs in a consecutive `docker run`? that should work, right? — OrenIshShalom, Jan 10 '23 at 18:39

score 0 · Answer 3 · answered Jan 11 '23 at 03:14

Here is a complete answer based on the concepts of slava-kuravsky and mello:

$ docker build --tag host --file Dockerfile .
$ docker run -d -t --name pg host
$ docker exec pg bash run.sh

The script can do what I want, currently it only lists the databases:

$ cat run.sh # <--- copied during docker build                                 
echo "Hello Postgres World"
psql --host=localhost -l

The Dockerfile does only initialization:

$ cat Dockerfile 
FROM postgres
USER postgres
COPY run.sh . # <--- the testing script
RUN sleep 2   # <--- probably not needed anymore
RUN initdb
RUN sleep 3   # <--- probably not needed anymore

When I perform the three commands above I get what I expect :

$ docker build --tag host --file Dockerfile .
[+] Building 7.2s (10/10) FINISHED                                                                                                     
# ... omitted for brevity ...
$ docker run -d -t --name pg host       
608ac7324e838924c9a5d0cfe65c8000e33350b86faf9df4511ef5fcf7440597
$ docker exec pg bash run.sh                 
Hello Postgres World
                                                List of databases
   Name    |  Owner   | Encoding |  Collate   |   Ctype    | ICU Locale | Locale Provider |   Access privileges   
-----------+----------+----------+------------+------------+------------+-----------------+-----------------------
 postgres  | postgres | UTF8     | en_US.utf8 | en_US.utf8 |            | libc            | 
 template0 | postgres | UTF8     | en_US.utf8 | en_US.utf8 |            | libc            | =c/postgres          +
           |          |          |            |            |            |                 | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.utf8 | en_US.utf8 |            | libc            | =c/postgres          +
           |          |          |            |            |            |                 | postgres=CTc/postgres
(3 rows)

You don't need initdb in your dockerfile, initdb is a part of entrypoint of postgres image and will be executed as soon you start a pg container — Slava Kuravsky, Jan 11 '23 at 07:50
Actually you can copy run.sh from the host directly to the running container `pg`: `docker cp run.sh pg:/root/run.sh`. So you don't need your dockerfile at all. — Slava Kuravsky, Jan 11 '23 at 07:54
@SlavaKuravsky would you like to edit your original answer to have the 1 liner you just suggested? I'd be happy to accept that ! — OrenIshShalom, Jan 11 '23 at 08:44

create temporary postgres server in docker

3 Answers3