I'm trying to build a Laravel application (LEMP Stack) in Docker Swarm. I'm using HaProxy, as described in Dockers networking documentation on ingress networks, to load balance from the manager nodes public IPV4 between all worker nodes.
My current configuration looks like so:
frontend http_front
bind *:80
stats uri /haproxy?stats
default_backend http_back
frontend http_front_ssl
bind *:443
stats uri /haproxy?stats
default_backend http_back_ssl
backend http_back
balance roundrobin
server vps1 xx.xx.xxx.xxx:8080 check
server vps2 xx.xx.xxx.xxx:8080 check
server vps3 xx.xx.xxx.xxx:8080 check
backend http_back_ssl
balance roundrobin
server vps1 xx.xx.xxx.xxx:4043 check
server vps2 xx.xx.xxx.xxx:4043 check
server vps3 xx.xx.xxx.xxx:4043 check
I have then a custom image that provides and exposes a PHP-FPM on port 9000:
docker service create \
--name php-fpm \
--replicas=3 \
--network app-net \
--secret source=proxy.conf,target=/etc/nginx/conf.d/site.conf \
--secret site.key \
--secret site.crt \
--mount type=volume,src=web,dst=/var/www \
jaquarh/php-fpm:alpha
The keys are for self-signed SSL (testing Https). The Dockerfile to build this is:
FROM php:7.4-fpm
COPY laravel/composer.lock laravel/composer.json /var/www/
WORKDIR /var/www
RUN apt-get update && apt-get install -y \
build-essential \
libpng-dev \
libjpeg62-turbo-dev \
libfreetype6-dev \
locales \
zip \
jpegoptim optipng pngquant gifsicle \
vim \
unzip \
git \
curl
RUN apt-get clean && rm -rf /var/lib/apt/lists/*
RUN docker-php-ext-install pdo_mysql
RUN docker-php-ext-configure gd --with-freetype --with-jpeg
RUN docker-php-ext-install gd
RUN pecl install memcached; \
docker-php-ext-install intl; \
docker-php-ext-install exif; \
echo "extension=memcached.so" >> /usr/local/etc/php/conf.d/memcached.ini; \
docker-php-ext-enable exif;
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
RUN groupadd -g 1000 www
RUN useradd -u 1000 -ms /bin/bash -g www www
COPY . /var/www
COPY --chown=www:www . /var/www
USER www
EXPOSE 9000
CMD ["php-fpm"]
I am then building an nginx service as the proxy for the php-fpm service.
docker service create \
--name nginx \
--secret site.key \
--secret site.crt \
--secret source=site.conf,target=/etc/nginx/conf.d/site.conf \
-p 8080:80 \
-p 4043:443 \
--replicas=3 \
--network app-net \
--mount type=volume,src=web,dst=/var/www \
nginx:latest \
sh -c "exec nginx -g 'daemon off;'"
Inside of my web volume, I hold all the laravel files. I am only testing Http at the moment so my nginx configuration looks like so:
server {
listen 80;
listen [::]:80;
server_name example.co.uk; # this is my actual domain for testing
index index.php index.html;
error_log /var/log/nginx/error.log;
access_log /var/log/nginx/access.log;
root /var/www/public;
add_header X-Frame-Options "SAMEORIGIN";
add_header X-XSS-Protection "1; mode=block";
add_header X-Content-Type-Options "nosniff";
charset utf-8;
location / {
try_files $uri $uri/ /index.php?$query_string;
gzip_static on;
}
location = /favicon.ico {
access_log off;
log_not_found off;
}
location = /robots.txt {
access_log off;
log_not_found off;
}
error_page 404 /index.php;
location ~ \.php$ {
try_files $uri =404;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass php-fpm:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;
include fastcgi_params;
}
# ready for when letsencrypt is introduced
location ~ /.well-known/acme-challenge {
allow all;
root /var/www/html;
}
}
Finally, I have a pre-built SQL database volume and service ready to use which the laravel .env uses.
docker service create \
--name mysql \
--replicas=3 \
--network app-net \
--secret site.key \
--secret site.crt \
--mount type=volume,src=dbdata,target=/var/lib/mysql/ \
--env MYSQL_DATABASE=xxxxxx \
--env MYSQL_ROOT_PASSWORD=xxxxxx \
--env MYSQL_USER=xxxxx \
--env MYSQL_PASSWORD=xxxxx \
-p 3306:3306 \
mysql:latest
The issue I am facing is that VPS1 (in this case the manager) gets these files perfectly fine because the volumes exist in the manager VPS. Both VPS2 and VPS3 show these volumes have been mounted correctly but are not copying the data over.
As seen in the images, on my development pc (which is also a worker for testing) I can see the mount has been successfully created and my docker is assigned the nginx container however, when I inspect the directories in bash, only the directory housing the laravel application exists in the manager node.
What am I doing wrong? How can I copy this volume across my worker nodes? If I inspect the volume on the manager it looks like this:
ubuntu@vps-438f3f94:~$ docker volume inspect web
[
{
"CreatedAt": "2021-04-23T09:29:23Z",
"Driver": "local",
"Labels": {},
"Mountpoint": "/var/lib/docker/volumes/web/_data",
"Name": "web",
"Options": {},
"Scope": "local"
}
]
Update: If I look at the HaProxy stats, I can see my development PC is DOWN because I'm not port forwarded (as its just for me to debug containers) but I'm assuming this has no real effect on volumes but just to give you an insight to my environment better.
http_back
Queue Session rate Sessions Bytes Denied Errors Warnings Server
Cur Max Limit Cur Max Limit Cur Max Limit Total LbTot Last In Out Req Resp Req Conn Resp Retr Redis Status LastChk Wght Act Bck Chk Dwn Dwntme Thrtle
vps1 0 0 - 0 3 0
1 - 38
38 25m15s 14191 23441 0 0 0 0 0 25m30s UP L4OK in 0ms 1 Y - 28 10 0s -
vps2 0 0 - 0 1 0
1 - 30
30 25m6s 10055 16896 0 0 0 0 0 25m28s UP L4OK in 0ms 1 Y - 30 10 0s -
vps3 0 0 - 0 0 0
0 - 0
0 ? 0 0 0 0 0 0 0 13h5m DOWN * L4TOUT in 2001ms 1 Y - 1 1 13h5m -
Backend 0 0 0 8 0 1 26212 121
68 25m6s 43399 52050
0 0 53 0 0 0 25m30s UP 2 2 0 9 2h20m
