Hello, hope you are envoying your Kubernetes journey !
I wanted to try this on my kind (Kubernetes in docker) cluster locally. So this is what I've done:
First I have setup a kind cluster locally with this configuration (info here: https://kind.sigs.k8s.io/docs/user/quick-start/):
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: so-cluster-1
nodes:
- role: control-plane
image: kindest/node:v1.23.5
- role: control-plane
image: kindest/node:v1.23.5
- role: control-plane
image: kindest/node:v1.23.5
- role: worker
image: kindest/node:v1.23.5
- role: worker
image: kindest/node:v1.23.5
- role: worker
image: kindest/node:v1.23.5
after this I created my cluster with this command:
kind create cluster --config=config.yaml
Next, i have created a test namespace (manifest obtained with: kubectl create ns so-tests -o yaml --dry-run):
apiVersion: v1
kind: Namespace
metadata:
name: so-tests
From there, i got my environment setted up, so I used your deployment config and replaced the deploymentName, appName and hpaName occurences by "so-71823613" (stack-overflow and you question id), but for the test, I decided to not use the hpa config.
next, since you did not provide the image you are using for apache, I used the dockerhub image httpd:2.4.53 (https://hub.docker.com/layers/httpd/library/httpd/2.4.53/images/sha256-10ed1591781d9fdbaefaafee77067f12e833c699c84ed4e21706ccbd5229fd0a?context=explore)
again, since i dont have your configmap config, i decided to comment out the part where you get env variables from the configmap.
since the default user in httpd image is "www-data", I first deployed the pod without any securityContext just to get the id of that user:
❯ k exec -it pod/so-71823613-555d8b454-z5ks5 -- id www-data
uid=33(www-data) gid=33(www-data) groups=33(www-data)
Once that i knew what was the id of the www-data user, I modified the securityContext. I kept the rest of the configuration (probes, volume etc.) as you configured them, here is the manifest now:
In the configuration file, the runAsUser field specifies that for any Containers in the Pod, all processes run with user ID 33(www-data).
The runAsGroup field specifies the primary group ID of 33 for all processes within any containers of the Pod. If this field is omitted, the primary group ID of the containers will be root(0).
Any files created will also be owned by user 33 and group 33 when runAsGroup is specified.
Since fsGroup field is specified, all processes of the container are also part of the supplementary group ID 33. The owner for volume "/var/lock" and any files created in that volume will be Group ID 33.
...
fsGroupChangePolicy - fsGroupChangePolicy defines behavior for changing ownership and permission of the volume before being exposed inside a Pod. This field only applies to volume types that support fsGroup controlled ownership and permissions. This field has two possible values:
OnRootMismatch: Only change permissions and ownership if permission and ownership of root directory does not match with expected permissions of the volume. This could help shorten the time it takes to change ownership and permission of a volume.
Always: Always change permission and ownership of the volume when volume is mounted.
( description from here: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/)
So, once i deployed my configuration using:
kubectl apply -f deployment.yaml
deployment.apps/so-71823613 created
I got this error:
k logs -f pod/so-71823613-7c5b65df4d-6scg5
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 10.244.1.2. Set the 'ServerName' directive globally to suppress this message
(13)Permission denied: AH00072: make_sock: could not bind to address [::]:80
(13)Permission denied: AH00072: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
AH00015: Unable to open logs
So, first to fix the first line error, I reconnected into the pod to fetch the httpd.conf file with:
k exec -it pod/so-71823613-555d8b454-fgjcs -- cat /usr/local/apache2/conf/httpd.conf > httpd.conf
once i get the http.conf file, I modified it, by adding:
ServerName localhost:8080
(cf https://ixnfo.com/en/solution-ah00558-apache2-could-not-reliably-determine-the-servers-fully-qualified-domain-name.html)
Then I put the new httpd.conf file into a configmap named "httpconf", and modified the deployment to mount the configmap into the right place, to replace the first one (here -> "/usr/local/apache2/conf/httpd.conf") with:
...
volumeMounts:
...
- name: "config"
mountPath: "/usr/local/apache2/conf/httpd.conf"
subPath: "httpd.conf"
volumes:
...
- name: "config"
configMap:
name: "httpconf"
...
❯ kubectl apply -f configmap.yaml -f deployment.yaml
configmap/httpconf created
deployment.apps/so-71823613 created
Then i got this error remaining:
(13)Permission denied: AH00072: make_sock: could not bind to address [::]:80
So, to fix it, I changed to listening port of apache directly into the configmap http.conf file (according to this: https://askubuntu.com/questions/338218/why-am-i-getting-permission-denied-make-sock-could-not-bind-to-address-when)
Listen 8080
ServerName localhost:8080
since I am now exposing the 8080 port, I also modified the probes and the port in consequence:
...
ports:
- containerPort: 8080
readinessProbe:
tcpSocket:
port: 8080
initialDelaySeconds: 180
periodSeconds: 60
livenessProbe:
tcpSocket:
port: 8080
...
After reapplying my config I got this new error:
❯ k logs -f pod/so-71823613-7dd7bdb66d-qtf9t
[Wed Apr 20 05:50:57.863971 2022] [core:error] [pid 1:tid 139771999915328] (13)Permission denied: AH00099: could not create /usr/local/apache2/logs/httpd.pid.KYUI5g
[Wed Apr 20 05:50:57.864061 2022] [core:error] [pid 1:tid 139771999915328] AH00100: httpd: could not log pid to file /usr/local/apache2/logs/httpd.pid
To fix that issue, i used your workaround with the emptyDir and added this:
volumeMounts:
...
- mountPath: /usr/local/apache2/logs/
name: apache2-logs
volumes:
...
- name: apache2-logs
emptyDir: {}
here are the manifests:
apiVersion: apps/v1
kind: Deployment
metadata:
name: so-71823613
spec:
replicas: 1
selector:
matchLabels:
app: so-71823613
template:
metadata:
name: main
labels:
app: so-71823613
spec:
securityContext:
fsGroup: 33
runAsNonRoot: true
runAsUser: 33
runAsGroup: 33
fsGroupChangePolicy: "OnRootMismatch"
containers:
- name: so-71823613
image: httpd:2.4.53
ports:
- containerPort: 8080
readinessProbe:
tcpSocket:
port: 8080
initialDelaySeconds: 180
periodSeconds: 60
livenessProbe:
tcpSocket:
port: 8080
initialDelaySeconds: 300
periodSeconds: 180
imagePullPolicy: Always
tty: true
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
# envFrom:
# - configMapRef:
# name: so-71823613
resources:
limits:
cpu: 1
memory: 2Gi
requests:
cpu: 1
memory: 2Gi
volumeMounts:
- mountPath: /var/lock
name: var-lock
- mountPath: /usr/local/apache2/logs/
name: apache2-logs
- name: "config"
mountPath: "/usr/local/apache2/conf/httpd.conf"
subPath: "httpd.conf"
volumes:
- name: var-lock
emptyDir: {}
- name: apache2-logs
emptyDir: {}
- name: "config"
configMap:
name: "httpconf"
---
apiVersion: v1
kind: ConfigMap
metadata:
name: httpconf
data:
httpd.conf: |
ServerRoot "/usr/local/apache2"
Listen 8080
LoadModule mpm_event_module modules/mod_mpm_event.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authn_core_module modules/mod_authn_core.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule access_compat_module modules/mod_access_compat.so
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule reqtimeout_module modules/mod_reqtimeout.so
LoadModule filter_module modules/mod_filter.so
LoadModule mime_module modules/mod_mime.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule env_module modules/mod_env.so
LoadModule headers_module modules/mod_headers.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule version_module modules/mod_version.so
LoadModule unixd_module modules/mod_unixd.so
LoadModule status_module modules/mod_status.so
LoadModule autoindex_module modules/mod_autoindex.so
<IfModule !mpm_prefork_module>
</IfModule>
<IfModule mpm_prefork_module>
</IfModule>
LoadModule dir_module modules/mod_dir.so
LoadModule alias_module modules/mod_alias.so
<IfModule unixd_module>
User www-data
Group www-data
</IfModule>
ServerAdmin you@example.com
ServerName localhost:8080
<Directory />
AllowOverride none
Require all denied
</Directory>
DocumentRoot "/usr/local/apache2/htdocs"
<Directory "/usr/local/apache2/htdocs">
Options Indexes FollowSymLinks
AllowOverride None
Require all granted
</Directory>
<IfModule dir_module>
DirectoryIndex index.html
</IfModule>
<Files ".ht*">
Require all denied
</Files>
ErrorLog /proc/self/fd/2
LogLevel warn
<IfModule log_config_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
<IfModule logio_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
</IfModule>
CustomLog /proc/self/fd/1 common
</IfModule>
<IfModule alias_module>
ScriptAlias /cgi-bin/ "/usr/local/apache2/cgi-bin/"
</IfModule>
<IfModule cgid_module>
</IfModule>
<Directory "/usr/local/apache2/cgi-bin">
AllowOverride None
Options None
Require all granted
</Directory>
<IfModule headers_module>
RequestHeader unset Proxy early
</IfModule>
<IfModule mime_module>
TypesConfig conf/mime.types
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
</IfModule>
<IfModule proxy_html_module>
Include conf/extra/proxy-html.conf
</IfModule>
<IfModule ssl_module>
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
</IfModule>
# ---
# apiVersion: autoscaling/v1
# kind: HorizontalPodAutoscaler
# metadata:
# name: so-71823613
# spec:
# maxReplicas: 1
# minReplicas: 1
# scaleTargetRef:
# apiVersion: extensions/v1beta1
# kind: Deployment
# name: so-71823613
# targetCPUUtilizationPercentage: 60
after waiting the initialDelaySeconds of the probes, I finally get my pod up and running correctly:
Every 1.0s: kubectl get po,svc,cm -o wide DESKTOP-6PBJAOK: Wed Apr 20 03:15:02 2022
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/so-71823613-897768549-mcmb4 1/1 Running 0 4m13s 10.244.4.4 so-cluster-1-worker3 <none> <none>
NAME DATA AGE
configmap/httpconf 1 4m14s
Bonus:
I then decided to expose the http deployment with a service, here is the manifest (obtained from " k expose deployment so-71823613 --port 80 --target-port 8080 --dry-run=client -o yaml":
apiVersion: v1
kind: Service
metadata:
name: so-71823613
spec:
ports:
- port: 80
protocol: TCP
targetPort: 8080
selector:
app: so-71823613
as you can see, I port-forwarded the 8080 pod port to 80 in the service (you can also use an ingress controller to expose the service outside of the cluster )
tried this on my machine:
❯ k port-forward service/so-71823613 8080:80
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080
Handling connection for 8080
and here is the result:
TADA !
To conclude, I tried to reproduce the best i could with your provided information (It was kinda cool), so if this does not work for you, it means that i need more information.
Thank you for your lecture.
bguess.
