I pushed ceph/ceph:v17 to lcoal docker registry. and tag as astute-tec.com:4000/ceph/ceph:v17
I modify local /usr/bin/cephadm code quay.io -> astute-tec.com:4000; and modify always add '--tls-verify=false' when use podman.
this is part I modified.
# Default container images -----------------------------------------------------
DEFAULT_REGISTRY = 'astute-tec.com:4000' # this
DEFAULT_IMAGE = f'{DEFAULT_REGISTRY}/ceph/ceph:v17'
DEFAULT_IMAGE_IS_MASTER = False
DEFAULT_IMAGE_RELEASE = 'quincy'
DEFAULT_PROMETHEUS_IMAGE = f'{DEFAULT_REGISTRY}/prometheus/prometheus:v2.33.4'
DEFAULT_LOKI_IMAGE = f'{DEFAULT_REGISTRY}/grafana/loki:2.4.0'
DEFAULT_PROMTAIL_IMAGE = f'{DEFAULT_REGISTRY}/grafana/promtail:2.4.0'
DEFAULT_NODE_EXPORTER_IMAGE = f'{DEFAULT_REGISTRY}/prometheus/node-exporter:v1.3.1'
DEFAULT_ALERT_MANAGER_IMAGE = f'{DEFAULT_REGISTRY}/prometheus/alertmanager:v0.23.0'
DEFAULT_GRAFANA_IMAGE = f'{DEFAULT_REGISTRY}/ceph/ceph-grafana:8.3.5'
DEFAULT_HAPROXY_IMAGE = f'{DEFAULT_REGISTRY}/ceph/haproxy:2.3'
DEFAULT_KEEPALIVED_IMAGE = f'{DEFAULT_REGISTRY}/ceph/keepalived:2.1.5'
DEFAULT_SNMP_GATEWAY_IMAGE = f'{DEFAULT_REGISTRY}/maxwo/snmp-notifier:v1.2.1'
-----------------------------------
if isinstance(ctx.container_engine, Podman):
cmd.append('--tls-verify=false') # this
if os.path.exists('/etc/ceph/podman-auth.json'):
cmd.append('--authfile=/etc/ceph/podman-auth.json')
cmd_str = ' '.join(cmd)
I use cephadm bootstrap --mon-ip 192.222.9.81 --skip-monitoring-stack it work. Then I have podman images and ceph containers.
[root@ceph1 ~]# podman ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6157c1d7b9e6 astute-tec.com:4000/ceph/ceph:v17 -n mon.ceph1 -f -... 2 hours ago Up 2 hours ceph-b7492ab6-465b-11ee-aaf0-5e11ca73e3ff-mon-ceph1
c6b8f3f3be4c astute-tec.com:4000/ceph/ceph:v17 -n mgr.ceph1.lxeo... 2 hours ago Up 2 hours ceph-b7492ab6-465b-11ee-aaf0-5e11ca73e3ff-mgr-ceph1-lxeove
But when I add another host, it failed. use follow command.(I had add ceph2 ceph2 ceph3 in /etc/hosts)
ssh-copy-id -f -i /etc/ceph/ceph.pub ceph2
cephadm shell -- ceph orch host add ceph2
failed log is:
cephadm ['--image', 'astute-tec.com:4000/ceph/ceph:v17', 'pull']
2023-08-29 12:55:46,930 7fe780e4f740 INFO Pulling container image astute-tec.com:4000/ceph/ceph:v17...
2023-08-29 12:55:46,967 7fe780e4f740 INFO Non-zero exit code 125 from /usr/bin/podman pull astute-tec.com:4000/ceph/ceph:v17
2023-08-29 12:55:46,967 7fe780e4f740 INFO /usr/bin/podman: stderr Trying to pull astute-tec.com:4000/ceph/ceph:v17...
2023-08-29 12:55:46,967 7fe780e4f740 INFO /usr/bin/podman: stderr Error: initializing source docker://astute-tec.com:4000/ceph/ceph:v17: pinging container registry astute-tec.com:4000: Get "https://astute-tec.com:4000/v2/": http: server gave HTTP response to HTTPS client
2023-08-29 12:55:46,967 7fe780e4f740 ERROR ERROR: Failed command: /usr/bin/podman pull astute-tec.com:4000/ceph/ceph:v17
how to fix it?
I known why it failed, because podman need --tls-verify=false to pull from local docker registry, I have change ceph2 node usr/bin/cephadm code to use --tls-verify=false, it doesnot work.
I think it may use the cephadm in podman image(astute-tec.com:4000/ceph/ceph:v17) to pull image, But I do not want to modify it, it is offical image.
Is there a way to solve this problem?
-----------------------------------------------------------
20230830 I change the podman images to create a new images
[root@host151 change_cephadm]# cat Dockerfile
FROM astute-tec.com:4000/ceph/ceph:v17
RUN sed -i 's/insecure=False/insecure=True/g' /usr/sbin/cephadm
RUN sed -i 's/%(thread)x/%(thread)x %(lineno)d/g' /usr/sbin/cephadm
then I recreate the ceph cluster, but add host failed,too.The log is
2023-08-30 02:22:37,528 7f27365de740 DEBUG --------------------------------------------------------------------------------
cephadm ['--image', 'astute-tec.com:4000/ceph/ceph:v17', 'pull']
2023-08-30 02:22:37,562 7f27365de740 INFO Pulling container image astute-tec.com:4000/ceph/ceph:v17...
2023-08-30 02:22:37,601 7f27365de740 INFO Non-zero exit code 125 from /usr/bin/podman pull astute-tec.com:4000/ceph/ceph:v17
2023-08-30 02:22:37,601 7f27365de740 INFO /usr/bin/podman: stderr Trying to pull astute-tec.com:4000/ceph/ceph:v17...
2023-08-30 02:22:37,601 7f27365de740 INFO /usr/bin/podman: stderr Error: initializing source docker://astute-tec.com:4000/ceph/ceph:v17: pinging container registry astute-tec.com:4000: Get "https://astute-tec.com:4000/v2/": http: server gave HTTP response to HTTPS client
2023-08-30 02:22:37,601 7f27365de740 ERROR ERROR: Failed command: /usr/bin/podman pull astute-tec.com:4000/ceph/ceph:v17
I found it doesnot show the lineno, the console cephadm shell command can show the lineno.
# cephadm shell -- ls
2023-08-30 02:18:16,304 7f19597e6740 9607 DEBUG --------------------------------------------------------------------------------
cephadm ['shell', '--', 'ls']
2023-08-30 02:18:16,327 7f19597e6740 2151 DEBUG Using default config /etc/ceph/ceph.conf
so, the question is, where file to execute the pull command? I search ceph project for Pulling container image, only one place in src/cephadm/cephadm.
the ceph project search for text
