This all started yesterday after I added a second IP address for port 443 to the "ips and ports" list in Froxlor. As soon as Froxlor's cron job ran, Apache failed to restart. Ever since then, nothing I try will get Apache to stay running with SSL enabled in Froxlor.
System Config:
- Ubuntu 20.04.2 LTS (focal)
- Apache 2.4.41
- Froxlor 0.10.27
Output from sudo systemctl start apache2
:
Job for apache2.service failed because the control process exited with error code.
See "systemctl status apache2.service" and "journalctl -xe" for details.
Output from systemctl status apache2.service
:
● apache2.service - The Apache HTTP Server
Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Mon 2021-07-19 13:33:31 UTC; 41s ago
Docs: https://httpd.apache.org/docs/2.4/
Process: 17629 ExecStart=/usr/sbin/apachectl start (code=exited, status=1/FAILURE)
systemd[1]: Starting The Apache HTTP Server...
apachectl[17641]: AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName'>
apachectl[17629]: Action 'start' failed.
apachectl[17629]: The Apache error log may have more information.
systemd[1]: apache2.service: Control process exited, code=exited, status=1/FAILURE
systemd[1]: apache2.service: Failed with result 'exit-code'.
systemd[1]: Failed to start The Apache HTTP Server.
Output from sudo journalctl -u apache2.service --since today --no-pager
:
systemd[1]: Starting The Apache HTTP Server...
apachectl[17169]: AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message
apachectl[17165]: Action 'start' failed.
apachectl[17165]: The Apache error log may have more information.
systemd[1]: apache2.service: Control process exited, code=exited, status=1/FAILURE
systemd[1]: apache2.service: Failed with result 'exit-code'.
systemd[1]: Failed to start The Apache HTTP Server.
systemd[1]: apache2.service: Unit cannot be reloaded because it is inactive.
"Address already in use" error
Initially I was also getting an error that said apachectl[16500]: (98)Address already in use: AH00072: make_sock: could not bind to address
on port 443. Running netstat -anp | grep 443
did not reveal any other processes hogging that port, so I suspected that Apache was trying to use port 443 twice (which tracks with my configuration goof). I managed to get into the database and delete the ip/port record (which had not been assigned to any sites yet) and this particular error went away because Froxlor stopped creating an extra conf file containing Listen 443
.
If I comment out both instances of Listen 443
within /etc/apache2/ports.conf
, then this particular error goes away but Apache still fails to load.
# If you just change the port or add more ports here, you will likely also
# have to change the VirtualHost statement in
# /etc/apache2/sites-enabled/000-default.conf
Listen 80
#<IfModule ssl_module>
# Listen 443
#</IfModule>
#<IfModule mod_gnutls.c>
# Listen 443
#</IfModule>
# vim: syntax=apache ts=4 sw=4 sts=4 sr noet
(This post with a similar issue offered some insight on this bit)
Output from sudo grep "443" /etc/apache2/*
grep: /etc/apache2/conf-available: Is a directory
grep: /etc/apache2/conf-enabled: Is a directory
grep: /etc/apache2/htpasswd: Is a directory
grep: /etc/apache2/mods-available: Is a directory
grep: /etc/apache2/mods-enabled: Is a directory
/etc/apache2/ports.conf:# Listen 443
/etc/apache2/ports.conf:# Listen 443
grep: /etc/apache2/sites-available: Is a directory
grep: /etc/apache2/sites-enabled: Is a directory
Misc. remarks about Froxlor:
- If I comment out
\Froxlor\Cron\MasterCron::run();
inside of/var/www/froxlor/scripts/froxlor_master_cronjob.php
, then the Froxlor cron job is effectively disabled. Can be useful for troubleshooting, but doesn't fix anything. - Running
sudo /usr/bin/php /var/www/froxlor/scripts/froxlor_master_cronjob.php --force
will trigger Froxlor to execute its cron job immediately
Current Status: After many hours of troubleshooting, here is what I know:
- when no ip is configured with port 443/SSL, Apache will start.
- deleting
/etc/apache2/sites-enabled/
directory allows Apache to start, until Froxlor's cron job regenerates it. - likewise, just deleting the *.443.conf files and any ssl.conf files from
/etc/apache2/sites-enabled/
also temporarily allows Apache to start (until the Froxlor cron job runs) - removing Froxlor from the server allows Apache to start, but the problem comes back immediately after configuring port 443 within Froxlor.
TLDR: Something broke when I opened Froxlor and added a second IP with a port that was already in use (port 443). Now Apache won't start unless I delete any .conf file involving SSH. Removing Froxlor (including deleting the database) and deleting sites-enabled
before reinstalling Froxlor did not resolve the issue.
EDIT: Regenerated my security certificates and now all is good.