59

haproxy does not start anymore, it shows the error

bind <ip>:443' : unable to load SSL private key from PEM file ...

We did not change anything on the certificates or configuration. Since the last start we only made normal updates to the system.

To find the error, I generated a completely new certificate (self signed) but the error still exists.

This is the structure of the PEM file:

-----BEGIN CERTIFICATE-----
MIIDXjCCAkY...
-----END CERTIFICATE-----
-----BEGIN RSA PRIVATE KEY-----
MIIEpgIBAAKC....
-----END RSA PRIVATE KEY-----

I also tried to convert the private key with

openssl pkcs8 -topk8 -inform pem -in server.key -outform pem -nocrypt -out server_new.key

but haproxy still shows the same error.

I'm trying for hours now but I can not find the reason. Please help! Thank you!

Update:

The problem has something to do with file access. The PEM file was stored at /data/ssl/domainname/domainname.pem. File rights are ok. When I move the PEM file to /etc/haproxy then everything is ok.

efdev1234
  • 845
  • 1
  • 10
  • 18
  • If you use a RSA certificate the key needs to have at least 1024 bits, on ubuntu even more; found this in a note here https://access.redhat.com/solutions/6813181. i.e. `openssl genrsa -out key 2048` does work – Simon Grätzer Mar 23 '23 at 09:58

10 Answers10

56

The order in which the cert and key files appear in the pem is important. Use the following to create the pem file.

cat example.com.crt example.com.key > example.com.pem
Tombart
  • 30,520
  • 16
  • 123
  • 136
Justin Hourigan
  • 763
  • 1
  • 4
  • 9
  • 1
    this is the order in my pem file as you can see in my question...but thanks – efdev1234 Jan 15 '15 at 11:24
  • 3
    This answer solved my problem. Thank you with the same error! I forgot to concatenate files. – esengineer Jun 25 '15 at 13:56
  • 1
    This may have changed because I got it working with the private key coming before the public cert in the PEM file. I had been getting the same error, but in my case it was because I was running HAProxy in Docker but forget to add a volume to the container so HAProxy could see the PEM. – NickRamirez Jan 22 '16 at 18:50
  • Thanks. I have the same issue while I am giving the server.pem file to haproxy – NIrav Modi Dec 26 '19 at 08:08
24

The problem I was running into on CentOS was SELinux was getting in the way. To test if SELinux is the problem execute the following as root: setenforce 0, then try restarting the haproxy. If it works, there is an SELinux problem. (You can re-enable SELinux now and try to fix the underlying problem with the command setenforce 1).

Since I have the certificates in the folder /etc/haproxy/certificates, the following command worked to get the right permissions on the files restorecon -v -R /etc/haproxy (depending on your OS and SELinux config this may or may not work).

Tim Ludwinski
  • 2,704
  • 30
  • 34
20

For me the problem was caused by this line in combined PEM file:

-----END CERTIFICATE----------BEGIN RSA PRIVATE KEY-----

After I split it I could start HaProxy and load it OK:

-----END CERTIFICATE-----
-----BEGIN RSA PRIVATE KEY-----
Michael Mrozek
  • 169,610
  • 28
  • 168
  • 175
TBONE
  • 201
  • 2
  • 2
18

For the latest version of letsencrypt certbot,fullchain.pem and privkey.pem files will be generated for you in /etc/letsencrypt/live/example.com folder. They need to be combined in order to HAProxy to read it properly.

cat fullchain.pem privkey.pem > example.com.pem

In HAProxy configuraion /etc/haproxy/haproxy.cfg

bind *:443 ssl crt /etc/letsencrypt/live/example.com/example.com.pem
Denis Besic
  • 3,067
  • 3
  • 24
  • 35
6

I also encountered this error. You might want to try to remove the passphrase from the private key before you begin ripping your hair out. It solved the problem for me. I think HAProxy is supposed to ask you for the password on restart, but it didn't in my case using 'sudo /etc/init.d/haproxy restart

To remove the password, try 'openssl rsa -in [PRIVATE_KEY_FILE] -out nopassphrase.key'

Is passphrase necesssary? There's a discussion in the link below. https://security.stackexchange.com/questions/70495/ssl-certificate-is-passphrase-necessary-and-how-does-apache-know-it

Community
  • 1
  • 1
brunettdan
  • 997
  • 9
  • 7
4

Did you append your certificate's private key to the end of the file?

HAProxy requires a "full chain" - certificate, intermediate authority (if you have one), and then private key. E.g.:

cat cert.pem cert.key > /haproxy/certs/fullchain.pem
RobM
  • 8,373
  • 3
  • 45
  • 37
2

The problem for me was a strange character at the beginning of the key.

This character did not show up when I cated the file because the character was <feff> otherwise known as the UTF-8 BOM (Byte Order Mark). It only showed up when I opened the file in vim.

I wouldn't expect this to be very common, but hopefully it saves someone some headache.

Peter Klipfel
  • 4,958
  • 5
  • 29
  • 44
1

Just for information, in my case I had space character in front of "-----BEGIN RSA PRIVATE KEY-----" sequence and that broke the pem file.

Igor Tverdovskiy
  • 847
  • 1
  • 7
  • 5
1

I'd like to add, for people which join here and have the same issue, that you have to keep your intermediate certificates in the chain as well... So if you have a chain with some layers, don't only take the rootca but also the intermediate certificates into your pem file

Sebastian Geschwill
  • 41
  • 7
1

SElinux was the problem for me as well. HAProxy reported it could not read the file due to permissions even though the permissions matched other pem files in the folder. Our process is automated which is likely why SELinux is involved. The solution that seems to work for me so far (leaving SELinux running) is:

#!/bin/sh

if [ "$2" == "add" ]; then
  sudo touch /etc/haproxy/ssl/$1
  sudo cat $1 > /etc/haproxy/ssl/$1
  sudo chmod 644 /etc/haproxy/ssl/$1
fi
if [ "$2" == "delete" ]; then
  sudo rm /etc/haproxy/ssl/$1
fi

echo "performed $2 on $1";
Flair
  • 2,609
  • 1
  • 29
  • 41
David Martineau
  • 11
  • 1