Using client certificates for security is a pretty cool idea! You can protect an entire application or even just a specific Uniform Resource Identifier (URI) to only those that provide a valid client certificate. It'll not only confirm it's valid but also supports revoking certificates when compromised.

A brilliant and common use case for this would be a corporate intranet or other internal application such as SharePoint. You may be looking for an additional level of security but want to avoid common methods such as obfuscating the URL or adding additional passwords to the site. Instead, we can make use of SSL certificates to allow only clients that provide a non-expired or revoked certificate that was issued by our server access. Users will need to add this client certificate to their web browsers personal certificate store to be allowed access to either the whole site or specific parts of it.

The Certificate Revocation List (CRL) is key to making this a sensible security approach when dealing with many users. Without it, should a certificate become compromised you would need to re-issue the Certificate Authority (CA) and any client certificates. However, with a CRL we can revoke a certificate allowing sane user management for access to your backend application.

Client-Certificate-Authentication-with-HAProxy---Bouncer2

A simple config example for locking down an entire application is below (this config example is reduced for simplicity):

listen VIP
    bind 172.16.200.85:443 transparent ssl crt /etc/haproxy/cert/server.pem ca-file /etc/haproxy/cert/ca.crt verify required crl-file /etc/haproxy/cert/root_crl.pem
    ...
    server RIP 172.16.203.35:80  weight 100  cookie RIP  check port 80 inter 4000  rise 2  fall 2  minconn 0  maxconn 0  on-marked-down shutdown-sessions

With the above config, only a valid client certificate will gain you access to the site(s) behind "listen VIP", you’ll notice we are using the statement “verify required” on the bind line. If we want to allow users without a client certificate to use this service we’ll need to change that to “verify optional”.

verify options:

required - A client cert must be provided.
optional - Prompt for client cert but allow access without one.
none - Don't prompt for client cert, useless for web browsers but good for curl...

Now the problem is that people with the client certificate can use the site as before but anyone else can also! So, what about if we need to allow those with the client certificate to access a specific URI keeping it private from those without the certificate? Then we need a config like this instead:

listen VIP
    bind 172.16.200.85:443 transparent ssl crt /etc/haproxy/cert/server.pem ca-file /etc/haproxy/cert/ca.crt verify optional crl-file /etc/haproxy/cert/root_crl.pem
    ...
    acl restricted_path path_beg,url_dec -m beg -i /secure/
    http-request deny if restricted_path !{ ssl_c_used 1 } || restricted_path !{ ssl_c_verify 0 }
    server RIP 172.16.203.35:80  weight 100  cookie RIP  check port 80 inter 4000  rise 2  fall 2  minconn 0  maxconn 0  on-marked-down shutdown-sessions

The above config allows all users to access the whole site except the protected area “/secure” requests to “https://<My-Site>/secure/<whatever>” will result in a “403 Forbidden” error. You may consider using "verify optional" on your public facing website but be warned! Users will be prompted to select a certificate when first accessing the site, most users wouldn't notice this as web browsers tend not to show the dialogue if you have no personal certificates installed. However, some users may so consider seperating the configuration so reqests to the secure part of the site can use "verify required" on one frontend/listen section while the unsecure part is on another. The easiest way to handle this is by using a different IP, port or subdomain for the secure part of your site. However, for a more advanced example using SNI so both the secure and unsecure site can be behind the same IP and port without users being prompted for a certificate check out this excellent example by Lukas Tribus: https://discourse.haproxy.org/t/how-to-set-ssl-verify-client-for-specific-domain-name/1489/3

We could also add custom error pages for revoked or expired certificates like so:

listen VIP
    bind 172.16.200.85:443 transparent ssl crt /etc/haproxy/cert/server.pem ca-file /etc/haproxy/cert/ca.crt verify optional crt-ignore-err all crl-file /etc/haproxy/cert/root_crl.pem
    ...
    acl restricted_path path_beg,url_dec -m beg -i /secure/
    redirect location /certmissing.html if restricted_path !{ ssl_c_used 1 }
    redirect location /certexpired.html if restricted_path { ssl_c_verify 10 }
    redirect location /certrevoked.html if restricted_path { ssl_c_verify 23 }
    redirect location /othererrors.html if restricted_path !{ ssl_c_verify 0 }
    server RIP 172.16.203.35:80  weight 100  cookie RIP  check port 80 inter 4000  rise 2  fall 2  minconn 0  maxconn 0  on-marked-down shutdown-sessions

You'll note that we've added "crt-ignore-err all" to the bind line to allow invalid certificates access, we can then provide different error pages depending on the "ssl_c_verify" status. Client verify 10 represents an expired certificate while 23 is revoked. I had a 26 once during testing which I think meant the certificate had an invalid purpose while 21 seems to return when you supply an entirely unrelated certificate.

Next, we can add in various headers containing client certificate information:

listen VIP
    bind 172.16.200.85:443 transparent ssl crt /etc/haproxy/cert/server.pem ca-file /etc/haproxy/cert/ca.crt verify required crl-file /etc/haproxy/cert/root_crl.pem
    ...
    http-request set-header X-SSL                       %[ssl_fc]
    http-request set-header X-SSL-Client-Verify         %[ssl_c_verify]
    http-request set-header X-SSL-Client-SHA1           %{+Q}[ssl_c_sha1]
    http-request set-header X-SSL-Client-DN             %{+Q}[ssl_c_s_dn]
    http-request set-header X-SSL-Client-CN             %{+Q}[ssl_c_s_dn(cn)]
    http-request set-header X-SSL-Issuer                %{+Q}[ssl_c_i_dn]
    http-request set-header X-SSL-Client-Not-Before     %{+Q}[ssl_c_notbefore]
    http-request set-header X-SSL-Client-Serial         %{+Q}[ssl_c_serial,hex]
    http-request set-header X-SSL-Client-Version        %{+Q}[ssl_c_version]
    server RIP 172.16.203.35:80  weight 100  cookie RIP  check port 80 inter 4000  rise 2  fall 2  minconn 0  maxconn 0  on-marked-down shutdown-sessions

Which will add the following fetches to the headers of the traffic so it can be logged or even used by your backend application:

X-SSL: 1
X-SSL-Client-Verify: 0
X-SSL-Client-SHA1: .+......W....%0By%7F...d
X-SSL-Client-DN: /C=AU/ST=Some-State/O=Internet Widgits Pty Ltd
X-SSL-Client-CN: Bob Roberts
X-SSL-Issuer: /C=AU/ST=Some-State/O=Internet Widgits Pty Ltd
X-SSL-Client-Not-Before: 170531090028Z
X-SSL-Client-Serial: 02
X-SSL-Client-Version: 1

There's a lot more information that we can get if needed, the possible fetches are mentioned here in the HAProxy manual: https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#7.3.4

You can also use a certificate revocation list should a cert become compromised for example when employees move on, you can simply revoke their certificate, re-create the CRL PEM (Privacy-Enhanced Mail) file and reload HAProxy to remove a users access.

So, we’ve discussed what we can do and why we might do it, so lets get to the how:

First, we need to create the CA and client certificates followed by the certificate revocation list - I’m doing this on CentOS 6 in a fairly generic way so it should work on most systems including the Loadbalancer.org Enterprise appliance!

Okay, let's create a directory for our CA and other certificate files under the HAProxy directory:

mkdir /etc/haproxy/cert
cd /etc/haproxy/cert

Create the CA which will be used for signing client certificate:

openssl genrsa -out ca.key 4096
openssl req -new -x509 -days 1826 -key ca.key -out ca.crt

Example answers:

Country Name (2 letter code) [AU]:UK
State or Province Name (full name) [Some-State]:HANTS
Locality Name (eg, city) []:Portsmouth
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Loadbalancer.org Ltd
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:Loadbalancer.org CA
Email Address []:

Create the OpenSSL configuration file:

nano -w ca.conf

Next paste the prepared and very simplified config file:

[ ca ]
default_ca=     dft_ca  # Configuration files may have more than one CA
                        # section for different scenarios.

[ crl_ext ]
# issuerAltName=issuer:copy  #this would copy the issuer name to altname
authorityKeyIdentifier=keyid:always

[ dft_ca ]
DIR= ./                                 # Default directory.
unique_subject= no

certificate=    ${DIR}/ca.crt           # The CA certificate.
database=       ${DIR}/index.txt        # Keeps tracks of valid/revoked certs.
                                        # administrative purposes.

new_certs_dir=  ${DIR}/                 # Copies of signed certificates, for
                                        # administrative purposes.

private_key=    ${DIR}/ca.key           # The CA key.
serial=         ${DIR}/serial           # Should be populated with the next
                                        # cert hex serial.

default_md = sha1

# These govern the way certificates are displayed while confirming
# the signing process.
name_opt=       ca_default
cert_opt=       ca_default

default_days= 730       # How long to sign certificates for.
default_crl_days= 730   # The same, but for CRL.

policy=         dft_policy      # The default policy should be lenient.
x509_extensions= cert_v3        # For v3 certificates.

[ dft_policy ]
# A value of 'supplied' means the field must be present in the certificate,
# whereas 'match' means the field must be populated with the same contents as
# the CA certificate. 'optional' dictates that the field is entirely optional.

C=      supplied        # Country
ST=     supplied        # State or province
L=      optional        # Locality
O=      supplied        # Organization
OU=     optional        # Organizational unit
CN=     supplied        # Common name

[ cert_v3 ]
# With the exception of 'CA:FALSE', there are PKIX recommendations for end-user
# certificates that should not be able to sign other certificates.
# 'CA:FALSE' is explicitely set because some software will malfunction without.

subjectKeyIdentifier=   hash
basicConstraints=       CA:FALSE
keyUsage=               nonRepudiation, digitalSignature, keyEncipherment
basicConstraints=       CA:false
subjectKeyIdentifier=   hash
authorityKeyIdentifier= keyid:always

Now, for a few extra requirements:

touch index.txt
echo 01 > serial

Next we create our first client certificate:

openssl genrsa -des3 -out client1.key 1024
openssl req -new -key client1.key -out client1.csr
openssl ca -batch -config ca.conf -notext -in client1.csr -out client1.crt

Example answers:

Country Name (2 letter code) [AU]:UK
State or Province Name (full name) [Some-State]:HANTS
Locality Name (eg, city) []:Portsmouth
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Loadbalancer.org Ltd
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:Bob Roberts
Email Address []:

Finally we can generate the CRL PEM file for HAProxy to use:

openssl ca -config ca.conf -gencrl -keyfile ca.key -cert ca.crt -out root_crl.pem

So, assuming we've done that all correctly we should now be able to start HAProxy and test our setup:

curl --insecure --cert-type pem --cert client1.pem "https://172.16.200.85/secure/index.html"

Accessing the secure area without the client cert:

[root@lbmaster cert]# curl --insecure "https://172.16.200.85/secure/index.html"
Enter PEM pass phrase:
<html><body><h1>403 Forbidden</h1>
Request forbidden by administrative rules.
</body></html>

Accessing the secure area when providing the certificate we now gain access:

[root@lbmaster cert]# curl --insecure --cert-type pem --cert client1.pem "https://172.16.200.85/secure/index.html"
Enter PEM pass phrase:
TEST SECURE

Revoking a certificate:

openssl ca -config ca.conf -revoke client1.crt -keyfile ca.key -cert ca.crt
openssl ca -config ca.conf -gencrl -keyfile ca.key -cert ca.crt -out root_crl.pem
service haproxy reload

Let's try again with the newly revoked certificate:

[root@lbmaster cert]# curl --insecure --cert-type pem --cert client1.pem "https://172.16.200.85/secure/index.html"
Enter PEM pass phrase:
<html><body><h1>403 Forbidden</h1>
Request forbidden by administrative rules.
</body></html>

Finally, you may ask how to remove the certificate password, how do I make a PEM file as used in the examples and how to use this in a clients web browser...

The password was set when we created the private key and can be removed although it can be useful as an extra security layer in some circumstances but we will remove it for our use:

openssl rsa -in client1.key -out client1_nopass.key

The PEM file is also nice and simple to generate with it just being a combined form of the .key and .crt files we created earlier, to combine these files into a PEM file we issue the following two commands:

cat client1_nopass.key > client1.pem
cat client1.crt >> client1.pem

Although other than for testing it may be more useful to create a PFX file file that can be imported into your web browser:

openssl pkcs12 -export -clcerts -in client1.crt -inkey client1.key -out client1.p12

Using the client certificate in your web browser, many have covered this and it's best done with lots of screenshots that I can't be bothered to make so here's some links instead:

http://www.binarytides.com/client-side-ssl-certificates-firefox-chrome/
http://www.jscape.com/blog/firefox-client-certificate

After importing the certificate you will usually need to restart your web browser before it will prompt you for the certificate. Equally, if you choose not to use a certificate when prompted you often need to do the same to get the browser to re-ask you again!

Client-Certificate-Authentication-with-HAProxy-1

Here are some links below for further reading:

https://www.haproxy.com/blog/ssl-client-certificate-management-at-application-level/
https://raymii.org/s/tutorials/haproxy_client_side_ssl_certificates.html
https://www.haproxy.com/blog/ssl-client-certificate-information-in-http-headers-and-logs/