Background

I run several services from my home Linux server to support the home software development I do. This includes services such as Subversion source control, Nexus Maven repository, JIRA issue tracking, Jenkins continuous integration build server to name a few. To provide a clean and consistent entry point into each service, I use separate vhosts in my Apache 2 server, each with its own dedicated sub-domain e.g. nexus.server.com, jira.server.com etc. These vhosts then proxy onto the underlying service using a mixture of HTTP, DAV and AJP13 proxying. [NOTE: I’m using server.com instead of my real domain here simply as an example]

As I want all of these services to be available securely externally I also ensure that they are all served over an SSL connection. Apache is responsible for the SSL handling – SSL is terminated at Apache and then plaintext proxying is used to the underlying service (SVN, Nexus etc.)

The use of SSL necessitates SSL certificates for each vhost, which is pretty straightfoward to implement. I have my own CA certificate which I use to issue certs for each of my services and by adding this CA cert to the truststores on client machines, the certification trust chain can be completed and the SSL connection is verified as valid.

I’ve recently gone through a process of renewing the various SSL certs for my services and moving them to their own dedicated sub-domains (previously they were simply sub-contexts from my main www.server.com domain e.g. www.server.com/nexus, www.server.com/jira etc. rather than having their own sub-domains e.g. nexus.server.com, jira.server.com etc.) Configuring the vhosts in Apache was trivial and on the face of it everything appeared to be working just fine after the changes. Accessing each service from a browser, the SSL certificates were being verified as valid and the web applications were completely functional. It wasn’t until I tried to access my Nexus Maven repository from a Maven command line build that I noticed there was a problem…

Where is that SSL cert coming from?!

On attempting to establish an SSL connection with my Nexus server, the connection was being rejected with a the following exception:

java.security.cert.CertificateException: No name matching nexus.server.com found

I knew that the SSL cert configured in my Nexus vhost was correctly issued for that sub-domain so I couldn’t understand why it was complaining. To investigate further I invoked mvn with the extremely useful Java network debugging property:

-Djavax.net.debug=ssl

This results in the full SSL handshake being logged which can be invaluable when investigating SSL problems. Alternatively, if you’re not using a Java based client and have openssl installed you can use the following command to inspect details of the SSL handshake:

openssl s_client -connect nexus.server.com:443 -status

By doing this I could see that the subject (and specifically the CN element) of the cert being returned by Apache was www.server.com instead of nexus.server.com! Woah – what was going on here…?

The only place that the www.server.com SSL cert was mentioned in my Apache configuration was in the www.server.com vhost – but it couldn’t be using that, could it? As a simple test I tried changing this vhost configuration to use the nexus.server.com cert and then restarted Apache. The next time I ran my mvn command it verified the SSL connection to nexus.server.com with no problems i.e. the SSL server cert now being returned was nexus.server.com! This was very strange I thought… until I started to think more about how Apache name based vhosts actually work…

Limitations of Apache name based virtual hosts

I started Googling on this subject and came across this very interesting Wiki article. This article explains the problem very well, but to summarise… it’s all to do with the way that name based hosts work in Apache: It relies on the Host: header in the HTTP request to determine which vhost this request should be serviced by. In the case of an HTTPS request, this only takes place after the initial SSL connection has been established. Therefore, there isn’t enough information available to Apache in that initial connection establishing phase in order to determine which SSL server cert to return in the SSL handshake. As a result, Apache uses the first SSL cert it finds in the ordered list of vhost configurations. In my case, this was for my www.server.com vhost which explains why it was using the www.server.com SSL cert!

But how could I get round this problem if Apache will only ever use one SSL cert in its SSL handshake?

Introducing wildcard SSL certs

The referenced article mentions the use of wildcard SSL certs i.e. those with a subject name of the form *.server.com. This will only work if all of the subsequent vhosts use genuine server.com sub-domains, but in my case that is true.

To be honest, I’d not really heard of wildcard SSL certs before so after a bit more reading I tried creating one for my domain – with a subject CN of *.server.com. After configuring this in each of my Apache vhosts and a bit of testing everything seems to work just fine. The only difference between now and my previous configuration is that the SSL certs associated with each vhost as less specific than before but I can’t see any problem with that at all.

So, wildcard SSL certs it is from now on!