Skip to main content

Apache seems to be using old expired certificate even though new one is installed



Apache 2.2.3/mod_ssl/CentOS 5.5 VPS



Our certificate expired on 2011-10-06, and even though we have seemingly installed the new one correctly, browsing to the site still shows an expired certificate! I've tried deleting my browser cache and using several different browsers. Relevant lines from the ssl.conf file (I've excluded those commented out.):



Listen 127.0.0.1:443
SSLSessionCache shmcb:/var/cache/mod_ssl/scache(512000)
SSLSessionCacheTimeout 300

# Note - I tried disabling SSLSessionCache with the "none" setting but it didn't help.

SSLEngine on
SSLProtocol all -SSLv2
SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+LOW
SSLCertificateFile /var/certs/gentlemanjoe.com/new2011/gentlemanjoe.com.crt
SSLCertificateKeyFile /var/certs/gentlemanjoe.com/new2011/gentlemanjoe.com.key
SSLCertificateChainFile /var/certs/gentlemanjoe.com/new2011/gd_bundle.crt
SetEnvIf User-Agent ".*MSIE.*" \
nokeepalive ssl-unclean-shutdown \

downgrade-1.0 force-response-1.0
CustomLog logs/ssl_request_log \
"%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
ServerAdmin webmaster@donotemailme.com
DocumentRoot /var/www/gentlemanjoe.com
ServerName gentlemanjoe.com

AllowOverride All
Order deny,allow
allow from all





Things I've Checked



First I tried moving the old cert and key files to a completely different folder to make sure Apache wasn't still grabbing them somehow. Nothing changed. For fun I tried temporarily renaming the new cert and key files, and Apache dutifully complained and refused to start.



Then I tried to make sure I wasn't being fooled by editing the wrong config file. Using "locate" I found only one httpd.conf file under /etc/httpd/conf/httpd.conf. I also used "locate" to verify that there is only one ssl.conf file, /etc/httpd/conf.d/ssl.conf. The key file is what I generated using OpenSSL, following the instructions GoDaddy gave for generating the CSR.




I have verified that I am working with the right site by uploading a test.html file to the /var/www/gentlemanjoe.com folder and verifying that I can browse to it. But if I try to view the test file in HTTPS, I get the same certificate expiration warning.



I verified that the cert itself has the right expiration date:



openssl x509 -in /var/certs/gentlemanjoe.com/new2011/gentlemanjoe.com.crt -noout -text

Certificate:
Data:
Version: 3 (0x2)
Serial Number:

07:e7:49:69:97:96:16
Signature Algorithm: sha1WithRSAEncryption
Issuer: C=US, ST=Arizona, L=Scottsdale, O=GoDaddy.com, Inc., OU=http://certificates.godaddy.com/repository, CN=Go Daddy Secure Certification Authority/serialNumber=07969287
Validity
Not Before: Oct 21 17:37:55 2011 GMT
Not After : Oct 8 21:16:03 2013 GMT
Subject: C=CA, ST=BC, L=Burnaby, O=Diamond Bailey Consolidated Commercial Services Ltd, OU= , CN=www.gentlemanjoe.com


I tried re-keying the cert at GoDaddy with a new CSR and everything seems to work, but I get the same result in the browser.




Possible Clue #1



Whenever I do "apachectl restart", I see this in the error_log file:



[Fri Oct 21 18:03:33 2011] [notice] SIGHUP received.  Attempting to restart
[Fri Oct 21 18:03:33 2011] [notice] Digest: generating secret for digest authentication ...
[Fri Oct 21 18:03:33 2011] [notice] Digest: done
[Fri Oct 21 18:03:33 2011] [info] APR LDAP: Built with OpenLDAP LDAP SDK
[Fri Oct 21 18:03:33 2011] [info] LDAP: SSL support available

[Fri Oct 21 18:03:33 2011] [info] Init: Seeding PRNG with 256 bytes of entropy
[Fri Oct 21 18:03:33 2011] [info] Init: Generating temporary RSA private keys (512/1024 bits)
[Fri Oct 21 18:03:33 2011] [info] Init: Generating temporary DH parameters (512/1024 bits)
[Fri Oct 21 18:03:33 2011] [info] Shared memory session cache initialised
[Fri Oct 21 18:03:33 2011] [info] Init: Initializing (virtual) servers for SSL
[Fri Oct 21 18:03:33 2011] [warn] RSA server certificate CommonName (CN) `www.gentlemanjoe.com' does NOT match server name!?
[Fri Oct 21 18:03:33 2011] [info] Server: Apache/2.2.3, Interface: mod_ssl/2.2.3, Library: OpenSSL/0.9.8e-fips-rhel5
[Fri Oct 21 18:03:34 2011] [notice] Apache/2.2.3 (CentOS) configured -- resuming normal operations
[Fri Oct 21 18:03:34 2011] [info] Server built: Aug 30 2010 12:28:40



The GoDaddy techs tell me that the www vs non-www should not matter, and I tend to agree, since the security warning in my browser is not complaining about a server name mismatch, but rather an expiration, indicating that the old certificate is still being loaded somehow.



Possible Clue #2



The HTTP Server response header for http://gentlemanjoe.com says "Andromeda" rather than "Apache". This seems weird to me since my Googling of "Andromeda" turns up a media-server type project, which wouldn't be installed on this server (but I can't say that with certainty since I didn't set any of this up, the usual admin/developer is on holiday and I'm just helping a friend with his site.) Also, the httpd.conf file does not contain the string "Andromeda" indicating that it hasn't been modified to spit this out. So it might be the Magento e-commerce platform he's using, but what would be the point of replacing the standard Apache response header?


Answer



Something's in front of Apache. Check out that config:



Listen 127.0.0.1:443

....



It's listening on localhost only, so internet clients aren't hitting this service directly - they're likely getting proxied.



For the sanity check that Apache's loading the right cert, hit the service directly on Apache's listener: openssl s_client -connect 127.0.0.1:443 -showcerts



Not sure about the Andromeda header, so, let's find the process: lsof -i.




Apache will have 127.0.0.1:443, while some other service has 0.0.0.0:443 (or the VPS's public address :443) - that's the one that needs the new cert.


Comments

Popular posts from this blog

linux - iDRAC6 Virtual Media native library cannot be loaded

When attempting to mount Virtual Media on a iDRAC6 IP KVM session I get the following error: I'm using Ubuntu 9.04 and: $ javaws -version Java(TM) Web Start 1.6.0_16 $ uname -a Linux aud22419-linux 2.6.28-15-generic #51-Ubuntu SMP Mon Aug 31 13:39:06 UTC 2009 x86_64 GNU/Linux $ firefox -version Mozilla Firefox 3.0.14, Copyright (c) 1998 - 2009 mozilla.org On Windows + IE it (unsurprisingly) works. I've just gotten off the phone with the Dell tech support and I was told it is known to work on Linux + Firefox, albeit Ubuntu is not supported (by Dell, that is). Has anyone out there managed to mount virtual media in the same scenario?

hp proliant - Smart Array P822 with HBA Mode?

We get an HP DL360 G8 with an Smart Array P822 controller. On that controller will come a HP StorageWorks D2700 . Does anybody know, that it is possible to run the Smart Array P822 in HBA mode? I found only information about the P410i, who can run HBA. If this is not supported, what you think about the LSI 9207-8e controller? Will this fit good in that setup? The Hardware we get is used but all original from HP. The StorageWorks has 25 x 900 GB SAS 10K disks. Because the disks are not new I would like to use only 22 for raid6, and the rest for spare (I need to see if the disk count is optimal or not for zfs). It would be nice if I'm not stick to SAS in future. As OS I would like to install debian stretch with zfs 0.71 as file system and software raid. I have see that hp has an page for debian to. I would like to use hba mode because it is recommend, that zfs know at most as possible about the disk, and I'm independent from the raid controller. For us zfs have many benefits,

apache 2.2 - Server Potentially Compromised -- c99madshell

So, low and behold, a legacy site we've been hosting for a client had a version of FCKEditor that allowed someone to upload the dreaded c99madshell exploit onto our web host. I'm not a big security buff -- frankly I'm just a dev currently responsible for S/A duties due to a loss of personnel. Accordingly, I'd love any help you server-faulters could provide in assessing the damage from the exploit. To give you a bit of information: The file was uploaded into a directory within the webroot, "/_img/fck_uploads/File/". The Apache user and group are restricted such that they can't log in and don't have permissions outside of the directory from which we serve sites. All the files had 770 permissions (user rwx, group rwx, other none) -- something I wanted to fix but was told to hold off on as it wasn't "high priority" (hopefully this changes that). So it seems the hackers could've easily executed the script. Now I wasn't able