Unable to connect NSX to Lookup Service when using a vSphere 6 subordinate certificate authority (VMCA)
Sam McGeownAfter deploying a new vSphere 6 vCenter Server Appliance (VCSA) and configuring the Platform Services Controller (PSC) to act as a subordinate Certificate Authority (CS), I was unable to register the NSX Manager to the Lookup Service. Try saying that fast after a pint or two!?
Attempting to register NSX to the Lookup Service would result in the following error:
NSX Management Service operation failed.( Initialization of Admin Registration Service Provider failed. Root Cause: Error occurred while registration of lookup service, com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain not verified )
Initially I thought that the NSX manager needed to somehow import the VMCA certificate to trust the Lookup Service certificate, however after reaching out to the NBSU ambassadors list I had a reply from Julienne Pham, a Technical Solutions Architect and CTO Ambassador with VMware Professional Services, who pointed me to the correct solution.
It seems that changing the PSC and vCenter certificates (even with the Certificate Manager tool) does not correctly update the service registration information. To quote VMware KB 2109074:
…the vCenter Server system uses a new certificate, but the service registration information on the Platform Services Controller is not updated
To resolve this issue, we need to use the ls_update_certs.py script to register the services correctly.
Retrieve the old SSL certificate’s thumbprint
If you haven’t updated the VCSA certificate yet, you can just view the vCenter certificate and find the sha1 thumbprint value. If, like me, you’ve already updated it, you’ll need to use the Managed Object Browser (MOB) to view it.
Open a web browser and go to:
https://
e.g. https://mgmt-vcsa.definit.local/lookupservice/mob?moid=ServiceRegistration&method=List
Enter valid administrator credentials and then edit the “VALUE” field to have just the
Use the browser’s built in search function (CTRL+F) to find the FQDN of your VCSA server - you’re looking for entries like the below image, where the sslTrust value is a Base64 encoded string which represents the old certificate.
Copy the Base64 string into a text editor and save it somewhere.
Open the saved certificate and find the SHA1 Thumbprint:
Copy the thumbprint back to your text editor and edit string to replace the spaces:
Update the Lookup Service registration certificates
You need to have a copy of your PEM encoded certificate chain on your VCSA. If you followed Derek Seaman’s excellent guide, you’ve already enabled bash and changed the root user’s default shell, and you also have the certificate chain in PEM format (under /root/ssl). If you haven’t, check out the “Install VMCA Certificate (VCSA PSC)” step on the linked article.
Connect to the VCSA with your SSH client and use the following command to update the lookup service certificate:
/usr/lib/vmidentity/tools/scripts/ls_update_certs.py -url
/usr/lib/vmidentity/tools/scripts/ls_update_certs.py -url https://mgmt-vcsa.definit.local/lookupservice/sdk -fingerprint 68e9819264334d6e61adf0cd96c9578a1b9b1990 -certfile /root/ssl/root_signing_chain.cer -user [email protected] -password Pa55w0rd!
Note: If your password has an exclamation mark (!) in it, or any other bash-unfriendly character, you will need to escape it with a backslash ("")
Once the command is run, it will take a few minutes and spew a lot of output on the screen, eventually though, you see the success message:
This should now allow you to connect NSX (or any other solution that uses the Lookup Service) successfully:
A big thank you to Julienne Pham for the help with this one!