As a consultant I’ve had the opportunity to design, install and configure dozens of production vRealize Automation deployments, from reasonably small Proof of Concept environments to globally-scaled multi-datacenter fully distributed behemoths. It’s fair to say, that I’ve made mistakes along the way - and learned a lot of lessons as to what makes a deployment a success.
In the end, pretty much everything comes down to getting the pre-requisites right. Nothing that I’ve written here is not already documented in the official documentation, and the installation wizard does a huge amount of the work for you.
For the purposes of this post, I am working with the following components, which have been pre-deployed on a single flat network.
vRA Appliances
Server |
vra-app-1 |
vra-app-2 |
Server |
vra-web-1 |
vra-web-2 |
vra-man-1 |
vra-man-2 |
vra-dem-1 |
vra-dem-2 |
vra-sql |
A and PTR (forward and reverse) DNS records are required for all components deployed - appliances and IAAS servers. DNS is a critical component of the application, you must be able to resolve the FQDN and the short name of all components, as well as reverse lookups. Use nslookup to validate from each server, to all other servers.
In addition to FQDNs configured for all the components, an A record will be required for each of the load balanced URLs.
The FQDNs need to be included in the Subject Alternative Names in the SSL certificates.
vra-app-1.definit.local -> vRA Appliance 1
vra-app-2.definit.local -> vRA Appliance 2
vra-web-1.definit.local -> vRA Web Server 1
vra-web-2.definit.local -> vRA Web Server 2
vra-man-1.definit.local -> vRA Manager Server 1
vra-man-2.definit.local -> vRA Manager Server 2
vra-dem-1.definit.local -> vRA DEM and vSphere Agent 1
vra-dem-2.definit.local -> vRA DEM and vSphere Agent 2
vra-sql-1.definit.local -> MSSQL Server (or cluster VIP)
vra-app.definit.local* -> Cafe Load Balancer IP
vra-web.definit.local* -> Web Load Balancer IP
vra-man.definit.local* -> Manager Load Balancer IP
I typically create a CNAME for the load balancer VIPs first, then move to A records later - see the Load Balancer section below.
It’s important to use a common time source for all components - domain joined Windows VMs for IaaS will use use the Domain Controllers. My DCs use my local NTP server, so the vRA Appliances are synchronised with the local NTP server directly too. You can use VMware Tools to set the time on each VM - if you do this ensure that the ESXi hosts have a common time source. Use one method or the other, but not a mix!
NTP for the vRA Appliance is configured when you run the installation wizard, and can be configured via the VAMI interface.
Since it’s introduction in vRA 7, the installation wizard does a fantastic job configuring the IaaS Windows Servers for their roles, and this massively improves the chance of a successful deployment first time. There are a few things that I always do for each Windows Server.
Update the IaaS servers with the latest patches before doing anything - typically this means maintaining an up-to-date template and deploying from that, whatever the method, ensure that the server is patched and that no updates will interfere with the deployment.
One common problem is that cloned VMs don’t generate a unique ID for MSDTC. This causes communication issues when you’re deploying vRA. You can either remove MSDTC from your template, then install once you’ve deployed it, or reinstall on the cloned VM. This issue does not affect Windows servers deployed by other methods than cloning.
[code]msdtc -uninstall && msdtc -install[/code]
MSDTC should be configured on all the IaaS boxes to allow communication with the MSSQL Server. You can use the following PowerShell command, or manually set it from Component Services.
[code]Set-DtcNetworkSetting -DtcName “Local” -RemoteAdministrationAccessEnabled:$True -RemoteClientAccessEnabled:$True -InboundTransactionsEnabled:$True -OutboundTransactionsEnabled:$True -LUTransactionsEnabled:$True -Confirm:$False[/code]
Create a service account to run the IaaS components. From the VMware documentation, it must meet the following requirements:
The account must be a domain user.
The account does not need to be a domain administrator, but must have local administrator permission, before installation, on all IaaS Windows servers.
The account password cannot contain a double quotation mark ( " ) character.
The Management Agent installer for IaaS Windows servers prompts you for the account credentials.
The account must have Log on as a service permission, which lets the Manager Service start and generate log files.
The account must have dbo permission on the IaaS database. If you use the installer to create the database, add the account login to SQL Server before installation. The installer grants the dbo permission after it creates the database.
If you use the installer to create the database, in SQL, add the sysadmin role to the account before installation. The sysadmin role is not required if you choose to use a pre-existing empty database.
In my lab environment, I’ve used [email protected]
To quickly add the service account to all Windows servers, you can run the following command:
[code]net localgroup administrators [email protected] /add[/code]
Typically I disable the Windows Firewall on all IaaS components during installation using the following command - it’s too easy to miss a required rule and break the installation:
[code]netsh advfirewall set allprofiles state off[/code]
Once the deployment is complete create rules for the well defined communications, based on IaaS Windows Server Ports documentation
A supported version of Microsft SQL server should be installed and configured for TCP/IP communication. As specified above, the IaaS service account should have dbo permission on the IaaS database, and if you want to create the database from the installer then you also need sysadmin role, which can be revoked after installation.
Certificates need to be generated for the vRA appliances, the Web service and the Manager service. There are a couple of ways to do this that will work, some are more manual than others. The method I use is to generate a Certificate Signing Request (CSR) and Private Key using OpenSSL, then sign them using my Microsoft Certificate Services Certificate Authority. Any standards compliant Certificate Authority, public or private, should be able to sign your certificates.
Getting the certificates minted up front and correctly is
The following configuration template can be used to generate the CSRs for each component, make sure to change the subjectAltName values to match the components you’re issuing for. The commonName can actually be anything you like, since it’s not used for the SSL, but I prefer to keep the load balanced name.
[code]
[ req ]
default_bits = 2048
default_keyfile = rui.key
distinguished_name = req_distinguished_name
encrypt_key = no
prompt = no
string_mask = nombstr
req_extensions = v3_req
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment, dataEncipherment, nonRepudiation
extendedKeyUsage = serverAuth, clientAuth
subjectAltName = DNS:vra-app.definit.local,DNS:vra-app-1.definit.local,DNS:vra-app-2.definit.local
[ req_distinguished_name ]
countryName = GB
stateOrProvinceName = West Sussex
localityName = Horsham
0.organizationName = DefinIT
organizationalUnitName = Lab
commonName = vra-app.definit.local
[/code]
When I’ve generated certificates and requests, I will have the following files:
During the install wizard you need the .key (private key), .cer (certificate) and CA certificate files.
During the installation phase of a distributed vRA deployment I always very strongly suggest using a CNAME pointed to the primary nodes for the Cafe, Web and Manager components. This is because the configuration of the load balancer is so critical to success that any misconfiguration can cause errors and a failed deployment. If you use a CNAME you can guarantee the deployment succeeds first time, and introduce the load balancer later.
When it comes to configuring the load balancer, Jad El Zein has a great resource in his Distributed Install Guide
Once the pre-requisites are in place, running the install wizard should be straightforward - I have run through the wizard below to prove that it can be done right, first time! I hope this post will save you some time and effort, and maybe help to not trip over the same mistakes I made!