Using Commercial Certificates in vSphere 5.5

I would like to share my experience and findings when working with commercial certificates in vSphere 5.5.  I’ve been working on a customer project where there is a requirement to replace the self-signed certificate of the Web Client with a commercial one. This particular project involved using a Verisign certificate.

I initially used the Certificate Automation Tool to generate the CSR (rui.csr) and the private key (rui.key). Oddly, Verisign is giving the Error 4824 below with the generated CSR:

The error points to the Subject Alternative Name (SAN) and specifically because there is a Shortname and Private IP in the SAN field of the CSR which Verisign does not allow. I remember when using the Certificate Automation Tool, the shortname and IP address are fields that must be populated. With the help of my colleague Frank Buechsel (http://fbuechsel.eu/), he recommended to generate the CSR’s manually using KB 2044696.

Following the KB article, I created an OpenSSL configuration file for the Web Client using the following format:

To remove the shortname and IP address, edit the subjectAltName and remove the ServerShortName and IP. It should look like this afterwards:

subjectAltName = DNS:server.domain.com

We can now generate the CSR using the modified configuration file. Following the same KB article, we generated the new CSR using the below commands (openssl is available inside the extracted Certificate Automation Tool):

After performing the above steps, we were able to successfully submit the CSR to Verisign.

After receiving the certificate in X.509 format, we are ready to install it using the Certificate Automation Tool. I renamed the certificate to rui.crt and attempted the replacement but encountered an error in the tool with a message that the chain cannot be validated. I then checked the downloaded certificate and saw that it does not contain the Intermediate CA in it. So it means that the tool won’t let you install the certificate if the chain to the intermediate CA is missing. We then proceeded to  download the Intermediate certificate from Verisign and again referencing the same KB article, created the chain.pem file in order to have the certificate chain.

Using the Certificate Automation Tool with the new chain.pem file, the certificate replacement successfully went through.

To cut the long story short, the key takeaways when using commercial certificates are:

• Ensure that only FQDN is in the SAN field of the CSR else CSR submission will fail
• Once you receive the certificate from your commercial CA, verify that the chain to the intermediate CA is present before proceeding with replacement

 

 

 

Removing Unwanted Plugins in vSphere

I always use my lab to prepare for projects and to learn different VMware products. Of course I don’t have the privilege of keeping all products running due to limited amount of compute resources . As time goes, I have to remove some of the appliances like VDP, vSphere Replication and vShield Manager.

Now that I am introducing NSX into my lab, I have noticed that logging into the Web Client now becomes painfully slow. It now takes about 3 to 4 minutes before I can get into the Web GUI. Once inside, everything is normal. Since NSX can only be configured in the Web client, I need to get this issue resolved.

Searching for similar issues in the web, I saw this one which is the closest to what I’m experiencing. I remember that I have lots of plugins that I no longer use: VDP, vSphere Replication Management and vShield Manager. To remove those plugins, I used this KB article.

Below, I captured the screenshots when I removed my VDP 5.1 plugin.

Login to http://<vcenter_name_or_IP>/mob and click content

Click ExtensionManager

Select and copy the extension that you are removing. In my case I’m removing the VDP 5.1 extension which is com.vmware.vdp

Click UnregisterExtension

Paste the name of the plugin and click Invoke Method to remove the plugin

You should get the Method Invocation Result: void message which tells you that the plugin has been removed.

I did the same procedure for my vSphere Replication Management (com.vmware.vcHms) and vShield Manager (com.vmware.vShieldManager) plugins and tested login again. And what have you, my web client login is back to normal.

Invalid Configuration for device ‘0’ Error Message

First time I’ve encountered this issue in my environment after a DRS initiated vMotion of a VM. I’ve noticed that the VM’s network card is disconnected and whenever I tried to connect it, the Invalid Configuration for device ‘0’  error pops up. From the looks of it, vMotion had a bandwidth issue but it did migrate successfully after another attempt.

This exact issue is documented in KB 2014469 and although vSphere 5.5 is not in the product list, I gave it a try to see if it will resolve this issue.

I did Option 1 of the KB article which is to move the VM into a different and unused dVPort. From the screenshot below, my VM Win2K8R2 is using port 104 of distributed switch vDS-DMZ.

To change this, edit the VM setting and highlight the network adapter. Under network connection, click Switch to advanced settings. In the Port ID box, change the Port to a new unused port. In the screenshot below, I changed the port from 104 to 105

Click OK then go back to Edit Settings and connect the NIC card. This resolved my issue and the VM is back into the network.

VM Replication status is “Not Active” after upgrade to vSphere 5.5/VR 5.5

Weird issue I encountered after I deployed vSphere Replication appliance on my upgraded vSphere 5.5 lab. I was able to configure vSphere replication for a VM but for strange reasons, the VM replication shows as Not Active. This worked fine when I was still using vSphere 5.1/VR 5.1. The hostd.log file shows the following error:

The configuration for vSphere Replication all looks good, all are green so I can’t figure out what’s wrong here.

Luckily, I found a  VM community post which is similar to my issue found here

I remember that in 5.1, I configured a vSphere Replication vmkernel port tick box in web client. This vmkernel port tick box is now removed in vSphere 5.5. As far as I know, that tick box is useless in vSphere 5.1 coz even though its checked, VR traffic will always default to the management vmkernel. See Sunny Dua’s article on properly configuring a different management network for vSphere replication

Going back to my issue, I followed the suggestion to remove the lines similar to the following in esx.conf of the ESXi host:

/net/vmkernelnic/child[0001]/tags/4 = “true”

The number [0001] may change depending on the environment. On mine its [0002]. After making the change and rebooting the ESXi hosts, vSphere Replication started working fine again.

Upgrading vCenter 5.5 to 5.5 U1 with vCenter Heartbeat installed [Part Three]

This is the last part of this 3 part series. In Parts One and Two, I discussed about this new “re-cloning” method for upgrading vCenter Server with heartbeat and started configuring the Primary vCenter server. In this part we will continue the configuration focusing on the Secondary vCenter Server.

1. Start the newly cloned vCenter Server. Make sure that both Principal and Channel NIC’s are disconnected before powering ON the VM. Also make sure that the vCenter Server Heartbeat service is stopped and set to Manual.

2. Reconfigure the IP address of the  Channel Nic from 1.1.2.1 to 1.1.2.2

3. Launch the vCenter Server Heartbeat Configure Server wizard and click the Machine tab. Change the physical server identity to Secondary.

4. Click the Public tab, configure the correct IP addresses, and click Finish. In my lab, I am changing the Management IP back to 192.168.11.108

5. Change the server name to the Secondary node’s previous Management name and add it to a Workgroup. Reboot when requested. In the screenshot below, I am changing the hostname back to vc02.

6. Review the Principal NIC’s IP Address before connecting the Principal NIC. This is to make sure we won’t have an IP conflict. The Channel NIC can also be connected.

7. Using the Service Control Manager, configure the VMware vCenter Server Heartbeat service Startup Type to Automatic. DO NOT Start the service just yet!

8. Join the server to the domain and reboot when requested. So now I am joining my vc02 server from WORKGROUP back to the lab.local domain.

9. When the secondary comes back up, we have basically brought up the 2 vCenter Nodes back to its original states but now upgraded to Update 1. Initiate Full replication and synchronization from Primary to Secondary

          

10. Uninstall and re-install the vCenter plugin.Navigate to the Applications: Plug-ins page, select the VirtualCenterNFPlugin and click Uninstall.

11. After the plug-in has been removed from the plug-ins list, click Install. Navigate to the location of the vCenter Server plug-in (the default location is C:\Program Files\VMware\VMware vCenter Server Heartbeat\R2\plugins\VMwareVirtualCenter\201.5.4.13406) and select the VirtualCenterNFPlugin.dll file.

12. After the plug-in has been added, select it, click Edit, then enter the credentials used to connect to vCenter Server.

That’s it folks!! To cover all grounds, do a failover test to check that everything works just fine.

Upgrading vCenter 5.5 to 5.5 U1 with vCenter Heartbeat installed [Part Two]

In Part One , I mentioned about KB2070181 that demonstrates a different method of upgrading vCenter Server when Heartbeat is installed. In this article, I will show the steps I did to perform this task on the Primary vCenter Server. Details of my environment are as follows:

Primary vCenter Server:
Hostname: vc01.lab.local
Management IP address: 192.168.11.107/24
Channel IP Address: 1.1.2.2/24

Secondary vCenter Server:
Hostname: vc02.lab.local
Management IP address: 192.168.11.108/24
Channel IP Address: 1.1.2.2/24

vCenter Server Public Name: vc.lab.local
vCenter Server Public IP Address: 192.168.11.106

 

Both vCenter Servers are VM’s in a LAN deployment. The SQL Server database in my environment is in a different VM and is not protected by Heartbeat. The version of Heartbeat is 6.6 which is a supported version from  the KB. In a production environment, always make sure you have a backup.  Here are the steps:

1. Make the Primary Node active and verify.

2. To verify, double click the Configure Server icon on the desktop and go to machine tab

3.  Shutdown the secondary node. The KB article mentioned to delete the secondary vCenter Server but I won’t do that just yet. I did a snapshot of my Primary VM and in case something goes wrong I can just revert to the snapshot and still use both the primay and secondary vCenters.

4. Rename the Primary node from the Management Name to the Public Service Name and reboot when requested.  In below screenshot, I renamed my primary vCenter from vc01 to vc.

5. Shut down vCenter Server Heartbeat, leaving protected applications running.

6. Using the Service Control Manager, configure the VMware vCenter Server Heartbeat service Startup Type to Manual.

7. Upgrade all protected applications. In my environment, I upgraded the following components to 5.5 U1 in the following order: SSO, Web Client, Inventory Service and vCenter Server. If you have Update Manager, upgrade it as well. The upgrade went smoothly, the installer detected an existing installation and prompted that they will be upgraded to 5.5 U1 version.

8. Start vCenter Server Heartbeat and run protected service discovery by navigating to the vCenter Server Heartbeat Console Application tasks page.

8. Clone the server (do not start the newly cloned node yet)

9. Using the Service Control Manager, configure the VMware vCenter Server Heartbeat service Startup Type to Automatic.

10. Rename the server back to the Primary node’s previous Management Name, and reboot when requested.

That’s it for the Primary vCenter Server. In part 3 of this series, I will continue on with configuring the Secondary node which actually is just the clone of the newly upgraded primary vCenter Server.

administrator@vsphere.local and SSO Password Policy

I was recently asked by a customer if the SSO account administrator@vsphere.local is affected by the Password Policy which by default expires after 90 days. A quick check on the documentations gave me two conflicting answers:

This one states  that it will expire after 90 days but this one says otherwise. After checking internally, it is confirmed that the SSO password policy only applies to  user created accounts in the sphere.local domain. The administrator@vsphere.local account is considered a System account so the password policy is ignored.

[Edit] The documentation team has been informed and the first link will be corrected soon.