Wednesday, May 18, 2011

Setup SSL Certificate on Cisco 3000 Series VPN Concentrator

In this case we went with a SSL Certificate from GoDaddy.com because of price. The problem arose from the fact that the CSR (Certificate Signing Request) wasn’t generated from the Cisco VPN Concentrator itself. However, in retrospect, this was probably the only way to do it properly because there doesn’t seem to be a way to export the private key from the Cisco. This would have posed a problem in our case because we purchased a Wildcard SSL Certificate (*.domain.tld) that can be installed on unlimited servers in our domain, and secure any host server within our domain.
Required Resources: The following software was used to complete this project:
Resolution: These are the general steps necessary to set this up, and should work for most major Certificate Authorities (CAs):
  1. Certificate Signing Request (CSR) generated on IIS Server. The CSR contains the Requested Public Key. The Private Key is left on the server.
  2. CSR submitted to Certificate Authority (CA). They generate the SSL Certificate, and provide the Cert, along with their CA Cert and Intermediate Cert.
  3. The CA Cert and Intermediate Cert are installed as CA Certs on the Cisco VPN Concentrator 3000.
  4. Here’s the tricky part. The Cisco Concentrator requires the SSL Cert to be in PKCS8 format, and contain the Private key and SSL Cert.
  5. I exported the Private/Public Key pair from the IIS Server, using the Windows Certificate Export Wizard; selecting to export both keys, and saving withOUT ‘high security’, and a password.
  6. This generates an encrypted PKCS12 file.
  7. At the unix command line (I used CYGWIN), I used OpenSSL (thanks to this site for OpenSSL basics)to first convert the PKCS12 file to standard format:
    openssl pkcs12 -in CERTIFICATE_NAME.pfx -out CERTIFICATE_NAME.pem
    The command prompts for the password used to export the key file from the IIS server. Then asks for a new password.
  8. I then converted the standard file to PKCS8 format for the Cisco:
    openssl pkcs8 -in CERTIFICATE_NAME.pem -topk8 -out CERTIFICATE_NAME.pk8
    Again the command prompts for the ‘New Password’ from the last export, and asks again for a newer password.
  9. Back on the Cisco Concentrator, I import a SSL certificate manually with Private Key for the Private Interface. Use the ‘copy and paste’ method.
  10. One other issue we have is that our CA uses an ‘Intermediate Certificate’. Thus creating a ‘chain’ of 3 trusted certificates: Ours, the ‘intermediate’ CA and the ‘root’ CA.
  11. In a text editor open both the CERTIFICATE_NAME.pk8 Private Key file you generated, along with the CERTIFICATE_NAME.cer SSL Certificate file provided to you by the CA.
  12. Copy and paste the Private key into the text box on the concentrator.
  13. Then immediately after, copy and paste the SSL Certifcate. Avoid any excess spaces, or blank lines.
  14. Then copy and paste the Issuing certifacte after the SSL Certificate.
  15. Finally copy and paste the Root Certificate at the end.
  16. The whole thing should look something like this:
    -----BEGIN ENCRYPTED PRIVATE KEY-----
    MIICiyuqweiSDuryiGquweryiDFuqweyGrqour

    9bgt3ouiiDnmbweFmnriorGuweioruu8u=
    -----END ENCRYPTED PRIVATE KEY-----
    -----BEGIN CERTIFICATE-----
    MIIEZkjasdlkDFajljkFyasdf8kGlyysfklysk

    asklfjsadkyy7DuklasyFdjfklyaGsdfjld=
    -----END CERTIFICATE-----
    -----BEGIN CERTIFICATE-----
    MIIEQkjasdlkDFajljkFyasdf8kGlyysfklysk

    asklfjsadkyy7DuklasyFdjfklyaGsdfjld=
    -----END CERTIFICATE-----
    -----BEGIN CERTIFICATE-----
    MIIC5jasdlkDFajljkFyasdf8kGlyysfklysk

    asklfjsadkyy7DuklasyFdjfklyaGsdfjld=
    -----END CERTIFICATE-----
  17. Enter the same sequence of data again for the Public interface.

Friday, May 13, 2011

How To Backup ESXi Configuration – The Missing Piece

This came up on #VMware on Freenode this weekend. Basically the concern was “How do I Backup my ESXi USB Key?” Other than ripping the USB key out of a production machine… how was the user to do this? Well, vMA and the vCLI provide a method for this:

Backing up your ESXi Configuration:

To backup your ESXi configuration you’ll be using the vicfg-cfgbackup.pl command as follows:
  • Download either the vMA or vCLI
  • Launch vicfg-cfgbackup.pl:
    C:\Program Files\VMware\VMware vSphere CLI\bin>vicfg-cfgbackup.pl –save –server 192.168.15.253 –username root –password password backup.bak
  • Note: The backup will be stored relative to your user “AppData” path:
    C:\Users\Username\AppData\Local\VirtualStore\

Restoring your ESXi Configuration:

Restoring your ESXi config can be done after you have the host up and responding over the network again by using the following:

C:\Program Files\VMware\VMware vSphere CLI\bin>vicfg-cfgbackup.pl –load –server 192.168.15.253 –username root –password password backup.bak

Note: You will be asked to reboot the host on restore.

Backing up multiple hosts! – There is a script to backup multiple ESXi hosts on the VMware communities site here. Also in PowerCLI here!

[Edit: Added link to backup multiple ESXi hosts from William in the comments. Thanks William!]
[Edit 2: Added PowerCLI link from NiTRo. Site is in French, PowerCLI is not]


ESXi and USB failure?

Interesting Article

In recent years, servers with embedded USB storage have become common practice. Today, all major hardware vendors deliver servers with embedded ESXi. Even in my home lab, servers are equipped with an onboard USB connector, USB stick and ESXi. Recently, on one host, the USB stick was moved to an external connector.  I was wondering, what would happen with an ESXi host with USB stick failure. Or even worse, pulling the USB stick.

So, after booting up my 2 node cluster, I made a fresh backup of a few important VMs and checked the vCenter Service Status. Now, it is time to remove the USB stick from one host. And this is what happened:
  • VMs on the affected host are still running.
  • Task & Events of the affected host shows this message “Lost connectivity to storage device mpx.vmhba32:C0:T0:L0. Path vmhba32:C0:T0:L0 is down. Affected datastores: “Hypervisor1”, “Hypervisor2”, “Hypervisor3”.”.
  • Followed by 3 Alarms “Cannot connect to storage”.
  • Another message in Tasks & Events is “Boot partition /bootbank cannot be found (0:02:33:03.304 cpu1:30722)”.
  • Time for some testing, all these actions do work: Power On a VM, Migrate a VM, host in Maintenance Mode, Exit Maintenance Mode (HA Agent is configured correctly).
  • Also the ESXi console is doing fine, System Customization is in place, and so are the System Logs.
  • From time to time above messages are repeated and in some occasions while migrating VMs “The Operation is not allowed in the current state” messages are received.
  • After 24 hours, the host is still running, and performing. So finally, I decided to enter the host in Maintenance Mode and shut it down. The power down took about 10 minutes ( less then 2 minutes is normal).
  • After insertion of the USB stick, the host was powered on and was automatically reconnected to the cluster.
At this time, my tentative conclusion is that failure, or even an missing USB stick does not have much impact on a ESXi host. Thanks for reading and I’m very interested in your experience and opinions concerning this subject.
P.S. A few days after posting, I stumbled onto this post, written by Alan Renouf. In the first part it is explained why ESXi keeps running without USB boot device.

Script from Alan to backup ESXI host.

############################
$RootFolder = "C:\Support\"
Get-VMHost | Foreach {
Write-Host "Backing up state for $($_.Name)"
$Date = Get-Date -f yyyy-MM-dd
$Folder = $RootFolder + $Date + "\$($_.Name)\"
If (-not (Test-Path $Folder)) {
MD $Folder | Out-Null
}
$_ | Get-VMHostFirmware -BackupConfiguration -DestinationPath $RootFolder
# Next line is a workaround for -DestinationPath not working correctly
# with folder names with a - in them.
MV ($RootFolder + "*") $Folder -ErrorAction SilentlyContinue
########################################

Thursday, May 5, 2011

To check the number of cores for a CPU in a virtual machine, you can use one of these utilities:
  • Coreinfo
    Coreinfo is a Microsoft command-line utility, developed by Mark Russinovich. It displays the mapping between logical processors and the physical processor, NUMA node, and socket on which they reside. It also provides information on the cache assigned to each logical processor.

    To check the distribution of cores across socket, use the coreinfo -c -s command. To download and install coreinfo, click
    here.
  • CPU-Z utility
    CPU-Z is a freeware application for Microsoft Windows operating systems and it provides information about CPU, Processor, Cache, Memory, System board, Graphics, and other hardware features. To download and install CPU-Z, see
    http://www.cpuid.com/.
In the figure below, the cpuid.coresPerSocket is set to 4 and, therefore, the number of cores per CPU is 4.
 
 
For information about setting the number of cores per socket in a virtual machine, see Setting the number of cores per CPU in a virtual machine (1010184).

Additional Information

  • CPU – Is the portion of a computer system that performs the instructions of a computer program. It is the primary element that carries out the computer’s functions. 
  • Core – Is a logical execution unit containing an L1 cache and functional units needed to execute programs. Cores can independently execute programs or threads. 
  • Socket – Is a physical connector on a computer motherboard that accepts a single physical chip.

Wednesday, May 4, 2011

Setting the number of cores per CPU in a virtual machine

Some operating system SKUs are hard-limited to run on a fixed number of CPUs. For example, Windows Server 2003 Standard Edition is limited to run on up to 4 CPUs. If you install this operating system on an 8-socket physical box, it runs on only 4 of the CPUs. The operating system takes advantage of multi-core CPUs so if your CPUs are dual core, Windows Server 2003 SE runs on up to 8 cores, and if you have quad-core CPUs, it runs on up to 16 cores, and so on.

Virtual CPUs (vCPU) in VMware virtual machines appear to the operating system as single core CPUs. So, just like in the example above, if you create a virtual machine with 8 vCPUs (which you can do with vSphere) the operating system sees 8 single core CPUs. If the operating system is Windows 2003 SE (limited to 4 CPUs) it only runs on 4 vCPUs.
 
 
Note: Remember that 1 vCPU maps onto a physical core not a physical CPU, so the virtual machine is actually getting to run on 4 cores.
 
Considering that 1 vCPU is equal to 1 CPU is an assumption for the sake of simplification, since vCPUs are scheduled on logical CPUs which are hardware execution contexts. These tasks can take a while in the case of a single core CPU, CPUs that have only 1 thread per core, or could be just a thread in the case of a CPU that has hyperthreading.
Consider this scenario:
In the physical world you can run Windows 2003 SE on up to 8 cores (using a 2-socket quad-core box) but in a virtual machine they can only run on 4 cores because VMware tells the operating system that each CPU has only 1 core per socket.
VMware now has a setting which provides you control over the number of cores per CPU in a virtual machine.
This new setting, which you can add to the virtual machine configuration (.vmx) file, lets you set the number of cores per virtual socket in the virtual machine.
 
To implement this feature:
  1. Power off the virtual machine.
  2. Right-click on the virtual machine and click Edit Settings.
  3. Click Hardware and select CPUs.
  4. Choose the number of virtual processors.
  5. Click the Options tab.
  6. Click General, in the Advanced options section.
  7. Click Configuration Parameters.
  8. Include cpuid.coresPerSocket in the Name column.
  9. Enter a value (try 2, 4, or 8) in the Value column.Note: Ensure that the number of vCPUs is divisible by the number of cpuid.coresPerSocket in the virtual machine. That is, when you divide the number of vCPUs by the number of cpuid.coresPerSocket, it must return an integer value. For example, if your virtual machine is created with 8 vCPUs, coresPerSocket can only be 1, 2, 4, or 8.

    The virtual machine now appears to the operating system as having multi-core CPUs with the number of cores per CPU given by the value that you provided in step 9.
  10. Click OK.
  11. Power on the virtual machine.

For example:
Create an 8 vCPU virtual machine and set cpuid.coresPerSocket = 2. Window Server 2003 SE running in this virtual machine now uses all 8 vCPUs. Under the covers, Windows sees 4 dual-core CPUs. The virtual machine is actually running on 8 physical cores.
 
Note:
  • Only values of 1, 2, 4, 8 for the cpuid.coresPerSocket are supported for the multi-core vCPU feature in ESX 4.x.
  • In ESX 4.0, if multi-core vCPU is used, hot-plug vCPU is not permitted, even if it is available in the UI.
  • Only HV 7 virtual machines support the multi-core vCPU feature.
Important: When using cpuid.coresPerSocket, you should always ensure that you are in compliance with the requirements of your operating system EULA (Regarding the number of physical CPUs on which the operating system is actually running).

Monday, May 2, 2011

Difference between Plenum & Riser Cat5/Cat6 cable

Plenum vs. Riser

Cable Ratings

Plenum (CMP) Rated Cable
Complies with NFPA-262 and UL-910. Only cable allowed in spaces defined as air plenums such as raised flooring systems and air handling ducts. Plenum cables must self extinguish and not reignite. They also produce less smoke than traditional PVC cables. The smoke and fumes are toxic.

Riser (CMR) Rated Cable
Comples with UL-1666. Defined for usage in vertical tray applications such as cable runs between floors through cable risers or in elevator shafts. These spaces cannot be used for environmental air. These cables must self extinguish and must also prvent the flame from traveling up the cable in a vertical burn test.

Other Cable Ratings:
Low Smoke Zero Halogen (LSZH) Rated Cable
Used in shipboard applications and computer networking rooms where toxic or acidic smoke and fumes can injure people and/or equipment. Examples of Halogens include Fluorine, Chlorine, Bromine, and Iodine. These materials when burned produce acidic smoke that can harm people and computer equipment. Low Smoke means the cable does not produce the heavy black soot and smoke common with PVC cables. These cables will self extinguish but cannot pass UL-910 or UL-1666 for a plenum or riser rating. 

General Purpose (CM, CMG, CMx) Cable
Complies with UL-1581 testing. Will burn and partially self extinguish. Not for use between build floors or in air plenum spaces. Often these cables are used for workstation cables and patch cords.