Showing posts with label ESXi. Show all posts
Showing posts with label ESXi. Show all posts

05 September 2016

Building a vSphere lab using ESXi linked clones

Update: William Lam has posted the ultimate nested homelab deployment script. I highly recommend it! I'll leave this blogpost up for historical sake but any and all information in it has been superseeded with the script mentioned above.

--------Original post-----------
When I'm setting up a nested vSphere lab, I don't wont to spend a lot of time doing the actual setup and start playing as soon as possible. Up until now I've used the ESXi appliance distributed by William Lam. My current workflow looks like this:

  1. Import OVF to vCenter using OVF customization
  2. Boot the imported VM so the parameters get picked up by the guest OS
  3. Done!

With a simple PowerShell script it is possible to deploy the OVF, add some advanced parameters to the VMX and have a working nested ESXi after the initial boot. All in one simple "wizard" like experience.
Most of the waiting time is used to deploy the OVF itself so I thought to myself: why not shorten the time needed to set up a nested lab! And if I do so using linked clones, I get the shortest deployment time possible, since it's just a thin clone. It should be possible since the script imports the same OVF multiple times anyway. I'll try to import the OVF once, make a snapshot and use the snapshot as the base template for the linked clones. Once the linked clone is created I'll add the advanced parameters to the VMX and get the nested lab off and running! So the new workflow should look like:

  1. Make linked clone
  2. Add advanced parameters to the VMX
  3. Boot the imported VM so the parameters get picked up by the guest OS
  4. Done!

Unfortunately it's not possible to make a linked clone using the vSphere Web Client. PowerCLI to the rescue! With the cmdlet New-VM we should be able to make a new virtual machine out of an existing snapshot. So for this to work, you'll need to import William Lam's nested ESXi appliance OVA and make one snapshot. Be sure to skip all the OVF customization since we'll do that later.

After taking the snapshot, we can see the virtual machine files and the snapshot in the datastore browser. The main VMDK that contains the ESXi install itself is about 540MB large. The snapshot delta file contains no data so that's nice and small. The new clone ends up taking under 3MB. As soon as you start it up, it will grow a bit but only the new written blocks are kept in the snapshot delta file.



Since I set up most of my labs the same way, there are some entries added to the script variables that don't change between script runs. The only variable that has to be entered into the clone script each and every time is the number of nested ESXi hosts I want. This changes every time I set up a lab so I've used a Read-Host prompt for that. I mostly use VSAN or some type of shared storage for the lab, so I set the createvmfs variable to false. If I don't use VSAN, I usually set up a Starwind Virtual SAN. It's an easy to set up iSCSI target with a free 2-node license for many IT professionals. It also offers VAAI support and some storage acceleration, so that's nice.

Creating one ESXi clone takes about 30 seconds until boot. During the first boot it runs through some configuration scripts and is ready to be used in under two minutes.


Next up will be a blogpost on how to automate a vCenter deployment into this lab and a script to add the nested ESXi hosts to the fresh vCenter server. Adding all of these together should give you a quick and easy way to deploy a nested vSphere lab.

New script (for the linked clones)


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
# Script by Hans Lenze Kaper - www.kaperschip.nl
# heavily inspired by William Lam - www.virtuallyghetto.com

# Variables for connecting to vCenter server
$viServer = "192.168.2.200"
$viUsername = "root"
$viPassword = "password"

# Variables for the lab host
$sourceVM = 'nestedESXi-template'
$sourceSnapshot = '20160904'
$destDatastore = 'SSD1'
$destVMhost = "192.168.2.230"
$numberNestedESXiInput = read-host -Prompt "How many nested ESXi hosts do you want to deploy?"

# Variables for the nested lab
$iprange = "192.168.10"
$netmask = '255.255.255.0'
$gateway = '192.168.10.254'
$dns = '192.168.10.254'
$dnsdomain = 'test.local'
$ntp = '192.168.2.254'
$syslog = '192.168.10.100'
$password = 'password'
$ssh = "True"
$createvmfs = "False" # Creates a Datastore1 VMFS volume on every host if true

# Actions - pay attention when making changes below - things may break #

$numberNestedESXi = (100 + $numberNestedESXiInput)
. 'C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI\Modules\VMware.VimAutomation.Core\VMware.VimAutomation.Core.ps1'
Connect-VIServer -Server $viServer -User $viUsername -Password $viPassword

101..$numberNestedESXi | Foreach {
   $ipaddress = "$iprange.$_"
    # Try to perform DNS lookup
    try {
        $vmname = ([System.Net.Dns]::GetHostEntry($ipaddress).HostName).split(".")[0]
        write-host "Resolved $vmname"
    }
    Catch [system.exception]
    {
        $vmname = "vesxi-$ipaddress"
        write-host "Set VMname to $vmname"
    }
    # Make my nested ESXi VM already!
    Write-Host "Deploying $vmname ..."
    $vm = new-vm -Name $vmname -Datastore $destDatastore -ReferenceSnapshot $sourceSnapshot -LinkedClone -VM (get-vm $sourceVM) -vmhost $destVMhost

    # Add advanced parameters to VMX
    New-AdvancedSetting -Name guestinfo.hostname -Value $vmname -Entity $vm -Confirm:$false
    New-AdvancedSetting -Name guestinfo.ipaddress -Value $ipaddress -Entity $vm -Confirm:$false
    New-AdvancedSetting -Name guestinfo.netmask -Value $netmask -Entity $vm -Confirm:$false
    New-AdvancedSetting -Name guestinfo.gateway -Value $gateway -Entity $vm -Confirm:$false
    New-AdvancedSetting -Name guestinfo.dns -Value $dns -Entity $vm -Confirm:$false
    New-AdvancedSetting -Name guestinfo.dnsdomain -Value $dnsdomain -Entity $vm -Confirm:$false
    New-AdvancedSetting -Name guestinfo.ntp -Value $ntp -Entity $vm -Confirm:$false
    New-AdvancedSetting -Name guestinfo.syslog -Value $syslog -Entity $vm -Confirm:$false
    New-AdvancedSetting -Name guestinfo.password -Value $password -Entity $vm -Confirm:$false
    New-AdvancedSetting -Name guestinfo.ssh -Value $ssh -Entity $vm -Confirm:$false
    New-AdvancedSetting -Name guestinfo.createvmfs -Value $createvmfs -Entity $vm -Confirm:$false
    
    $vm | Start-Vm -RunAsync | Out-Null
    Write-Host "Starting $vmname"

}


Old script (for deploying the OVF)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
# William Lam
# www.virtuallyghetto.com

. "C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI\Scripts\Initialize-PowerCLIEnvironment.ps1"
$vcname = "192.168.2.200"
$vcuser = "root"
$vcpass = "password"

$ovffile = "%userprofile%\Desktop\Nested ESXi\Nested_ESXi_Appliance.ovf"

$cluster = "VSAN Cluster"
$vmnetwork = "Lab_network"
$datastore = "SSD1"
$iprange = "192.168.10"
$netmask = "255.255.255.0"
$gateway = "192.168.10.254"
$dns = "192.168.10.254"
$dnsdomain = "test.local"
$ntp = "192.168.10.254"
$syslog = "192.168.10.150"
$password = "password"
$ssh = "True"

#### DO NOT EDIT BEYOND HERE ####

$vcenter = Connect-VIServer $vcname -User $vcuser -Password $vcpass -WarningAction SilentlyContinue
erver $vcenter -Confirm:$false
$datastore_ref = Get-Datastore -Name $datastore
$network_ref = Get-VirtualPortGroup -Name $vmnetwork
$cluster_ref = Get-Cluster -Name $cluster
$vmhost_ref = $cluster_ref | Get-VMHost | Select -First 1

$ovfconfig = Get-OvfConfiguration $ovffile
$ovfconfig.NetworkMapping.VM_Network.value = $network_ref

190..192 | Foreach {
    $ipaddress = "$iprange.$_"
    # Try to perform DNS lookup
    try {
        $vmname = ([System.Net.Dns]::GetHostEntry($ipaddress).HostName).split(".")[0]
    }
    Catch [system.exception]
    {
        $vmname = "vesxi-vsan-$ipaddress"
    }
    $ovfconfig.common.guestinfo.hostname.value = $vmname
    $ovfconfig.common.guestinfo.ipaddress.value = $ipaddress
    $ovfconfig.common.guestinfo.netmask.value = $netmask
    $ovfconfig.common.guestinfo.gateway.value = $gateway
    $ovfconfig.common.guestinfo.dns.value = $dns
    $ovfconfig.common.guestinfo.domain.value = $dnsdomain
    $ovfconfig.common.guestinfo.ntp.value = $ntp
    $ovfconfig.common.guestinfo.syslog.value = $syslog
    $ovfconfig.common.guestinfo.password.value = $password
    $ovfconfig.common.guestinfo.ssh.value = $ssh

    # Deploy the OVF/OVA with the config parameters
    Write-Host "Deploying $vmname ..."
    $vm = Import-VApp -Source $ovffile -OvfConfiguration $ovfconfig -Name $vmname -Location $cluster_ref -VMHost $vmhost_ref -Datastore $datastore_ref -DiskStorageFormat thin
    $vm | Start-Vm -RunAsync | Out-Null
}


17 August 2016

AHCI controller passthrough with a Supermicro Xeon-D motherboard

So someone asked me if it is possible to passthrough the onboard AHCI SATA controller from a Supermicro Xeon-D motherboard to a VM. Since I use ESXi, that's what I'll use to show you this. Hyper-V 2016 features discrete device assignment in the latest technical preview builds, you can try this with Hyper-V too. Maybe I'll do a blogpost on that some other time.

[10 feb 2021 - Update] This still works in ESXi 7.0.1 but the passtru.map has been placed in the subfolder /etc/vmware/.

I started by having a look in the list of devices that are available for passthough. The Lynx Point AHCI Controller was not on the list of devices since SATA controllers are unsupported for passthrough. Let's fix that!

Log in to your host using SSH or use the DCUI locally.
Find out the PCI ID using the following command:
esxcli storage core adapter list

Your onboard SATA controller is usually listed as vmhba0. The ID we're looking for is listed in the Description. So in this case, it's 0000:00:1f.2.

Enter the next command to find the PID. Substitute the PCI ID with your own ID if needed.
lspci -n | grep 0000:00:1f.2
In this case, the PID we're looking for is 8c02. Add this PID at the bottom of /etc/passthrou.map like so.
Edit: In ESXi 7.01, this file can be found here: /etc/vmware/passthru.map
Save your changes and reboot the ESXi host.

After the reboot, you'll be able to select the Lync Point AHCI controller.

Mark the device for passthrough and reboot the host.

When the host is up again, edit the VM settings to add the PCI device.


After adding the PCI device to the VM, boot it to see the result.












In my case, the VM I added the controller to is a Windows 7 virtual machine. To show you the controller and the connected disks, I added a screenshot of the device manager. I have two Samsung 850 Pro SSDs connected to the onboard SATA controller.
Reminder: Since the files for the VM itself have to be stored somewhere, you'll need another storage device. In my case, all the VM files are stored on an NVMe SSD. This method can be used to create a dedicated storage VM with a number of hard drives. It's a great way to give a FreeNAS VM direct access to a number of SATA drives.

Many thanks to Hilko for showing me the commands in the "Energy efficient ESXi server" thread on got.tweakers.net. Most of this blogpost is a shameless copy of his post.