Homelab part 3: Management

In my opinion, a homelab should be volatile. Most of my lab is used in a simple cycle:

1. You think of something to try, test or wreck.
2. You build it as fast as possible while cutting as little corners as you can in order to make sure the results are valid.
3. Execute your test plan.
4. Evaluate the results to see if they are as expected. Troubleshoot or repeat the tests as necessary.
5. Decide if you need your setup for more tests. If yes, shut it down or safe. If in doubt or you no longer need the setup, delete all the bits.

This cycle does not mean I try one thing at a time. What it does mean is that I try to remove clutter as much as possible.

In order to set up a lab as fast as possible, there are some parts of the lab I don't rebuild with every test. This is the management part of the lab. To make sure the management stuff doesn't get wiped, I put up a separate physical server just to host the management roles. So what are these roles?

• vCenter server for the deployment of templates and tracking of performance over longer periods of time. I also use the vSphere Web Client to manage the lifecycle of the virtual machines. 
• Sexilog to collect logs and alerts and display them on a dashboard.
• A Windows virtual machine to use as an RDP jump box to the lab when I'm not at home. This is also the Windows server that runs all the PowerCLI/PowerShell scripts in the homelab.
• A virtual NAS to store ISOs, templates and random bits of data. This is also the primary data storage device in the house containing all the photos, documents and other important data.

I specifically chose not to set up a domain controller because I prefer to set up a fresh copy for every test (a simple PowerCLI/PowerShell workflow makes this really easy). This way I know for certain that specific settings I use for one setup don't interfere with another.

The resources needed for these workloads are quite modest. This enables me to look at a low-power option that is affordable (usually, the nice low-power servers that have some grunt are costly. The Supermicro SYS-E200-8D for example). After a lot of contemplation I decided to try a very low cost option and see if I could make it work. I had 16GB DDR3 SO-DIMM and a PicoPSU lying around. That should be enough to run my management workloads. I went looking for a low-power motherboard with a few CPU cores so I wouldn't have to worry about CPU contention. Because of this reason, a quad core CPU was preferable to a dual core option.
I found an Asrock N3700-ITX and I decided to give it a shot. It looked a bit underpowered with a quad core Atom (Braswell) processor, passive cooling and four SATA ports. The N3700 provides a little higher turbo speed over the N3150. No idea if this really helps but the price difference is small enough to try. If I didn't have a PicoPSU, I'd have bought the Asrock N3150DC-ITX because it has a 12V DC input and comes with the appropriate 65W adapter.

The first attempts to get ESXi to run on the system were unsuccessful. Many thanks to Antonio Jorba for solving the problem. Deploying the vCenter appliance was simple enough once I figured out how to connect to the Host Client and such. An SSD stores all of the virtual machines and two connected 5TB disks take care of storage for the virtual NAS. Running just ESXi 6.0 idle with a single SSD connected uses 11W (balanced power management). The complete system with the disks and the virtual machines running uses around 25W. That equals about 50 euros a year in power if I leave it running 24/7. So it meets the requirements.

Shopping list:

• Asrock N3700-ITX
• PicoPSU 90W
• 80W 12V Leike adapter
• 2x 8GB Corsair Vengance
• Samsung SM843T - 480GB
• Random shoe box I dug out of the waste paper bin

I'm still looking for a nice case to put the board, SSD and two HDDs into. Something the size of an average shoe box would be perfect. If you have a good suggestion, let me know!

Homelab part 2: router and networking

I've been an ADSL/VDSL internet customer with the same telco provider for many years. They offer me a simple internet connection with an Arcadyan VGV7519 modem/router/wireless access point style device. The telco I chose does not limit the functions of their modem or limit access to the webinterface. This allows me to make any configuration changes I want (requiring frequent resets to factory defaults when I first started tinkering). There's one specific feature I'm really happy about: bridge mode! This feature puts the all-in-one device into a modem only mode. The first Gigabit ethernet port on the modem becomes an unfiltered TCP/IP connection with a public DHCP IP address from my provider. Why does this feature make me happy? Because the Arcadyan is not as stable as I'd like. During the time I've used it as a modem/router, I had to reset it about once every week. Always around 21:00 in the evening. This doesn't sound to bad but the wife agreed it was an annoyance. Internet problems while watching a movie or her favorite series is a no-go. Time for improvements! Hence bridge mode. Being able to provide a gigabit ethernet cable to a router of choice is a big plus.
While exploring the wonderful world of cheap routers I came across Mikrotik. This Eastern European
company makes network devices that can be described as true jack-of-all-trades. Most models they make combine a hardware switch chip with a processor, some RAM and a wireless antenna. Throw in some Linux based software and a GUI with a gazillion buttons and you have an ultimate nerd device. If you want it to be a simple managed switch, it can do that. If you want it to be a router with multiple routing protocols (MPLS, BGP and OSPF, to name a few) it can do that too. Being able to do all I need from my network in a single affordable box is a big plus. Requirement: Hardware reset possible is met by allowing the wife to pull a single plug to reset internet access.

The Mikrotik rb2011uias-2hnd-in:

• Offers 5Gbit and 5 Fast Ethernet ports (plenty for my lab)
• Does NAT routing to the internet with minimal CPU load
• Does DHCP for all the networks
• Hosts some DNS zones for the lab
• Terminates my SSTP VPN tunnels (both for site2site tunnels and remote access when I'm not at home)
• Splits my network into two VLANs: normal network and lab network
• Offers separate SSIDs for normal network, lab network and a guest wifi network that is isolated and has a limited bandwidth
• Routes between the networks
• Creates graphs of the network traffic on every interface
• Firewalls internet traffic based on ports and mangle rules
• Has the ability to run virtual RouterOS or OpenWRT instances (multiple routing instances, yeeh!)
• Has a small touchscreen for quick interface configuration or graphs
• Uses about 10 Watts

So far I'm really happy with it and I keep thinking of new things I can do with it. Next up is trying to set it up as a wireless access point controller. I've been eyeing the RBwAPG-5HacT2HnD, a dual band AC wireless access point to make this work throughout the house.

Homelab part 1: requirements

I run a homelab where I play with a lot of new technology and I like to tell you about my setup. I have a number of demands (honestly, most of them are my wife's demands) that I have to adhere to:

• Low Power - It's nice to have a full enterprise environment to play with at home but there's a limit to how much I want to pay for such a playground. Power costs around 2 euros for every Watt burned 24/7. To meet this requirement I've decided to split up my lab into two distinct parts with a different purpose. Part 1 is the always-on stuff. The equipment that offers the core infrastructure at home (also used by the wife, so it has to be stable and easy to reset). Part 2 is the lab itself, my playground where I can build and tear down to my hearts content. Since this equipment only runs when I'm actively using it, it can be a more power hungry setup.
• Low Noise - I like silence - so hearing a jet engine-like sound in the background when I'm at home playing with my lab is not something I want. A homelab has to be silent! More about this under the next bullet.
• Smallish - The best room 'in the house' to host my equipment is the shed. While this may sound like bad idea, it's not. The shed is underneath the kitchen in one corner of the house. It's dry, has a relatively constant temperature and is connected to the house for power and networking. Since the kitchen has a heavy and solid floor that offers excellent noise isolation. This means I can house noisier stuff, hooray! The kitchen floor is built using big wooden beams that offer a nice space between them. While this space will comfortably fit a number of 2u rack servers, there is a limit to what it can accommodate in size and weight. 
• Fast - There's no joy in waiting for installations or configurations. I usually want to try and replicate a very specific setup and I tear down the virtual setup as soon as my tests are done. This usually means I start with an empty slate every time I decide to try something. There's no joy in having to invest multiple hours every time I want to see the effects of a single configuration change. The faster I can build and set up the test environment, the better!
• Hardware Reset Possible - My wife has to be able to restore internet connectivity without using a single web interface or login. This means that all the devices used for the internet connectivity have to cope with a reset by power plug removal. If I'm not at home and the wife calls to tell Netflix isn't working, I want to be able to say "Don't worry darling, just pull the plug to reset it." This requirement eliminates the possibility for virtual appliances to deliver core network services. With a virtual appliance I cannot say "See the red box? Reset it by pulling the plug."

As a general rule of thumb I like integration where possible and separation where needed to get to a homelab setup that is as simple as I can make it without sacrificing functionality. Putting all the equipment behind a single power supply is a big plus as it drives efficiency. Separating lab and important data is a must as I regularly wipe and rebuild the lab to try different hardware based products.
If price was no object, I'd probably buy a nice 4 node in 2u appliance with a lot of SSDs. If it were possible to make a heterogeneous appliance that'd be my dream. One Xeon-D based low-power node for the always-on part and three Xeon E5-26XXv4 nodes with lots of compute power and memory to run beastly virtual labs.

Start of a blog

So this is the start of a blog. What do I hope to achieve by maintaining this blog? Mostly an archive of my own findings and solutions while doing my work. If you like what you read, great! Maybe you'll like some other posts as well. Have a look.

About me

I work for a Managed Service Provider in the IT space for ten years and counting. While most content will be work related, opinions are my own and do not reflect the position and/or opinion of my employer.
You can also find me on

-- Linkedin here --

.
Content
Since I work for an MSP (Managed Service Provider) in the Netherlands, most of the content will cover IT topics and related technology. In recent years I've been focusing on storage and server virtualization. Since this is the corner of IT where I spend most of my time, I expect the majority of content to cover this area.
Homelabs in general and improving mine in particular takes up a lot of my time at home. I spend many evenings tinkering away at systems I'll probably never use. Some of this research and/or interesting findings will be posted as well. Some posts might be considered failures from the start because of the ludicrous ideas that are tried and tested anyway. Warning, not all tests have a happy ending.