vDS Design Decisions and Consequences ...

I wanted to share an example of unexpected problems with Network design in vSphere.

These are not problems you probably experience in a properly protected production cluster, but I like to fix things that are broken.

When I deployed my lab machines I built a vSphere distributed switch and then followed the recommendations to build out my PortGroups.

One of the selection points is the Port Binding method. The selections are Static or Ephemeral.

Normally I just leave this as Static so that the system will provision ports from the virtual switch as needed to serve the Guests.

http://www.vmwarearena.com/vsphere-distributed-switch-part-14/

However , this caused me some trouble.

At some point , I crashed my cluster and had some trouble recovering vCenter. I tried to get it started, but I couldn’t get it to connect to the network.

I had to create a “Recovery” port group on a new standard switch and assign it to an open Network adapter. I then changed the Network on the Guest to “recovery”

With the guest attached to this network I was allowed to start it and get networking….. So I wonder, what was the problem….

Well , it turns out that without vCenter , Static port binding can’t function because it doesn’t have awareness of all the resources.

http://www.vstellar.com/2017/10/12/distributed-switch-port-group-bindings/

http://www.joshodgers.com/2014/11/23/example-architectural-decision-port-binding-setting-for-a-dvportgroup/

“4. New Virtual machines cannot be powered on and connected to a dvPortGroup (VDS) when vCenter is down.”

It seems this also applies when a virtual machine has been registered on another host without vCenter.
So , I got the system running for a few months like that and forgot to migrate it back to the vDS… which wasn’t a big deal because I didn’t need vCenter to move to any other hosts any way…. Until I did.


vCENTER FAILURE :

This weekend I had a problem with Host 1… the single host that vCenter could run on. The host was severely crippled and I could not restart any guests properly.
So, I found I had one option – Unregister all the machines on Host 1 and Re-Register them on Host 3.

I re-registered my PSC , vCenter and DC, but once I started them , I realized that I only had networking on my vCenter…

Well , that was because I created the “Recovery” port group and moved the Cable from Host 1 to Host 3…

So , I shut everything down.

Assigned the DC to the “Recovery” network – Powered On

Waited for start and confirmed networking.

Assigned the PSC to the “Recovery” network – Powered On

Waited for start and confirmed Access to VAMI

Powered on vCenter – Waited for start and confirmed Login.

FINALLY , I was able to get into vCenter.

I login and look around for signs of problems… ironically , my vSAN appears healthy, the HOST 1 is ok (DRS is disabled) … But there are a handful of “GUEST NAME(Orphaned)” machines. There doesn’t seem to be any way to remove them from the inventory or delete the reference.

I realize that they will probably clear up if I put HOST1 into maintenance mode. It takes a few minutes, but all the machines register on other hosts and are no longer orphaned.

CLEANUP:

In order to avoid problems in the future, I identified that my “Critical” virtual machines should be associated with a vDS port group that can survive this type of issue. While the Port Binding was Static , it was not helping when the VM was disconnected from the switch

I migrated all other machines off the 192-PtG and  changed to Ephemeral Port Binding. I was then able to  save the configuration.

I then migrated my VM’s from the Recovery Network to the vDS port group.

Completing this wizard moves all 3 machines with no errors.

Now I can go Shutdown Host 1 to troubleshoot and turn DRS back on in my cluster without fear of machines moving to bad hosts.


CAUTION: There are warnings about using Ephemeral port binding in situations where security is paramount. This example was from my lab and this solves a problem in that environment. Always research the appropriateness of a solution for your use case before making changes....Protect your vCenter and this is not an issue.

https://kb.vmware.com/s/article/1022312

http://www.yellow-bricks.com/2011/06/02/ephemeral-ports/

Thank you,

Jason Valentine

MCP, VCP-DCV 2019, vExpert 2019, CMNO

Hello World!

This is my first of many discussions on Vmware solutions , virtualization use cases and advancements and  the computing industry as a whole.

I am a long time veteran of VMware products starting with my first introduction professional introduction to the products in 2006 when the first VMware Server for Windows product was released free.

I built several use cases in the years following , deploying Vmware server 1.03 , Server 2.0 , ESXi 35 and Finally deploying ESXi 4.1 in production in July of 2010.

I lived in Florida for more than 32 years and developed most of my Vmware experience with PGT Industries, a window and door manufacturing company in Venice FL.

In 2013 I moved out to Denver,CO to provide better experiences for my family but continued to manage the environment remotely until 2015 when I had the opportunity to work with a small MSP in Denver. I gained a significant level of experience with evaluating and creating solutions for clients in the healthcare, retail as well as the oil and gas space.

In 2017, I returned to working with PGT to help manage and integrate some VMware sites that they acquired and have continued to help refresh and integrate those environments.

In 2018 I started to participate in the VMware community by answering questions in the VMware forum on Spiceworks.com. This proved to be an interesting and challenging experience and I continue to frequent their forums.
I had also been a long time attendee of the VMUG local meetings since moving to Denver and took the chance to apply as a co-leader and have been volunteering time with that organization.
I applied for vExpert the same year and was granted my first star.

I have long been an advocate for the VMware learning program and have continually challenged myself to become and stay VCP certified. I first trained in 2011, then Certified in 2015 - VCP-DCV 5.

In 2017 I had to recertify and passed the 5.5 test

This year I have completed my renewal for the 2nd time and on the 6.5 test. I am currently VCP-DCV 2019 certified.

Thank you for visiting my blog, I hope to find useful content and discussions to post here.

Jason