Broadcom announces VMWare licensing Changes

 Today we wake up with the answer to the long awaited question of how Broadcom would change the license requirements for VMware products.

Related : VMware by Broadcom Dramatically Simplifies Offer Lineup and Licensing Model - VMware News and Stories

In the Monday announcement Broadcom has eliminated perpetual licensing and a significant number of product variations.

The Register summarized some of these changes and the Compare Editions page has now been updated.

It still remains to be seen how this will impact customers in the long term, but overall I think this is a better overall outcome then we all feared.

The change to subscription was anticipated, although I think many of us assumed it would be simple elimination of SKU's.

With the revision of the bundles and the way vSphere is presented overall I anticipate that we will be required to analyze the value of the components more effectively for our customers as we present expansions to the current environment or re-evaluate the SnS renewals.

I don't want to go to much deeper in this post but I did want to bookmark these sources since I am certain they will be valuable for future discussions, or so that we can track changes if this plan evolves.

Jason

VMware VCAP-DCV Design

I reached a milestone last week! I passed the VCAP-DCV Design 2023 and finally completed my VCIX-DCV path.

For those that are looking for study resources I have included my recommendations below.

For the Deploy exam I drew most of my study from lab time. You can use the VMware Hands-On-Labs or refer to your Production environment. There are a handful of scenarios that you have to solve. If you want to see examples you can check out this YouTube playlist from the VMUG in SriLanka - Question:01 - YouTube

If you are ambitious enough to create your own HomeLab whether it is Physical hardware or using VMware Workstation (included in Advantage) on your desktop - don't forget that VMUG has the Advantage program that give you access to a huge number of products for a full year.

» VMUG Advantage Membership

For the Design exam I started with this site: 

VMware vSphere 8.x Advanced Design Exam (3V0-21.23) Study Links – vcdx133.com

The first 2 links really are the most important.

I also added any or ALL of the IT Architect Series Books - 

Amazon.com: IT Architect Series: Foundation in the Art of Infrastructure Design: A Practical Guide for IT Architects: 9780996647755: Arrasjid, VCDX-001 John Yani, Gabryjelski, VCDX-023 Mark, McCain, VCDX-079 Chris: Books

IT Architect Series: The Journey: A Guidebook for Anyone Interested in IT Architecture: Palmer, Melissa: 9781483473635: Amazon.com: Books

IT Architect Series: Stories from the Field, Vol. 1: Arrasjid, John Yani, Wood, Matthew, Gabryjelski, Mark, Dangerous Age, Ioannis: 9780999092927: Amazon.com: Books

I didn't read this one but it was recommended by others - IT Architect Series: Designing Risk in IT Infrastructure, Behr, Daemon, eBook - Amazon.com

One of the things that I was really stuck on was how to properly Calculate and perfectly right size the number of hosts required when given the existing CPU/ Memory or Storage utilization. 

This book did a great job of explaining overhead requirements and N+1 or N+2 architecture. - The Complete VMware vSphere Guide: Design a virtualized data center with VMware vSphere 6.7: 9781838985752: Computer Science Books @ Amazon.com

I had access to some of these through an Oreilly.com subscription so you might have more affordable ways to get to some of these.

If you want to follow some YouTube videos, I would highly recommend the vBrownbag EMEA Series that was done for vSphere 6, the information is still HIGHLY relevant and Rebecca Fitzhugh's video on Objective 1.3 (old objectives) had enough examples that I was able to better understand the relationship between the Functional and Non-Functional Requirements and their respective Design Qualities.

The Design exam is going to be retired on Version 7 at the end of this year and the 8 will be available in January or February so the first link is updated with the 8 info.

Installing ESXi 8.0 on legacy hardware.

VMware deprecated a significant amount of hardware and operating systems with the ESXi 8.0 release. However, it is still possible to install on unsupported hardware at your own risk.

This information is primarily intended to support the homelab community but could be used as a reference for other use cases.

The initial part of setup works as expected. You set your default keyboard language.

Set the root password

and then you are presented with the following screen .......

ESXi 8 supports TPM 2.0 and above for advanced security. TPM 1.2 or lower is not supported by these functions. However, you may be offered the option to override if other functionality is supported.

Acknowledging the warning sends the next dialog...

press Enter to confirm that you want to Force the Installation and the installer will continue.

You will be able to complete the steps for setup. Some features may not be available and your system will likely be at risk. 

WARNING: This is NOT recommended for a production environment. Make a business case and spend the money on the proper hardware before using this option in a critical setting.

Hope you found this useful!

Jason

installing VMware ESXi version 8 over existing partition

When installing ESXi or vSphere on a host which has an existing bootable partition from am older version or perhaps vSan data on existing partitions, some special steps may need to be take .

This is primarily for homelab users, but can serve as a reference for other situations.

During the installation process, the installer will notify you if partitions already exist. In my case I had one drive that was for booting the prior version and 2 vSAN drives which I plan on rebuilding. The installer flags these with the * and # symbols as noted in the key above.

I want to install over my prior boot media, so I select the drive that "* Contains a VMFS partition", but you have to be cautious here. VMFS partition doesn't just mean prior boot media, it could mean a VMFS datastore. Some systems may have been configured with a local datastore, overwriting this partition would cause a permanent loss of data. I am hopeful that you do not need this data and the point is moot, but be vigilant none the less.

Once the drive is selected and confirmed, a new dialog appears if a prior version is detected. You are offered the choice to "Upgrade" or "Install" if upgrading is a supported option.

As I am intending on overwriting this media, I select "Install" and press Enter to confirm.

You will be presented with the choice for the default keyboard and the installer will continue from this point as if it was a new system.

Hope this helps!

Jason

Why can't I shut down my vSAN cluster?

 I ran into a bit of an odd issue today.

After having a load of trouble with a bad boot media on vSAN cluster, I had actually managed to stabilize it and had been running it for a few months.

Today I wanted to shut it down and do some maintenance, dusting , wireing , expansion, that type of stuff.

I followed my usual process of placing the hosts into Maintenance mode with "No Data Migration"

https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.vsan-monitoring.doc/GUID-31B4F958-30A9-4BEC-819E-32A18A685688.html

This follows along with the steps I had done many times before in my homelab, but today was different.

In this case I had joined the hosts to a vCenter hosted on another cluster. Previously I would have had the vCenter on the vSAN cluster. I would have shutdown the vCenter, then powered off the hosts.

I was able to place Host 1 and 2 into maintenance, but Host 3 would NOT go into maintenance. The tasks showed that it was waiting for the VMs to be powered off or migrated.

I had already disabled DRS, I had disabled HA. The VMs were powered off. So I figured I would just remove them from inventory... they weren't important to me.

Yet the issue persisted.

So I clicked on Host 3 and checked the VMs

Lo and behold there were 2 vCLS machines still sitting on the host.

So now I am trying to shutdown the vCLS systems , but they are throwing Access Denied errors.

I little bit of digging into the previous article uncovers this note..

"For vSphere 7.0 U1 and later, enable vCLS retreat mode. For more information, see the VMware knowledge base article at https://kb.vmware.com/s/article/80472."

In order to cleanly shutdown my vSAN cluster, I would need to enable a vCLS retreat mode.

The expectation is that with the addition of an "Advanced Setting" I will be able to enable or disable the Cluster Services in the future. The article makes it pretty clear that it is not meant to be run "disabled" so they also offer advice for scripting the change to make it easier to build into a "shutdown" or "startup" workflow.

Hope this article helps, I will update with pictures later as I go through the process.

Jason

VCAP - DCV Deploy 2021

 I Passed!

Absolutely beside myself. Could not have done it without all the support from my family and my team. On to the next challenge!

https://www.linkedin.com/posts/jasonvalentine_vmware-certified-advanced-professional-activity-6861835156651618304-lIVn

VCP-DCV 2021

 I am very excited to have completed my 4th refresh of the VCP-DCV Certification on Oct 20th. I even passed with my highest score ever. 412.

I was fortunate enough to get a Tech+ Pass to VMworld 2021 gifted through the VMUG program and I vowed not to let it go to waste.

So after a brief discussion with my wife, I determined that I could complete 2 certification tests using the discounted vouchers between October 4th and November 4th.

One would be a VCP refresh and the other would be a VCAP-Deploy attempt.

I know some folks like stories, so here is how I prepped and passed this test in only 2 weeks.

Let me preface by saying the test was not at all 'easy'. I have now tested on 3 Major version of ESXI , 5.1 , 6.5 and 7.0

The older tests were very much focused on bare technical specifications, troubleshooting and detailed configuration tasks. The current test adds in so much more content for the "modern hybrid cloud datacenter". Trying to remember the technical, the troubleshooting and the configuration while ADDING information about technologies that you may not be seeing yet is no easy feat.

I was fortunate though that there had been an updated release of the Official Certification guide from Pearson. Where the prior guides were very heavy on screenshots and processes, this seemed to be just walls and walls of text. That is not to say that it is bad, it was extremely useful. The alternative is to RTFM on docs.vmware.com , according to the exam blueprint. At the least the Certification guide provided "Key Points" to narrow the focus a bit.

I read it cover to cover and found myself recharged when reading processes that I was familiar with.

Sometimes though, the best way to remember something is to see it done..

So I FIRED UP THE LAB.... and it died... hmmm.. probably should have swapped out that USB boot media when I had the chance. Probably will need to figure out how to recover vSAN with a lost host...

So I FIRED UP THE OTHER LAB ! And I performed an upgrade of the vCenter to 7.0.3!

I tried to upgrade the hosts, but the boot media on these were internal SD cards... error "not enough space for upgrade"

So I powered off and removed one of the blades to get a closer look at the SD cards. 2GB Mirror in the Dell M520 blades. 2GB?! ah , ok , wow.

So I slotted it back in with the intention of addressing that again later, and thats when one of the RAID controllers failed.

I only know that because I tried to refresh vCenter after repowering the blade and realized it was not loading. A quick look at the hosts showed why. The snapshot chain had become corrupt. Trying to commit the snapshots failed. Trying to consolidate the virtual machine, failed.

At least the domain controller survived..... nope. Black screen , restarted, crashed on boot. Direct to Windows recovery options. I could get into directory services restore mode, but no repairs would stick. Dang.

So vSAN lab is down,  Dell Lab is down.. This would be a great opportunity to learn some things... but I am short on time. 

So, on a prior recommendation from a friend and colleague , I checked out a 7 day trial of CBT nuggets. Fortunately they had an updated course for the VCP-DCV 2021 2V0-21.20 exam presented by Keith Barker. I wanted to see some specific topics so I jumped around a bit, but I did find Keiths style very engaging. More than a few times things didn't go 'as planned', sometimes this was on purpose , other times it was the lab he was using, but it was interesting that some of these things were left in.. because, honestly things do not always go right and it would be wrong to present only 100% success every time.

Ultimately I was able to salvage a domain controller from the vSAN lab, (host 2 is dead in the water though) ... 

I built a new vCenter and Domain controller on the Dell kit.

Read some more, watched Keith for a few hours, ran a couple of scenarios on my labs, ran a couple of scenarios on hol.vmware.com.

Then, I got a good nights rest.

I went to the testing center, sat for the exam, second guessed myself a few times. Reviewed the questions that I had "marked" and determined that it was probably more harm than good to go back to them (it was only 9.. and of those 4 were marked because I thought I might need them)

I clicked the button to submit. AND BANG! Screen says congratulations.

Very relieved. I don't have to take it again, I can sit for the VCAP-DCV Deploy now... so it starts all over again..

Wish me luck on November 2nd!

Jason

"INVALID" machines after vSAN shutdown.

 In some business environments or homelabs it may be necessary to shutdown a vSAN cluster entirely.

VMware has a proper procedure outlined on the documents site.

Shutting Down and Restarting the vSAN Cluster (vmware.com)

Once you have completed your maintenance tasks and fire everything up, you may be startled when you login to each of the hosts and attempt to verify your VMs....

They will probably all display as an "INVALID" status with the GUID Path in the "name" field.

fig.1 - Host 1 - 2 "invalid" machines

fig.2 - Host 2 - 2 "Invalid" machines

This message certainly caught one of my peers off guard during a recent datacenter update. That quick rush of butterflies in your gut that tells you all of your data is now corrupt.

However, I quickly reassured him that nothing was lost and the issue was simply that all of the hosts were still in maintenance mode.

When the vSAN nodes are in maintenance mode after a reboot, none of the stats are correct since none of the vSAN components are allowed to work together. 

fig.3 - 0Byte vsanDatastore

If you have powered on all of the vSAN nodes in the cluster and waited a sufficient amount of time for all the vSAN nodes to communicate, the remediation is actually quite swift.

Remove all the Hosts from Maintenance mode.

fig.4 - Host 2 - Still in Maintenance Mode!

The names will re-appear for each of the Guest objects.

fig.5 - Host 1 - Proper names!

fig.6 - Host 2 - Proper names again!

If all of your workload is on these hosts, you can start your DNS server, then the vCenter server.

fig.7 - vCenter operational again.

Before you know it , you are back in business.

Congratulations to the vExperts!

 

Today , VMware released the names of new and returning vExperts for 2021.

vExpert 2021 Award Announcement

I was honored and humbled to be able to stay with this program for my 4th year.

I believe in the VMware eco system and have seen how transformational it has been in my business and I enjoy sharing stories and advice with others on the subject.

I was fortunate enough to join the VMUG in Denver at a great time and found a group of people that understood my language. After being a member for several years, I volunteered my time to hep organize and plan activities for the local groups along with 2 other amazing leaders. The planning and organization of the 2018 Denver UserCon was intense, but attending on the event day was awe inspiring. So many VMware enthusiasts that I spoke with that day thanked us for helping to provide them with so much information about the products and services that could help advance their careers and businesses.

I had to relocate for work to Orlando and immediately sought out the local chapter and immediately volunteered. Once again I found myself connected with passionate leaders and VMware advocates. I was fortunate enough to attend a few in person meetings before the pandemic fully took hold , and I am looking forward to the day that we can get together again.

I am looking for new ways to support and add to the community.

I have started this blog as a way of documenting things that I find interesting and new.

I am looking at the option to livestream some lab sessions.

I answer questions on the Spiceworks forums for those that find themselves in a bind.

Jason Valentine's SpiceWorks

Having access to a community that understands your needs and daily challenges is such a profound help and I am hopeful that I give as much  as I take.

Jason

Updating the DNS Search domains and FQDN Hostname for an ESXi Host.

 Here is a quick tip for anyone that has deployed an ESXi host from a home network or received DHCP values that didn't match your intended configuration.

I deployed a machine that I want to use in my home lab, but my default Comcast domain is displayed as the FQDN.

I have already changed this in the DCUI , but it persists in the vSphere client.

The configuration is found on the default TCP/IP stack, Click Networking

Select the TCP/IP Stacks tab

the DNS configuration shows my Domain name as hsd1.fl.comcast.net ,but my proper domain name is in the search domains box. Lets clean this up.

Edit the Default TCP/IP Stack - note the Domain name and Search Domain fields.

Update the Domain name and Search Domains fields to remove and replace the incorrect domain.

Refreshing the host client shows the updated values!

Hope it helps!

Jason

Reusing Disks from a Previous vSAN

 This post may be mostly relevant to home-lab builders, but if you have previously configured vSAN on the disks used in host, you may not be able to easily reuse them in a new vSAN.

Lets take a look at my specific use case.

I want to install vCenter 7 and I want to establish vSAN on the host during this process.

For this installation I am going to install on host ESX03 , I previously used the internal disks for a vSAN cluster, but I am upgrading from hybrid to all flash using a 1TB SSD that I recently acquired.

In the datastore part of the installer , I select "Install on a new vSAN cluster containing the target host"

This will create a standalone vSAN node, configuration of the cluster is completed later when additional hosts are added.

Once I get to the section to claim my disks, I can see that I only have one disk and it is the new 1TB disk. My existing 120GB drive does not appear.

On the host I can see that this 120GB disk does exist.

Selecting the disks shows me that there is an existing vSAN partition table. We need to clear this partition.

Select Actions -> Clear Partition Table

Note the warning, erasing this partition is destructive and will remove the data from the disk that is essential for the previous vSAN.

I know that I will not need that data, so I can proceed.

There are no partitions on the disk.

Refreshing the vCenter installer by going back and then forward again. I can now see my 120GB drive available to claim

I select my 1TB drive

and claim it for the capacity Tier. I can now proceed with the remainder of the vCenter deployment

Hope this helps someone 

Jason

Probably a bright spot for HomeLabs in the ESXi 7.0 U1C release.

 I was looking up the release notes for the 7.0U1C update and spotted this bit of good news.

"With ESXi 7.0 Update 1c, you can use the installer boot option systemMediaSize to limit the size of system storage partitions on the boot media."

https://docs.vmware.com/en/VMware-vSphere/7.0/rn/vsphere-esxi-70u1c.html

I remember this being a design consideration for the new stack that I was deploying at work. I needed to have more than 150GB for the boot media.

Considering that I have 512GB per host there, it probably doesn't make sense to use this new option...

but on my homelab with 24GB of RAM per host, or an embedded deployment in VMware workstation... this could save money and space.

Jason

Happy New Year!

 This year I am continuing my journey with VMware solutions and I want to share more with the community along the way.

I am currently studying to attempt the VCAP tests for DataCenter Design and Deploy.

I have an expanded homelab with some upgraded components and I will be doing some live streams as I study for the test. 

Jason

vDS Design Decisions and Consequences ...

I wanted to share an example of unexpected problems with Network design in vSphere.

These are not problems you probably experience in a properly protected production cluster, but I like to fix things that are broken.

When I deployed my lab machines I built a vSphere distributed switch and then followed the recommendations to build out my PortGroups.

One of the selection points is the Port Binding method. The selections are Static or Ephemeral.

Normally I just leave this as Static so that the system will provision ports from the virtual switch as needed to serve the Guests.

http://www.vmwarearena.com/vsphere-distributed-switch-part-14/

However , this caused me some trouble.

At some point , I crashed my cluster and had some trouble recovering vCenter. I tried to get it started, but I couldn’t get it to connect to the network.

I had to create a “Recovery” port group on a new standard switch and assign it to an open Network adapter. I then changed the Network on the Guest to “recovery”

With the guest attached to this network I was allowed to start it and get networking….. So I wonder, what was the problem….

Well , it turns out that without vCenter , Static port binding can’t function because it doesn’t have awareness of all the resources.

http://www.vstellar.com/2017/10/12/distributed-switch-port-group-bindings/

http://www.joshodgers.com/2014/11/23/example-architectural-decision-port-binding-setting-for-a-dvportgroup/

“4. New Virtual machines cannot be powered on and connected to a dvPortGroup (VDS) when vCenter is down.”

It seems this also applies when a virtual machine has been registered on another host without vCenter.
So , I got the system running for a few months like that and forgot to migrate it back to the vDS… which wasn’t a big deal because I didn’t need vCenter to move to any other hosts any way…. Until I did.


vCENTER FAILURE :

This weekend I had a problem with Host 1… the single host that vCenter could run on. The host was severely crippled and I could not restart any guests properly.
So, I found I had one option – Unregister all the machines on Host 1 and Re-Register them on Host 3.

I re-registered my PSC , vCenter and DC, but once I started them , I realized that I only had networking on my vCenter…

Well , that was because I created the “Recovery” port group and moved the Cable from Host 1 to Host 3…

So , I shut everything down.

Assigned the DC to the “Recovery” network – Powered On

Waited for start and confirmed networking.

Assigned the PSC to the “Recovery” network – Powered On

Waited for start and confirmed Access to VAMI

Powered on vCenter – Waited for start and confirmed Login.

FINALLY , I was able to get into vCenter.

I login and look around for signs of problems… ironically , my vSAN appears healthy, the HOST 1 is ok (DRS is disabled) … But there are a handful of “GUEST NAME(Orphaned)” machines. There doesn’t seem to be any way to remove them from the inventory or delete the reference.

I realize that they will probably clear up if I put HOST1 into maintenance mode. It takes a few minutes, but all the machines register on other hosts and are no longer orphaned.

CLEANUP:

In order to avoid problems in the future, I identified that my “Critical” virtual machines should be associated with a vDS port group that can survive this type of issue. While the Port Binding was Static , it was not helping when the VM was disconnected from the switch

I migrated all other machines off the 192-PtG and  changed to Ephemeral Port Binding. I was then able to  save the configuration.

I then migrated my VM’s from the Recovery Network to the vDS port group.

Completing this wizard moves all 3 machines with no errors.

Now I can go Shutdown Host 1 to troubleshoot and turn DRS back on in my cluster without fear of machines moving to bad hosts.


CAUTION: There are warnings about using Ephemeral port binding in situations where security is paramount. This example was from my lab and this solves a problem in that environment. Always research the appropriateness of a solution for your use case before making changes....Protect your vCenter and this is not an issue.

https://kb.vmware.com/s/article/1022312

http://www.yellow-bricks.com/2011/06/02/ephemeral-ports/

Thank you,

Jason Valentine

MCP, VCP-DCV 2019, vExpert 2019, CMNO

Hello World!

This is my first of many discussions on Vmware solutions , virtualization use cases and advancements and  the computing industry as a whole.

I am a long time veteran of VMware products starting with my first introduction professional introduction to the products in 2006 when the first VMware Server for Windows product was released free.

I built several use cases in the years following , deploying Vmware server 1.03 , Server 2.0 , ESXi 35 and Finally deploying ESXi 4.1 in production in July of 2010.

I lived in Florida for more than 32 years and developed most of my Vmware experience with PGT Industries, a window and door manufacturing company in Venice FL.

In 2013 I moved out to Denver,CO to provide better experiences for my family but continued to manage the environment remotely until 2015 when I had the opportunity to work with a small MSP in Denver. I gained a significant level of experience with evaluating and creating solutions for clients in the healthcare, retail as well as the oil and gas space.

In 2017, I returned to working with PGT to help manage and integrate some VMware sites that they acquired and have continued to help refresh and integrate those environments.

In 2018 I started to participate in the VMware community by answering questions in the VMware forum on Spiceworks.com. This proved to be an interesting and challenging experience and I continue to frequent their forums.
I had also been a long time attendee of the VMUG local meetings since moving to Denver and took the chance to apply as a co-leader and have been volunteering time with that organization.
I applied for vExpert the same year and was granted my first star.

I have long been an advocate for the VMware learning program and have continually challenged myself to become and stay VCP certified. I first trained in 2011, then Certified in 2015 - VCP-DCV 5.

In 2017 I had to recertify and passed the 5.5 test

This year I have completed my renewal for the 2nd time and on the 6.5 test. I am currently VCP-DCV 2019 certified.

Thank you for visiting my blog, I hope to find useful content and discussions to post here.

Jason