2017 VMUG Storage Policy Based Management

Editor's Notes

• #3: Data, and most especially what you do with it to offer new/better experiences for your customers, is going to be the key differentiator between you and your competition
• #4: Self-driving cars – Other projections state that they will generate 1GB of data per second. Equifax – personal data from 143 million US citizens. Cost CxOs their jobs. Hurricane Irma in the US, – Are you prepared for Disaster Recovery? Now what if you put these 2 together? What if someone hacked a self-driving car?
• #8: VVOLS KB - https://kb.vmware.com/kb/2113013 Storage providers inform vCenter Server about specific storage devices, and present characteristics of the devices and datastores (as storage capabilities).
• #10: Storage Policy-Based Management (SPBM) is the foundation of the VMware SDS Control Plane and enables vSphere administrators to over come upfront storage provisioning challenges, such as capacity planning, differentiated service levels and managing capacity headroom, whether using vSAN or Virtual Volumes (VVols) on external storage arrays. SPBM provides a single unified control plane across a broad range of data services and storage solutions. The framework helps to align storage with application demands of your virtual machines. SPBM is about ease, and agility. Traditional architectural models relied heavily on the capabilities of an independent storage system in order to meet protection and performance requirements of workloads. Unfortunately the traditional model was overly restrictive in part because standalone hardware based storage solutions were not VM aware, and were limited in their abilities to unique settings to various workloads. Storage Policy Based Management (SPBM) lets you define requirements for VMs or collection of VMs. This SPBM framework is the same framework used for storage arrays supporting VVOLs. Therefore, a common approach to managing and protecting data can be employed, regardless of the backing storage. ---------------------------------- Overview: Key to software defined storage (SDS) architectural model SPBM is the common framework to abstract traditional storage related settings away from hardware, and into hypervisor Applies storage related settings for protection and performance on a per VM, or even per VMDK level ----------------------------------
• #11: https://blogs.vmware.com/vsphere/2014/10/vsphere-storage-policy-based-management-overview-part-2.html
• #13: Common Rules – these come from I/O Filters on hosts (VMCrypt, SIOCv2, VAIO) Rule-Sets come from storage, either vSAN or VVOls.
• #16: http://cormachogan.com/2013/09/06/vsan-part-5-the-role-of-vasa/
• #17: Defining a policy will let vSAN use “what if” APIs so that you can see the “result” of having such a policy applied to a VM of a certain size. Very useful as it gives you an idea of what the “cost” is of certain attributes. Mirroring = RAID-1 Erasure Coding = RAID-5/RAID-6
• #18: Key Message/Talk track: Creating a storage policy is nothing more than defining what your requirements are for a VM, or a collection of VMs. These requirements are typically around protection and performance of the VM. A new policy can be created and applied to a VM, or an existing policy can be adjusted. The VM will adopt the new performance and protection settings without any down time. ---------------------------------- Overview: Policies define levels of protection and performance Applied at a per VM level, or per vmdk level vSAN currently provides five unique storage capabilities to vCenter Server ---------------------------------- Details: Storage policy rules available (in 6.6) are: Number of disk stripes per object Flash read cache reservation (%) Primary level of failures to tolerate (PFTT - for stretched clusters) Secondary level of failures to tolerate (SFTT – for local protection) Failure Tolerance method Affinity IOPS limit for object Disable object checksum Force provisioning Object space reservation (%) Defining a policy will let vSAN use “what if” APIs so that you can see the “result” of having such a policy applied to a VM of a certain size. Very useful as it gives you an idea of what the “cost” is of certain attributes. ----------------------------------
• #19: Key Message/Talk track: After a policy is created, it can easily be applied to an individual VMDK of a VM, an entire VM, or a collection of VMs in the data center. Applying at a VMDK level can be useful for applications that have different needs within defined drives of of the guest OS. For instance, a drive dedicated for the database may have different requirements than the drive dedicated for transaction logs. ---------------------------------- Overview: When the policy is selected, vSAN uses it to place/distribute the VM to guarantee availability and Performance Policies can be changed without any interruption to the VM ---------------------------------- Details: Defining a policy will let vSAN use “what if” APIs so that you can see the “result” of having such a policy applied to a VM of a certain size. Very useful as it gives you an idea of what the “cost” is of certain attributes. Only one SPBM policy is allowed to be applied. vSAN does not support the appended SPBM policies. Policies can also be assigned by rules or tags. An example might be all VMs with “Prod-SQL” in the VM name or resource group might be set at RAID-1 and an FTT=2. VM named “Test-Web” would never be applied to this SPBM policy, and would adopt the default policy for the environment. ----------------------------------
• #20: Key Message/Talk track: Failures to Tolerate (FTT) is a rule that defines how many failures can be tolerated to let the VM or other object continue to run in the event of a failure. This is one of the key pillars behind vSAN’s ability to protect a VM from failure of a fault domain (disk, disk group, host, defined fault domain, or site) ---------------------------------- Overview: “FTT” defines the number of hosts, disk or network failures a storage object can tolerate. For “n” failures tolerated, “n+1” copies of the object are created and “2n+1” host contributing storage are required! Primary Failures to Tolerate (PFTT) defines the number of sites that can accept failure. (0, 1) Secondary Failures to Tolerate (SFTT) defines the number within a site that can accept failure (0, 1, 2, 3) ---------------------------------- Details: FTT can and will be dependent on a number of factors. A few important factors include: The number of hosts in the vSAN cluster The Failure Tolerance Method (FTM) that is defined for the object. Using a RAID-1 (mirroring) Fault Tolerance Method (FTM), an FTT of 2 would mean that a minimum number of hosts in a cluster would be 5. FTT=3 would require 7 hosts. Number of Failures Mirror copies Witnesses Min. Hosts Hosts + Maintenance 0 1 0 1 host n/a 1 2 1 3 hosts 4 hosts 2 3 2 5 hosts 6 hosts 3 4 3 7 hosts 8 hosts ---------------------------------------------------------------------------------------------- There is also Primary and Secondary Failures to Tolerate (PFTT and SFTT) are for vSAN stretched clusters PFTT defines the number of sites failures (0, 1) SFTT defines the number of failures within a site (0, 1, 2, 3)
• #21: Key Message/Talk track: This policy, sometimes known as “stripe width” defines the minimum number of capacity devices across which each replica of a storage object is distributed. Increasing the predefined number of stripes per object beyond 1 is intended to help performance. ---------------------------------- Overview: Defines the minimum number of capacity devices across which each replica of a storage object is distributed. Higher values may result in better performance. Stripe width can improve performance of write Destaging, and fetching of uncached reads Higher values may put more constraints on flexibility of meeting storage compliance policies To be used only if performance is an issue ---------------------------------- Details: Most beneficial on the following scenarios: A non cached read on a hybrid configuration, where one is typically reliant on the rotational latencies of a single spinning disk. Reads on an all-flash configuration, where fetching I/O may be able to be improved in some situations. Destaging buffered writes to persistent tier (all flash, or hybrid). This will relieve some of the backpressure that could be induced by large amount of write activity, whether they are sequential or random in nature. vSAN may create more stripes than what is defined. With DD&C, component A with a strip width of 1 will not necessarily live just on disk 1, but rather, be sprinkled around the various capacity disks of a disk group. It becomes an implicit stripe width setting, but will not show up in the UI as a traditional change in a stripe width. Component size can impact stripe width, as an object over 255GB will be split into two components. This however could end up on the same disk, or a different disk group. ----------------------------------
• #22: Key Message/Talk track: A failure tolerance method (FTM) is the way data will maintain redundancy. The simplest FTM is a RAID-1 mirror. This would have a mirror copy of objects/components across multiple hosts. Another FTM is RAID-5/RAID-6, where data is striped across multiple hosts with parity information written to provide tolerance of a failure. Parity is striped across all hosts. When done over the network using software only, this is sometimes referred to as erasure coding. This is done inline; there is no post-processing required. VMware’s implementation of erasure coding stripes the data with parity across the minimum number of hosts in order to comply with the policy. RAID-5 will offer a guaranteed 30% savings in capacity overhead compared to RAID-1 ---------------------------------- Overview: Available in all-flash configurations only Example: FTT = 1 with FTM = RAID-5 3+1 (4 host minimum, 1 host can fail without data loss) 5 hosts would tolerate 1 host failure or maintenance mode state, and still maintain redundancy 1.33x instead of 2x overhead. 30% savings 20GB disk consumes 40GB with RAID-1, now consumes ~27GB with RAID-5 ---------------------------------- Details: RAID-5/6 does have I/O amplification on writes (only). RAID-5. Single write operation results in 2 reads and 2 writes RAID-6. Single write operation results in 3 reads and 3 writes (due to double parity) RAID-5/6 only supports the FTT of 1, or 2 (implied by choosing RAID-5 or RAID-6). Will not support FTT=0, or FTT=3 The realized dedup & compression ratios will be different when employing RAID-5/6 than when using RAID-1 mirroring. Space efficiency using erasure codes will be more of a guaranteed space reduction because of the lack of implied multiple full copies. Even if the DD&C ratio may be less on objects that use RAID-5/6, the effective overall capacity used will be equal to, if not better than RAID-1 with DD&C. FTM can and will be dependent on a number of factors. A few important factors include: The number of hosts in the vSAN cluster Stripe width defined for the objects Using a RAID-5, and an implied FTT of 1 would mean that a minimum number of hosts in a cluster would be 4. With 4 hosts, 1 host can fail without data loss (but will lose redundancy). To maintain full redundancy with a single host in maintenance mode, the minimum would be 5 hosts. Cluster sizes for RAID-5 need to be 4 or more hosts. Not multiples of 4 hosts. Since VMware has a design goal of not relying on data locality, this implementation of erasure coding does not bring any negative results by distributing the RAID-5/6 stripe across multiple hosts. ---------------------------------- Internal:
• #23: Key Message/Talk track: VMware’s RAID-6 is a dual parity version of the erasure coding scheme used in the RAID-5 FTM. An FTM of RAID-6 will imply an ability to tolerate 2 failures (e.g. FTT=2) and maintain operation. RAID-6 will offer a guaranteed 50% savings in capacity overhead compared to RAID-1 using an FTT of 2. Just as with RAID-5 erasure coding, this is all done inline, with no post processing required. Parity is striped across all hosts. VMware’s implementation of erasure coding stripes the data with parity across the minimum number of hosts in order to comply with the policy. RAID-6 will offer a guaranteed 50% savings in capacity overhead compared to RAID-1 and an FTT of 2. ---------------------------------- Overview: Available in all-flash configurations only Example: FTT = 2 with FTM = RAID- 4+2 (6 host minimum. 1 host can fail without data loss 7 hosts would tolerate 1 host failure or maintenance mode state, and still maintain redundancy 1.5x instead of 3x overhead. 50% savings 20GB disk consumes 60GB with RAID-1, now consumes ~30GB with RAID-6 ---------------------------------- Details: RAID-5/6 does have I/O amplification on writes (only). RAID-5. Single write operation results in 2 reads and 2 writes RAID-6. Single write operation results in 3 reads and 3 writes (due to double parity) RAID-5/6 only supports the FTT of 1, or 2 (implied by choosing RAID-5 or RAID-6). Will not support FTT=0, or FTT=3 The realized dedup & compression ratios will be different when employing RAID-5/6 than when using RAID-1 mirroring. Space efficiency using erasure codes will be more of a guaranteed space reduction because of the lack of implied multiple full copies. Even if the DD&C ratio may be less on objects that use RAID-5/6, the effective overall capacity used will be equal to, if not better than RAID-1 with DD&C. FTM can and will be dependent on a number of factors. A few important factors include: The number of hosts in the vSAN cluster Stripe width defined for the objects Using a RAID-6, and an implied FTT of 2 would mean that a minimum number of hosts in a cluster would be 6. With 6 hosts, 2 hosts can fail without data loss (but will lose redundancy). To maintain full redundancy with a single host in maintenance mode, the minimum would be 7 hosts. Cluster sizes for RAID-6 need to be 6 or more hosts. Not multiples of 6 hosts. Since VMware has a design goal of not relying on data locality, this implementation of erasure coding does not bring any negative results by distributing the RAID-5/6 stripe across multiple hosts. ----------------------------------
• #24: We get a lot of questions about whether vSAN is available for prime-time production use. With 10,000 customers, vSAN is now used everywhere for all manner of applications. Here is one such example, where vSAN is used in a mission critical role.
• #25: VVol 2.0" refers to additional functionality supported in vSphere for VVol targets written specifically for it, notably replication. Many VVol solutions still offer only what you might call "VVol 1.0". Regardless, the vSphere Compatibility Guide will tell you whether a given VVol storage system is certified to work with vSphere 6.5, which could be "VVol 1.0" or "VVol 2.0". To be clear vSphere 6.5 does NOT REQUIRE "VVol 2.0" on the storage side.
• #26: The IO Blender effect – lots of different I/O types – random/sequential, read/write, different block sizes, being handled by the same LUN. All sorts of mechanisms were introduced to alleviate this situation, such as RAID, wide-striping, QoS, etc. On the vSphere side of things, we introduced SIOC, SDRS, etc. Many customers kept spreadsheets of what VMs were supposed to be on which LUNs for performance and data service purposes.
• #28: VASA providers the Control Plane. PEs provide the Data Plane
• #29: https://blogs.vmware.com/virtualblocks/2016/11/30/vasa-provider-considerations-controller-embedded-vs-virtual-appliance/ VASA Provider in VVols: Provides storage awareness services Centralized connectivity for ESXi and vCenter Servers Responsible for creating Virtual Volumes (VVols) Provide support for VASA APIs used for ESXi Responsible for defining binding operations Offloading VM related operations directly to array
• #30: Why the concept of a PE? In today’s LUN-Datastore world, the datastore has two purposes – It serves as the access point for ESXi to send IO to and it also serves as storage container to store many VM files (VMDKs). This dual-purpose nature of this entity poses several challenges. It should not be necessary to have so many access points to the storage. Because of the rigid nature of the size of the datastore, and the fewer number of datastores, multiple VMs are stored together in the same datastore even if the VMs have different requirements. This leads to the so-called IO blender effect. So, how about we separate out the concept of the access point from the storage? This way, we can fewer number of access points to several number of storage entities. And hence the introduction of PE.
• #31: NFS v41 support statement: https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.storage.doc/GUID-AAA99054-4D81-49F8-9927-65E9B08577AD.html
• #32: During a rescan ESX will identify PE and maintain then in DBs. Multi-pathing on the PE ensures high availability Concept of queue depth in a PE? Yes, PEs are given queue depth of 128. Compare with a LUN which only had 32 or 64, and how many VMs per LUN.
• #33: Need at least 1 SC per array. You can have as many as the array can support. An SC cannot span across array
• #35: Login to UI. Select Administration. Select vSphere Integration. Populate VC info. Select plugins – in this case, web client and VASA Provider.
• #36: Note that not all VASA implementations give you this level of detail. Also, others may take a different approach to configuring PEs and Storage Containers.
• #37: Octo is the name of a “group” on the Nimble Array which I provided as part of the registration – it could be anything.
• #39: Storage = Nimble Storage Add a Rule e.g. encryption Add another rule e.g. protection
• #40: Compatible = nimble. Other refs: https://www.hpe.com/h20195/v2/getpdf.aspx/4AA5-6907ENW.pdf (HPE and Vvols)
• #41: Figures provided by HPE – August 2017 (VMworld 2017 Las Vegas)
• #42: https://code.vmware.com/programs/vsphere-apis-for-io-filtering
• #43: https://code.vmware.com/programs/vsphere-apis-for-io-filtering IO request moving between the guest operating system (Initiator), located in the Virtual Machine Monitor(VMM), and a virtual disk (Consumer) are filtered through a series of two IO Filters (per disk), one filter per filter class, invoked in filter class order. For example, a replication filter executes before a cache filter. Once the IO request has been filtered by all the filters for the particular disk, the IO request moves on to its destination, either the VM or the virtual disk. Partner will develop IO Filter plug-ins to provide filtering virtual machines. Each IO Filter registers a set of callbacks with the Filter Framework, pertaining to different disk operations. If a filter fails an operation, only the filters prior to it are informed of the failure. Any filter can complete, fail, pass, or defer an IO request. A filter will defer an IO if the filter has to do a blocking operation like sending the data over the network, but wants to allow further IOs to get processed as well. If a filter performs a blocking operation during the regular IO processing path, it would affect the IOPS of the virtual disk, since we wouldn't be processing any further IOs until the blocking operation completes. If the filter defers an IO request, the Filter Framework will not pass the request to subsequent filters in the class order until the filter completes the request and notifies the Filter Framework that the IO may proceed.
• #44: Available since vSphere 6.5.
• #45: https://www.vmware.com/resources/compatibility/search.php?deviceCategory=vaio 6 certified partner VAIO products, out of which 3 are Cache and 3 are Replication. Cache accelerators using local flash devices (or some memory) to accelerate reads, and sometimes writes.
• #46: Available since vSphere 6.5.
• #47: This is before I added the I/O Accelerator from Infinio. These are provided by default in vSphere.
• #48: When the policy has been created, it may be assigned to newly deployed VMs during provisioning, or to already existing VMs by assigning this new policy to the whole VM (or just an individual VMDK) by editing its settings.
• #49: What is the relationship between vCenter Server and KMS Server? VMware vCenter now contains a KMIP client, which works with many common KMIP key managers (KMS). VMware does not own the KMS. Plan for backup, DR, recovery, etc., with your KMS provider. You must be able to retrieve the encryption keys in the event of a failure, or you may render your VMs unusable. Administrators should not encrypt their vCenter Server. Possible “chicken-and-egg” situation where you need vCenter to boot (KMS client) so it can get the key from the KMS to unencrypt its files, but it will not be able to boot as its files are encrypted. vCenter Server does not manage encryption. It is only a client of the KMS. With VM Home encrypted, only administrators with ‘encryption privileges’ can access the console of the virtual machine. One misconception: VM Home folder is not encrypted. Only some files in the VM Home folder are encrypted. Some (non-sensitive) VM files and log files are not encrypted. Core dumps are encrypted on ESXi hosts with encrypted VMs. Encrypted virtual machines cannot be exported to an OVF, nor can they be suspended.
• #50: The VM Encryption and SIOC are available by default. Infinio is a third party plugin for cache acceleration - I installed this separately.
• #51: http://www.infinio.com/sites/default/files/resources/Case%20Study%20-%20UG%20Center%20and%20Hotel%20-%20FINAL.pdf
• #53: Screenshots courtesy of http://www.virtualjad.com/2017/05/scoop-vrealize-automation-7-3.html https://blogs.vmware.com/virtualblocks/2017/05/23/storage-policy-based-management-vrealize-automation/
• #54: I don’t know much about this, but I believe that changing the policy will also Storage vMotion the VM to another datastore that meets the policy requirements – checking with Jad.
• #56: When you use Virtual SAN, Horizon defines four virtual machine storage requirements, such as capacity, performance, and availability, in the form of default storage policy profiles and automatically deploys them for virtual desktops onto vCenter Server.  The policies are automatically and individually applied per disk (Virtual SAN objects) and maintained throughout the lifecycle of the virtual desktop. Storage is provisioned and automatically configured according to the assigned policies. You can modify these policies in vCenter.  Horizon creates vSAN policies for linked-clone desktop pools, instant-clone desktop pools, full-clone desktop pools, or an automated farm per Horizon cluster.