Mid term report

Enhancing vSphere Networking Automation Framework Ashok P

Abstract:
Virtual Networking is software mainly used for Virtualization of Networking system in
ESX, Testing of this Virtual Networking manually is a time consuming and tedious job hence we
use a automation framework called ―vdNet Framework‖ which is internally developed by
VMware to automate testing of this Virtual Networking system.

Driver Test Manager (DTM) is Microsoft’s test automation framework for Windows
device driver testing & certification tests. It provides automated scheduling and execution of the
driver test that hardware and device driver developers are required to pass for Certification and
―Certified for Windows‖ logo use.
Even though DTM is an automated test suite few initial steps (Setup part) are manual and
doing all those setup part manually is a time consuming and difficult job. Hence integrating
DTM into vdNet Framework gradually reduces the time required for testing drivers in DTM.

Manipal Centre for Information Science, Manipal Page 1


Contents:
1. Introduction 3

2. Product Overview 6

3. Virtual Networking 9

4. vdNet Framework 11

5. Driver Test Manager 14

6. Contribution and Details of Proposed Work 20

7. Bibliography 21



1. Introduction
Virtualization is an abstraction layer that decouples the physical hardware from the operating
system to deliver greater IT utilization and flexibility. It is a framework or methodology of
dividing the resources of a computer into multiple execution environments, by applying one or
more concepts or technologies such as hardware and software partitioning, time-sharing, partial
or complete machine simulation, emulation, quality of service, and many others.

Virtualization allows multiple virtual machines, with heterogeneous operating systems to run in
isolation, side by side on the same physical machine. Each virtual machine has its own set of
virtual hardware (e.g., RAM, CPU, NIC, etc.) upon which an operating system and applications
are loaded. The operating system sees a consistent, normalized set of hardware regardless of the
actual physical hardware components. Virtual machines are encapsulated into files, making it
possible to rapidly save copy and provision a virtual machine. Full systems (fully configured
applications, operating systems, BIOS, and virtual hardware) can be moved, within seconds,
from one physical server to another for zero-downtime maintenance and continuous workload
consolidation.

VMware virtualization solutions have enabled IT organizations of all sizes to deliver resources
and applications when and where they’re needed, enabling business to respond to market
dynamics more quickly and more efficiently then ever.

Virtualization is the catalyst that makes IT as a Service a reality. It is the enabling technology on
which cloud computing architectures are and will be built. Our customer proven VMware
vSphere virtualization platform dramatically expands IT effectiveness and efficiency, going
beyond basic virtualization to deliver critical management and quality-of-service capabilities that
provide a complete cloud infrastructure solution. Working in concert with our partners, we’re
helping businesses of all sizes accelerate their transition to cloud computing, a flexible new
approach to delivering applications and services at the lowest cost and with the highest
availability.
Only VMware provides an evolutionary approach to cloud computing—the fastest and most
pragmatic path— that extends existing applications and preserves IT skill sets. VMware offers
the most trusted solutions for transforming your current IT environment to deliver optimal agility



and flexibility, while preserving and even enhancing security and control. Chosen by both
enterprises and public cloud providers as their preferred cloud infrastructure platform, VMware
vSphere provides compatibility and portability between a secure private cloud and VMware
enabled public clouds.
We typically describe three key properties of virtual machines that are responsible for their
powerful benefits. Let’s examine each one of those properties
1. The first key property is partitioning. Virtual machines allow a single computer to be
divided into separated partitions that can each run an operating system and application
stack concurrently. In fact, those virtual machines can be running completely different
operating systems and software because they each have their own virtual storage
locations, memory spaces and networking interfaces. A component of the VMware
virtualization layer called the virtual machine monitor manages the concurrent execution
of each virtual machine on the host system hardware. Typically we’d actually see a ratio
of about 4 to 8 running virtual machines per physical CPU. The networking and storage
features of virtual machines let you use them just as you would real machines in
networked configurations or joined together in clusters for high-availability.
2. Our second critical feature is isolation which is critical for safe and reliable server
consolidation.
VMware Virtual machine monitors use the hardware protection features of the
CPU to isolate the virtual machines from each other and monitor. By basing our
isolation on the hardware protection we get very strong isolation. In other words,
there is unlikely to be a hole. Each virtual machine is isolated from the host and
other VMs, in the sense that it doesn’t share a kernel or processes.
In a real environment, what this means is that applications in one virtual machine
can encounter viruses or blue screen their operating system, and there is no effect
on any other virtual machine. In fact, we had the U.S. National Security Agency
try to hack from one virtual machine to another for over a year and they couldn’t
find any weaknesses to exploit. That proven isolation strength has led the NSA to
approve VMware technology for running insecure off-the-shelf software on their
secure machines.



Resource controls & isolation features give application owners full protection
from the stability and performance problems of other applications. A virtual
machine with an application leaking memory or a runaway process consuming
CPU can only use as much of the host resources as you’ve allocated to that virtual
machine. The neighboring virtual machines will retain their allocations of CPU,
memory, disk I/O and network I/O.
3. Our third primary feature of virtual machines is encapsulation. The complete state of a
virtual machine – memory, disk storage, I/O device and CPU state, and virtual hardware
configuration – is stored in a small set of files. These files are hardware independent so
you can move a virtual machine from one x86 system – say a Dell server– to another –
say an IBM server – and that virtual machine will run with no changes necessary as long
as the VMware virtualization layer is present. An encapsulated virtual machine is at a
minimum just the virtual machine configuration file (a small text file defining the virtual
machine’s properties) and the virtual disk file that contains its installed operating system.
A snapshot of a running virtual machine would add files encapsulating the memory and
processor state of the virtual machine so that a point-in-time image of a running virtual
machine can be saved and reverted to at any time. Encapsulation means that your ability
to copy, save, and move virtual machines wherever and whenever you need them is as
simple as copying a directory of files.

Fig .1.1 Without virtualization Fig.1.2 With virtualization



2. Product Overview
VMware provides variety of products to IT industry. VMware build an agile, efficient
and self-service infrastructure with virtualization and deliver IT as a service. Deploy highly
available applications with VMware, the leader in virtualization, from the desktop to the data
center to the cloud. VMware products can be mainly grouped into two major lines. They are
Server and Datacenter Virtualization Products and Desktop Virtualization products.

2.1 Server and Datacenter Virtualization Products

Turn your datacenter into a flexible cloud infrastructure with the performance and reliability
needed to run enterprise applications on the platform trusted by the most demanding datacenters
around the world. Leverage existing assets and applications while offering self-service
deployment and provisioning through virtualization. Create a private cloud and deliver IT
infrastructure as an easily accessible service.

Deliver IT as a service, when and where needed, while reducing capital and operating costs.
Reduce power needs while freeing your IT staff from administrative tasks so they can focus on
innovative solutions to your changing business needs. The available products in this category
are:

2.1.1 VMware vSphere
VMware vSphere is VMware's first cloud operating system, able to manage large pools
of virtualized computing infrastructure, including software and hardware. Offer the highest
levels of availability and responsiveness for all applications and services with VMware vSphere,
the industry’s most complete and robust platform for virtualization in the most demanding
datacenters around the world. Create resource pools so you can deliver the highest levels of
application service agreements with the lowest total cost per application workload. Virtualizes
your business-critical applications for unprecedented flexibility and reliability.

2.1.2 VMware vCenter Server
VMware vCenter Server provides a scalable and extensible platform that forms the foundation
for virtualization management. VMware vCenter Server, formerly VMware Virtual Center,



centrally manages VMware environments allowing IT administrators dramatically improved
control over the virtual environment compared to other management platforms. VMware vCenter
Server:
Provides centralized control and visibility at every level of virtual infrastructure
Unlocks the power of vSphere through proactive management
Is a scalable and extensible management platform with a broad partner ecosystem

Fig.2.1. VMware vCenter Server

2.1.3 VMware vSphere Hypervisor

VMware vSphere Hypervisor is based on VMware ESXi, the hypervisor architecture that sets the
industry standard for reliability, performance and ecosystem support. VMware vSphere
Hypervisor is the easiest way to get started with virtualization—and it’s free. Consolidate your
applications onto fewer servers and start saving money through reduced hardware, power,
cooling and administration costs. With VMware vSphere Hypervisor, you can:
Run multiple applications on a single server
Run a greener datacenter and reduce energy costs
Back up and recover applications more easily
Virtualizes even business-critical applications



2.2 Desktop Virtualization product
Deploy desktops faster and improve business continuity and disaster recovery while
reducing capital and operating system costs through desktop and application virtualization.
Migrate and upgrade operating systems and applications with less downtime, eliminate the need
to recode, retest and recertify applications, and get more out of your existing desktop assets.
Reduce the need for IT admin staff at remote and branch offices by delivering virtual
desktops to remote and temporary users while protecting corporate data. Centralize desktop
management and speed up desktop deployments while reducing both operating and support
costs.

2.2.1 VMware Workstation
VMware Workstation 7 is optimized for maximum performance when running on 32-bit
and 64-bit Windows 7 PCs. Supports Flip 3D and Aero Peek to show live thumbnails of your
virtual machines. Run legacy application with 3D graphics, faster performance, and tighter
desktop integration better than Windows XP Mode. Get shared folders and drag and drop
convenience.
Automate and streamline tasks to save time and improve productivity. Run applications
in Linux, Windows, and more at the same time on the same PC with no rebooting. Evaluate and
test new operating systems, applications and patches in an isolated environment. Demonstrate
complex software applications on a single laptop in a repeatable, reliable manner. Rich
integration with Visual Studio, Eclipse, and the Spring Source Tool Suite make it incredibly easy
to debug applications on multiple platforms.



3. Virtual Networking
3.1 VMware Virtual Networking Concepts
VMware Infrastructure provides a rich set of networking capabilities that integrate well with
sophisticated enterprise networks. These networking capabilities are provided by VMware ESX Server
and managed by VMware Virtual Center. With virtual networking, you can network virtual machines in
the same way that you do physical machines and can build complex networks within a single ESX Server
host or across multiple ESX Server hosts, for production deployments or development and testing
purposes.
Virtual switches allow virtual machines on the same ESX Server host to communicate with each
other using the same protocols that would be used over physical switches, without the need for additional
networking hardware. ESX Server virtual switches also support VLANs that are compatible with standard
VLAN implementations from other vendors.
A virtual machine can be configured with one or more virtual Ethernet adapters, each of which
each has its own IP address and MAC address. As a result, virtual machines have the same properties as
physical machines from a networking standpoint. In addition, virtual networks enable functionality not
possible with physical networks today.
This guide is for VMware Infrastructure 3 users who want a clearer understanding of the
basic design of the networking capabilities in VMware Infrastructure 3 and how that design
affects deployment in the datacenter.



Fig 3.4 virtual switches and virtual NICs

3.2 VMware Virtual Networking Components
In discussions of VMware Infrastructure 3, you may see references to as many as five different
virtual network adapters. Three of those are virtual Ethernet adapters used by virtual machines.
In most cases, a virtual machine uses only one of the three types of virtual adapters. The three
types of adapters available for virtual machines are:
vmxnet — a paravirtualized device that works only if VMware Tools is installed in the
guest operating system. A paravirtualized device is one designed with specific awareness
that it is running in a virtualized environment. The vmxnet adapter is designed for high
performance. In most cases, when you select the flexible network adapter, this is the
adapter used after VMware Tools is installed in the guest operating system.
vlance — a virtual device that provides strict emulation of the AMD Lance
PCNet32Ethernet adapter. It is compatible with most 32-bit guest operating systems. This
adapter is used when you select the flexible network adapter but VMware Tools is not
installed in the guest operating system.
e1000 — a virtual device that provides strict emulation of the Intel E1000 Ethernet
adapter. This is the virtual Ethernet adapter used in 64-bit virtual machines. It is also
available in 32-bit virtual machines.
The other virtual network adapters are:
vswif — a paravirtualized device similar to vmxnet that is used only by the ESX Server
service console.
vmknic — a virtual device in the VMkernel, the software layer that manages most of the
physical resources on the ESX Server hosts. The vmknic is used by the TCP/IP stack that
services VMotion, NFS and software iSCSI clients that run at the VMkernel level, and
remote console traffic.
All five of the virtual network devices share the following characteristics:
They have their own MAC addresses and unicast/multicast/broadcast filters.
They are strictly Layer 2 Ethernet adapter devices.



4. vdNet Framework
vdNet framework is developed to verify all the networking features (vmkernel + virtual
devices). Automation setup includes a launcher machine, which is also called as Master
Controller (MC), ESX hosts and multiple test VMs among which one VM is called System under
Test (SUT) and other VMs are called "helpers". Selected tests are executed on the SUT. Helper
VMs are used to send/receive/verify networking traffic as part of executing the tests. The MC
machine has to be a Linux VM (specifically RHEL 5.2 32-bit VM or the pre-configured MC) or
a physical Linux host that is connected to the same corporate network as Host and Test VMs.

4.1 Network Setup
vdNet framework is designed and implemented based on the following assumption on the
network setup. Framework is not guaranteed to work if the network setup is not done as
mentioned here.



The Master controller is connected to the corporate network (VM Network) which should be able
to access any hosts/VMs under test.
2 isolated networks (broadcast domains) are created. One is control network which is under
corporate network and other one is non-routable test network.
VM Network port group in ESX is by default connected to corporate network.
Only one adapter in each VM/host should be under control network. All the other virtual and
physical adapters should be connected to the test network.

4.2 STAF Automation
STAF is an Open Source automation framework designed around the idea of reusable
components. It is intended to make it easier to create automated test cases and workloads. STAF can
help you increase the efficiency, productivity, and quality of your testing by improving your level of
automation and reuse in your individual test cases as well as your overall test environment..

STAF runs as a daemon process (called STAFProc) on each system. So, for example, if you
wanted to run STAF on your office machine and 5 test machines in a lab, you would install STAF on
all 6 systems. Then, to use STAF in this environment, you would start STAFProc on all 6 machines.
The collection of machines on which you have installed STAF is referred to as the STAF
Environment.

Fig 3.7 STAF Working environment



STAF operates in a peer-to-peer environment; in other words, there is no client-server
hierarchy among machines running STAF. Fig 3.7 illustrates that the STAFProc daemons serve as
the communication mechanism over the network. Hence VMware uses the STAF in order to
automate the testing process and also to run the tests in the remote machines also.

4.3 Perl Scripting Language

Perl is an acronym, short for Practical Extraction and Report Language. It was designed
by Larry Wall as a tool for writing programs in the UNIX environment and is continually being
updated and maintained by him.

Like shell script languages, Perl does not require a special compiler and linker to turn the
programs you write into working code. Instead, all you have to do is write the program and tell
Perl to run it. This means that Perl is ideal for producing quick solutions to small programming
problems, or for creating prototypes to test potential solutions to larger problems.

In VMware, we use Perl scripting language in order to write Perl scripts, by using these
scripts we can automate the testing process.



5. Driver Test Manager
5.1 Microsoft Driver testing

Driver Test Manager (DTM) is Microsoft’s test automation framework for Windows device driver
testing & certification tests. It provides automated scheduling and execution of the driver test that
hardware and device driver developers are required to pass for
Certification and “Certified for Windows” logo use.

For device drivers after successfully passing the certification test,
Microsoft creates a digitally signed certificate file that when included in
the driver installation package prevents Windows from displaying a
warning message stating the driver has not been certified by Microsoft.
(See WinXP version of pop-up on the right) Figure 1 – Example Pop-up warning

5.2 VMware and DTM
The goal is to use Microsoft’s DTM test kit at VMware for two main purposes:

Third party QA test suite to insure VMware’s Windows drivers are being developed and tested
to Microsoft’s standards throughout the driver and driver environment development cycles.
Certification test kit to obtain digital signatures avoiding pop-ups which provide customers
increased confidence that VMware has performed all certification tasks required for a Microsoft
compliant certified driver.

5.3 Microsoft Test Kit History

HCT: Released during NT 4.0 days (1996), used until late 2006 when Windows Vista was
released. This was a single machine, single instance test kit.

DTM/WLK: Released in late 2006 along side with the Vista release. At that time, DTM was part
of Windows Driver Kit (WDK). Later DTM was separated from WDK and changed to its current
name, Windows Logo Kit (WLK); this is why it has 2 commonly used names. This is the first
standalone test kit for Windows certification that requires multiple machines and a database to
run tests.



VMware started certifying drivers since WS 4.0 and has continued since. ESX products starting
requiring certified drivers since ESX 3.0. Certified drivers allows for silent installs, and also acts
as proof to our customers that we are delivering quality products that are tested and certified to
meet MSFT Windows OS requirements and guidelines.

5.4 Architecture

How DTM Works

DTM Clients

DTM Studio DTM Controller

…

Up to 150 DTM Clients
Copyright © 2005 VMware, Inc. All rights reserved.

5.5 Driver Test Tracks
Unclassified: Also called DRS (Driver Reliability Signature). This is the program most of our
drivers are tested and certified under. These are tests that generically stress a driver but do not
run any feature specific tests (unless it’s a USB device on Win7/Win2k8 R2 or later OS, then
some basic USB tests will be enumerated). Setup for this program is simple. Takes ~1-2 days to
setup and run all tests for one driver.
LAN -> Networking: Includes all of the ―Unclassified‖ tests, as well as a handful NDIS
tests. Setup for this program is complex, requiring an additional VM to support each
system under test and modifications to the ESX kernel and networking settings for server
testing. Takes ~3-4 days to setup and run all tests for one driver.
Storage -> Adapter (SAS): ): Includes all of the ―Unclassified tests, as well as a handful
of Storage specific stress and functionality tests. Setup for this program is complex,
requiring additional software and disk management/setup. Takes ~6-7 days to setup and
run all tests for one driver.



5.6 DTM Automation
Even though DTM is an automated test suite few initial steps are still manual and yet to
be automated. End to End DTM automation and integrating it to the vdNet framework reduces
the time required to setup the test environment.

Following are the steps to initiate DTM tests and needs to be automated:

Note: Assuming that VM repository contains a VM with a DTM controller and DTM
studio already installed in it.

Clone (Linked clone) the DTM controller VM into your ESX box and power on the VM.
Check whether DTM controller/DTM Studio VM is connected to Public vSwitch if not
connect it to Public vSwitch.
There are two types of submission Unclassified Test Procedure and Networking Test
Procedure.
For Unclassified Test we need only one DTM client VM.
For Networking Test, two DTM clients must be used. One DTM client (NDIS Client
VM) is needed to hold the ―Device under Test‖. Another DTM client (NDIS Server VM)
is needed to receive and send data to the device under test while testing is in progress.
Picking up the VM’s from the repository
 For Unclassified Tests pick up one VM from the repository and install DTM
client.
 For Networking Tests pick up 2 VM’s (Win2k8 RTM/R2 or Win7) from the
repository and install DTM client on both the VM’s (NDIS Client – Device under
test, NDIS Server – Helper machine).
 Disable all firewalls (if applicable).
 Disable all virus scanning software (if applicable).
 Disable all malware scanning software (if applicable).
 Install all security updates and/or SP updates from Windows Update (if applicable).
Depending upon whether it is Unclassified Test or Networking Test, clone (Linked clone)
the VM’s into your ESX box.
All the VM’s should be on same subnet.
Install VMware debugging tools on ESX server and configure vsish parameter with
following command
 set config/Misc/intOpts/VmkStressEnable 0
Disable ESX firewall and configure security policy to ―Allow Promiscuous Mode‖.
Configure VLAN ID to 4095(All).
Make sure your VMs are setup with the following .vmx configuration file
chipset.onlineStandby ="true"
For Networking Tests configure the First VM (NDIS Client VM) with three vNICs (2 in
public network, 1 in private network) and second VM (NDIS Server VM) with two



vNICs (1 public, 1 private). These vNICs must be of the same device (i.e., vmxnet2 or
vmxnet3) with same driver version installed on all of these vNICs.
Check whether all VM’s (DTM controller, NDIS client and NDIS server) are
communicating with each other.
Launch the DTM studio in the DTM controller VM.
Check whether both client and server machines are present in the default machine pool
(One VM in case of unclassified test).
 If any one VM in not present the test case fails.
Create the machine pool of your own.
Move the client and server VM’s from default machine pool into the newly created
machine pool.
Change the status of both the VM’s to reset mode if not able to change the status to reset
mode then the test case fails.
Go to device console and select New submission
 Select OS type, category, logotype, qualification level.
 Name the submission and select the machine pool.
 Load the corresponding INF file.



ESX Server
DTM Controller pNIC 1
vNIC 1

pNIC 2

Public vSwitch pNIC…

Test device Remote support

NDIS Client NDIS Server
vNIC 1
VM VM

vNIC 2 vNIC 1

Local support
vNIC 3 vNIC 2

Local message Remote Message
Private vSwitch

For Networking Tests start all the test cases and properly choose the following adapters
appropriately as shown in the following diagram
 Local Support: - This device (vNIC) communicates with the DTM controller. It also
communicates test status information to the remote support device in the remote test
machine via the support network and connected to public vSwitch to acquire IP address
from DHCP server.
 Local Message: - This vNIC is connected to Private vSwitch and need to assign manual
IP address (192.168.0.1/2 or similar).
 Remote Support: - This device communicates with the DTM controller. It also
communicates test status information to the local support device in the local test machine
and connected to public vSwitch to acquire IP address from DHCP server.



 Remote Message :- This vNIC is connected to Private vSwitch and need to assign
manual IP address((192.168.0.1/2 or similar).

Save the settings and schedule the jobs.
Once the testing is complete create a submission package (.cpk format) and save it and
launch DTM log viewer (already installed) in DTM controller VM and pass the .cpk file
to view the log file.

·



6.Contribution Details
1) Initial ramp up

 Understanding Networking Concepts

 Running Sample test cases in vdNet Framework

 DTM Setup

 Training on vSphere 4.1

2) Understood Basics of Network Driver Interface Specification (NDIS).

3) Wrote Perl Script to automate DTM (Standalone) for WIN2K3.

4) Running DTM test suits.

Details of Proposed Work
1) Integrating standalone DTM automation into vdNet Framework.
2) Running WLK test suit for different GOS’s.
3) Getting more familiar with vdNet Framework and fixing PR’s.



7. Bibliography
1. http://en.wikipedia.org/wiki/Windows_Hardware_Certification_Kit

2. http://staf.sourceforge.net/current/STAFPerl.htm


Mid term report

More Related Content

What's hot (20)

Similar to Mid term report (20)

Recently uploaded (20)

Mid term report