Introducing

Cisco Unified
Computing System
Learn Cisco UCS with Cisco UCSPE

Stuart Fordham

Aprcss
Stuart Fordham
Introducing Cisco Unified
Computing System
Learn Cisco UCS with Cisco UCSPE

Apress®
Stuart Fordham
Bedfordshire, UK

ISBN 978-1-4842-8985-3 e-ISBN 978-1-4842-8986-0

https://doi.org/10.1007/978-1-4842-8986-0

© Stuart Fordham 2023

This work is subject to copyright. All rights are solely and

exclusively licensed by the Publisher, whether the whole or
part of the material is concerned, specifically the rights of
translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other
physical way, and transmission or information storage and
retrieval, electronic adaptation, computer software, or by
similar or dissimilar methodology now known or hereafter
developed.

The use of general descriptive names, registered names,

trademarks, service marks, etc. in this publication does not
imply, even in the absence of a specific statement, that
such names are exempt from the relevant protective laws
and regulations and therefore free for general use.

The publisher, the authors, and the editors are safe to

assume that the advice and information in this book are
believed to be true and accurate at the date of publication.
Neither the publisher nor the authors or the editors give a
warranty, expressed or implied, with respect to the
material contained herein or for any errors or omissions
that may have been made. The publisher remains neutral
with regard to jurisdictional claims in published maps and
institutional affiliations.

This Apress imprint is published by the registered company

APress Media, LLC, part of Springer Nature.
The registered company address is: 1 New York Plaza, New
York, NY 10004, U.S.A.
To my family.
Introduction
This book is a guide on how to set up and configure a UCS
(Unified Computing System). The beauty of it is that you
don’t need to run out and buy one; you can use the UCS
Platform Emulator running on a virtual machine on a
laptop!
We start with setting up the virtual machine, and then
look at how a real-life install would be completed. From
there we will dive into the hardware that makes up a UCS
system and then look at the networking requirements to
make your UCS talk to the rest of the network. Once we
have this in place, we move on to creating policies and
profiles, which are needed by our servers. After this, we
finish off with security and troubleshooting.
This book is intended for people looking to get an
introduction to Cisco’s UCS.
Acknowledgments
Thanks as always to my family and my amazing team at
work.
Table of Contents
Chapter 1:​Setting Up UCSPE
Setting Up UCSPE
Importing UCSPE into VMWare
Starting UCSPE
Real-World UCS Setup
Summary
Chapter 2:​The UCS Components
Managing UCSPE Hardware
Adding and Removing Devices
Removing UCS Devices
Fabric Interconnects
Adding Devices
Chassis
Blade Servers
FEX
Rack Servers
Direct Attach Mode
Single Wire Management
Dual Wire Management
Enclosures
Summary
Chapter 3:​Northbound Networking and SAN
UCS networking
 Uplink ports
Summary
Chapter 4:​Policies
Creating the UCS Organization
Storage Policies
Dynamic vNIC Connection Policies
Creating VLANs
vNIC/​vHBA Placement
vMedia Policies
Server Boot Policies
Maintenance Policies
Server Pool Policies
Operational Policies
Management IP Addresses
KVM Management Policy
Scrub Policies
UUID Pool
MAC Pools
WWNN
VSAN
Summary
Chapter 5:​Service Profiles and Templates
Creating Service Profile Templates
Summary
Chapter 6:​UCS Security
AAA
Hardening the Web Interface
Summary
Chapter 7:​UCS Troubleshooting
Call Home
SNMP
Logging and Events
SYSLOG
Techsupport Files
Summary
Index
About the Author
Stuart Fordham
, CCIE 49337, is the Network
Manager and Infrastructure Team
Leader for SmartCommunications
SC Ltd, which is the only provider
of a cloud-based, next-generation
customer communications
platform. Stuart has written a
series of books on SD-WAN, BGP,
MPLS, VPNs, and NAT, as well as
a CCNA study guide and a Cisco
ACI Cookbook. He lives in the UK
with his wife and twin sons.
About the Technical Reviewer
Luca Berton
is an Ansible Automation Expert,
who has been working with the
Red Hat Ansible Engineer Team
for three years. With more than
15 years of experience as a
system administrator, he has
strong expertise in infrastructure
hardening and automation. An
enthusiast of the open source, he
supports the community by
sharing his knowledge in different
events of public access. Geek by
nature, Linux by choice, Fedora of course.
© The Author(s), under exclusive license to APress Media, LLC, part of
Springer Nature 2023
S. Fordham, Introducing Cisco Unified Computing System
https://doi.org/10.1007/978-1-4842-8986-0_1
1. Setting Up UCSPE
Stuart Fordham1
(1) Bedfordshire, UK

The Cisco UCS (Unified Computing System) is an extensive

system in terms of size and cost. Because of this, it is
difficult to get hands-on experience within the home.
Thankfully, Cisco has released an emulator, UCSPE (UCS
Platform Emulator), which runs happily on a laptop!
There are some limitations to UCSPE when compared to
the whole system. We cannot install operating systems on
the blades and rack mount computers, we cannot connect
to other networking equipment, such as switches and SAN
storage, and we cannot perform tasks such as setting up
LDAP authentication.
These limitations will not stop us from having some fun
though, as we can learn a lot about how to operate and
maintain a UCS, without even having to leave the house!
So, let’s start by downloading and setting up UCSPE.
Setting Up UCSPE
UCSPE is a free download from Cisco.com. You will need a
Cisco ID, so sign up if you do not have one already
(https://id.cisco.com/signin/register). You can easily
find UCSPE by searching for it on the main Cisco page
(Figure 1-1).

1
CISCO
1
1 1 1 1 . Products and Services Solutions Support Learn

Refine results I Sorted by: Relevancy v

Filter by:

All Categories All Types All Products

-
community.cisco.com/t5/unified-computing- system/ucs - platform - emulator - downloads ucspe - 4 - 2 - 2a - ucspe -4-1 -2c/ta - p/3648 1 77
UCS Platform Emulator Downloads: UCSPE 4.2(2a) / UCSPE 4.1(2c)
26 May 2022 - Emulator (UCSPE) to physical UCS Manager domains is not recommended or supported by Cisco TAC. UCSPE backups
...
should only be imported to UCSPE domains. Cisco Cisco UCS Platform Emulator version 4.1(2cPE1): UCSPE 4.1(2cPE1) Release Page
Downloads: Download UCSPE 4.1(2c) OVA or ZIP (Requires Cisco.com login) Cisco ... Cisco UCS Platform Emulator version 4.0(4ePE1):...

Figure 1-1 Searching for UCSPE

Click on the link (as shown in Figure 1-1) and you will be
taken to the main page for UCSPE where you can click the
link for the OVA and Zip file downloads (Figure 1-2).
UCS Platform Emulator Downloads: UCSPE 4.2(2a) / UCSPE 4.1 (2c)
UCS Platform Emulator Jf Unified Computing Syste... «> 495499< 140 1*4 123
VIEWS HELPFUL COMMENTS

a ericwill 12-12-2013 08:46 AM

Edited On: 05-26-2022 1 1 :45 AM

The following Cisco UCS Platform Emulators are available for download from Cisco:

Current Cisco UCS Platform Emulators

CONFIGURATION IMPORT NOTE: Importing configuration backups (All, System, or Logical) taken from the UCS Platform Emulator (UCSPE) to
physical UCS Manager domains is not recommended or supported by Cisco TAC. UCSPE backups should only be imported to UCSPE domains.

Cisco UCS Platform Emulator 4.2(2aPE1) - UCS 62xx/63xx/64xx Fabric Interconnect, C4200, S3260, Mini:

• UCSPE 4.2(2aPE1) Release Page

Downloads:

• Direct downloads from this site are not supported. The following links should be used for downloads:

• OVA and ZIP file downloads: UCSPE_4.2(2a) Downloads

• Note that download and use Of the Platform Emulator is subject to the UCS Platform Emulator License Agreement

Figure 1-2 The UCSPE page

You will then be prompted to sign in using your Cisco

ID. Once you have signed in, you can download the UCSPE
software (Figure 1-3).

Software Download
Downloads Home

(q Swch... UCS Manager

( Expand All | Collapse All Release 4.2(2a)_beta [ beta Related Links and Documentation
- No related links or documentation -
Selected Releases

4.2(2a)_bota

File Information Release Date Size

UCSPE 4.2(2a) OVA 26-May-2022 2355.42 MB

UCSPE_4.2.2aS9.ova

UCSPE 4.2(2a) ZIP 26- May- 2022 2278.06 MB

UCSPE_4.2.2aS9.»p

Figure 1-3 The UCSPE download options

 The OVA file is the easiest option to use. You will need to
accept the Cisco license agreement to download it. At the
time of writing, the current version is 4.2(2a).
Importing UCSPE into VMWare
To run UCSPE, you will need the following available
hardware:
1 CPU
2048MB memory (2GB)
This is hardly resource-intensive, so it should run
happily on most modern computers. Our platform will
primarily use VMWare (Fusion), but UCSPE will also run
fine on VirtualBox or other hypervisors. Installing the
VMWare software is not covered in the book.
Firstly, start VMWare, and select the option to import
(Figure 1-4).

Name

Windows 7 x64
Ol Wi r—
B Migrate Your PC... [if Ubuntu 64-bit
IB Import...
B Folder
B Scan

Figure 1-4 Importing into Fusion

In the window that pops up, click on “Choose File…”

(Figure 1-5).
 tU

Choose an Existing Virtual Machine

V

I
Choose Virtual
-o-
Configuration
-o-
Importing
o
Finish
Machine

Recent items:

Choose File...

? Cancel Go Back Continue

Figure 1-5 Select the file

Select the UCSPE file you downloaded earlier and click

on “Open.” Click Continue again (Figure 1-6).
 1
o©o
Choose an Existing Virtual Machine

Choose Virtual Configuration

- o -
Importing
o
Finish
Machine

Recent items:

ucspe-4.1.2c
Size: Unknown
Type: Other Show in Finder I

UCSPE_4.2.2aS9
Size: Unknown
Type: Other ; Show in Finder j

Choose File...

7 Cancel Go Back Continue

Figure 1-6 Click Continue

In the next window, you can rename the VM and select

where to store it (Figure 1-7). Click “Save.”
 Save As: UCSPE_4.2.2aS9-2

Tags:

Where: Virtual Machines 0 v

Share this virtual machine with other users on this Mac

Some features will be limited when sharing a virtual machine.
Sharing is only available when the virtual machine is saved in a
shared folder.

Cancel Save

Figure 1-7 Save the virtual machine

The virtual machine will start to import into VMWare

(Figure 1-8).
 ©
•
©

Importing the Virtual Machine

Choose Virtual Configuration Importing

o
Finish
Machine

Importing UCSPE_4.2.2aS9-2

Cancel Go Back Continue

Figure 1-8 Complete the import

Once the import has been completed, you can customize

the settings or click Finish (Figure 1-9).
 Finish
The configuration of the virtual machine is now complete.

Choose Virtual Configuration Importing Finish

Machine

Virtual Machine Summary

Guest Operating System CentOS version 5 and earlier
Memory 2 GB
Networking Share with my Mac (NAT)
Networking Share with my Mac (NAT)
Networking Share with my Mac (NAT)
Device Summary USB Controller

To change the default virtual machine settings, click Customize Settings. To run the
virtual machine now, click Finish.
Customize Settings

? Finish
Cancel Go Back

Figure 1-9 The final VM settings

Starting UCSPE
Start the UCSPE virtual machine by right-clicking on it and
selecting Start.
When you start UCSPE for the first time, it runs through
a quick installation and then starts the services, which can
take a few minutes. Subsequent starts do not trigger an
installation. The VM will then do some basic tasks, like
using DHCP to gain an IP address (well, three actually; one
for each of the Fabric Interconnects and one for the virtual
IP, or VIP) (Figure 1-10).

Cisco UCS Platform Emulator 4 .Z(Z«S*JPE1)

Connect to IP: 192.168.68.135

THE UCSPE IS PROVIDED AS IS, WITHOUT ANY WARRANTIES OH REPRESENTATIONS EXPRESS. IMPLIED OR STATUTORY, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF QUALITY. PERI
ORMANCE, NON INFRINGEMENT. fKRCIIANBILlTY OR FITNESS

Available login: user 'uespe', password 'uespe' (console « ssh)

ncspe login:

Figure 1-10 Starting UCSPE

In the next stage, UCSPE will complete the backend

tasks, such as setting up SSH and GUI access, as well as
generating the hardware catalog, which we will look at in
the next chapter (Figure 1-11).
 ••• OD ¡3 <+ < > < •■> Q Q n < 6 ucspe-4.1.2c

Resetting UCSPE Configuration ...I UK J

Harting ape_seruer.pl ... [OK J
f
Harting NTP ...( OK 1
Prepping UCSPE . . .
Gleaning up . . . [ OK ]
Cluster Mode: [ CLUSTERED J
JAM DNE Port : Í 00 1
JCSPE DNE Port: [ 8881 J
JCSPE UI Port: ( 888Z 1
SSL Port: [ 443 1
Jerifying Network Interfaces ... [OK J
JTP: ( 172. 16. Z. 134 1
JCSPE Switch Type : <8>UCSPE Switch Type at End : <64108>Gcncrating SwitchType: <64108>. Cluster: <ha> FI Config: Calling fi Binary to generate it.. <4GFI-HD
:witchld> = 9
Inside ur 1 tetopsystem
Serial fro« db: FD0231387E9 for side= A

Serial from db: FDDZ31388E9 for side= B

[ OK ]
Jpdating UCSPE Catalog ...[ OK ]
JCSPE UU1D: dbc99%8-db69-llec-88?d-0e8c29?92d0?
hatching d bunch of files ...mu: cannot move 'zapedlsk/apezzopt/sanv'binzsuitchPartionSize.sh. Imp’ to ‘zlsanzbinzswitchPortionSlze.sh' : No such file or directory
ihmod: cannot access 'zisanzbln/suitchPartlonSize.sh' : No such file or directory
’atching Headers(HTTP) . . .
’atching Headers. (HTTPS).. .
’atching for Device Connector ( https )... .
Include /aped iskzapczzopt/apachczconf/cxtra/httpd -cloud .conf
[ UK J
Generating SAN config . . . [ OK 1
Generating APE config ... [UK 1
Generating SSH config ...I OK J
Setup UCS FI Connector...! Í1K 1
JCSPE Iters ion: I HERCULEAN HAPLORHINI J
JCSM Database . . . [ J
Checking environment configurations ... Í UK 1
Removing old logs ... 1 UK 1
Gleaning SAN and APE DB . . . ( UK J
Starting SAN DNE and AG (right) ... 1 OK J
Starting SAN DNE and AG (left) ... Starting APE DNE ... ( OK 1
>rt subjec. Is zCN=UCSPE-172-16-Z-134Sz
Starting APE HTTPD ... t OK 1
[ UK 1
Sunning Post -startup tasks
Setting Power Budget State ... [OK J
leaertlng device» ... J

Figure 1-11 The VM is starting

Once everything has been completed, you will see the

login details, which are the IP address of the VIP and the
username and password (which are both “ucspe”) (Figure
1-12).

Connect to IP: 17Z.16.Z.134

THE UCSPE IS PROUIDED AS IS, WITHOUT ANY WARRANTIES OR REPRESENTATIONS EXPRESS,

ORMANCE, NON INFRINGEMENT, MERCHANBILITY OR FITNESS

Available login: user 'ucspe' , password 'ucspe' (console & ssh)

iicspe login:

Figure 1-12 The login details

You can login and use the UCSPE VM console to show

you more network details, the general status, and to
perform functions such as resetting the system, rebooting,
or shutting down (Figure 1-13).

Cisco UCS Platform Emulator 4.2(2aS9)

Choose an option:
a : Show Status
c: Login to CL I shell
i: Configure UCS Intersight Connection
n: Modify Network Settings
t: Modify System Settings
s: Restart UCSPE Processes

f : Perform a Factory Reset

r: Reboot the UM
x: Logout user
z: Shutdown the UM

Figure 1-13 The UCSPE Console

You can now browse to the GUI using HTTPS (in this
instance, it would be https://172.16.2.134) (Figure 1-
14).
 4- c a Q & Mips <172.16 2 m. con'lg 0 ± =

Figure 1-14 The UCSPE GUI

You can see that we have a list of our equipment on one

side, and on the other we can log into UCS Manager (using
the default username and password “ucspe”).
When UCSPE starts up, it will generate a fairly random
environment, and that is where we will start in the next
chapter after we look at how a UCS would be set up in the
real world.
Real-World UCS Setup
If you are setting up a physical UCS, you will need to
allocate three IP addresses; one for the management of
each of the Fabric Interconnects (FICs), and one for the
virtual IP (VIP) that will be used for the cluster
management.
Once you have racked the FICs, we need to do the
essential cabling for them, which will be management
(LAN) interfaces that will connect to your upstream or
management switches, and then we need to connect the
cluster interfaces. These are the L1 and L2 interfaces, and
we cable L1 on the first FIC to L1 on the other FIC and L2
to L2.
In this section, we will be configuring two FICs. FIC A
will have the IP address 10.99.1.10, with a /24 subnet mask
(255.255.255.0). FIC B will use the IP address 10.99.1.11.
The VIP will be 10.99.1.200, and the default gateway will
be 10.99.1.1. We will configure a DNS server IP address of
10.99.1.5. The cluster will be called “Mastering-UCS.”
Only power up the first FIC at first. We only power up
the second one once the first FIC has been configured.
Connect your computer to the first FIC using a console
cable, open your terminal software (like PuTTY or
SecureCRT) and get on to the serial port using the
following settings:
9600 baud
8 data bits
No parity
1 stop bit
After connecting, you will see the following prompt:

—- Please read the following carefully —-

At the request of the publisher, I have had to
change
the output shown in the Cisco console, because
it is also shown on a website. Even though what
you would
Have seen would have come straight from the Cisco
device.
This was done to avoid any copyright issues.
So, I hope you understand that (for this chapter
only)
Things are going to get super weird.
Everything goes back to normal in the next chapter
though, so please bear with the incoming
strangeness, but they wouldn't listen to me.

We are going to use the console for our configuration, so

type “console” and press enter.

Choose a config technique. (console/gui) ? console

Type in “setup” next, as we are not restoring from a

backup.

Here you choose a setup mode; New or rebuild from

backup. (setup/restore) ? setup

This is a new Fabric Interconnect, so type “Yes” to start

a fresh setup.

Thou hast elected to make a new Fabric

interconnect. Resume? (y/n): y

Type in a new password for the admin user when

prompted. The complexity needs to be at least eight
characters.

Choose a p@ssw0rd for "admin": Admin123

Type it in again "admin": Admin123

We will set up a cluster, so type in “yes” when

prompted.

Might this interconnect device be half of a

cluster(select "ney" for lonesome-mode)? (yes/no)
[n]: yes

Because this is the primary switch that we are setting

up, it will be the “A” of the pair. The secondary switch will
be “B.”

Pop in the switch fabric (A/B) []: A

Next, we get to name our cluster. The name we chose

will also be used on each Fabric Interconnect, with them
getting either -A or -, depending on if they are the first FI,
or the second.

Enter the system name: Mastering-UCS

Set the IP address, subnet, and gateway for the fabric

interconnect.

Corporeal Switch Management0 IPv4 address :

10.99.1.10
Corporeal Switch Management0 IPv4 netmask :
255.255.255.0
IPv4 address of the gateway : 10.99.1.1

Now we type in the cluster IP address. It is this address

that will be used by the primary Fabric Interconnect and is
the IP you use when you fire up the UCS Manager.

Gathering IPv4 addy : 10.99.1.200

 You can then configure your DNS servers. This is
optional.

Wanna use a DNS Server? (yes/no) [n]: y

Pop in its numbers : 10.99.1.5

We are not going to configure a default domain name, so

choose “no” for the next section.

Probably don't need a default domain name?

(yes/no) [n]:n

You will see a summary of the settings we have entered

so far. Either type “y” to save the settings and restart the
fabric interconnect, or select “n” to go back and make any
modifications that you need to.

The following configuration will be pertained:

Switcharoo Role=A
UCS Designation=Mastering-UCS
Corporeal Switch Management0 IP Addy=10.99.1.10
Corporeal Switch Management0 IP
Netmask=255.255.255.0
Gateway=10.99.1.1
Nameserver=10.99.1.5
Gathering Enabled=yes
Gathering IP Addy=10.99.1.200

Save it and use this config (select "no" if you

want to re-enter)? (yes/no): yes

Sorting this out for you. Standby.

Configuration file – Ok

Our first fabric interconnect is now completed. Connect

to the console on the second device and start it up.
 From this point, it’s just a matter of following the same
steps.

—- Please read the following carefully —-

Still with us? Marvelous. Not much more to do now,
And we can get back to how things actually look
when
you configure a UCS. Again, I'd like to stress
that I did try
to keep everything as you'd see it on screen, but
it was a
case of either change it, or not get it published.
I hope you are well. Been to any good music gigs
recently? How's the family? I wish you a bright
and happy future.
Let's finish this off. Cheers.

Which do you prefer? (console/gui) ? console

The installation will detect the presence of Fabric

Interconnect 1 (assuming you have connected the
heartbeat ports together), and this switch will be added to
the existing cluster. Type “y” to do this.

The setup is like a Jedi and has felt the aura of

another Fabric interconnect. Wanna add this switch
to the cluster? Carry on (y/n) ? y

Pop in the admin p@ssw0rd of the other Switcharoo

thing: Admin123
Attaching to the other switch… finito
Stealing info from other switch… finito
Other switch Management0 IP Addy: 10.99.1.10
Other switch Management0 subnet: 255.255.255.0
Gathering IP addy : 10.99.1.200
 We now set the IP address for this fabric interconnect.

Corporeal Switch Management0 IPv4 thing :

10.99.1.11

Save the configuration.

Wanna keep the config (type "negative" if you

don't like it)? (yes/no): yes
Saving config. Give it a second or three.
Finally, we're done – Thanks

Navigate to https://10.99.1.200 using a browser to

log into the UCS.
Summary
We started this chapter by downloading the latest version
of UCSPE from the Cisco website, importing it into
VMWare, and starting it up. We then compared this to how
a real UCS installation works in a clustered environment.
In the next chapter, we will configure UCSPE according
to a topology (instead of the random assortment of
hardware that it generates) and, as we do this, we will look
in greater depth at the various components of a UCS
environment.
© The Author(s), under exclusive license to APress Media, LLC, part of
Springer Nature 2023
S. Fordham, Introducing Cisco Unified Computing System
https://doi.org/10.1007/978-1-4842-8986-0_2
2. The UCS Components
Stuart Fordham1
(1) Bedfordshire, UK

When UCSPE starts up, it generates a new inventory. This

comprises two chassis with five blades, one enclosure with
two nodes, two fex, and ten rack servers. While the naming
of the devices will vary between what you may see on your
screen and the screenshots in this book, any differences
should be minor.
We can see the equipment that has been created for us
by clicking on the Equipment link on the left-hand side.
Managing UCSPE Hardware
Adding and Removing Devices
If you want to edit the auto-generated hardware that
UCSPE has provided, then you can remove and add
devices. You don’t have to remove (or add anything) here,
this is more for reference if you want to create your own
setup. However, it does give us an excellent, logical, way of
introducing all the different components of the UCS and
how they are connected. If you do go ahead and follow the
following steps, then you might want to do a factory reset
on the VM before the next chapter, which will set us up
with a brand new UCS again before we move on to the next
chapter.

Removing UCS Devices

Before we can remove a piece of hardware, we need to
disconnect it first. The easiest way to do this is to click on
the broken chain-link icon at the top right-hand corner
(Figure 2-1). This will disconnect all the devices (apart
from the Fabric Interconnects).

Disconnect all devices

"I
Figure 2-1 Disconnecting the devices

We can disconnect individual devices by clicking on the

red circle next to the line item we want to remove if we
only want to remove one piece of equipment.
If you have chosen to remove all of them, once they
change their green circle to a red one, click on the red
circle at the top (Figure 2-2).

Remove all devices

Figure 2-2 Removing the devices

This will remove the devices. If you find that (after

waiting a few minutes) nothing has changed, then click on
the trashcan icon on the device line item to delete them one
by one. Refreshing the page is also useful here, as is
clicking on the Equipment link, to update any changes that
have been made.
The only devices that are left will be the fabric
interconnects (Figure 2-3).
Hardware Inventory

Fabric Interconnect

UCSM Id $ Name Side Vendor

switch-A A Cisco Systems, Inc.

switch-B B Cisco Systems, Inc.

Rack Server ® t

UCSM Id Name Vendor

Fabric Extender +)

UCSM Id Name Side Vendor

Chassis ® t

UCSM Id # Name t Vendor

Blade Server

Slotld Vendor Model

Enclosure ® t

UCSM Id Name Vendor

Enclosure NODE

Slotld Vendor Model

Figure 2-3 The hardware inventory

 If the devices do not appear to change state, then click
the Equipment link on the left-hand side and the screen
should refresh. You may have to do this a lot with UCSPE
as it can be slow to pick up changes, such as when we
come to add hardware shortly.
Now that we have an empty canvas (so to speak) we can
start with the Fabric Interconnects.
Fabric Interconnects
The Fabric Interconnects (also referred to as “FICs” or
“FIs.” FI is better to use to avoid confusion, as FIC sounds
much like FEX, which we will cover shortly) is where all the
magic happens. This is where we manage the UCS estate
as this is where the UCS software is held.
Generally, we would have two Fabric Interconnects,
though you may also encounter a UCS-Mini. The UCS-Mini
can handle between two and fifteen servers (a maximum of
eight blade servers and seven rack servers), and places the
FI (the UCS 6324 model) within the chassis, rather than
them being separate hardware.
The FI runs a version of the Cisco Nexus software
providing northbound connectivity to the rest of the
network as well as connectivity to storage.
We can change the FI model if we want to by clicking on
the cog icon at the top (Figure 2-4).

i IM ? 0A
Change Cluster State

Hardware Inventory
Change Fl Serial

Fabric Interconnect
Change Fabric
Interconnect
UCSM Id t Name
UCSPE Restart settings
switch-A items, Inc.

switch-B B Cisco Systems, Inc.

Figure 2-4 Changing the Fabric Interconnect

 The available models are shown in Table 2-1.
Table 2-1 Fabric Interconnect models

Model Size 100G 40/100G 40G 10/25G 10G PSU Fans

(RU) ports ports ports ports ports
UCS-FI-M- N/A - - 1 - 4 N/A N/A
6324
UCS-FI- 2 - - - - 48+481 2 2
6296UP
UCS-FI- 1 - - 24 - 162 1+1 2+2
6332-16UP
UCS-FI- 1 - - - - 32+161 2 1+1
6248UP
UCS-FI- 1 543 6 - - - 2 3+1
6454
UCS-FI- 2 1083 12 - 96 - 2 2+1
64108
UCS-FI- 1 - - 32 - - 1+1 2+2
6332

If you do change the FI, then you will have to restart

UCSPE (Figure 2-5).

Figure 2-5 Restarting UCSPE

The interconnects come with two power supplies (PSUs)

and four fans (Figure 2-6).
’ Equipment
Fabric Interconnect 2)
Fabric Interconnect A
Fl-A
Fl-B Model: UCS-FI-64108
Serial:FDO231 307E9
Vendor: Cisco Systems, Inc.
Description: Cisco UCS 96x10/25G + 12x40G/100G Port 2RU Fabric Interconnect

’ PSU (2)
E PSU Slot 1:
AC PSU For UCS-FI-6332
UCS-PSU-6332-AC

El PSU Slot 2:
AC PSU For UCS-FI-6332
UCS-PSU-6332-AC

’ Fans (4)
Fan Slot 1:
Fan Module For UCS 6332 Fabric Interconnect
UCS-FAN-6332
El Fan Slot 2:
Fan Module For UCS 6332 Fabric Interconnect
UCS-FAN-6332
Fan Slot 3:
Fan Module For UCS 6332 Fabric Interconnect
UCS-FAN-6332
Fan Slot 4:
Fan Module For UCS 6332 Fabric Interconnect
UCS-FAN-6332

Figure 2-6 The Fabric Interconnects

Adding Devices
Next, we will come to our chassis.
Chassis
The chassis holds our blade servers. The chassis model
options we have are
UCSS-S3260 – a modular storage server with dual M5
server nodes.
UCSC-C3X60 – similar to the S3260 but is now
discontinued. Both are optimized for large datasets.
UCSB-5108-DC.
UCSB-5108-DC2.
N20-C6508.
UCSB-5108-AC2.
The 5108s are 8-slot 6RU chassis with two I/O bays. The
N20-C6508 is the same as the previous, but is now
discontinued.
You can add a chassis, such as the UCSB-5108-AC2, by
clicking on the plus sign next to the word “Chassis” on the
Equipment page. Enter the name for the chassis, select the
model and click on “Add” (Figure 2-7).

Chassis © t

Name: Chassisl Pió V UCSS-S3260|Cisco Systems Inc Add

UCSC-C3X60-BASE|Cisco Systems Inc
UCSM Id Name UCSB-5108-DC|Cisco Systems Inc Serfs
UCSB-5108-DC2|Cisco Systems Inc
Blade Server N20-C6508|Cisco Systems Inc
UCSB-5108-AC2|Cisco Systems Inc
Slotld Vendor

Figure 2-7 Adding a Chassis

The chassis will appear in our inventory on the left-hand

side (Figure 2-8).
 ’ Equipment
’ Chassis (1 )

Chassis CH51
* Fabric lnterconnect(2)
* Stash Servers

Figure 2-8 The new Chassis

Now that we have our first chassis, we need to fill it with

the components that connect it to our FIs and make it hum
gently4 in the data center (power supplies and fans).
If we select the chassis and click on the edit button to
the right on the item line, then we can see that we have
many options of components we can add (Figure 2-9).

Chassis CH52: Chassisl Manage bnks of Chassis <? Stash

_ ,. . , Peer Devree Peer Device Peer Slot/Peer Edit Delete

Source(dcvice_xVport)
Pon Port Lmk
Modet UCS8S108AC2
UCSM ChMM ID UiassijoW
Señal: CMS2
Oseo AC Slide Server Chassis, 6U with Eight
Description Blade Server Slots

Blade Psu Fan lom Template

Figure 2-9 Chassis hardware

We are not going to add any blade servers at the

moment, but we do need to add some power. We do this by
clicking on “Psu” (not sure why Cisco didn’t capitalize all of
“PSU,” but there we are), selecting an appropriate model
(such as the Platinum II AC power supply), and dragging it
up to the chassis, above where the model is shown and
underneath the plus and minus buttons (Figure 2-10).

Chassis CH52: Chassisi

Platinum II AC Power
C) W ® Supply for or Stash
Model: UCSB-5108-AC2chass¡s
UCSM Chassis ID: Unassigned
Serial: CH52
Cisco AC Blade Server Chassis, 6U with Eight
Description: Blade Server Slots
Figure 2-10 Adding a PSU to a chassis

Once you let go, you can select how many to add.
UCSPE will tell us how many available slots we have
available to fill. We can decide which slot to add an item to
by typing in the slot number (such as “1”) or a range, by
typing in “1-4” and pressing enter. You should now have
four power supplies (Figure 2-11):
 Chassis CH52: Chassisl

Successfully added PSU to chassis.

Model: UCSB-5108-AC2
UCSM Chassis ID: Unassigned
Serial: CH52
Cisco AC Blade Server Chassis, 6U with Eight
Description: Blade Server Slots

’ PSU 4
®PSU Slot 1:
Platinum II AC Power Supply For UCS 5108 Chassis
UCSB-PSU-2500ACDV
®PSU Slot 2:
Platinum II AC Power Supply For UCS 51 08 Chassis
UCSB-PSU-2500ACDV
®PSU Slot 3:
Platinum II AC Power Supply For UCS 5108 Chassis
UCSB-PSU-2500ACDV
®PSU Slot 4:
Platinum II AC Power Supply For UCS 51 08 Chassis
UCSB-PSU-2500ACDV
Figure 2-11 Our Chassis has power!

Chassis also require fans and we add these in the same

manner, by clicking on the Fan link next and adding eight
fans (Figure 2-12). In the box, you can type “1-8” to add all
eight fans in one go.
Chassis CH52: Chassisl

& LU

Successfully added Fan to chassis.

Model: UCSB-5108-AC2
UCSM Chassis ID: Unassigned
Serial: CH52
Cisco AC Blade Server Chassis, 6U with Eight
Description: Blade Server Slots
Fans 8
Fan Slot 1:
Fan Module For UCS 5108 Blade Server Chassis
N20-FAN5

El Fan Slot 2:
Fan Module For UCS 5108 Blade Server Chassis
N20-FAN5

Fan Slot 3:
Fan Module For UCS 5108 Blade Server Chassis
N20-FAN5 X
_ Fan Slot 4:
Fan Module For UCS 5108 Blade Server Chassis
N20-FAN5

; Fan Slot 5:
Fan Module For UCS 5108 Blade Server Chassis
N20-FAN5

Fan Slot 6:
Fan Module For UCS 5108 Blade Server Chassis
N20-FAN5

El Fan Slot 7:
Fan Module For UCS 5108 Blade Server Chassis
N20-FAN5

J Fan Slot 8:
Fan Module For UCS 5108 Blade Server Chassis
N20-FAN5

PSU 4
Figure 2-12 Adding chassis fans

Next, we can add the IOMs. The IOMs are “In/Out

Modules.” These are also known as FEXs. They are the line
cards that connect our chassis to our fabric interconnects.
They also provide the interface connections to the blade
servers, and CMC (Chassis Management Controller), which
is used for monitoring our components, such as fans, power
supplies, and temperatures and this is also the component
that is responsible for monitoring blade insertion and
removal. Lastly, they also provide Chassis Management
Switch (CMS), which gives us the KVM (Keyboard, Video,
Mouse), Serial over LAN (SoL), and Intelligent Platform
Management Interface (IPMI) abilities to our blades.
The IOM options we have are

Model Fabric ports Server ports Throughput

2304 4x40GE 8x40Gbps 320Gbps5
2208XP 8x10Gbps 32x10Gbps 80Gbps
2204XP 4x10Gbps 16x10Gbps 40Gbps
2408 8x25GE 32-10Gbps 400Gbps5

If we add two 2408 IOMs and click the word

“Equipment” in the left-hand pane, then we should see the
chassis change, listing the (currently disconnected) IOM
ports (Figure 2-13).
Chassis CH52: Chassisl Manage Links of Chassis & Stash

Model:
©0

UCS.M Chassis <X

• UCS8-S108AC2
Unassigned
tom 1/1
lype

disconnected
Peer Device
Senai/Side
disconnected/
Peer Slot/Pe«r
Port
disconnected
Edit
Port
S
Delete
Link
0
Serial: CH52
tom 1/2 disconnected disconnected/ disconnected 0
Cisco AC Made Server Chassis. 6U with Eight iom 1/3 disconnected disconnected/ disconnected S 0
Description: Blade Server Slots
iom 1/4 disconnected disconnected/ disconnected S s
>0M
GiOMSIot 1:
2
iom 1/5 disconnected disconnected/ disconnected e
Cisco UC5 2409 8 Pon VO Module iom 1/6 cSsconnected disconnected/ disconnected 0
UCS-IOM-2408 iom 1/7 disconnected disconnected/ disconnected 0
&I0M Slot 2:
Cisco DCS 2408 8 PWt VO Module
iom 1/8 disconnected disconnected/ disconnected a
UCS40M 240B iom 2/1 disconnected disconnected/ disconnected 0
» Fans 8
iom 2/2 disconnected disconnected/ disconnected 0
• PSU
tom 2/3 disconnected disconnected/ disconnected 0
tom 2/4 disconnected disconnected/ dsconnected 0
tom 2/5 disconnected disconnected/ disconnected 0
tom 2/6 disconnected disconnected/ disconnected 0
tom 2/7 disconnected disconnected/ disconnected 0
tom 2/8 disconnected disconnected/ disconnected 0

Figure 2-13 IOMs in our Chassis

We are not going to configure the IOMs just yet, instead,

we are going to see how we can quickly create two more. If
we click back onto the main Equipment list, then we can
click on the duplicate icon next to our chassis, and again to
create two more chassis.
We should have three chassis now (Figure 2-14).

Chassis ® t

UCSM Id t Name Vendor Model Serial Insert/Remove

Unassigned

Unassigned
Chassisl

DUPLChassisSZ
Cisco Systems Inc

Cisco Systems Inc

UCSB-5108-AC2

UCSB-51O8-AC2
CHS1

CH52
•
•
Unassigned

Figure 2-14 Three Chassis

DUPLChassis53 Cisco Systems Inc UCSB-5 1 08-AC2 CH53
•
This ability to duplicate can save a lot of time if we need
to add multiples of the same hardware component and is
especially useful when adding servers.
We can now connect our chassis to our Fabric
Interconnects. We can do this by editing the first chassis
and clicking the pencil icon under “Edit Port” on port 1/1
(IOM 1, port 1). The Peer Device Type needs to be set to
“fi.” Select FI A and select a free port (such as 1/20).
Repeat the process for iom 1/2, selecting the next available
port on the same FI (1/21).
 Next, edit iom 2/1 (IOM 2, port 1) selecting FI B, and
the same port number as used on iom 1/1 (port 1/20), and
repeat for 2/2, selecting the next port (1/21) (Figure 2-15).

Manage Links of Chassis U O

Successfully created a new Link Connection.

Peer Device Peer Device Peer Slot/Peer Edit Delete

Source(device_id/port)
Type Serial/Side Port Port Link
iom 1/1 fi FDO231307E9/A 1/20

iom 1/2 fi FDO231307E9/A 1/21 s

iom 1/3 disconnected disconnected/ disconnected Ulf
iom 1/4 disconnected disconnected/ disconnected 0
iom 1/5 disconnected disconnected/ disconnected W
iom 1/6 disconnected disconnected/ disconnected LUJ
iom 1/7 disconnected disconnected/ disconnected W
iom 1/8 disconnected disconnected/ disconnected

iom 2/1 fi FDO231308E9/B 1/20

iom 2/2 fi FDO231308E9/B 1/21

iom 2/3 disconnected disconnected/ disconnected

iom 2/4 disconnected disconnected/ disconnected ¡3

iom 2/5 disconnected disconnected/ disconnected
iom 2/6 disconnected disconnected/ disconnected

iom 2/7 disconnected disconnected/ disconnected UÜ

iom 2/8 disconnected disconnected/ disconnected 0

Figure 2-15 Chassis 1 IOM connectivity.

Repeat the process on the other chassis, following Table

2-2.
Table 2-2 IOM connectivity
Chassis IOM IOM Port Peer Device Peer Port
2 1 1/1 FI A 1/24
2 1 1/2 FI A 1/25
2 2 2/1 FI B 1/24
2 2 2/2 FI B 1/25
3 1 1/1 FI A 1/28
3 1 1/2 FI A 1/29
3 2 2/1 FI B 1/28
3 2 2/2 FI B 1/29

The reason we leave gaps between the ports of one IOM

and the ports of another (as they go into the FI) is that if
we want to increase bandwidth later on, we can keep
things nice and neat and ordered. Also, bear in mind that
the cabling is one IOM to one FI. The IOM, essentially,
becomes part of the FI, so we never cross the streams. This
isn’t Ghostbusters. Bad things really will happen. Maybe
not end-of-the-world type stuff, but certainly a call to Cisco
TAC (Technical Assistance Center)!
If you were to start UCS now, this is what your systems
would look like (Figure 2-16).

Chassis 3
Figure 2-16 Our topology so far
Blade Servers
Our chassis are fairly useless if we have no (B-series)
servers to run in them. So, add some servers by dragging
the server from the menu at the bottom into the chassis.
Each server will need CPU, memory, and storage, so add
these as well. When adding servers, plan them out
carefully. For example, if you have three chassis and
servers that will perform different functions (such as ESXi
hypervisors, database servers, application servers, and so
on), share these out across all three chassis so that if one
chassis has an issue, the servers in the other chassis can
continue to server your data an environment as required.
Once you have added your servers, click on the red
button next to each of the chassis (to remove them) and
then click the green button to insert them again. The red
button should turn green, as well as the green button next
to each of the servers. You may need to wait a few minutes
before you can insert it again.
Our chassis will look something like this (Figure 2-17):
 >•••••• jR
—
>••••••. —-
¿^¡^•••^ToVoWTmWmT

CISCO

Figure 2-17 Our chassis

Now that we have some blade servers, we should add

some rack servers. Before we do this, however, we are
going to add some FEXs.
FEX
FEX stands for “Fabric Extender.” These allow us to
increase the number of ports we have at our disposal. The
options we have in UCSPE are

Model Server ports Uplinks

N2K-C2232TM-E-10GE 32x 1/10GBASE-T 8x10GE
N2K-C2232TM-10GE 32x 1/10GBASE-T 8x10GE
N2K-C2148T-1GE 48x 1G Base-T 4x SFP+
N2K-C2232PP-10GE 32x 1/10GE SFP/SFP+ 8x10GE
N2K-C2348UPQ-10GE 48x 1/10 Gigabit Ethernet (SFP/SFP+) 6x 40GB
N9K-C93180YC-FX3 48x 1/10/25-GBps fiber 6x 40/100GB
Rack Servers
In UCSPE, the rack servers (C-series) connect to the FEXs
and there is a wide variety of servers to choose from. Way
too many to list here with all their differences, but similarly
to the blade servers, you will need to add CPU, memory,
disks, I/O adapters, storage controllers, and PSUs.
When we connect rack servers, we need to consider how
we connect them; we have options of “Direct Attach
Server,” “Single Wire Management,” or “Dual Wire
Management.”

Direct Attach Mode

In Direct Attach mode (available with UCS version 3.1 or
later), the servers attach directly to the fabric
interconnects, bypassing the need to have FEXs.

Single Wire Management

As the name suggests, single wire management uses a
single cable into the FEX for management and data traffic.

Dual Wire Management

In dual wire mode, separate cables are used for data and
management.
Enclosures
UCS enclosures, such as the UCSC-C4200-SFF can host up
to four “nodes,” such as the C125. These are designed for
dense compute form factor with high core densities, where
the ability to scale out with compute-intensive machines is
critical.
We do need to add fans and power to the enclosures, but
not IOMs. We can add the nodes by dragging them onto the
chassis as we do with the other hardware. Once we have
added the nodes (as well as the CPU, memory, I/O
adapters, and disks), we can connect the node to the FEX
by clicking the chain icon where it says “Manage Links of
Node.”
We can see how this all looks by clicking the Equipment
link at the top left-hand corner and then clicking the UCS
icon at the end of the row of icons. We can log in using the
username and password of “ucspe.”
The default UCSPE-generated layout will look a little
like this (Figure 2-18):

Figure 2-18 Our UCS environment

Summary
In this chapter, we looked at how to add and remove the
components available to us in UCSPE, as well as how to
connect them all together.

Footnotes
1 Using an expansion module

2 It is technically sixteen 1 and 10Gbps and FCoE, or 4-, 8- and 16-Gbps Fibre
Channel unified ports

3 These are 10/25/40/100Gbps and FCoE ports

4 When I say “hum gently” I am joking. Without all four PSUs plugged in these
things sound like a plane taking off!

5 Across two IOMs

© The Author(s), under exclusive license to APress Media, LLC, part of
Springer Nature 2023
S. Fordham, Introducing Cisco Unified Computing System
https://doi.org/10.1007/978-1-4842-8986-0_3
3. Northbound Networking and
SAN
Stuart Fordham1
(1) Bedfordshire, UK

In the first two chapters, we set up UCSPE and looked at

how to connect it physically, both to its own components
and to the rest of the network. The physical cabling we
looked at in the first chapter was, however, purely for
management. This would allow us to control our UCS, but
the blades and rack servers would have no connectivity to
the rest of the network. In this chapter, we will be focusing
on how to add the networking components that will let our
UCS talk to the rest of the world.
UCS networking
At the moment, our UCS servers will be disconnected from
the rest of the network, so we need to add means for them
to pass packets to the rest of the network. We have a
couple of ways to achieve this. We can use “Uplink” ports,
or port-channels. We will start by looking at uplink ports.

Uplink ports
We start to configure uplink ports by going to the
Equipment tab in UCS manager, scrolling down to the
fabric interconnect, and then into the ports. Uplink ports
can either be configured on the fixed module (fixed ports
that are part of the FI) or on the expansion module (a
module purchased separately and installed in the FI). We
select the port we want to configure and then, from the
“Reconfigure” menu, set it as an Uplink port (Figure 3-1).
 All Equipment / Fabric Interconnects / Fabric Interconnect

General Faults Events FSM Statistics

’ Equipment
Chassis
Fault Summary
Rack-Mounts

Fabric Interconnects ® 0 0 0
0 0 0 0
* Fabric Interconnect A (primary) ®
Fans
Status
» Fixed Module

Overall Status 1 Admin Down

’ Ethernet Ports
Additional Info
Port 1
Admin State : Disabled
Port 2

Port 3 Actions

Port 4

Port 5

Port 6 Reconfigure

Port 7 Configure as Uplink Port

Port 8 Configure as FCoE Uplink Port

Port 9 Configure as Server Port

Configure as FCoE Storage Port

Port 10
Configure as Appliance Port
Port 11

Figure 3-1 Reconfiguring a port as an Uplink port

We will then need to confirm that we do wish to

reconfigure the port, and then acknowledge the action once
completed.
The next step in creating our uplinks is to head into the
network tab in UCS Manager, and select the LAN option
from the left-hand side. We will be able to see our uplink
interface listed under the relevant fabric (“Fabric A” for FI-
A, “Fabric B” for FI-B) (Figure 3-2).
 All

LAN

LAN Cloud

Fabric A

Port Channels

Uplink Eth Interfaces

Eth Interface 1/1

VLANs

VP Optimization Sets

Figure 3-2 Our Uplink interface

On the other side of the page, we have options we can

set for the interface (Figure 3-3). Firstly, we can give it a
label, which can be helpful to quickly identify which
network device and port we are connected to.
LAN / LAN Cloud / Fabric A / Uplink Eth Interfaces / Eth Interface 1/1

General Faults Events

Actions Properties

Enable Interface ID

Disable Interface Slot ID

Fabric ID

User Label

Transport Type
Port

Flow Control Policy

Link Profile

Admin Speed

FEC
Figure 3-3 Uplink interface settings

Next, we can set a flow control policy, which controls

how the port acts (in the sending and receiving of pause
frames) when the receive buffer is full. We create a flow
control policy by going to LAN ➤ Policies ➤ root ➤ Flow
Control Policies and clicking “Add.” We give it a name, set
the priority and Receive and Send to “on” (otherwise the
flow-control packets won’t be sent or received) (Figure 3-
4).

Create Flow Control Policy

Cancel

Figure 3-4 Flow Control Policy

We then assign this to the interface (Figure 3-5).

 Properties

ID : 1
Slot ID : 1
Fabric ID : A

User Label :

Transport Type : Ether

Port : sys/switch-A/slot-1/switch-ether/port-1

Flow Control Policy : FlowControl-Pol

Link Profile : default

Admin Speed : 1 Gbps O 10 Gbps Q25 Gbps (« Auto

FEC : ©Auto QCI74 QCI91

Figure 3-5 Interface Flow control

The Link Profile controls how the interface works with

UDLD (UniDirectional Link Detection). We can create a
new Link Profile by going to LAN ➤ Policies ➤ LAN Cloud ➤
Link Profile ➤ default and clicking “Create UDLD Link
Policy.” Here we name the policy and set it to enabled and
set the mode, either normal or aggressive (Figure 3-6).
Normal mode will detect mis-cabling issues, whereas
aggressive mode will detect when a link has become
unidirectional. Normal mode is not used very much, as
Aggressive mode will give us the good stuff that we need,
such as “bad” ports being disabled so that failover can
happen.
Figure 3-6 UDLD modes

Now that we have a UDLD policy, we can create a link

policy by going up one level (LAN ➤ Policies ➤ LAN Cloud
➤ Link Profile) and clicking “Add.” We can name the policy
and assign the UDLD-Aggressive link policy to it (Figure 3-
7).
Figure 3-7 Link Profile

The next step is to assign this profile to our interface

(Figure 3-8), making sure that we click “Save Changes” at
the bottom of the screen.
 Properties

ID : 1
Slot ID : 1
Fabric ID : A

User Label :

Transport Type : Ether

Port : sys/switch-A/slot-1/switch-ether/port-1

Flow Control Policy : FlowControl-Pol

Link Profile : LinkProfile

Admin Speed : Q1 Gbps Q 10 Gbps Q 25 Gbps (J) Auto

FEC ©Auto QCI74 QCI91
Figure 3-8 Assigning a link profile to an interface

The following settings control the port speed and the

Forwarding Equivalence Class (FEC), which is a form of
quality of service.
Clearly, one uplink interface alone will not be enough;
we should, at a very minimum, add an uplink on the second
FI. Ideally, we would have second interfaces on each FI,
going to the other upstream switch to provide a level of
redundancy (Figure 3-9).
 Fl-A Fl-B
Figure 3-9 FI redundant uplinks

While two interfaces are good, we are not making the

best of our capabilities. With uplink ports, traffic is pinned
to one of these links. One isn’t much fun when we could use
all four cables at the same time (turning single 40GBps
links into a combined 80GBps link).
To do this, we need to create another uplink on FI-A
(1/2). We can do this without going to the Equipment tab,
we just need to go to LAN ➤ LAN Cloud ➤ Uplink Eth
Interfaces and click on “Add,” and select Port 2, by double
clicking on it, under the fixed module (Figure 3-10):
 LAN / LAN Cloud / Fabric A / Uplink Eth Interfaces

Uplink Eth Interfaces

’ LAN
- LAN Cloud

Fabric A
Tz Advanced Filter

Name
+ Export
Fabric ID
>5 Print
Slat

Port Channels
Configure Uplink Ports
Uplink Eth Interfaces
@ Ports
Eth Interface 1/1

VLANs Name

VP Optimization Sets Fabric Interconnect A (primary)

Fabric B Fixed Module

QoS System Class Port 1

LAN Pin Groups

Port 2
> Thmchnlri Dnlirioc

Figure 3-10 Adding another uplink

We also need to add two uplinks (eth1/1 and eth 1/2) to

FI-B, using the same method.
Now we have an even number of links, we can create
our port channels.
We create the port channels by going top LAN ➤ LAN
Cloud ➤ Fabric A ➤ Port Channels. Click the “Add.” We set
the port channel number, and give it a name (Figure 3-11).
Click “Next”
 Create Port Channel
Set Port Channel Name

Name : PO-1
Add Ports

Cancel

Figure 3-11 Creating the port-channel

In the next window, select the interfaces to add to the

port channel (Figure 3-12).

Create Port Channel

Set Port Channel Name Ports Ports in the port channel

Slot ID Aggr. Po... Port MAC Slot ID Aggr. Po... Port MAC
Add Ports
1 1 00:00:0... No data available

1 0 2 00:00:0...

Figure 3-12 Adding interfaces to the port-channel

Click the double arrow button to move them into the

port channel (Figure 3-13).
 Create Port Channel ? X

Set Port Channel Name Ports Ports in the port channel

Slot ID Aggr. Po... Port MAC Slot ID Aggr. Po... Port MAC
Add Pons
No data available 1 0 1 00:00:0...

1 0 2 00:00:0...

< Prev Finish Cancel

Figure 3-13 Added interfaces

Once you have added the interfaces, click “Finish.” You

will receive a message to say that the port channel has
been created (Figure 3-14).
Figure 3-14 The port channel has been created.

Repeat the process, adding port channel 2 to FI-B.

Our UCS port channel setup will look like this (Figure 3-
15):

Fl-A Fl-B
Figure 3-15 Port-Channel topology

From the point of view of the Nexus switches above our

FI’s, the configuration would look like this (Figure 3-16):
Figure 3-16 Completed Port-Channels

The Nexus interfaces and port channels would be

configured as follows:

NX-9K01# sh run int eth1/1

interface Ethernet1/1
description ### FI-A PORT 1 UPLINK ###
switchport
switchport mode trunk
 switchport trunk allowed vlan all
spanning-tree port type edge
spanning-tree bpduguard enable
speed 40000
no negotiate auto
channel-group 1 mode active
no shutdown

NX-9K01# sh run int eth1/2

interface Ethernet1/2
description ### FI-B PORT 1 UPLINK ###
switchport
switchport mode trunk
switchport trunk allowed vlan all
spanning-tree port type edge
spanning-tree bpduguard enable
speed 40000
no negotiate auto
channel-group 2 mode active
no shutdown

NX-9K01# sh run int po 1

interface port-channel1
description ### VPC to FI-A ###
switchport
switchport mode trunk
switchport trunk allowed vlan all
speed 40000
no negotiate auto
no lacp suspend-individual
vpc 1

NX-9K01# sh run int po 2

interface port-channel2
 description ### VPC to FI-B ###
switchport
switchport mode trunk
switchport trunk allowed vlan all
speed 40000
no negotiate auto
no lacp suspend-individual
vpc 2

And the other switch configuration:

NX-9K02# sh run int eth 1/1

interface Ethernet1/1
description ### FI-A PORT 2 UPLINK ###
switchport
switchport mode trunk
switchport trunk allowed vlan all
spanning-tree port type edge
spanning-tree bpduguard enable
speed 40000
no negotiate auto
channel-group 1 mode active
no shutdown

NX-9K02# sh run int eth 1/2

interface Ethernet1/2
description ### FI-B PORT 2 UPLINK ###
switchport
switchport mode trunk
switchport trunk allowed vlan all
spanning-tree port type edge
spanning-tree bpduguard enable
speed 40000
no negotiate auto
 channel-group 2 mode active
no shutdown

NX-9K02# sh run int po 1

interface port-channel1
description ### VPC to FI-A ###
switchport
switchport mode trunk
switchport trunk allowed vlan all
speed 40000
no negotiate auto
no lacp suspend-individual
vpc 1

NX-9K02# sh run int po 2

interface port-channel2
description ### VPC to FI-B ###
switchport
switchport mode trunk
switchport trunk allowed vlan all
speed 40000
no negotiate auto
no lacp suspend-individual
vpc 2

In this configuration, we can set the interfaces to run

together (instead of singularly) and also benefit from a
considerable speed increase. While this is not something
we can achieve within the sandboxed environment that is
UCSPE (as our port channel status will show as
“Indeterminate,” we can see this in a real-life example
(Figure 3-17):
Admin Speed [Q 1 Gbps Q 10 Gbps (*)40 Gbps 0'25 Gbps Q Gbps O' Auto
Operational Speed(Gbps) : 80

Figure 3-17 80 Gbps port channel

Summary
In this chapter, we configured uplink ports to connect our
UCS to the rest of the network. In the next chapter, we will
start configuring the policies we need for our servers.
© The Author(s), under exclusive license to APress Media, LLC, part of
Springer Nature 2023
S. Fordham, Introducing Cisco Unified Computing System
https://doi.org/10.1007/978-1-4842-8986-0_4
4. Policies
Stuart Fordham1
(1) Bedfordshire, UK

At this stage, you are probably hungry to do some actual

UCS server configuration, I am, and this is where it all
starts; with policies.
Policies are used to create service profile templates, and
from these templates we can assign service profiles to our
servers. Before we start though, we should create our UCS
organization.
Creating the UCS Organization
We create UCS organizations to simplify our management.
They offer us a hierarchical way of organizing our policies
(as well as our pools and service profiles). We create the
organization by going to the Servers tab, expanding any
one of the options, such as Servers ➤ Policies ➤ root ➤ Sub-
Organizations, and selecting “Add.” Give the organization a
name, and click “OK” (Figure 4-1).

Figure 4-1 Creating the Organization

You will receive an acknowledgment that the

organization has been created (Figure 4-2).
 Create Organization

J Successfully created organization LearningUCS.

Figure 4-2 The organization has been completed

You will also notice that the same organization has been
created under Service Profiles and also under Service
Profile Templates, as well as Pools. You will also find the
new organization in the LAN tab, the SAN tab, the Storage
tab, and the Chassis tab.
Now, we can start to create our policies. We will not
cover every single policy option, as there are a lot of them.
Instead, we will focus on the ones required to create a
service profile template, which will then be applied to our
servers.
Storage Policies
Our storage policy is going to be quite simple; we will just
create a mirrored RAID volume from our two disks. To do
this, go to “Servers ➤ Policies ➤ root,” right-click “Local
Disk Config Policies” and select the pop-up option to create
one. Call it “LocalDiskPol” and set the mode to “RAID 1
Mirrored” (Figure 4-3).
 Create Local Disk Configuration Policy

Name LocalDiskPol

Description

Mode RAID 1 Mirrored

Protect Configuration
If Protect Configuration is set, the local disk configuration is preserved if the service profile is disassociated
with the server. In that case, a configuration error will be raised when a new service profile is associated with
that server if the local disk configuration in that profile is different.
FlexFIash
FlexFIash State : (•) Disable Q Enable
If FlexFIash State is disabled, SD cards will become unavailable immediately.
Please ensure SD cards are not in use before disabling the FlexFIash State.
FlexFIash RAID Reporting State : (•) Disable Q Enable
FlexFIash Removable State : Q Yes Q No (•) No Change

If FlexFIash Removable State is changed, SD cards will become unavailable temporarily.

Please ensure SD cards are not in use before changing the FlexFIash Removable State.

Cancel

Figure 4-3 Local Disk policy

Dynamic vNIC Connection Policies
Dynamic vNICs are not applicable to us (in our sandboxed
environment), as these are used for determining
connectivity between virtual machines and dynamic vNICs
running on servers with VIC adapters. However, if we were
running through the service profile wizard (which we will
do in the next chapter), this is where we would set up this
connectivity. In the same wizard though, is the VLAN
creation, which is where we are going to sidestep to.

Creating VLANs
UCSPE has already created some VLANs for us, but we will
create some more, by going to LAN ➤ LAN Cloud ➤ VLANs.
We can create VLANs on a per-Fabric basis, or on both
fabrics at the same time. Click “All” and then click “Add..
Our first VLAN will be called “DB,” and will have a VLAN
ID of 10 (Figure 4-4).

LAN / LAN Cloud

LAN
LAN Uplinks VLANs Server Links MAC Identity Assignment IP Identity Assignment

LAN Cloud Dual Mode Fabric A Fabric B VLAN Groups VP Optimization Sets

Fabric A

Port Channels
Create VLANs
Port-Channel 1 PO-1 0 VLAN Name/Prefix DB
Uplink Eth Interfaces
Create Multicast Policy
Multicast Policy Name : <not se(>
VLANs

VP Optimization Sets
• Common/Global Fabric A Fabric B Both Fabrics Configured Differently
Fabric B You are creating global VLANs that map to the same VLAN IDs in all available fabrics.

Port Channels
.
Enter the range of VLAN IDs.(e.g. " 2009-2019" " 29,35,40-45" , " 23" , " 23.34-45" )
VLAN IDs: 10|
Uplink Eth interfaces

VLANs
Sharing Type : • None Primary Isolated Community

Figure 4-4 Creating the DB VLAN

As the GUI shows, we can use this to create ranges of

VLANs on both fabrics (Common/Global), individual
fabrics, or we can configure the fabrics differently. The
latter option allows us to specify different VLAN IDs for
each fabric (though the name we give this particular VLAN
will be the same across both fabrics, just the VLAN ID will
be different).
The sharing type is for setting up private VLANs
(PVLAN) and allows us, if we should so desire, to isolate
ports. We create a primary VLAN and one (or more)
secondary VLANs, which can either be an isolated or a
community VLAN. Isolated ports can only communicate
with the associated port in the primary VLAN, not even
with each other. Community ports – communicate with
each other and with promiscuous ports. For both Isolated
and Community VLANs, we must create a primary VLAN
first.
Repeat the process, creating a VLAN 13, which is our
DMZ. Our VLANs should look like Figure 4-5.
LAN ¡ LAN Cloud

LAN Uplinks VLANs Server Links MAC Identity Assignment IP Identity Assignment QoS Global Policies Faults Events FSM

All Dual Mode Fabric A Fabric B VLAN Groups VP Optimization Sets

v, Advanced Filter f Export Print

Name ID Fabric ID Type Transport Native VLAN Sharing

VLAN DB(10) 10 Dual Lan Ether No None

VLAN default (1) 1 Dual Lan Ether Yes None

VLAN default (1) 1 B Lan Ether No None

VLAN default (1) 1 A Lan Ether No None

VLAN DMZ (13) 13 Dual Lan Ether No None

VLAN finance (3) 3 B Lan Ether No None

VLAN finance (3) 3 A Lan Ether No None

VLAN human-reso... 5 B Lan Ether No None

VLAN human-reso... 5 A Lan Ether No None

© Add '
Delete O Info

Figure 4-5 Our VLANs

vNIC/vHBA Placement
UCS blades have a component called a “Mezzanine” card.
Mezzanine cards can give us storage acceleration, port
expansion, GPUs (Graphics Processing Units) and VICs
(Virtual Interface Cards). We also have mLOMs (modular
LAN on Motherboard) cards, which offer VIC expansion.
The UCS, as we spoke about back in Chapter 2, has an
IOM and each IOM has a defined internal bandwidth (the
bandwidth that goes to the blades). The 2104 has 2x 10GB,
the 2204 has 4x 10GB, and the 2208 has 8x 10GB. This
means that a blade can get 80Gb-KR bandwidth across a
pair of IOMs.
The “KR” in this equation is a data rate specification
across a backplane medium (K), using a 64B/66B (R)
coding scheme (which is all to do with the electrical
encoding at the physical layer) in a single lane
configuration. For a deeper dive into this, check out this
very good blog post: www.tbijlsma.com/2012/03/how-ucs-
achieves-80gbe-of-bandwidth-per-blade/
We can control how each of our vNICs are assigned to
these lanes through a “Placement Policy,” allowing us to
utilize the hardware capacity to its fullest. Such as all
having all vNICs on one card and all vHBAs (virtual Host
Bus Adaptors) on another card; this could be due to
compatibility reasons, or card speed.
To create a placement policy we would go to Servers ➤
Policies ➤ root ➤ Sub-Organizations ➤ LearningUCS ➤
vNIC/vHBA Placement Policies. Although we don’t need to
create one ourselves, we would do so by clicking on the
“Add” button (Figure 4-6).
Figure 4-6 Placement Policies

The options we have are

All: the vCON (virtual network interface connection) is
used for all vNICS and vHBAs that are assigned to it, not
assigned to it, or are dynamic.
Assigned only: Only vNICs and vHBAs are assigned to
the vCON.
Exclude-Dynamic: The vCON cannot be used for
dynamic vNICS or vHBAs.
Exclude-Unassigned: the vCON can only be used for
vNICs or vHBAs assigned to it, or dynamic vNICs and
vHBAs.
 Exclude usNIC: The vCON cannot be used by user-space
NICs.
User-space NICs bypass the kernel when sending
packets, improving the performance of the software. For a
great blog post on actual use-cases, have a read of
https://jeremywaldrop.wordpress.com/2010/08/26/cis
co-ucs-vnicvhba-placement-policies/
vMedia Policies
vMedia policies allow us to boot our servers from ISO
images stored on a share. We create these by going to
Servers ➤ Policies ➤ root ➤ Sub-Organizations ➤
LearningUCS ➤ vMedia Policies. To create one, click “Add”
and enter the details, such as those in Figure 4-7.

Figure 4-7 A vMedia policy

In the preceding policy, we would be loading a CD ISO

image called Linux.iso from
https://san.domain.local/ISOs/Linux. Well, depending
on our server boot policy, that is.
Server Boot Policies
Server boot policies control how we boot our servers and in
what order we try these options. We configure a boot policy
by going to Servers ➤ Policies ➤ root ➤ Sub-Organizations
➤ LearningUCS ➤ Boot Policies. Click on “Add” to create a
new policy. In Figure 4-8, we are creating a policy to first
boot from a CD (or DVD) mounted via the CIMC. It will
then try to boot from a local LUN if no CD or DVD is found.

Create Boot Policy ? X

Name : BootPolicy

Description :I
Reboot on Boot Order Change : O
Enforce vNIC/vHBA/iSCSI Name : Q
Boot Mode : <• Legacy Uefi
WARNINGS:
The type (primary/secondary) does not indicate a boot order presence.
The effective order of boot devices within the same device class (LAN/Storage/iSCSI) is determined by PCIe bus scan order.
If Enforce vNIC/vHBA/iSCSI Name is selected and the vNIC/vHBA/iSCSI does not exist, a config error will be reported.
If it is not selected, the vNlCs/vHBAs are selected if they exist, otherwise the vNlC/vHBA with the lowest PCIe bus scan order is used.

@ Local Devices Boot Order

+ — Y/ Advanced Filter f Export Print O

Name Order * vNL. Type LU- WWN Slot... Boo... Boo... Des...

Add Local JBOD CIMC Mounted CD/DVD 1

Add SD Card Local LUN 2

Add internal USB

Add External USB

Add Embedded Local LUN

Add Embedded Local Disk

Add CD/DVD I Move Down

Move Up Delete
Add Local CD/DVD

Add Remote CD/DVD

Add Floppy

Add Local Floppy

Add Remote Floppy

Cancel

Figure 4-8 Server Boot Policy

Maintenance Policies
We will, from time to time, have to perform maintenance on
our UCS, usually in the way of upgrading the firmware. You
may upgrade at a particular time, taking the inevitable
reboots of the fabric and IOMs as you go. However, you
may not want the blades to reboot at the same time, so,
unless you want to cause an outage, it’s a good idea to
implement a maintenance policy. Head to Servers ➤
Policies ➤ root ➤ Sub-Organizations ➤ LearningUCS ➤
Maintenance Policies and click “Add.” Create a
maintenance policy that will (at a very minimum) require a
user acknowledgment before rebooting the servers (Figure
4-9).

Figure 4-9 Maintenance Policy

Server Pool Policies
Server Pool are used for servers that share characteristics,
such as type, amount of memory, drive configuration, or the
type of CPU. We create a pool first (Servers ➤ Pools ➤ root
➤ Server Pools). We start by naming our server pool as
shown in Figure 4-10.

Create Server Pool

Set Name and Description Name : MyServerPool

Description : |
Add Servers

Figure 4-10 Creating a server pool

Next, we add our servers, selecting them in the first

window (Figure 4-11).
 Create Server Pool ? X

Set Name and Description Servers Pooled Servers

O 0
Add Servers

3 1 UCS8-... No data available

3 2 UCSB-... »
«
3 3 UCSB-...

4 1 UCSB-...

4 2 UCS0-...

1 UCSC-...

2 UCSC-...

3 UCSC-...

4 UCSC-...

5 UCSC-—

6 UCSC-...

7 UCSC-...

Model: UCSB-B200-M5 Model:

Serial Number; SRV122 Serial Number.
Vendor: Cisco Systems Inc Vendor:

< Prev Finish Cancel

Figure 4-11 Selecting the servers for the pool

Once we have added the servers (Figure 4-12), click

“Finish.”
 Create Server Pool ? X

Set Name and Description Servers Pooled Servers

Add Servers
Chassis ID Slot ID Rack...- PID C... SI- R... U... PID A... S- C-

3 3 UCSB-.. 3 1 U- U- S- 8

4 1 UCSB--. » 3 2 U_ U- S... 8

4 2 UCSB-...

1 UCSC-...

2 UCSC-...

3 UCSC-...

4 UCSC-...

5 UCSC-...

6 UCSC-...

7 UCSC-...

8 UCSC-...
9 UCSC-...

E££L •±1*

Model: Model:
Serial Number: Serial Number:
Vendor: Vendor:

< Prev Finish Cancel

Figure 4-12 Our server pool

We can also create a pool qualification, which will, as we

just mentioned, pool servers based on characteristics. We
do this from Servers ➤ Policies ➤ root ➤ Sub-Organizations
➤ LearningUCS ➤ Server Pool Policy Qualifications (Figure
4-13).
Figure 4-13 Server pool policy qualifications

In our qualification, we are going to keep it simple and

just match against the server product ID (PID), as shown in
Figure 4-14.
Figure 4-14 A server PID qualification

Once we have added this (Figure 4-15), we can click on

“Finish” to create the qualification.
Qualifications

+ — Tz Advanced Filter Export

*Model
Print O
Name * Max From To Archit... Speed Step... Powe...

Server PID Qualification UCSB-B200-M5

@ Add Delete Info

Figure 4-15 Our completed qualification

The last step is to create a policy to tie these all

together. We do this by going to Servers ➤ Policies ➤ root
➤ Sub-Organizations ➤ LearningUCS ➤ Server Pool
Policies. We name our policy and either assign the policy to
a pool or we can select the qualification, but not both
(Figure 4-16). While we can set both when we create the
policy, once we go back into it, we will find the pool empty.
Pool assignments are fairly static, whereas qualifications
are more dynamic in nature.

Create Server Pool Policy

Name

Description :

Target Pool :

Qualification :

Figure 4-16 Our Server Pool Policy

The last policies we are going to cover are some small

but very important ones!
Operational Policies
Operational policies cover aspects of the servers like BIOS,
IPMI, management IP addresses, power control, scrub
policies, KVM management and graphics card policies.
There are three that we should cover, starting with
management IP addresses.

Management IP Addresses
The management IP addresses come from a defined pool of
IP addresses and it is to one of these addresses we connect
to when we launch the KVM from the UCS GUI. We create
the pool by going to LAN ➤ Pools ➤ root ➤ Sub-
Organizations ➤ LearningUCS ➤ IP Pools. We can create
them under LAN ➤ Pools ➤ root ➤ IP Pools as well, if you so
desire. Create a new IP pool called “KVM-IP-Pool” (Figure
4-17).

Figure 4-17 The KVM pool

 Click “Next,” and assign a block of IP addresses (Figure
4-18). This needs to be large enough to cover all the
servers you have (and any future ones).

Figure 4-18 The KVM Pool IP range

Pick a range that doesn’t overlap with anything (such as

your DHCP scope) otherwise this could cause issues in your
environment. The pool will appear in the GUI (Figure 4-19).

Create IP Pool ? X

1 Define Name and Description + — Tz Advanced Ffltor + Expon * Print 0

Name From To Subnet Default Gateway Primary DNS Secondary DNS
2 Add IPv4 Blocks
[172.16.31.1 ... 172.16.3110 172.16,31.59 255 255.255.0 17216.31 1 172.16.31.4 172.16.31.5

3 Add IPv6 Blocks

Figure 4-19 The finished IP pool

As we are not adding an IPv6 pool, click “Next,” and

then click “Finish.” Now that we have our port range, we
need to specify which port we will be using.
KVM Management Policy
The default KVM port is 2068, but we can change that by
going to Servers ➤ Policies ➤ root ➤ Sub-Organizations ➤
LearningUCS ➤ KVM Management Policy. Create a new
policy called KVM-Port-Policy, setting the port to 3099
(Figure 4-20).

Create KVM Management Policy

Default KVM Port is 2068. Please select the port range between 1024-49151

Figure 4-20 The KVM management port policy

Onto our final policy.

Scrub Policies
The last set of policies we are going to implement are scrub
policies. These control how the disks on a server will be
treated in scenarios such as moving blades. For example,
you are balancing the blades in your UCS chassis, evening
out three application blades across three chassis. You have
arranged the downtime, attached the service profile to the
empty destination slot, and removed the blade. When you
put it in the new chassis slot, the blade is picked up and
once it’s booted up, you find that (due to the configured
Scrub policy) the disks have been wiped.
This is where scrub policies will save you. Head to
Servers ➤ Policies ➤ root ➤ Sub-Organizations ➤
LearningUCS ➤ Scrub Policies. Set all the options to “No”
(Figure 4-21).

Create Scrub Policy

Cancel

Figure 4-21 Our scrub policy

Now, we can keep our data safe if we move a blade!

Before we move onto the next chapter, however, we need
to create a few more items, namely our pools and a VSAN

UUID Pool
We need to be able to identify our servers in UCS, well,
more specifically, the UCS systems need to identify our
servers. While we can name them (by giving them labels) in
the UCS GUI, the backend systems have a different way of
referencing the servers, and this is through a UUID. The
UUID (Unique Identifier) is a 128-bit reference. We can
create a pool of UUIDs, saving us from manually assigning
them to each of our servers. To create the pool, go to
Servers ➤ Pools ➤ root ➤ Sub-Organizations ➤
LearningUCS ➤ UUID Suffix Pools. Click “Add” and create
a block of 30 UUIDs, as in Figure 4-22.
Figure 4-22 Creating a UUID Suffix pool

Once we have created our pool, we can see the

sequential suffixes (Figure 4-23).

Sub¬
Servers / Pools / root / X.
Organizations / LeamingUCS / UUID Suffix Pools / Pool UUID-Pool

General | UUID Suffixes UUID Blocks Faults Events

Tz Advanced Filter + Export Í Print

UUID Suffix Assigned Assigned To

0000-000000000001 No

0000-000000000002 No

0000-000000000003 No

0000-000000000004 No

0000-000000000005 No

0000-000000000006 No

Figure 4-23 Our UUID pool

MAC Pools
In the same way that our servers need a unique identifier,
so do our network interfaces. We do this through MAC
pools. Navigate to LAN ➤ Pools ➤ root. Click “Add” and
name the MAC pool (such as “MyMacPool,” as in Figure 4-
24).

Figure 4-24 MyMacPool

Click “Next” to add the MAC addresses (Figure 4-25).

Cisco suggests that the block uses 00:25:B5:xx:xx:xx for
compatibility reasons.
 Create MAC Pool

1 Define Name and Description Tz Advanced Filter f Export # Print 0

Name From To
Add MAC Addresses
[00:25:B5:00:00:0... 00:25:B5:00:00:00 00:25:B5:00:00:13

Create a Block of MAC Addresses

First MAC Address : 00:25:B5:00:00:00 Size : | 20|

To ensure uniqueness of MACs in the LAN fabric, you are strongly encouraged to use the following MAC
prefix:
00:25:B5:xx:xx:xx

Prev Finish Cancel

Figure 4-25 Our MAC address block

WWNN
In the same way that we created blocks of IDs for our
servers and MAC addresses for our network cards, our SAN
fabric will also need some uniqueness. We do this through
the WWNN (World Wide Node Names) pool, which has a
number of WWNs (World Wide Names). Navigate to “SAN
➤ Pools ➤ root ➤ Sub-Organizations ➤ LearningUCS” and
right-click WWNN Pools, choosing the option to create a
new one. Name the pool “wwnn-pool” (Figure 4-26) and
click “Next.”
Figure 4-26 Our WWNN pool

Create a block of sixty WWNs, following the naming

advice of Cisco (20:00:00:25:b5:xx:xx:xx), as shown in
Figure 4-27.
Figure 4-27 Our WWN block
VSAN
The last component we are going to create is our VSAN.
This will enable us to separate our storage traffic. We will
be using the 2000 and 2001 as our VSAN and FCoE (Fibre
Channel over Ethernet) IDs (as these are the ones Cisco
suggests), as shown in Figure 4-28.

Create VSAN

Name : MyVSAN
FC Zoning Settings

FC Zoning : • Disabled Enabled

Do NOT enable local zoning if fabric interconnect is connected to an upstream FC/FCoE switch.

Common/Global Fabric A Fabric B • Both Fabrics Configured Differently

You are creating a single VSAN that maps to A VLAN can be used to carry FCoE traffic and can be mapped to this
a different VSAN ID in each available fabric. VSAN.

Enter the VSAN IDs that map to this VSAN. Enter the VLAN ID that maps to this VSAN.

Fabric A Fabric A
—
VSAN ID 2000 FCoE VLAN : 2000

Fabric B Fabric B

VSAN ID 2001 FCoE VLAN : 2001

Figure 4-28 Our VSAN

Summary
In this chapter, we have created the policies and pools to
control our servers. In the next chapter, we will start
assigning these to our servers.
© The Author(s), under exclusive license to APress Media, LLC, part of
Springer Nature 2023
S. Fordham, Introducing Cisco Unified Computing System
https://doi.org/10.1007/978-1-4842-8986-0_5
5. Service Profiles and
Templates
Stuart Fordham1
(1) Bedfordshire, UK

In this chapter, we will be creating a Service Profile

template from the pools and policies we created in Chapter
4.
Creating Service Profile Templates
Navigate to “Servers ➤ Service Profile Templates ➤ root ➤
Sub-Organizations ➤ LearningUCS ➤ Service Profiles ➤
Service Profiles” and click “Add” (Figure 5-1).

Servers / Service Profiles / root / Sub-Organizations / LearningUCS

Servers General Sub-Organizations Service Profiles Pools Policies FC Zones Faults Events

Service Profiles Service Profiles Associated Blades Associated Racks Pooled Servers Service Profile Templates
root
Sub-Organizations + — f Export Print

LearningUCS Name

Sub-Organizations Service Profiles

Service Profile Templates

root

Sub- Organizations
LearningUCS
Sub-Organizations
Policies

root (?) Add

Figure 5-1 The Service Profile page

Choose the expert option from the pop-up menu (Figure

5-2).

Create Service Profile (expert)

Create Service Profiles From Template
Create Service Profile

© Add Delete Info

Figure 5-2 Choose the expert option

Name the service profile “B200-template” and from the

UUID Assignment drop-down, select the UUID-Pool (Figure
5-3).
 Create Service Profile (expert) ? X
You must enter a name for the service profile. You can also specify how a UUID will be assigned to this profile and enter a description
of the profile.

Identify Service Profile

Name : B200-template

Storage Provisioning The service profile will be created in the following organization, its name must be unique within this organization.
Where : org-root/org-LeamingUCS
Specify how the UUID will be assigned to the server associated with this service profile.
Networking UUID

SAN Connectivity
UUID Assignment; UUID-Pool(30/30)

Zoning
Create UUID Suffix Pool
The UUID will be assigned from the selected pool.
vNIC/vHBA Placement The available/total UUlDs are displayed after the pool name.

vMcdia Policy
Optionally enter a description for the profile. The description can contain information about when and where the service profile should be
used.
Server Boot Order

Maintenance Policy

Server Assignment

Operational Policies

Next > Finish

Figure 5-3 Naming the template

Click “Next” to move on to the next page. On this page,

we will set our storage options, we can select a storage
profile, or a policy, or a local disk configuration policy,
which is the option we will use (Figure 5-4).
Select “LocalDiskPol” from the dropdown box.
 Create Service Profile (expert)
Optionally specify or create a Storage Profile, and select a local disk configuration policy.
Identify Service Profile

Storage Provisioning Specific Storage Profile Storage Profile Policy Local Disk Configuration Policy |
Local Storage: LocalDiskPol
Networking

Mode : RAID 1 Mirrored

SAN Connectivity Create Local Disk Configuration Policy
Protect Configuration : No
If Protect Configuration is set. the local disk configuration is
Zoning preserved if the service profile is disassociated
with the server. In that case, a configuration error will be
raised when a new service profile is associated with
vNIC/vHBA Placement that server if the local disk configuration in that profile is
different.
FlexRash
vMedia Policy FlexFIash State : Disable
If FlexFIash State is disabled. SD cards will become
unavailable immediately.
Server Boot Order Please ensure SD cards are not in use before disabling the
FlexFIash State.
FlexFIash RAID Reporting State : Disable
Maintenance Policy
FlexFIash Removable State : No Change
If FlexFIash Removable State is changed. SD cards will
Server Assignment become unavailable temporarily.
Please ensure SD cards are not in use before changing the
FlexFIash Removable State.
Operational Policies

< Prov Next > Finish Cancel

Figure 5-4 Selecting the LocalDiskPol

Click “Next” to move on to the Networking page.

We did not create a dynamic vNIC policy in the last
chapter, but we did create some new VLANs. Assign eth0 to
the DMZ VLAN, and eth1 to the DB VLAN (Figure 5-5).
 Create Service Profile (expert)
Optionally specify LAN configuration information.
Identify Service Profile

Storage Provisioning Dynamic vNIC Connection Policy:

Networking

SAN Connectivity
How would you like to configure LAN connectivity?

• Simple
'
Expert No vNICs ( Hardware Inherited Use Connectivity Policy
Zoning
Specify the virtual network adapters (vNICs) that server should use to connect to a LAN. To specify more than two vNICs, select the
Expert configuration mode.
vNIC/vHBA Placement
vNIC 0 (Fabric A) vNIC 1 (Fabric B)

vMedia Policy Name : ethO Name : eth1

Select VLAN : DMZ - 13 (None) Select VLAN : DB- 10 (None)

Server Boot Order
Create VLAN Create VLAN
VLAN in LAN cloud will take the precedence over the VLAN in LAN cloud will take the precedence over the
Maintenance Policy Appliance Cloud when there is a name clash. Appliance Cloud when there is a name clash.

Server Assignment

Operational Policies

< Prev Noxt> Finish Cancel

Figure 5-5 The networking options

Clicking Next will take us to the SAN connectivity page.

Here, we will assign the wwnn-pool we previously created
(Figure 5-6).
 Create Service Profile (expert)
Optionally specify disk policies and SAN configuration information.
Identify Service Profile

How would you like to configure SAN connectivity?

Storage Provisioning
• Simple Expert No vHBAs Hardware Inherited Use Connectivity Policy
A server is identified on a SAN by its World Wide Node Name (WWNN). Specify how the system should assign a WWNN to the server
Networking associated with this profile.
World Wide Node Name

SAN Connectivity

WWNN Assignment: wwnn-pool(60/60)

Zoning

Create WWNN Pool

vNIC/vHBA Placement The WWNN will be assigned from the selected pool.
The avaiiabie/total WWNNs are displayed after the pool name.

vModia Policy

Specify the virtual host bus adapters (vHBAs) that the server should use to connect to a SAN. To specify more than two vHBAs, select
Server Boot Order the Expert configuration mode.
vHBA 0 (Fabric A) vHBA 1 (Fabric B)
Maintenance Policy
Name fcO Name : fc1

10 Server Assignment Select VSAN : default Select VSAN : default

।

Create VSAN Create VSAN

Operational Policios WARNING: there are not enough WWN addresses available in
WARNING: there are not enough WWN addresses available in
the default WWPN pool. This vHBA will be created with an
the default WWPN pool. This vHBA will be created with an
Invalid WWN address.
Invalid WWN address.

Next > Finish Cancel

Figure 5-6 The SAN connectivity page

Click on “Next” to move forward into the zoning tab

(Figure 5-7). We are not going to set up any HBA zoning
here, so click “Next” again.
 Create Service Profile (expert)
Specify zoning information
Identify Service Profile

Zoning configuration involves the following steps:

Storage Provisioning 1 . Select vHBA Initiator(s) (vHBAs are created on storage page)
2. Select vHBA Initiator Group(s)
3. Add selected Initiator(s) to selected Initiator Group(s)
Networking

Select vHBA Initiators Select vHBA Initiator Groups

SAN Connectivity
Name Name Storage Connection Policy _
Zoning No data available

fc1
vNIC/vHBA Placement

vMedia Policy

Server Boot Order

? © Add
Maintenance Policy

Server Assignment

Operational Policies

J < Prev Finish Cancel

Figure 5-7 The zoning options

On the vNIC/vHBA Placement tab, we can choose how

our network interfaces get their placement (Figure 5-8).

r- Create Service Profile (expert) ? X

Specify how vNICs and vHBAs are placed on physical network adapters
Identify Service Profile

o Storage Provisioning
vNIC/vHBA Placement specifies how vNICs and vHBAs are placed on physical network adapters (mezzanine)
In a server hardware configuration independent way.

Select Placement: Create Placement Policy

Let System Perform Placement
Networking
System will perform automatic placement of vNICs and vHBAs based on PCI order.

SAN Connectivity Name Address Order A

o Zoning
vNIC ethO

vNIC oth1
Derived

Derived
1

vNIC/vHBA Placement vHBA fcO Derived 3

vHBA fcl Derived 4

vMedia Policy

Server Boot Order

Move Down l Delete O Modify
Figure 5-8 Network interface placement

Select “vNIC eth0” and click “Modify” (Figure 5-9).

Select “MyMACPool” for the MAC address assignment,
assign eth0 to the DMZ VLAN, then click “OK.”

Modify vNIC
Name : ethO
MAC Address

MAC Address Assignment: MyMACPool(20/20)

Create MAC Pool

MAC Address : Derived
The MAC address will be automatically assigned from the selected pool.
The MAC address assignment change will be effective only after server reboot.

Use vNIC Template : O

Create vNIC Template

Fabric ID: ® Fabric A Q Fabric B Enable Failover

VLANs V'LAN Groups

Y, Advanced Filter f Export Print $

Select Name Native VLAN VLAN ID

DB o 10

default o 1

0 DMZ ® 13

,
finance o 3
r—

CDN Source : • vNIC Name User Defined

Figure 5-9 Configuring eth0

Do the same for eth1, assigning it to the DB VLAN

(Figure 5-10).
 Modify vNIC ? X

Name : eth1
MAC Address

MAC Address Assignment: MyMACPool(20/20)

Create MAC Pool

MAC Address : Derived
The MAC address will be automatically assigned from the selected pool.
The MAC address assignment change will be effective only after server reboot.

Use vNIC Template :

Create vNIC Template

Fabric ID: O Fabric A (•) Fabric B Enable Failover

VLANs VLAN Groups

Yz Advanced Filter f Export Print $

Select Name Native VLAN VLAN ID

0 DB ® 10

default o 1

DMZ 0 13

finance 0 3
r i

CDN Source : (• vNIC Name ( User Defined

—

<9 ( Cancel

Figure 5-10 Configuring eth1

For the fc0 and fc1 interfaces, assign them to the WWPN
pool and MyVSAN (Figures 5-11 and 5-12).
 Modify vHBA ? X

Name fcO
World Wide Port Name

WWPN Assignment: 20:00:00:25:B5:XX:XX:XX

Create WWPN Pool

WWPN : 20:00:00:25:B5:00:00:00|
Click here to verify if this WWPN is available

Use vHBA Template :

Create vHBA Template

Fabric ID : |®A QB
Select VSAN : MyVSAN Create VSAN

Pin Group <not set>

Create SAN Pin Group

Persistent Binding • Disabled Enabled

Max Data Field Size : 2048

oc Cancel

Figure 5-11 Configuring fc0

 Modify vHBA ’ X

Name : fd
World Wide Port Name

WWPN Assignment: 20:00:00:25:B5:XX:XX:XX

Create WWPN Pool

WWPN : 20:00:00:25:65:00:00:01

Use vHBA Template : O

Create VSAN

Pin Group <not set>

Cancel

Figure 5-12 Configuring fc1

Click “Next” once all four interfaces have been

configured.
The next screen is the vMedia tab. Here we will assign
the Linux vMedia policy we created (Figure 5-13), which
will set up the Linux.iso image to be used when we boot our
server (at least it would work in a real environment, but
we’ll just have to pretend in UCSPE).
 Create Service Profile (expert) ? x
Optionally specify the Scriptable vMedia policy for this service profile.
Identify Service Profile

Storage Provisioning vMedia Policy: unux

Networking Create vMedia Policy

Name Linux
SAN Connectivity
Description
Retry on Mount Failure Yes
Zoning
vMedia Mounts

vNIC/vHBA Placement
+ - Y> Advanced Filter f Export
* Print
Name Type Protocol Authenti... Server Filename Remote ... User Remap ... Writable
vMedia Policy
Linux... CDD HTTPS None san.do.. Linux.iso /ISOs/Li. . No No

Server Boot Order

Maintenance Policy

Server Assignment

Operational Policies

< Prev Next > Finish Cancel

Figure 5-13 Setting the vMedia policy

The next tab is where we set our server boot order

(Figure 5-14), first trying the CIMC mounted CD or DVD,
and then the local hard disk. Select “BootPolicy” from the
drop-down menu.
 Create Service Profile (expert)
Optionally specify the boot policy for this service profile.
Identify Service Profile

Select a boot policy.

Storage Provisioning
Boot Policy: BootPolicy
Create Boot Policy
Networking Name : BootPolicy
Description :
SAN Connectivity Reboot on Boot Order Change : No
Enforce vNIC/vHBA/iSCSI Name : Yes
Zoning Boot Mode : Legacy
WARNINGS:
The type (primary/secondary) does not indicate a boot order presence.
vNIC/vHBA Placement The effective order of boot devices within the same device class (LAN/Storage/iSCSl) is determined by PCIe bus scan order.
If Enforce vNIC/vHBA/iSCSI Name is selected and the vNIC/vHBA/iSCSI does not exist, a config error will be reported.
If it is not selected, the vNICs/vHBAs are selected if they exist, otherwise the vNIC/vHBA with the lowest PCie bus scan order is used.
vMedia Policy Boot Order

Advanced Filter f Export i“s Print $

Server Boot Order
Name Order vNIC/vH... Type LUN Name WWN Slot Nu„. Boot Na... Boot Path Descripti...

CIMC... 1
Maintenance Policy
Local ... 2
Server Assignment

Operational Policies

< Prev Next > Finish Cancel

Figure 5-14 The server boot order

Clicking “Next” takes us to the Maintenance Policy

window. Here, we are going to select “ServerMaintPol”
from the drop-down, as shown in Figure 5-15.
 Create Service Profile (expert)
Specify how disruptive changes (such as reboot, network interruptions, firmware upgrades) should be applied to the system.
Identify Service Profile

Storage Provisioning Q Maintenance Policy

Select a maintenance policy to include with this service profile or create a new maintenance policy that will be accessible to all service
Networking profiles.

Maintenance Policy: ^rverMaintroi

ServerMaintPol v
Cfeate Maintenance p0|lCy
SAN Connectivity
Name : ServerMaintPol
Description :
Zoning
Soft Shutdown Timer : 150 Secs
Storage Config. Deployment Policy : User Ack
vNIC/vHBA Placement Reboot Policy : User Ack

vMedia Policy

Server Boot Order

Maintenance Policy

Server Assignment

Operational Policies

< Prev Next > Finish Cancel

Figure 5-15 The server maintenance policy

Our penultimate window is for server assignment. We

can choose to assign our template to a server, as well as set
our firmware policies to control our BIOS, disk controllers,
and adaptors (Figure 5-16).
 Create Service Profile (expert) 7 X
Optionally specify a server or server pool for this service profile.
Identify Service Profile

You can select an existing server or server pod. or specify the physical location of the server you want to associate with this service
Storage Provisioning profile.

Server Assignment: Assign Later

Networking Create Server Pool

Select the power state to be applied when this profile is associated

SAN Connectivity with the server.
• Up Down
Zoning

vNIC/vHBA Placement The service profile is not automatically associated with a server. Either select a server from the list or associate the service
profile manually later.

vMedia Policy
@ Firmware Management (BIOS, Disk Controller, Adapter)

Server Boot Order

Maintenance Policy

Server Assignment

Operational Policies

Finish Cancel

Figure 5-16 Firmware policies

The final window is where we set our operational

policies, such as the KVM-IP-Pool for our management IP
addresses (Figure 5-17).
 Create Service Profile (expert)
Optionally specify information that affects how the system operates.
Identify Service Profile

Storage Provisioning @ BIOS Configuration

If you want to override the default BIOS settings, select a BIOS policy that will be associated with this service profile
Networking Create BIOS Policy
BIOS Policy : <notset>
*
SAN Connectivity

@ External IPMI/Redfish Management Configuration

Zoning

Q Management IP Address
vNIC/vHBA Placement

Outband IPv4 Inband

vMedla Policy

Management IP Address Policy: KVM-IP-Pool(50/50)

Server Boot Order
IP Address : 0.0.0.0
Maintenance Policy Subnet Mask : 255.255.255.0
Default Gateway : 0.0.0.0
The IP address will be automatically assigned from the selected pool.
Server Assignment

Operational Policies

Create IP Pool

< Prev Finish Cancel

Figure 5-17 Management IP address policy

This is where we also set our scrub and management

port policies, which are “ScrubPolicy” and “KVM-Port-
Policy,” respectively (Figure 5-18).

Q Scrub Policy
Create Scrub Policy
Scrub Policy : ScrubPolicy

@ KVM Management Policy

Create KVM Management Policy

KVM Management Policy : KVM-Port- Policy

Figure 5-18 Scrub and KVM port policies

 Once we have finished setting these, click “Finish.” We
should receive a notice that our service profile has been
successfully created (Figure 5-19).

Figure 5-19 Our template is complete

Now that we have a template, the next logical step is to

associate this with a server. Select the first server in the
first chassis, which, in this instance, is Server 1 in Chassis
3 (Equipment ➤ Chassis ➤ Chassis 3 ➤ Server 1). Select
the server and in the actions pane, we can see an option for
“Associate Service Profile” (Figure 5-20).
 Equipment / Chassis / Chassis 3 / Servers / Server 1

General Inventory Virtual Machines Installed

Fault Summary

0 0 0 0

Status

Overall Status : I Unassociated

@ Status Details

Actions

Create Service Profile

Associate Service Profile

Figure 5-20 Associating a template to a server

Clicking on this link will bring up a new window in

which we can select from the service profiles that we have
created. Select our template, and click “OK” (Figure 5-21).
 Associate Service Profile
Select an existing service profile to associate with the selected server.

Service Profiles

• Available Service Profiles Q All Service Profiles

Select Name Org Assoc State

Service Profile B200-template org-root/org-LearningUCS Unassociated

Figure 5-21 Selecting the service profile

Once we click OK, we will see a warning box pop up.

This will list any issues with our template. More
specifically, it will list any issues that may occur by
assigning the template to the particular piece of hardware
that we have chosen. The template may be perfectly fine,
but specific policies may have different effects on different
hardware variations – such as disk configurations or the
number of network interfaces we have, for example.
Hopefully, there should not be any issues, and we should
just get a warning that the blade will reboot (Figure 5-22).
 Associate Service Profile

Create: Server sys/chassis-3/blade-1 (prg-rppt/prg-LearningUCS/ls-B200-template/pn)

Will cause the Immediate Reboot of:

Service Profile B200-template (org-root/org-LearningUCS/ls-B200-template)\Serw. sys/chassis-
3/blade-1]

Are you sure you want to apply the changes?

Press Yes to disregard the warning and submit changes, No to quit the wizard
or Cancel to make changes to the current configuration.

Cancel

Figure 5-22 Associating the service profile

Once you click “Yes” to confirm, you will receive a final

dialog to say that the operation has started (Figure 5-23).

Associate Service Profile

The system is modifying Service Profile B200-template's association.

You can monitor the operation's progress on the server's FSM tab and in the Overall Status field.

OK

Figure 5-23 The service profile is now associated

We can watch the operation from the blade’s General

page. After a few moments, the Overall Status will change
from “Unassociated” (Figure 5-20) to “Config” (Figure 5-
24).
 Status

Overall Status : 0Config

@ Status Details

Figure 5-24 The Config status

Expanding the Status Details box will show us what task

is currently running (Figure 5-25).

Status

Overall Status : 0Config

@ Status Details

Current Task

Waiting for system reset(FSM-

STAGE:sam:dme:ComputePhysicalAssociate:
BootWait)

Configuration Error not-applicable

Admin State f In Service
Discovery State t Complete
Avail State 1 Unavailable
Assoc State 0Establishing
Power State f On
Slot Status f Equipped
Check Point Discovered
Figure 5-25 The Current Task

There is more information to be found, though, and this

is in the FSM tab. This shows in much greater detail all the
steps that occur when assigning a service profile to a
server (such as Figure 5-26). The FSM tab is useful in many
other scenarios, which we will look at in the
troubleshooting chapter (Chapter 7).

Equipment / Chassis / Chassis 3 / Servers / Server 1

General Inventory Virtual Machines Installed Firmware CIMC Sessions SEL Logs VIF Paths Health Diagnostics Faults Events | FSM Statist.
FSM Status : In Progress
Description :
Current FSM Name : Associate
Completed at

Progress Status

Remote Invocation Result

Remote Invocation Error Code : 1006
Remote Invocation Description : Walting for storage subsystem to initialize

@ Step Sequence

Order Name Description Status Timestamp Retried

1 Associate Download Images Download images from oper... Skip 2022-06-06T02:06:46Z 0

2 Associate Copy Remoto Copy images to peer node(F... Skip 2022-06-06T02:06:46Z 0

3 Associate Update IBMC Fw Update CIMC firmware of se... Skip 2022-06-06T02:06:46Z 0

4 Associate Wait For IBMC Fw- Wait for CIMC firmware com... Skip 2022-06-06T02:06:46Z 0

5 Associate Config User Access Configuring external user ac... Success 2022-06-06T02:06:47Z 1

asínrialn ArfHr9tn IRMP Cw» A mura tn PIMP ftrmitrara rd « «Wo •MYJ'j-nR-n«Tfi'>-nRxR7 n

Name : Associate Oob Poll Sas Expander Boot img Activate Status
Status : Skip
Description : Waiting for Sas expander boot firmware update to complete(FSM-STAGE:sam:dme:ComputePhyslcalAssoclate:OobPo)ISasExpanderBootlmgActivateStatus)

Order : 66
Retried : 0
Timestamp : 2022-06-06T02:07:21Z

Figure 5-26 The FSM tab

Once all the tasks have been completed, we will see the
status change to “Success,” and the server will be powered
on (Figure 5-27).
 Equipment / Chassis / Chassis 3 / Servers I Server 1

General Inventory Virtual Machines Installed Firmware CIMC Sessions SEL Logs V1F Paths Health Diagnostics Faults Events FSM | Statist > >
FSM Status Success
Description

Current FSM Mame Tumup

Completed at 2022-06-06102:07:522
Progress Status
Remote invocation Result Not Applicable
Remote Invocation Error Code . None
Remote invocation Description

© Step Sequence

Order Name Description Status Timestamp Retried

1 Tumup Check Power Availab. . Check if power can he alloc. Skip 2022-06-06T02:07:52Z 0

2 Tumup Power Deploy Wait Waiting for power allocation ... Skip 2022-06-0€T02.07:52Z 0

3 Tumup Execute Power-on server sys/chassis... Success 2022-06-06T02.07:52Z 1

Figure 5-27 The association is a success

Returning to the general tab, we will see that the server

now has an overall status of “OK” and that the server has
been powered on (Figure 5-28).

Status

Overall Status : t OK

@ Status Details

Configuration Error not-applicable

Admin State f In Service
Discovery State f Complete
Avail State Unavailable
Assoc State f Associated
Power State t On

Slot Status f Equipped

Check Point Discovered

Figure 5-28 The server is now associated with a service profile

 From the servers General tab, we can also see which
template has been applied (Figure 5-29). This will be shown
in its full path format (org-root/org-LearningUCS/ts-B200-
template).
Properties

Slot ID 1 Chassis ID : 3
Product Name Cisco UCS B200 M5 2 Socket Blade Server

Vendor Cisco Systems Inc PID : UCSB-B200-M5

Revision 0 Serial : SRV122

Manufacturing Date N/A
Asset Tag

Name

User Label

Unique Identifier 77396ece-e377-11ec-0000-00000000000f

Service Profile org-root/org-LeamingUCS/ls-B200-template

Health LED

Figure 5-29 The service profile is shown on the general tab

We can also check the success of applying our template

by checking the various settings applied to our server, such
as the boot order (Figure 5-30).

© Boot Order Details

| Configured Boot Order | Actual Boot Order

+ — T> Advanced Filter + Export if Print

Name Order A vNIC... Type LUN ... WWN Slot ... Boot... Boot... Des...

CIMC Mounted CD/DVD 1

Local LUN 2

Figure 5-30 The boot order settings

The RAID settings can be found in the Inventory, under

the Storage tab (Figure 5-31).
 General Inventory Virtual Machines Installed Firmware CIMC Sessions SEL Logs VIF Paths Health Diagnostics Faults Events FSM Statist

Motherboard CIMC CPUs GPUs Memory Adapters HBAs NlCs iSCSI vNICs Security Storage Persistent Memory

Controller LUNs Disks Security

+ — Advanced Filter t Export *> Print O

Name ID Type Subtype

Storage Controller SAS 1 1 SAS NA

General FSM Faults Events Statistics

btan nme : n/a tna nme : n/a

@ Local Disk Configuration Policy

Mode : RAID 1 Mirrored

Protect Configuration : No
If Protect Configuration is set, the local disk configuration is preserved if the service profile is disassociated
with the server. In that case, a configuration error will be raised when a new service profile is associated with
that server if the local disk configuration in that profile is different.

Figure 5-31 The RAID settings

The MAC address assignment can also be found in the

Inventory, this time under the NICs tab (Figure 5-32).

General Inventory Virtual Machines Installed Firmware CIMC Sessions SEL Logs VIF Paths Health Diagnostics Faults Events FSM

Motherboard CIMC CPUs GPUs Memory Adapters HBAs iSCSI vNICs Security Storage Persistent Memory

+ — Tz Advanced Filter f Export Print

Name vNIC Vendor PID Model Operability MAC Original MAC * ID

> NIC 1 ethO Cisco Systems... UCSB-MLOM-40G-03 Cisco UCS VIC 1340 Operable 00:25:B5:00:00:OF 00:00:00:00:0..

> NIC 1 eth! Cisco Systems... UCSB-VIC-M83-8P Cisco UCS VIC 1380 Operable 0O:25:B5:OO:OO:OE 00:00:00:00:0...

Figure 5-32 The MAC address assignment

We can check the management IP assigned to the server

from the General tab as well (Figure 5-33), which shows us
the pool name as well (KVM-IP-Pool).
 Q Management IP Address

Outband IPv4 Inband

Management IP Address Policy : Pooled

Pool Name KVM-IP-Pool
IP Pool Instance org-root/org-LearningUCS/ip-pool-KVM-IP-Pool

IP Address 172.16.31.59
Subnet Mask 255.255.255.0
Default Gateway 172.16.31.1

Reset Management IP Address

Figure 5-33 The management IP

We can also look in other places to check whether the

template assignment has been successful, such as the
UUID pools (Servers ➤ Pools ➤ root ➤ Sub-Organizations ➤
LearningUCS ➤ UUID Suffix Pools ➤ UUID-Pool) as shown
in Figure 5-34.
Servers / Pools I root / ^ub . u
Organizations
/ LeamingUCS / UUID Suffix Pools i Pool UUID-Pool

General | UUID Suffixes | UUID Blocks Faults Events

Advanced Filter f Export

* Print O
UUID Suffix Assigned * Assigned To Prev Assigned To

0O00-0OO00000O0OF Yes o«j-root/org-LeamingUCS/ls-B200-template org-rooVorg-LeammgUCS/ls-B200-template

0000-000000000001 No

Figure 5-34 The UUID pool usage

Summary
In this chapter, we created a service template using the
policies and pools we created in Chapter 4. We then
assigned this to a server and checked that the correct
template settings had been applied.
In the next chapter, we will look at securing the UCS.
© The Author(s), under exclusive license to APress Media, LLC, part of
Springer Nature 2023
S. Fordham, Introducing Cisco Unified Computing System
https://doi.org/10.1007/978-1-4842-8986-0_6
6. UCS Security
Stuart Fordham1
(1) Bedfordshire, UK

In this chapter, we will be securing our UCS system

through AAA for logins and hardening of the system.
AAA
Presently, we use the inbuild admin account, “USCPE.”
However, on a live system, you would naturally need to use
multiple accounts, which should be individual and
trackable. We do this through AAA (Authentication,
Authorization, and Accounting), whereby we connect to and
use the accounts from a directory service. The options we
have for this are LDAP, RADIUS, and TACACS+. This
chapter will look at how this would be achieved through
LDAP (specifically, Microsoft Active Directory). While
UCSPE is a self-contained environment, we can run
through the setup to see how it looks in real life.
When using AAA, we map groups of users to one of the
in-built UCS roles. These roles are

Role Description
name
aaa The AAA administrator role gives read-and-write access to the
users and roles as well as the AAA configurations. It has read
access to the rest of the UCS system.
admin As the name suggests, the admin role has full access to the entire
UCS system.
facility- The facility manager role has read/write access to the power
manager management portion of the system. It has read access to the rest
of the UCS system.
network This role has read/write access to the fabric interconnects and
also to network security functions. It has read access to the rest of
the UCS system.
operations Read/write access to logs and faults. It has read access to the rest
of the UCS system.
read-only Read-only access to the system.
server- Read/write access to the majority of the service profile settings.
compute Cannot create, modify, or delete vNICs or vHBAs.
server- Read/write access to physical server-related operations. It has
equipment read access to the rest of the UCS system.
Role Description
name
server- Read/write access to logical server-related operations. It has read
profile access to the rest of the UCS system.
server- Read/write access to server security-related operations. It has
security read access to the rest of the UCS system.
storage Read/write access to storage operations. It has read access to the
rest of the UCS system.

We start by creating an LDAP provider, by going to

“Admin ➤ User Management ➤ LDAP ➤ LDAP Providers”
and clicking “Add.” Enter the LDAP details specific to your
environment, such as those shown in Figure 6-1.

Create LDAP Provider

1 Hostname/FQDN (or IP Address) : DC01.domain.local

Order lowest-available
2
Bind DN CN=ucs-binduser,OU=svcAccounts,OU=Mgmt,DC=d

Base ON DC=domain,DC=local

Port : 389

Enable SSL

Filter : sAMAccountName=$userid

Attribute

Password

Confirm Password

Timeout : 30

Vendor ; |C Open Ldap

1 Í») MS AD

Finish

Figure 6-1 Creating the LDAP provider

 In the preceding, we are using the following settings
(details such as the Bind DN would be in the Active
Directory):
Hostname/FQDN (or IP address): DC01.domain.local
Order: lowest-available
Bind DN: CN=ucs-
binduser,OU=svcAccounts,OU=Mgmt,DC=domain,DC=lo
cal
Base DN: DC=domain,DC=local
Port: 389
Enable SSL: unticked
Filter: sAMAccountName=$userid
Attribute: empty
Password: $tr0ngP4ssw0rd!
Confirm Password: $tr0ngP4ssw0rd!
Vendor: MS AD
Click next and set group authorization to “Enable” and
group recursion to recursive (Figure 6-2). This means that
the UCS will use the target attribute (memberOf) to check
if the user authenticating is a member of a group. Group
recursion allows the UCS to look through the user directory
level by level until it finds the user.
Figure 6-2 The LDAP provider group settings

Leave the Target Attribute as “memberOf,” and “Use

Primary Group” unticked. Once you are done, click
“Finish.” You will get a notification that the LDAP provider
has been created (Figure 6-3).
 Create LDAP Provider

DC01.domain.local (Lowest Available) successfully created.

Figure 6-3 The LDAP provider has been created

Usually, you would have more than one LDAP provider

to offer a level of resiliency.
We now need to create an LDAP Provider Group. Go to
“Admin ➤ User Management ➤ LDAP ➤ LDAP Provider
Groups” and click “Add.” Name the provider group (Figure
6-4).

Create LDAP Provider Group

Name : domain.local

Figure 6-4 Creating an LDAP provider group

Select the LDAP providers you created in the first step,

and click the arrows to move it from LDAP Providers to
Included Providers (Figure 6-5).
Figure 6-5 The LDAP provider group

Click “OK” to save the group and you will see the notice
of completion (Figure 6-6).

Figure 6-6 The LDAP provider group has been created

 We will next create some LDAP group maps, to link our
AD groups to the UCS roles. We do this from “Admin ➤
User Management ➤ LDAP ➤ LDAP Group Maps.” Click
“Add” and add the LDAP group DN, and click on the
desired role, similar to Figure 6-7.

Create LDAP Group Map ? X

LDAP Group DN : CN=ucs-admins,OU=Groups,OU=Mgmt,DC=domain,

Roles Locales

aaa
¡V admin
facility-manager
network
operations
read-only
server-compute
server-equipment
server- profile
server-security
storage

O Cancel

Figure 6-7 Creating an Admin LDAP Group Map

Create another for the read-only role (such as Figure 6-

8).
Figure 6-8 Creating a read-only LDAP Group Map

Our two group mappings will be shown in the GUI

(Figure 6-9).
 All / User Management / LDAP / LDAP Group Maps

LDAP Group Maps

Yz Advanced Filter + Export Print

Name Roles

CN=ucs-admins,OU=Groups,OU=Mgmt,DC-domain,DC=l... admin

CN=ucs-readonly,OU=Groups>OU=Mgmt,DC=domain,DC... read-only

Figure 6-9 The completed LDAP group mappings

Now that we have our LDAP provider set up, we need to

tell the UCS how to use it. Go to “Admin ➤ User
Management ➤ Authentication ➤ Authentication Domains”
and click “Add.” In the Create a Domain window, name the
domain, and set the realm to LDAP, selecting the provider
group we created earlier (Figure 6-10).

Create a Domain

Realm O Local O Radius Q Tacacs (•) Ldap

Provider Group domain.local

Cancel
Figure 6-10 Creating the authentication domain

The next time we log in, the UCS will first search the
domain.local Active Directory for the user, before falling
back to the local user account database on the UCS. If the
user is not found in either Active Directory or the local
database, then the login will be denied.
Hardening the Web Interface
By default, UCS will use a self-signed certificate (in the
case of UCSPE, there is no certificate for the web
interface). We can’t assign a certificate in UCSPE (as this is
a closed-off playground), but we can run through the steps.
Navigate to “Admin ➤ Key Management” and click
“Trusted Points.” Trustpoints are the certification
authorities we are using. Click “Add.” Add the certificate
from your Certification Authority and click OK to create the
trust point (Figure 6-11).

Trusted Point

Name domain-CA

CA
certificate
Certificate Chain : goes
iere|

Figure 6-11 Creating the trust point

Click Key Management again, and then “Key Rings.”

Click “Add” and create a new key ring (Figure 6-12). Click
“OK.”
 Key Ring ? X

Name: MyKeyRing

Modulus ® Mod2048 D Mod2560 Q Mod3072 O Mod3584 : Mod4096

Figure 6-12 Creating a key ring

Select the new key ring and then click “Create

Certificate Request.” Enter the details (as shown in Figure
6-13).
 Create Certificate Request

Figure 6-13 Creating the certificate request

 We will see the final details in the GUI (Figure 6-14).

Properties

Name : MyKeyRing

Modulus : Mod2048

Certificate Status : Empty Cert

@ Request

DNS : myucs.domain.local
Locality
State : London
Country : UK
Organization Name : LearningUCS
Organization Unit Name :
Email : [email protected]
Subject : myucs

IPv4 IPv6

IP Address: 192.168.68.160
FI-AIP ; 192.168.68.161

Fl-B IP : 192.168.68.162

Figure 6-14 The final certificate settings

We would (on a real-life system) then copy the

certificate from the same page, and have this certificate
signed by our certificate authority.
Once we have the signed certificate, download it and
choose the Base64 encoded version, open it in a text editor,
and select and copy the entire text, then click “Certificate,”
select the trust point, and paste in the signed certificate.
The final step in the process is to select the key ring in the
Communication Services (Admin ➤ Communication
Management ➤ Communication Services). Under the
HTTPS settings, select the newly created, and newly
certified key ring (Figure 6-15).

Communication Services Events FSM

Telnet

Admin State : Q Enabled (•) Disabled

HTTPS

Admin State : (•) Enabled Q Disabled

Port : 443

Operational Port : 443

Key Ring : KeyRing default

Cipher Suite Mode : <not set> n Strength Q Low Strength Q Custom

KeyRing default
Cipher Suite : ORT40:!EXPORT56:!LOW:!R
KeyRing MyKeyRing
Allowed SSL Protocols : -
SSLv2 and SSLv3) O On|y TLSv1 2

Figure 6-15 Setting the key ring to be used for HTTPS GUI access

There are a few other steps we can take to harden our

UCS. Firstly, we should switch to HTTPS instead of
permitting HTTP, which we do under the HTTP settings
where we can redirect HTTP traffic to our UCS to the more
secure HTTPS (Figure 6-16).
HTTP

Figure 6-16 Redirecting HTTP to HTTPS

Next, we should disable Telnet, as anyone sniffing your

traffic will be able to capture your unencrypted password
(Figure 6-17).

Telnet

Admin State : O Enabled @ Disabled

Figure 6-17 Disabling Telnet

Finally, we can get rid of the less secure ciphers by

setting the cipher suite mode to “Custom” and setting the
cipher suite to use
“ALL:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!DSS:!MED
IUM” (which excludes ciphers such as low, triple-DES,
MD5, for example) and then set the “Allowed SSL
Protocols” to “Only TLSv1.2” (Figure 6-18).
HTTPS

Admin State : (•) Enabled Q Disabled

Port : 443

Operational Port : 443

Key Ring : KeyRing default

Cipher Suite Mode : O High Strength O Medium Strength Q Low Strength (•) Custom

Cipher Suite : ALL:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!DSS:!M

Allowed SSL Protocols : Q default (Allow all except SSLv2 and SSLv3) (S) Only TLSvl .2
Figure 6-18 Hardening HTTPS

These settings should satisfy most PCI QSAs and

security scans!
Summary
In this chapter, we secured access to our UCS system. In
the next chapter, we will look at UCS troubleshooting.
© The Author(s), under exclusive license to APress Media, LLC, part of
Springer Nature 2023
S. Fordham, Introducing Cisco Unified Computing System
https://doi.org/10.1007/978-1-4842-8986-0_7
7. UCS Troubleshooting
Stuart Fordham1
(1) Bedfordshire, UK

In this, our final chapter, we will look at ways we can

troubleshoot the UCS and enhance our monitoring of the
platform.
Call Home
Call home enables the UCS to send diagnostic data to
chosen recipients either periodically or in the event of an
issue, so in the event of a problem, Cisco can be notified
automatically and have relevant data sent to them.
We start by going to “All ➤ Communication Management
➤ Call Home” and enabling it (Figure 7-1).
All / Communication Management / Call Home

General Profiles Call Home Policies System Inventory Anonymous Reporting Events FSM

Admin

State : ©Off O°n

States

Figure 7-1 By default Call Home is disabled

Once we enable it, we need to enter our contact

information and contract details (Figure 7-2).
 All / Communication Management / Call Home

General Profiles Call Home Policies System Inventory Anonymous Reporting

Admin

State : Off ®On

Switch Priority : Debugging

Throttling : Off ®On

States

Contact Information

Contact : [email protected]

Phone : +441234123456

Email : [email protected]

Address : House, Street, City, Postcode

Ids

Email Addresses

From : [email protected]

Reply To : [email protected]

Figure 7-2 The Call Home details

After you have entered the email details, scroll down

and enter the SMTP server details, and click “Save
Changes” (Figure 7-3).
SMTP Server

Host (IP Address : 192.168.68.25

or Hostname)

Port 25

Save Changes

Figure 7-3 The SMTP server used for Call Home

Call Home uses profiles and these control who gets what
information. The inbuilt ones are shown in Figure 7-4.
All / Communication Management / Call Home

General | Profiles Call Home Policies System Inventory Anonymous Reporting Events FSM

+ — Tz Advanced Filter Export iji Print

Name Level Alert Groups

Profile CiscoTAC-1 Normal Cisco Tac

Profile full_txt Warning all,ciscoTac,diagnostic,environmental

Profile short_txt Warning all,ciscoTac,diagnostic,environmental

Figure 7-4 The Call Home profiles

We can create our own specific profiles by clicking the

“Add” button and entering the details (as shown in Figure
7-5). The predefined alert groups cover environmental
(power, fan, temperature alarms, and other similar issues),
diagnostic (such as a server’s POST details), and all the
other critical alerts (Cisco TAC).
Figure 7-5 A custom Call Home profile

We can also (if we want to) enable or disable call home

for specific causes (Figure 7-6).
Figure 7-6 A Call Home policy

Call home can also send the system inventory (either

periodically, or on-demand), as shown in Figure 7-7.
All / Communication Management / Call Home

General Profiles Call Home Policies | System Inventory Anonymous Reporting Events FSM

Actions Properties

Send System Inventory Now

Figure 7-7 Sending system inventory

Anonymous reporting is independent of how call home is

configured, in that call home may be disabled and
anonymous reporting can be enabled (and vice versa); this
sends a minimal error and health information about the
UCS (Figure 7-8).
All / Communication Management / Call Home

General Profiles Call Home Policies System Inventory Anonymous Reporting Events FSM

Actions Properties

View Anonymous Reporting Data Anonymous Reporting : Off • On

SMTP Server

Host (IP Address or Hostname) : 1 92.1 68.68.25

Port : 25

Figure 7-8 Enabling Anonymous Reporting

As we configure call home, we will see the actions we

have taken in the events tab, and this is where we can look
to see if call home has been triggered in the event of an
issue, or if we have periodic reporting turned on (Figure 7-
9).
All / Communication Management / Call Homo

General Piojos Cail Homo Policies System Inventory Anonymous Reporting | Evonts FSM

ACvanced F.w Excort * Print O Refresh No* “I Auto Refresh

Affected object Code ID Cause Created at - User Description

cell-borne E4 195331 108870 transition 2022-06-06T13:44 3 1Z admin [FSM:8EGIN]: can-home conftgutailon(FSM:sam:dme;CalihomeEpConfifiCa3»...

coll-home E4 195331 108871 transition 2022-06- 06ri3:44:31 Z admin [FSM STAGE END): (FSM-STAGE samdme:CaiihomeEpConf>gCaahome:begln)

Figure 7-9 The Call Home events

Finally, the call home settings will be replicated from

the primary FI to the secondary, and we can check the
progress of this in the FSM tab (Figure 7-10).
 General Profiles Call Home Policies System Inventory Anonymous Reporting Events FSM

FSM Status : In Progress

Description :
Current FSM Name : Config Callhome
Completed at :
Progress Status : 1 45%
Remote Invocation Result : Not Applicable
Remote Invocation Error Code : None
Remote Invocation Description :

0 Step Sequence

Order Name Description * Status Timestamp

1 Config Callhome Set Local call-home configuration on primary(FSM -STAGE:sam;dme:CallhomeEpCoofi .. Success 2022-06-06T13:53:29Z

2 Config Callhome Set Peer call-home configuration on secondary(FSM-STAGE:sam:dme:CallhomeEpCo In Progress 2O22-O6-O6T13:55:4OZ

Figure 7-10 The Call Home FSM

SNMP
SNMP (Simple Network Monitoring Protocol) also allows us
to be proactive in our management of the UCS. We can use
a pull method, whereby the UCS can be polled by a
network monitoring system (NMS) such as Zabbix, or a
push method utilizing SNMP traps to capture a specific
event and send them to an NMS.
To configure SNMP, go to “Admin ➤ Communication
Management ➤ Communication Services ➤ SNMP” and
click “Enabled” next to “Admin State.” We then need to set
the community or username, the protocol (TCP, UDP, or
both), and the contact and location.
The UCS also supports SNMPv3 for greater security.
Logging and Events
While the use of Call Home and SNMP is a somewhat
proactive approach, from time to time, we will need to go
digging a bit deeper to find what issues we have, and this is
certainly something we will need to do before we try and do
an upgrade as upgrades can and will fail if we have critical
issues with our UCS.
We can see how many issues we have from anywhere in
the GUI as we have a group of icons at the top of the
screen, as shown in Figure 7-11.

Figure 7-11 The GUI shows the active alerts

In the preceding, we have five critical issues, thirty-

three major, one minor, and 28 warnings. We can see what
these actually are by going to “Admin ➤ Faults, Events and
Audit Logs,” which shows the preceding categories as well
as “Info,” “Condition,” “Cleared,” “Soaking,” and
“Suppressed.” There is another state of “flapping.” The
difference between the states is shown in Table 7-1.
Table 7-1 The different fault levels

Level Description
Critical A service-affecting condition that requires immediate resolution
Major A service-affecting condition that requires urgent resolution
Minor A non-service-affecting condition that requires fixing to prevent a
more severe fault from occurring in the future
Warning A potential or impending service-impacting fault but which
currently has no significant effect on the system
Level Description
Info A basic notification or informational message
Condition An informational message about a condition
Cleared The fault has been cleared
Flapping A fault has been raised, cleared and raised again within a short
period of time.
Soaking A fault has been raised and cleared within a short time interval
(the flap interval), but it could be flapping, but if it doesn’t
reoccur then it moves into the cleared state
Suppressed SNMP traps and call home notifications have been stopped

By default we will see all the levels apart from Soaking

and Suppressed (Figure 7-12).
Faults, Events and Audit Log

| Faults
J Events Audit Logs Sysiog Core Fats TecftSupport Files Settings

Filers V Filter Tz Advanced Fmer + Export * Print Hide Fault Details

Severity Vo Severity Code O Affected object Cause Last Transition Description

Show All
O sye/rack-urut^board 2022-06 WT08:25:01Z
o Cnucal F2013 76389 identity-unestablishabte CPU ty pe on server 2 i„

O F2013 77278 sys/rack-und 3/ board Idenlity-unestaWishabie 2022-06-04T08:25:30Z CPU type on server 3 i_

V Major

Minor O F1932 77507 sys/rack-und-3 chassB-intrusron 2022-06-04T08:25:34Z Chassis enclosure for „

<► Warning
0 F2013 77964 syn/rack-mw- Hi board icferMity-unestabtishaOle 2022-06-04T08:25:S9Z CPU type on server 4 L_ |
O Mo
’4 Condition

0 Cleared
Total: 108 Selected: 1

Details
• r <0 10 I 25 | 50 I All
*

0 Soaking Summary Properties

0 Suppressed Seventy : O Info/None Affected object : sys/rack- unit-3

Category Ye Description ; Chassis enclosure for server 3 is open
Last Transition : 2022-06-04T08:2S:34Z
V All ID ! 77507 Type : equipment
>/ Generic Actions
Cause : chassis-intrusion Created at : 2022-06-04T08.25:34Z
Acknowledge Fault
Code : F1932 Number of Occurrences : 1
V Network
Operations Original severity ; Info

•J Sysdebug Previous seventy : Info Highest severity Info

Figure 7-12 The UCS faults

We can acknowledge the faults that have been cleared

and they will be removed from the list (Figure 7-13).
 Acknowledge Fault

Are you sure you want to acknowledge this fault? This fault will be removed if it is acknowledged and cleared.

Yes

Figure 7-13 Fault acknowledgment

Faults are only removed once they have been resolved,

so our UCSPE will remain in this state (unless we remove
most of the configurations that we have put in that will not
work in this sandboxed environment.
We can tackle faults as we see fit, such as starting with
the most important ones (Figure 7-14), and going through
them level by level.
Fault», Events and Audit Log

Faults | Events Audit Logs Syslog Core Files TechSuppcrt F-Jes Settings

Filters (x) Y Filter T, Advanced FJter f Export *■ Print Fault Detail O

Seven?/ Vo Seventy Code ID Affected oDject Cause Last Transition Description

Show All o F999674 107301 tiys/swiir.h-B/access-e. . fsm-fa:led 2O22-06-06TO8 43 5OZ (FSM FAILED]: internal .
V O Critical o F100O227 107306 -
sys/switch 9/locator- ted fsm-feiled 2022-06-06T08:44:20Z |FSM:FAILED1: sot Fl lo„.
v
Mmot
o F999619 107555 sysfidap-ext fsm-faied 2022-06-06T09:32:40Z |FSM:FAILED]: external
_
0 Warning
o F9W620 107571 sys/auth-realm f$m-1aiied 2022-06-06T09:38:40Z [FSMfAHED]: realm co...

o Info o F999710 109093 Ca«-tlOrtM! fsm-fa led 2022-06-06T14:20:20Z (FSMFAILED): cal-ho .

Figure 7-14 Filtering faults

We can also look at events, which show us the affected

object and the FSM description (Figure 7-15).
Faults, Events and Audit Log I Events

- Eran» Aj! ' ICO | Events |

T, AXuncad Filar Expos if- PrvX O Hvi'esh New Auto ttohosA

Evarita Aftoctod C0¡IKt Coda IO Camo Crnated nt User Dcscr goon

Aude Logs t»nnci server. £419'3052 S5975 WWrtiOn 2022-06- 04T IX IniurnW If SM STAGE REMOTE -ERROit)- Result Lrocrofied -tai Code ERR- BMC-*u-rcsnewl-errw Message CoUd not get f
S.stog tMxtcrswver- £4)95052 S59?a transiten 2022-<M4-04T17 miqmai IfSM STAGE RETRYit XMeiiyng a server n 4/4 voCWC-rSM-STAGE snm dmc:FatncCo»np«.»S»c<ip«cnt,»y Occi'r
Cera Filos •ntMic/wvcf— E419ÜD52 SS979 transition 2022-06-04TM internal IFSM SFAGE STALE-FAILl Kle’r/y.ng a server *14/4 va CMCtFSM- STAGE sarr CwFanncCcTouleSotEpdent.+ylx.
I*c*>s«wt r«w taDnc/seA«r_ £4)99052 sS9eo tiansiton 2022-OG-04TV micrrw |fSM STAGE REMOTE -ERRORJ ResuH ur.dern?ed tai Code t CoUd not get f
Settings 2022-O6-04T17 InttrmM
tunnc/scwrr E4)9b052 S99B1 transition |f SMSTAGE STALE- TAIL; KfcnrA/mg a server
* 4/4 via C1MCIFSM - STAGE sam OwcFabncCdvr nuieStorfpdcnt.V E«

Figure 7-15 The event log

 We also have the audit logs, allowing us to see who did
what and when (Figure 7-16).

iI
5 s

aI
X 5

MÍ o

S
i

HP i
i

Figure 7-16 The audit log

1

1
n
1
1

1
n u
li í* n n H H
« ¿ s
i8
AJ 8

a
KJ 8É u5

KJ 8í M g

KJé

M
KJ g
M8

P fc
KJ 8t U t
SYSLOG
Having faults, events, and audit logs in the GUI is fine, but
really all of this data should be sent to a more centralized
logging system, and we can do that by using a SYSLOG
server (Figure 7-17). We go to “Admin ➤ All ➤ Faults,
Events and Audit Log ➤ Syslog,” enabling the service and
entering the hostname or IP address of the SYSLOG server.

Remote Destinations

Server 1

Admin State : (•) Enabled Q Disabled

Level : critical (UCSM Critical)

Hostname (or IP Address) : 192.168.68.215

Facility : Local?

Server 2

Figure 7-17 Setting up remote syslog

Techsupport Files
No matter how proactive we are, there will be times that
we need to reach out to Cisco TAC and you will need to
send them technical support files. We generate these on
the UCS and download them to send over to Cisco.
Navigate to “Admin ➤ All ➤ Faults, Events and Audit Log ➤
TechSupport Files.” Click on “Add.” You will see that the
options are to “Create and Download a Tech Support File,”
or to just “Create a Tech Support File” (which you can
download at another time), as shown in Figure 7-18.

Create and Download a Tech Support File

Create a Tech Support File

© Add
Figure 7-18 Generating a techsupport file

Whichever option we choose, we get the same options

for what we want to generate. For the most part, UCSM
will be sufficient, but what tech support file we generate
does depend on the situation at hand. The options we have
are shown in Figure 7-19 and explained in Table 7-2.
 ?
Create and Download a Tech Support File
Options

Create and Download a Tech Support File

(•) ucsm O ucsm-mgmt Q chassis O fabric-extender Q rack-server Q server-memory

Technical support data for the entire UCSM instance will be created and downloaded to the default browser
download location.

J Exclude Commands
J Include Fabric Interconnect Trace Logs
Selecting " Exclude Commands" reduces the tech support collection time by excluding all the CLI commands
from the file. Do not select this option unless advised to by TAC.

Cancel

Figure 7-19 The techsupport file options

Table 7-2.

Option Description
ucsm Covers the entire UCS domain, but does not include chassis, fabric-
extender, rack-server, or server memory
ucsm- The UCS management services (but not the fabric interconnects)
mgmt
Chassis Either the CIMCs or I/O modules in a specific chassis
fabric- As the name suggests, this is for the FIs
extender
rack- The rack servers
server
Option Description
server- Includes B-series and C-series memory support data
memory

In this way, we can be more targeted and send the tech

support file for a particular rack server, for instance
(Figure 7-20).

7
Create and Download a Tech Support File
Options

Create and Download a Tech Support File

O ucsm O ucsm-mgmt O chassis O fabric-extender (•) rack-server Q server-memory

Rack Server ID : 2 Rack Server Adapter ID : all

Figure 7-20 A rack server techsupport file

The resulting files can then be uploaded to the Cisco

TAC case.
Summary
In this chapter, we looked at ways to help us troubleshoot
issues with the UCS.
Thanks for reading; I hope the book has been useful to
you.
Index
A
Admin LDAP Group Map
Anonymous reporting
Authentication, Authorization, and Accounting (AAA)
Authentication domain

B
Blade servers

C
Call home events
Call home FSM
Call home policy
Certificate request
Chassis
adding a PSU
fans
FEXs
5108s
hardware
inventory
IOMs
model options
UCSB-5108-AC2
Chassis Management Controller (CMC)
Chassis Management Switch (CMS)

D, E
Default UCSPE-generated layout
Direct Attach mode
Dynamic vNICs

F
Fabric Interconnects (FICs)
Fault acknowledgment
FCoE (Fibre Channel over Ethernet) IDs
FEXs
Firmware policies
Forwarding Equivalence Class (FEC)
FSM tab

G
Group recursion

H
HTTPS GUI access
HTTPS settings

I, J
In/Out Modules (IOMs)
IOM connectivity
IPv6 pool

K
KVM-IP-Pool
KVM management port policy
KVM pool
KVM-Port-Policy

L
LDAP group mappings
LDAP group maps
LDAP provider
LDAP provider group

M
MAC address assignment
MAC address block
MAC pools
Maintenance policies
Management IP addresses
Management IP address policy
Mezzanine cards

N
N20-C6508
Network monitoring system (NMS)

O
Operational policies
KVM management port policies
MAC pools
management IP addresses
scrub policies
UUID pool
WWNN pool

P, Q
Placement policies
Policies
dynamic vNIC connection
local disk policy
maintenance
operational
server boot
server pool
service profile templates
storage
UCS organizations, creation of
VLAN creation
vMedia policies
vNIC/vHBA placement
VSAN
Pool assignments
Private VLANs (PVLAN)
R
Rack servers
RAID settings
Read-only LDAP Group Map

S
Scrub policies
Sending system inventory
Server boot order
Server boot policies
Server PID qualification
Server pool policies
Service Profile templates
associating the service profile
“B200-template”
configuring eth0
configuring eth1
configuring fc0
configuring fc1
current task
expert option
firmware policies
FSM tab
general tab
“LocalDiskPol”, selecting
MAC address assignment
management IP
networking options
network interface placement
RAID settings
SAN connectivity page
scrub and KVM port policies
server boot order
server maintenance policy
service profile page
 service profile, selecting
template to a server
vMedia policy, setting
“vNIC eth0”
vNIC/vHBA Placement tab
zoning options
Simple Network Monitoring Protocol (SNMP)
Storage policy
SYSLOG server

T
TechSupport files
Telnet
Trust point

U
UCSB-5108-AC2
UCS components
blade servers
chassis
enclosures
FICs
FEX
hardware, UCSPE
rack servers
UCS enclosures
UCS faults
UCS-Mini
UCS networking
UCSPE hardware
devices, removal of
disconnecting the devices
hardware inventory
UCS Platform Emulator (UCSPE)
console
download options
 final VM settings
GUI using HTTPS
hardware
import, completion
importing into fusion
into VMWare
limitations
main page
real-world UCS setup
searching
selection, UCSPE file
setting up UCSPE
SSH and GUI access
starting UCSPE
virtual machine
UCS security
AAA
See Authentication, Authorization, and Accounting (AAA)
hardening HTTPS
hardening, web interface
UCS troubleshooting
call home
contact information and details
custom call home profile
fault levels
logging and events
SMTP server details
SNMP
SYSLOG server
Techsupport Files
Unified Computing System (UCS)
Uplink ports
FI redundant uplinks
flow control policy
80 Gbps port channel
 interface flow control
link profile
network tab in UCS Manager
Nexus interfaces
normal mode
port channel
reconfiguration
traffic
in UCS manager
UCS port channel setup
UDLD modes
UDLD policy
uplink interface
uplink interface settings
UUID (Unique Identifier) pool

V
Virtual IP (VIP)
Virtual network interface connection (vCON)
vMedia policies
VMWare

W, X, Y
Web interface
World Wide Node Names (WWNN) pool

Z
Zabbix