SlideShare a Scribd company logo
1April 2013
Considerations for Building
a Private Cloud
Folsom Update
Ryan Richard, RHCA
OpenStack Architect - Private Cloud
ryan.richard@rackspace.com
@rackninja
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Why Folsom?
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
What is a Private Cloud?
Elastic or Traditional Virtualization (hint: Pick One!)
Multi-tenant (Multi-Application)
Size
Private API endpoints
Limited inbound connectivity
Customized for specific workloads
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Build with the End in Mind
# of instances on this physical host?
Flavor: m1.tiny
Ram: 512MiB
vCPU: 1
Disk: 0 (size of image)
Total instances:
?
48
vCPU Ram Disk
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Build with the End in Mind
# of instances on this physical host?
Flavor: m1.medium
Ram: 4096MiB
vCPU: 2
Disk: 10GiB + 40GiB
Total instances:
24
vCPU Ram Disk
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Build with the End in Mind
# of instances on this physical host?
Flavor: ram heavy
Ram: 64GiB
vCPU: 8
Disk: 100GiB
Total instances:
2
vCPU Ram Disk
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Capacity Thoughts
Don’t allow disk size of 0
Public Cloud: Limit flavors to image size
Private Cloud: Can add flavors for specific
application profiles
Network Utilization
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Capacity Thoughts
(resource / resource in smallest flavor) = maximum # of
instances per machine
Fixed Network - Double or quadruple number of expected
instances
I can always add more machines....
But you can’t change or modify the fixed
network once instances are running
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Build with the End in Mind
2 networks (3 w/ floating IPs)
1) Host Network
2) Fixed Network
3) Floating network
Easy to add physical nodes and/or
networks
Don’t try to change the fixed
network once in production
Easy to add additional floating
networks
nova-network
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Images and Storage
Images
Building images talk tomorrow @1:50 in room C123
Format: raw qcow2 vdi ...
Container: bare ami bundle ...
Cloud tech: cloud-init none
Partitioning: fixed dynamic
Drivers: virtio vhost_net xen ...
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Images and Storage
Glance
File backed (local)
Alternatives: Swift, CloudFiles, NFS (locally mounted)
Snapshots?
qcow2 format: “snapshot_image_format=qcow2“
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Images and Storage
Glance Performance
Network Throughput: 1Gbps = 125MBps (expect ~112MB/s)
RAID 5 - large sequential read/writes
Disk Bandwidth > IOPS
Improve cache - Reduce # of images
Image Size Not Cached Cached
1.4GB 20secs 1sec
16.4GB 2min 21secs 1sec
*times from “creating
image” to “qemu-img
create”
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Images and Storage
Storage
4 focus points: Glance, Compute (local), Cinder, Swift
Glance - Space, Sequential read/write
Compute - random IO
Cinder - Performance, Density
Swift - JBOD, density
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Architecture Examples and Thoughts
1 - 20 physical servers
Single controller
Single API
Single network (1Gbps or 2Gbps)
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Architecture Examples and Thoughts
20-100 servers
HA controllers and load balanced APIs
Swift/CloudFiles for Glance
Availability Zones
Consider “frontend” and “backend”
networks
Compute Node metrics - Dedicated
Machine(s)
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Performance Considerations and Bottlenecks
IO
Random IO
Reduce IO per instance
Block Storage!
Review Hypervisor best practices
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Lessons Learned
Lessons learned
Floating IPs MUST be associated with the “public_interface”
Each piece of OpenStack has its own architecture
Folsom is stable
Migration (live, block) works but scenarios exist where it
doesn’t
OpenStack is still changing often, keep up to date with
current state of the projects
Don’t do Heterogeneous Nodes
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Operational Updates
nova hypervisor* calls
Image types in Glance
policy.json
Coming in Grizzly:
Cells
Quantum
Better LDAP/AD support
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
But....
But this is a design summit also
Open to
discussions/thoughts/questions
20
RACKSPACE® HOSTING | 5000 WALZEM ROAD | SAN ANTONIO, TX 78218
US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COM
RACKSPACE® HOSTING | © RACKSPACE US, INC. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM
Rackspace is hiring
www.rackertalent.com

More Related Content

PDF
Glauber Costa on OSv as NoSQL platform
Don Marti
 
PDF
Basic and Advanced Analysis of Ceph Volume Backend Driver in Cinder - John Haan
Ceph Community
 
PDF
Linux Block Cache Practice on Ceph BlueStore - Junxin Zhang
Ceph Community
 
PDF
Accelerating Ceph with iWARP RDMA over Ethernet - Brien Porter, Haodong Tang
Ceph Community
 
PDF
Global deduplication for Ceph - Myoungwon Oh
Ceph Community
 
PDF
Achieving the ultimate performance with KVM
StorPool Storage
 
PDF
RADOS improvements and roadmap - Greg Farnum, Josh Durgin, Kefu Chai
Ceph Community
 
PDF
Ceph Object Storage at Spreadshirt
Jens Hadlich
 
Glauber Costa on OSv as NoSQL platform
Don Marti
 
Basic and Advanced Analysis of Ceph Volume Backend Driver in Cinder - John Haan
Ceph Community
 
Linux Block Cache Practice on Ceph BlueStore - Junxin Zhang
Ceph Community
 
Accelerating Ceph with iWARP RDMA over Ethernet - Brien Porter, Haodong Tang
Ceph Community
 
Global deduplication for Ceph - Myoungwon Oh
Ceph Community
 
Achieving the ultimate performance with KVM
StorPool Storage
 
RADOS improvements and roadmap - Greg Farnum, Josh Durgin, Kefu Chai
Ceph Community
 
Ceph Object Storage at Spreadshirt
Jens Hadlich
 

What's hot (20)

PPTX
Ceph Day Chicago - Ceph Deployment at Target: Best Practices and Lessons Learned
Ceph Community
 
PDF
Performance tuning in BlueStore & RocksDB - Li Xiaoyan
Ceph Community
 
PDF
Ceph Day San Jose - From Zero to Ceph in One Minute
Ceph Community
 
PPTX
MySQL Head-to-Head
Patrick McGarry
 
PDF
Ceph Day San Jose - HA NAS with CephFS
Ceph Community
 
PDF
Automatic Operation Bot for Ceph - You Ji
Ceph Community
 
PDF
BlueStore: a new, faster storage backend for Ceph
Sage Weil
 
PDF
Ceph Day San Jose - All-Flahs Ceph on NUMA-Balanced Server
Ceph Community
 
ODP
Experience In Building Scalable Web Sites Through Infrastructure's View
Phuwadon D
 
PDF
Ceph and cloud stack apr 2014
Ian Colle
 
PPTX
Ceph Day San Jose - Ceph at Salesforce
Ceph Community
 
PPTX
SSD для вашей базы данных, Петр Зайцев (Percona)
Ontico
 
PDF
OSv at Cassandra Summit
Don Marti
 
PDF
Ceph Day Taipei - Bring Ceph to Enterprise
Ceph Community
 
PDF
Doing QoS Before Ceph Cluster QoS is available - David Byte, Alex Lau
Ceph Community
 
PDF
Migratory Workloads Across Clouds with Nomad
Philip Watts
 
PDF
Hosting MongoDB & our experience with database as a service
Grzegorz Gwoźdź
 
PDF
London HUG 14/4 - Deploying and Discovering at Scale with Consul and Nomad
London HashiCorp User Group
 
PPTX
Ceph Day London 2014 - Best Practices for Ceph-powered Implementations of Sto...
Ceph Community
 
PDF
Ceph QoS: How to support QoS in distributed storage system - Taewoong Kim
Ceph Community
 
Ceph Day Chicago - Ceph Deployment at Target: Best Practices and Lessons Learned
Ceph Community
 
Performance tuning in BlueStore & RocksDB - Li Xiaoyan
Ceph Community
 
Ceph Day San Jose - From Zero to Ceph in One Minute
Ceph Community
 
MySQL Head-to-Head
Patrick McGarry
 
Ceph Day San Jose - HA NAS with CephFS
Ceph Community
 
Automatic Operation Bot for Ceph - You Ji
Ceph Community
 
BlueStore: a new, faster storage backend for Ceph
Sage Weil
 
Ceph Day San Jose - All-Flahs Ceph on NUMA-Balanced Server
Ceph Community
 
Experience In Building Scalable Web Sites Through Infrastructure's View
Phuwadon D
 
Ceph and cloud stack apr 2014
Ian Colle
 
Ceph Day San Jose - Ceph at Salesforce
Ceph Community
 
SSD для вашей базы данных, Петр Зайцев (Percona)
Ontico
 
OSv at Cassandra Summit
Don Marti
 
Ceph Day Taipei - Bring Ceph to Enterprise
Ceph Community
 
Doing QoS Before Ceph Cluster QoS is available - David Byte, Alex Lau
Ceph Community
 
Migratory Workloads Across Clouds with Nomad
Philip Watts
 
Hosting MongoDB & our experience with database as a service
Grzegorz Gwoźdź
 
London HUG 14/4 - Deploying and Discovering at Scale with Consul and Nomad
London HashiCorp User Group
 
Ceph Day London 2014 - Best Practices for Ceph-powered Implementations of Sto...
Ceph Community
 
Ceph QoS: How to support QoS in distributed storage system - Taewoong Kim
Ceph Community
 
Ad

Similar to Considerations for building your private cloud folsom update (20)

PDF
Consideration for Building a Private Cloud
OpenStack Foundation
 
PDF
Be a Cloud Native
InnoTech
 
PDF
Considerations for Building Your Private Cloud.pdf
OpenStack Foundation
 
PDF
Considerations for building your private cloud folsom update 041513
OpenStack Foundation
 
PDF
[Rakuten TechConf2014] [F-4] At Rakuten, The Rakuten OpenStack Platform and B...
Rakuten Group, Inc.
 
PPTX
Intro to OpenStack - WAJUG
Kevin Jackson
 
PPTX
Openstack Summit Tokyo 2015 - Building a private cloud to efficiently handle ...
Pierre GRANDIN
 
PDF
Lessons Learned Running The Largest OpenStack Clouds
Kenneth Hui
 
PDF
Chef and OpenStack Workshop from ChefConf 2013
Matt Ray
 
PPTX
Sanger, upcoming Openstack for Bio-informaticians
Peter Clapham
 
PPTX
Flexible compute
Peter Clapham
 
PDF
OpenStack Operations Guide 1st Edition Tom Fifield
tuekamrasyid
 
PPTX
Scaling Xen Within Rackspace Cloud Servers
Rackspace
 
PDF
Txlf2012
Joe Brockmeier
 
PDF
How to Build a Compute Cluster
Ramsay Key
 
PDF
Liz Quilty – Security, Scaling & High End Hosting for WordPress sites
WordCamp New Zealand
 
PDF
MSST-2013 Openstack in the Land of Guilder
Joshua McKenty
 
PPTX
What are clouds made from
John Garbutt
 
PDF
Openstack Operations Guide 1st Edition Tom Fifield Diane Fleming
molaxmeizu
 
PPTX
Cf summit2014 roadmap
James Bayer
 
Consideration for Building a Private Cloud
OpenStack Foundation
 
Be a Cloud Native
InnoTech
 
Considerations for Building Your Private Cloud.pdf
OpenStack Foundation
 
Considerations for building your private cloud folsom update 041513
OpenStack Foundation
 
[Rakuten TechConf2014] [F-4] At Rakuten, The Rakuten OpenStack Platform and B...
Rakuten Group, Inc.
 
Intro to OpenStack - WAJUG
Kevin Jackson
 
Openstack Summit Tokyo 2015 - Building a private cloud to efficiently handle ...
Pierre GRANDIN
 
Lessons Learned Running The Largest OpenStack Clouds
Kenneth Hui
 
Chef and OpenStack Workshop from ChefConf 2013
Matt Ray
 
Sanger, upcoming Openstack for Bio-informaticians
Peter Clapham
 
Flexible compute
Peter Clapham
 
OpenStack Operations Guide 1st Edition Tom Fifield
tuekamrasyid
 
Scaling Xen Within Rackspace Cloud Servers
Rackspace
 
Txlf2012
Joe Brockmeier
 
How to Build a Compute Cluster
Ramsay Key
 
Liz Quilty – Security, Scaling & High End Hosting for WordPress sites
WordCamp New Zealand
 
MSST-2013 Openstack in the Land of Guilder
Joshua McKenty
 
What are clouds made from
John Garbutt
 
Openstack Operations Guide 1st Edition Tom Fifield Diane Fleming
molaxmeizu
 
Cf summit2014 roadmap
James Bayer
 
Ad

Considerations for building your private cloud folsom update

  • 1. 1April 2013 Considerations for Building a Private Cloud Folsom Update Ryan Richard, RHCA OpenStack Architect - Private Cloud [email protected] @rackninja
  • 2. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Why Folsom?
  • 3. RACKSPACE® HOSTING | WWW.RACKSPACE.COM What is a Private Cloud? Elastic or Traditional Virtualization (hint: Pick One!) Multi-tenant (Multi-Application) Size Private API endpoints Limited inbound connectivity Customized for specific workloads
  • 4. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Build with the End in Mind # of instances on this physical host? Flavor: m1.tiny Ram: 512MiB vCPU: 1 Disk: 0 (size of image) Total instances: ? 48 vCPU Ram Disk
  • 5. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Build with the End in Mind # of instances on this physical host? Flavor: m1.medium Ram: 4096MiB vCPU: 2 Disk: 10GiB + 40GiB Total instances: 24 vCPU Ram Disk
  • 6. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Build with the End in Mind # of instances on this physical host? Flavor: ram heavy Ram: 64GiB vCPU: 8 Disk: 100GiB Total instances: 2 vCPU Ram Disk
  • 7. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Capacity Thoughts Don’t allow disk size of 0 Public Cloud: Limit flavors to image size Private Cloud: Can add flavors for specific application profiles Network Utilization
  • 8. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Capacity Thoughts (resource / resource in smallest flavor) = maximum # of instances per machine Fixed Network - Double or quadruple number of expected instances I can always add more machines.... But you can’t change or modify the fixed network once instances are running
  • 9. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Build with the End in Mind 2 networks (3 w/ floating IPs) 1) Host Network 2) Fixed Network 3) Floating network Easy to add physical nodes and/or networks Don’t try to change the fixed network once in production Easy to add additional floating networks nova-network
  • 10. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Images and Storage Images Building images talk tomorrow @1:50 in room C123 Format: raw qcow2 vdi ... Container: bare ami bundle ... Cloud tech: cloud-init none Partitioning: fixed dynamic Drivers: virtio vhost_net xen ...
  • 11. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Images and Storage Glance File backed (local) Alternatives: Swift, CloudFiles, NFS (locally mounted) Snapshots? qcow2 format: “snapshot_image_format=qcow2“
  • 12. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Images and Storage Glance Performance Network Throughput: 1Gbps = 125MBps (expect ~112MB/s) RAID 5 - large sequential read/writes Disk Bandwidth > IOPS Improve cache - Reduce # of images Image Size Not Cached Cached 1.4GB 20secs 1sec 16.4GB 2min 21secs 1sec *times from “creating image” to “qemu-img create”
  • 13. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Images and Storage Storage 4 focus points: Glance, Compute (local), Cinder, Swift Glance - Space, Sequential read/write Compute - random IO Cinder - Performance, Density Swift - JBOD, density
  • 14. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Architecture Examples and Thoughts 1 - 20 physical servers Single controller Single API Single network (1Gbps or 2Gbps)
  • 15. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Architecture Examples and Thoughts 20-100 servers HA controllers and load balanced APIs Swift/CloudFiles for Glance Availability Zones Consider “frontend” and “backend” networks Compute Node metrics - Dedicated Machine(s)
  • 16. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Performance Considerations and Bottlenecks IO Random IO Reduce IO per instance Block Storage! Review Hypervisor best practices
  • 17. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Lessons Learned Lessons learned Floating IPs MUST be associated with the “public_interface” Each piece of OpenStack has its own architecture Folsom is stable Migration (live, block) works but scenarios exist where it doesn’t OpenStack is still changing often, keep up to date with current state of the projects Don’t do Heterogeneous Nodes
  • 18. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Operational Updates nova hypervisor* calls Image types in Glance policy.json Coming in Grizzly: Cells Quantum Better LDAP/AD support
  • 19. RACKSPACE® HOSTING | WWW.RACKSPACE.COM But.... But this is a design summit also Open to discussions/thoughts/questions
  • 20. 20 RACKSPACE® HOSTING | 5000 WALZEM ROAD | SAN ANTONIO, TX 78218 US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COM RACKSPACE® HOSTING | © RACKSPACE US, INC. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM Rackspace is hiring www.rackertalent.com

Editor's Notes

  • #2: Goal: To give ideas on how to build private clouds powered by OpenStack software This talk is to serve as a guide not and end all be all. Ultimately you’ll have to find the right solution for your company. Ask to hold questions till the end
  • #3: Plan for these talks: We’ve been using and deploying Folsom Private Clouds for 6 months, next Summit, we’ll give a Grizzly Talk Grizzly is quite a different ...”beast”... I’ll focus on some of the Grizzly features at the end
  • #4: Are you looking for a cheaper VMware or a truly elastic private cloud? OpenStack Can do both but it certainly isn’t a drop in replacement for VMware Private Clouds generally not multi-tenant where tenants are different companies. However they will most likely be Multi-application which is in a sense multi-tenant since different applications require different resources. Size for Talk: Less than 100 nodes Inbound Connectivity: Floating IPs or all instances get public IPs
  • #5: Problem: don’t know how many virtual machines due to flavors CPU/RAM ratio way off, disk most likely under utilized
  • #6: Problem: don’t know how many virtual machines due to flavors CPU/RAM closer
  • #7: Problem: don’t know how many virtual machines due to flavors CPU/RAM ratio way off
  • #8: You’re the service provider, you must plan for capacity. Similar issues as traditional virt but now users can spin up on demand without going through an ops team public model: limit flavors , can pre plan capacity easily private model: closely watch capacity trends since you can’t plan as well Don’t forget about network utilization
  • #9: pre-quantum you cannot add/change you fixed network -- Some ability to add different networks but users MUST pass which network they want to use at instance boot time fixed range defines maximum number of IPs per “cluster”
  • #10: Fixed network is the one thing you need to get right. tell why!! Assigning a floating IP completely changes the way inbound and outbound connectivity happens. I’m giving an entire talk about nova-network tomorrow at 11:50am
  • #11: format - ISO can be used to create initial image with VNC with Raw - use fixed partitioning qcow2 - gain space saving benefits without losing too much performance over raw
  • #12: Explain what glance is It’s hard to realistically guess how many images or size of images you’ll be using. Expect Users to snapshot It’s simpler to standardize on base images and use automation tools to configure services within the instances.
  • #15: SIngle Controller - MySQL, rabbitmq-server, keystone, glance registry/api, nova-scheduler, nova-os-api-compute, nova-cert, nova-vncproxy, horizon Network utilization will be different. Understand your current usage and plan accordingly. If worried about nic saturation, break out your nova services (glance, nova services) to a separate network, Aggregate Bonds
  • #16: Convey why to consider swift 100 nodes, 2000 instances (20 nodes per), any of them could be snapshotting. Will be a bottleneck frontend backend networks frontend for external connectivity backend for instance to instance, instance to non-openstack server (dedicated DB,
  • #17: If you’re not using another system (Cinder, SAN, NetApp. etc) for additional storage, IO will need to a top consideration
  • #18: As I mentioned when starting, every major slide on here could be an entire talk and most of these slides are “lessons learned”
  • #19: nova calls for stats and instances on a hypervisor new image types, includes ISO policy.json allows you to define which role is allowed to make a specific API call Cells - Mainly used for large clouds (500+ nodes) Quantum - default network manager working LDAP/AD integration expected