Building FOSS clouds

Building FOSS Clouds

Sebastien Goasguen, Kris Buytaert
LinuxCon EU, Nov 6th
Barcelona, Spain

Slides at: slideshare.net/sebastiengoasguen

Speakers…
• Joe stayed home to deal with the release,

• Kris Buytaert stepped in:

Tutorial Outline
• 10:10 – 10:45 FOSS Clouds and CloudStack
intro

• 10:45 – 11:00 DevCloud demo

• 11:00 – 11:50 FOSS Cloud Automation,
DevOps configuration and monitoring tools

Define:tutorial
• A tutorial is a method of transferring
knowledge and may be used as a part of a
learning process. More interactive and specific
than a book or a lecture…<snip>

• Therefore:

DevCloud
• A CloudStack sandbox
– Pass the USB keys around and get the Vbox appliance
devcloud.ova file

• Import the appliance in VirtualBox
– Play with CloudStack
– http://wiki.cloudstack.org/display/COMM/DevCloud
– https://cwiki.apache.org/confluence/display/CLOUDSTA
CK/CloudStack+devcloud+environment+setup

Objectives
• Learn the basic architecture of a cloud
• Learn a few new tools
• Explore CloudStack via its sandbox
• Discover some concepts of automation
• Get home with some ideas to try out
• Mileage may vary

NIST Definition of Cloud

• "cloud computing is a model for enabling
ubiquitous, convenient, on-demand network
access to a shared pool of configurable
computing resources (e.g., networks, servers,
storage, applications and services) that can be
rapidly provisioned and released with
minimal management effort or service
provider interaction.“ – NIST

NIST Service/Deployment Models
• Service Models:
• IaaS (e.g EC2, rackspace)
• PaaS (e.g GAE, Azure)
• SaaS (e.g Gapps)
• Deployment model:
• Private cloud
• Public cloud
• Hybrid cloud (Private than can use Public)

NIST Key Characteristics of Cloud
• Elasticity
• Expands and Shrinks as needed
• E.g Elastic Load Balancing, Elastic Map Reduce
• Measured Service
• Utility Pricing. You pay for what you use
• E.g $0.08 for one hour of small instance
• On-demand
• You get it when you ask for it
• E.g One api call and you get your machine, ~no
queue

“Guidelines”
• Virtualization and automation are key
enablers of the cloud characteristics
• Build your infrastructure to enable on-
demand elastic provisioning of –servers,
storage, network –
• Setup a PaaS that suits you and your
applications
• Then move to continuous delivery of
applications …

IaaS challenges
• Not out of the box by itself.
• Need a farm of hypervisors
– Xen, KVM, Vmware…
• Need storage
– For image catalogue
– For volume/snapshot management
• Need flexible network that can be configured
on-demand
– VLANS, no VLANS, existing net infra, SDN…

Announcing Apache CloudStack 4.0

Coming at 12:00 EST
• Apache CloudStack 4.0
• Check out the testing procedure:
– https://cwiki.apache.org/CLOUDSTACK/cloudstack
-40-test-procedure.html
• Or watch my amazing screencast:
– http://vimeo.com/52150218

A bit of History
• Original company VMOPs (2008)
– Founded by Sheng Liang former lead dev on JVM
• Open source (GPLv3) as CloudStack
• Acquired by Citrix (July 2011)
• Relicensed under ASL v2 April 3, 2012
• Accepted as Apache Incubating Project April
16, 2012
• First Apache (ACS 4.0) release expected now

Apache Process
• 100% community driven
• New ideas, decisions only taken on mailing lists.
Votes taken by community
• Project led by Project Management Committee
(PMC):
– http://www.apache.org/dev/pmc.html
– http://incubator.apache.org/guides/committer.html
• Non committers get invited as committers:
– http://community.apache.org/newcommitter.html

Contributions
Examples:

Sungard: Announced that
6 developers were joining
the Apache project
Schuberg Philis: Big
contribution in
building/packaging and
Nicira support
Go Daddy: Early proto of
Maven building
Caringo: Support for own
object store
Basho: Devcloud
packaging

A Very Flexible IaaS Platform
Compute Hypervisor

Storage Block & Object

Primary Storage Secondary Storage

Network Network & Network Services

http://www.slideshare.net/cloudstack/cloudstack-architecture

Architecture / Language
• Java application
• Tomcat6, Axis2, Maven build + ant
– Ant going away in 4.1

• Collaboration Conference, Nov 30th -Dec 2nd
– http://collab12.cloudstack.org/

Build and Run in 4.1
• git clone https://git-wip-
us.apache.org/repos/asf/incubat
or-cloudstack.git
• mvn clean
• mvn install
• mvn –P developer –pl developer
–Ddeploydb
• mvn –pl :cloud-client-ui
jetty:run

Cloud Interactions OVM Cluster Primary
Storage

Ap i
Xen vcenter
vcenter

Monitoring Primary
CS API Clu I
ste AP vSphere Cluster
Storage
r M n ter
End
End
gm vCe
t
User UI
User UI
Primary
XS Cluster Storage
Admin
Admin Clustered
Clustered
CloudStack XAPI
UI
UI CloudStack
Domain
Domain
CS Admin & CloudStack
CloudStack
CloudStack
CloudStack
End-user API Primary
Admin
Admin
UI
UI
Management
Management JSON KVM Cluster Storage
Server
Server NetConf
Juniper SRX
Cloud user Nitro API
{API client (Fog/etc)} VNC
JSON
ec2 API JSON Netscaler

Cloud user Console
Console
Console
{ec2 API client } ProxyConsole
VM
Proxy VM
Proxy VM
Proxy VM NFS
MySQL
MySQL Server
Sec. Storage
Sec. Storage NFS NFS
{Proxied} SSH Sec. VM
Storage
Sec. Storage
VM
Ajax
Ajax HTTPS VM
VM
Console
Console Router VM
Router VM HTTP (Template Download)
Router VM
Router VM HTTP (Template Copy)
Router VM
Router VM
Cloud user HTTP (Swift)
http://www.slideshare.net/cloudstack/cloudstack-architecture

Terminology
Zone: Availability zone,
aka Regions. Could be
worldwide. Different data
centers
Pods: Racks or aisles in a
data center
Clusters: Group of
machines with a common
type of Hypervisor
Host: A Single server
Primary Storage: Shared
storage across a cluster
Secondary Storage:
Shared storage in a single
Zone

Storage
• Primary Storage:
– Anything that can be mounted on the node of a cluster.
– Cluster LVM…iSCSI…
– Holds disk images of running VMs
– Support for CEPH with KVM hypervisors
• Secondary Storage:
– Available across the zone
– Holds snapshots and templates (image repo)
– Can use Openstack swift or any object store (Gluster FS…)
– New support for Caringo

• Can use NFS for both to start
• Storage Abstraction refactoring underway

Networking
• Extremely flexible to:
– Provide isolation with VLANs
– Provide isolation at L3 with shared L2 (scalability)
– Support hardware devices that exposes API
– Deployed on existing networking infrastructure
– Support new networking paradigm (SDN)
• Support for Nicira Virtual P
• Extensive use of Open VSwitch

A customizable GUI
AJAX + API

API
• Not really REST
• A set of methods available over http(s)
• Unauthenticated on integration port
• Authenticated on 8080 using Access and
Secret Key
• Python/Ruby clients available
• Internal Marvin client
• Cloudmonkey CLI
• Other clouds client support the API

Enabling EC2 and S3
• Via the GUI

• Via API call on integration API port 8096
http://localhost:8096/client/api?
command=updateConfiguration&name=enable.s3.api&value=true
http://localhost:8096/client/api?
command=updateConfiguration&name=enable.ec2.api&value=true

DevCloud
• Several use cases:
– Try CloudStack in an isolated sandbox. Runs within
the appliance
– Develop CloudStack on own machine, build locally
and deploy new version in DevCloud (Build and
test)
– Develop and Run locally, use DevCloud as Xen
hosts

DevCloud: self-contained
CloudStack runs in the appliance

Testing “4.0” code in DevCloud
• Deploy new CloudStack code in self-contained
DevCloud:
• mvn –P deps
• ant rdeploy
• ant rdeploydb
– Wipes database of mgt server, you will need to
reconfigure the “data center”
• ant rdebug

DevCloud: as Host
Run CloudStack on local machine
Use DevCloud to setup hosts

DevCloud to test 4.1 branch
• mvn -P developer clean
• mvn -P developer install
• mvn -P developer -pl developer –Ddeploydb
• mvn -P developer -pl tools/devcloud –
Ddeploydb
• mvn -pl :cloud-client-ui jetty:run

Configure infrastructure:
• mvn -P developer,deploysvr -pl
tools/devcloud -Ddeploysvr

Testing Framework –for the PyUG 
[environment]

• Marvin is a Python dns=10.147.28.6
mshost=10.147.39.69
based framework to run mysql.host=10.147.39.69

tests against a
[cloudstack]
private.gateway=10.147.40.1

CloudStack install private.pod.startip=10.147.41.121
private.pod.endip=10.147.41.160

• Could be used as a private.netmask=255.255.254.0
public.gateway=10.147.40.1
simulator of a public.vlan.startip=10.147.41.162

datacenter
public.vlan.endip=10.147.41.200
public.netmask=255.255.254.0

• Used to configure an
hypervisor=XenServer
host=10.147.40.10

infrastructure on a mgt host.password=password
#storage pools
server primary.pool=nfs://10.147.28.7:/expo
rt/home/automation/sadhu/primary
secondary.pool=nfs://10.147.28.6:/ex
port/home/automation/sadhu/secondary

Demo time
• http://vimeo.com/52150218

Info
• Apache incubator project
• http://www.cloudstack.org
• http://incubator.apache.org/cloudstack/
• #cloudstack on irc.freenode.net
• @cloudstack on Twitter
• http://www.slideshare.net/cloudstack
• http://cloudstack.org/discuss/mailing-lists.html

Welcoming contributions and feedback, Join the fun !

Building FOSS clouds

More Related Content

What's hot (20)

Viewers also liked (11)

Similar to Building FOSS clouds (20)

More from Sebastien Goasguen (20)

Building FOSS clouds

Editor's Notes