PUBLISHED: Facebook Cloud Computing

Cloud Computing

Facebook Cloud Computing
SaaS, PaaS, and IaaS:
Standards & Practices
It’s not easy supporting 500+ million friends.

WHAT YOU SHOULD KNOW: WHAT YOU WILL LEARN: DEFINITIONS

• Cloud computing concepts • Updates to the LAMP stack What is LAMP?
• Application development • Advantages of MemCacheD LAMP is an acronym for a software stack that
• LAMP architecture • Apache hadoop cluster consists of the following: Linux, Apache,
• Unix • Apache hive MySQL, and PHP.
• Database concepts (RDBMS) • RDBMS redistribution

INTRODUCTION CACHE
We are going to examine Facebook’s cloud computing envi- MemCacheD sort of sneaked into the LAMP stack. Here is
ronment. It’s not easy supporting 500+ million friends so how where MemCached fits into the stack:
exactly do they do it? First we need to take a look at the stack
that is now embraced by most of the world’s largest websites Operating System
(including Facebook). The term LAMP was first used in 1998 Webserver
in a German Computing Magazine in the hopes of promoting [CACHE: MemCached]
competition against Microsoft Windows NT. Database
Programming Language
LAMP
LAMP maps to the following functions: LANGUAGE
These days the P in LAMP (which use to exclusively mean
• Linux (Operating System) PHP) can mean anything-- Pearl, Python, and even Ruby.
• Apache (Web Server) Even Apache is not immune from radical reinterpretation.
• MySQL (Database Server) Some are swapping it out for EngineX or Lighttpd (pro-
• PHP (Programming Language) nounced as “Lightie”).

When all elements of this stack are working together, you can FACT
build truly scalable websites. The new thinking is to use open source components to build
scalable and successful websites.
INSTALL
LAMP is pervasive and easy to install. In some environments SCRIPTING LANGUAGES
like Ubuntu or Debian you can get started with LAP in as little The thing to remember when using scripting languages is that
as a single command line such as: they perform poorly in production environments.

apt-get install FACEBOOK & PHP
apache2 php5-mysql Facebook uses PHP extensively. The entire front end of Fa-
libapache2-mod-php5 cebook is written in PHP. Let’s take a closer look at what
mysql-server Facebook has done around PHP from a performance and
open source perspective to really help it scale. After all PHP
LUSTER is a scripting language and that means you get some instant
For the most part LAMP is considered ‘old’ since it has not engineering and productivity benefits in terms of being able
been updated to reflect the latest thinking. LAMP has been to create something, refresh it, and see it in your browser;
static for over 10 years and todays demanding environments but again, be prepared for some unacceptable performance
and even more demanding younger developers want more from hits when you are running it in production.
the stack.

1 Data Center 2/2011

Facebook Cloud Computing SaaS

HIPHOP FOR PHP They don’t use joins, complex queries, or pull multiple tables
Facebook’s solution to this problem was to go custom and cre- together using views or anything like that. One would believe
ate HipHop for PHP. that the database teams don’t care about the relational aspect
of the database. This is not true. They care a great deal, but
FACEBOOK CUSTOM SOLUTION they use the relational database capabilities in a nontraditional
Facebook created HipHop for PHP. Development took about way.
three plus years and was open sourced in 2010. HipHop for
PHP takes: FACT
At Facebook the fundamental ideas of relational databases
PHP SourceCode -> Transforms it into C++ -> have not gone away. They are just implemented in nonconven-
Compiles it using G++ -> Produces an executable binary. tional ways.

FACT ARCHITECTURE
Facebook has achieved a 50% reduction across the web tier When you look at the Facebook database architecture, it con-
using HipHop for PHP. sists of the following layers:

1) Web Servers
TIP 2) MemCacheD Servers (distributed secondary indexes)
You should evaluate HipHop for PHP for use in your enter- 3) Database Servers (MySQL - primary data store)
prise.
The folks at Facebook are running the innovation engine at JOINS
100% and have implemented two key changes around the LAMP When it comes to using joins, they use the webserver to com-
stack. The areas of key innovation are taking place around: bine the data. The join action takes place at the webserver
layer. This is where HipHop for PHP becomes so important.
• Databases Facebook web server code is CPU intensive because the code
• Large Data Analysis runs and does all these important things with data. This would
traditionally be handled by the relational database.
Data analysis is at the heart of understanding what a friend
“user” is doing on the Facebook website. People are trying to HISTORY
find areas of efficiency wherever they can even if that means at These are database issues that people talked about 30 or 40
the database level. Take for example the NO-SQL movement. years ago, except they are now being discussed and imple-
These days you have so many databases to choose from and mented at different layers in the stack. If you’re using MySQL,
the recommendation is that you choose the one that best suits or NoSQL, you’re not escaping the fact that you need to com-
your needs. bine all that data together-- you’ll still need a way to go and look
it up quickly.
ARCHITECTING THE WOLRDS LARGETS WEBSITE So if we look at the NoSQL technology stack, there are a
Facebook stores all of the user data in MySQL (except news- number of No-SQL families of databases:
feed) and Facebook uses 1000’s of nodes in MySQL clusters.
For the people whose responsibility it is to manage the data- • Document stores
base portion of the website, they don’t care that MySQL is a • Column family stores
relational database. • Graph databases
• Key vale pair db’s

Then you ask yourself, what problem am I trying to solve?
What family of No-SQL databases am I going to use?
Look closer. There are a number of differences inside. Take
one database category:

Cassandra & HBase
They have a lot of tradeoffs:

• From a consistency perspective
• From a replication perspective

So at the end of the day you need to return to your original
question:

• What is the problem I am trying to solve?
• What is the best database for me to use?

Facebook stores most of its data inside MySQL. Facebook
stores about 150 terabytes (1024 TB) of data inside cassandra
which is used for inbox search on the site. About 36 petabytes

Data Center 2/2011 2

Cloud Computing

of uncompressed data is stored within a apache hadoop clus- SILVER CLUSTER
ter overall. They then replicate the data to a second cluster which they call
the silver cluster. The silver cluster is where people can run ad
HADOOP & BIG DATA hoc queries.

Facebook runs a hadoop cluster with 2,200+ servers with about FACT
23,000 cpu cores inside of it. Facebook is seeing that the amount Facebook leadership, product management, and sales (a group
of data they need to store is growing rapidly. of 300 to 400 people) run hadoop and apache hive jobs every
single month in an attempt to make better business decisions.
FACT Facebook created this level of data access to help people of the
Facebook storage requirements grew by 70x between 2009- organization make better product decisions because the data
2010. is readily available.
By the time you read this article, Facebook will be storing over
50 petabytes of uncompressed information which is more than APACHE HIVE
all the works of mankind combined. All of this is a direct result of Apache hive is one of the other technologies that Facebook
increased user activity on Facebook. Currently over 500M peo- uses to provide a SQL interface. This technology sits on top of
ple use the site every month and over half use it every day. hadoop and provides a way for people to do data analysis. The
great thing is that all of these components are open source-
DATA ANALYSIS Things which you can use - Today!
Facebook has learned a lot about how important it is to con-
duct effective data analysis is the running of a large successful SUMMARY
website. The LAMP stack is evolving. You will now see the introduction
of a cache layer. The number of choices you have at the data-
KEY base layer is also increasing and the separation of relational da-
You need to look at things from a product perspective. tabase functions has also occurred. Facebook, like most com-
Take into consideration a common problem of corporate com- panies has decided to implement technologies from a number
munications. What is the most effective email that you could send of different providers to avoid lock-in to any one solution. Data
to users who have not logged on to the site for over a week? In analysis will continue to grow in importance (something global
other words, what email type would be the most compelling? enterprises have known for a very long time even though they
may have failed to act or may not have had enough time to act
• A- A email that is a combination of text, status updates and on that knowledge).
other data. Great tools are now available-- consider evaluating apache
• B- A email that is simple text saying “hello, there are 92 hadoop, apache hive, and scribe. Use these tools to really un-
new photos. derstand what’s happening at your site. Follow Facebook’s lead
and evaluate developments in the open source space-- Face-
The data shows that the ‘B’ email performs 3x better. book is committed to this and has even created an open source
website which can be found at facebook.com/open source.
ANALYSIS
Analysis is the key. Data helps the people at Facebook make better
product decisions. You can see clear examples of this in the emails RESOURCES
that they send raking ‘news feed’ and the addition of the ‘like’ but-
ton. Analysis of large data helps product managers understand • Facebook Open Source http://facebook.com/opensource
how the different features affect how people use the site. • The Delta Cloud Project http://www.deltacloud.org
• GoGrid http://www.gogrid.com
SCRIBE • Amazon Elastic Compute Cloud (Amazon EC2) http://aws.amazon.
com/ec2
Facebook uses an open source technology that they also cre-
• Microsoft Cloud Services http://www.Microsoft.com/Cloud
ated called scribe that takes the data from 10’s of thousands of
• NIST - National Institute of Standards and Technology http://www.
webservers and funnels it into the hadoop warehouse. nist.gov/index.html
• Book - The Challenge of the computer utility by Douglas Parkhill
PLATINUM CLUSTER • Book - The Mythical Man-Month: Essays on Software Engineering by
The initial problem they encountered was that too many web- Fred Brooks
sites were trying to funnel data into one place, so hadoop tries • InterviewTomorrow.Net - Helping America get to work. Free access
to break it out into a series of funnels and collect the data over to the 2011 executive recruiter database
time. The data is then pushed into the platinum hadoop clus-
ter about every 5 to 15 min. They also go and pull in data from
MySQL clusters on a daily schedule. ABOUT THE AUTHOR
The apache hadoop cluster is at the center of the Facebook Richard C. Batka - Business & Technology Executive. Author.
architecture and is vital to the business. It’s the cluster that if Mr. Batka is based in New York and provides advisory services
it went down, it would directly affect the business. Facebook to a select group of clients. Mr. Batka has worked for global le-
will need to focus redundancy efforts in this area. Currently it is aders Microsoft, PricewaterhouseCoopers, Symantec, Verizon,
highly maintained and monitored so everyone is taking a lot of Thomson Reuters and JPMorgan Chase. A graduate of New
time and care considering every query before they run it against York University w/ honors, he can be reached at rbusa1@gmail.com or follo-
the cluster. wed on Twitter at http://twitter.com/RichardBatka.

3 Data Center 2/2011

PUBLISHED: Facebook Cloud Computing

More Related Content

Viewers also liked (13)

Similar to PUBLISHED: Facebook Cloud Computing (20)

PUBLISHED: Facebook Cloud Computing