Hopping in clouds: a tale of migration from one cloud provider to another

Hopping
a tale of migration from one cloud provider to another
in
Clouds

Michele Orselli
CTO@Ideato
_orso_
micheleorselli / ideatosrl
mo@ideato.it

Let’s start from the beginning…

What is the national sport in
Italy?

“Italians lose wars as if they were football
matches, and football matches as if they
were wars”
Winston Churchill

Peaks on gen - jun - aug up to 70 M pg/mth
Peaks during big matches

PaaS Platform as a Service
(almost) Zero conﬁguration
Put the code “on the cloud” and you’re done

Hard limits on resource (e.g 50 db con)
Deploy via ftp (sf cache mess)
Blackbox: No realtime log, no access
PHP 5.3

Web: the main web (sf1)
Mobile: mobile version (sf components)
Vxl: community site (sf2 v2.3)

Talk: api for comments, votes, ratings (sf2 v2.3)
Adv: api for ads serving (sf2 v2.3)
Media: api for images mgnt (sf2 v2.3)

The problems began with talk…

Quick wins
Tuning the http response headers
Caching more endpoints
Optimize queries

1 $date = new DateTime();
2 $date->modify("+$lifetime seconds");
3
4 $response->setExpires($date);
5 $response->setMaxAge($lifetime);
6 $response->setSharedMaxAge($lifetime);

the ﬁrst candidate for migration was… the talk app

PHP from 5.3 to 5.6
Mysql from 5.0 to 5.6
Apache to nginx (+ php fpm)

Web servers ip are dynamic
Can connect only through bastion
Share session between servers

Share user sessions between servers

1 'hosts' => function () {
2 $c = Ec2Client::factory([
3 'profile' => 'calciomercato',
4 'region' => 'eu-central-1',
5 ]);
6
7 $ips = new GetInstancesIps($c);
8
9 return $ips->execute();
10 }

1 public function execute()
2 {
3 $instances = $this->ec2Client
4 ->describeInstances(
5 [
6 'DryRun' => false,
7 'Filters' => [
8 [
9 'Name' => 'instance.group-name',
10 'Values' => ['Web Public Auto-assign SG'],
11 ],
12 ],
13 ]);
14
15 return $instancesDescription->getPath(
16 'Reservations/*/Instances/*/NetworkInterfaces/*/
PrivateIpAddresses/*/PrivateIpAddress'
18 );
19 }

1 Host cmbastion
2 HostName xx.xx.xx.xx
3 User ec2-user
4 Port 9760
5 StrictHostKeyChecking no
6 UserKnownHostsFile /dev/null
7 IdentityFile ~/.ssh/cm_bastion.pem
8 LogLevel quiet

10 Host 10.0.14.*
11 User centos
14 IdentityFile ~/.ssh/cm_production.pem
15 ProxyCommand ssh -W %h:%p cmbastion
16 LogLevel quiet
17
18 Host 10.0.24.*
19 User centos
22 IdentityFile ~/.ssh/cm_production.pem
23 ProxyCommand ssh -W %h:%p cmbastion
24 LogLevel quiet

Share users sessions between servers

1 fastcgi_cache_key
"$scheme$request_method$host$request_uri";
2 fastcgi_cache_lock on;
3 fastcgi_cache_revalidate on;
4 fastcgi_cache_valid 3m;

1 if ($request_method ~ ^(POST|PUT|DELETE)$ ) {
2 set $no_cache 1;
3 }
4
5 if ($request_uri ~* "/api/queue") {
6 set $no_cache 1;
7 }
8
9 location ~ ^/(app|dev).php(/|$) {
[..]
17
18 # Enable fastcgi_cache
19 add_header X-Cache $upstream_cache_status;
20 fastcgi_cache CALCIOMERCATO_TALK;
21 fastcgi_cache_bypass $no_cache;
22 fastcgi_no_cache $no_cache;
23 }

8
9 location ~ ^/(app|dev).php(/|$) {
10 fastcgi_split_path_info ^(.+.php)(/.*)$;
11 include fastcgi_params;
12 fastcgi_param SCRIPT_FILENAME $realpath_root
$fastcgi_script_name;
13 fastcgi_param DOCUMENT_ROOT $realpath_root;
14 fastcgi_param HTTPS off;
15 fastcgi_index app.php;
16 fastcgi_intercept_errors on;
17
18 # Enable fastcgi_cache
[..]
23 }

Load test using old logs
Create AMI Images
Deploy latest version of the code
Switch dns

Now we have the platform running on
two clouds: RCS and AWS

Only stateless apis
Small database small trafﬁc
Infrastructure was already set
Easy peasy

Created 1 Cloudfront distribution
dynamic content (adv.calciomercato.com)

No database, it consumes data from
other services
High impact, 40% of the total trafﬁc

How to deal with static assets?

s3://com-calciomercato-cdn-mobile/

dynamic content (m.calciomercato.com)
static content (cdnmobile.calciomercato.com)

Sync asset to s3 via s3cmd
s3cmd  
-m text/javascript  
--no-preserve sync  
/var/www/mobile/content/js  
s3://com-calciomercato-cdn-mobile/

Deploy on a sample machine
Performance test based on log
Deploy
Rebuilding AMI
Switch DNS

Allows uses to create their own personal blog
First app that can be considered “complete”
Exposes api for user related stuff
Works as SSO

1 services:
2 memcache:
3 class: Memcache
4 calls:
5 - [ addServer, [%memc_host%, %memc_port% ]]
6 session.handler.memcache:
7 class:
8 SymfonyComponentHttpFoundationSession
StorageHandlerMemcacheSessionHandler
10 arguments: [
11 @memcache,
12 { prefix: %session_memcache_prefix%,
13 expiretime: %session_memcache_expire% }
14 ]

1 framework:
2 session:
3 handler_id: %session_handler_id%

Deal with User Generated Content

Created 2 Cloudfront distributions
dynamic content (vxl.calciomercato.com)
static content (cdnvxl.calciomercato.com)

Gaufrette
Filesystem abstraction layer
http://knplabs.github.io/Gaufrette/

1 knp_gaufrette:
2 adapters:
3 photo_storage:
4 aws_s3:
5 service_id: cdn.amazon_s3
6 bucket_name: %amazon_s3_bucket_name%
7 options:
8 directory: data
9 filesystems:
10 photo_storage:
11 adapter: photo_storage
12 alias: photo_storage_filesystem
13

8 cdn.amazon_s3:
9 class: AwsS3S3Client
10 factory_class: AwsS3S3Client
11 factory_method: 'factory'
12 arguments:
13 -
14 key: %cdn.amazon_s3.aws_key%
15 secret: %cdn.amazon_s3.aws_secret_key%
16 region: eu-central-1

7 class PhotoUploader
8 {
[..]
27 public function upload(File $file, $dir)
28 {
29 $fullPath = $dir.'/'.$file->getFilename();
30
31 if (!in_array($file->getMimeType(), self::$allowedTypes)) {
32 throw new InvalidArgumentException($file->getMimeType());
35 }
36
40 return $this->filesystem->write(
41 $fullPath,
42 file_get_contents($file->getPathname())
43 );
44 }

7 class PhotoUploader
8 {
[..]
27 public function upload(File $file, $dir)
28 {
29 $fullPath = $dir.'/'.$file->getFilename();
30
31 if (!in_array($file->getMimeType(), self::$allowedTypes)) {
32 throw new InvalidArgumentException($file->getMimeType());
35 }
36
40 return $this->filesystem->write(
41 $fullPath,
43 );
44 }
return $this->filesystem->write(
41 $fullPath,
43 );

Deploy on a sample machine
Performance test based on log
Deploy
Rebuilding AMI
Copy User Assets on S3
Switch DNS

Oldest and biggest codebase
Proxy for mobile calls
High trafﬁc 60%

PHP 5.6 not supported by symfony 1

Plan A: try to upgrade sf1 to support php 5.6
Plan B: deploy web on different machines

https://github.com/LExpress/symfony1

1 protected function camelize($text)
2 {
3 return preg_replace(array('#/(.?)#e', '/(^|_|-)+(.)/
e'), array("'::'.
4 strtoupper('1')",
"strtoupper('2')"), $text);
5 }
6
7 public static function camelize($text)
8 {
9 return strtr(ucwords(strtr($text, array('/' =>
10 ':: ', '_' => ' ', '-' => ' '))),
array(' ' => ''));
11 }

static content (cdnweb.calciomercato.com)

https://blog.cloudﬂare.com/zone-apex-naked-domain-root-domain-cname-supp/
calciomercato.com => cmelb-463612445.eu-central-1.elb.amazonaws.com

Only stateful api
Handles image thumbailing
Pretty big archive (70GB)

1 public function generateThumbAndUploadToCdn(File $file, $width,
$height)
2 {
3 $downloadedFile = $this->downloadFromFileManager($file);
4
5 $cdnKey = $this->generateThumbCdnKey($file, $width, $height);
6 $resizedFile = $this->resizeFilesystemImage($downloadedFile,
$width, $height);
8
9 $optimizedFile = $this->optimizeImage($resizedFile);
10
11 $this->uploadFileToCdn($optimizedFile, $cdnKey)
12
13 $this->updateFileInfoTumbs($file, $width, $height, $cdnKey);
14
15 $this->deleteTemporaryFile($downloadedFile);
16 $this->deleteTemporaryFile($optimizedFile);
17
18 return true;
19 }

Transfer from Rackspace CDN to S3

#!/bin/bash 
login="USERNAME_FTP" 
pass="FTP_PASSWORD" 
host="HOST_FTP_RACKSPACE" 
 
remote_dir='/web/content/data' 
local_dir=“/var/www/vhosts/media.calciomercato.pro/data" 
base_name="$(basename "$0")"
lftp -u $login,$pass $host << EOF 
set ftp:ssl-allow no 
set mirror:use-pget-n 5
mirror -c -P5 --log="/var/log/$base_name.log"
"$remote_dir" "$local_dir"
quit

1 public function slugifyFilename($text)
2 {
3 $text = preg_replace('~[^pLd]+~u', '.',
$text);
4 $text = trim($text, '-');
5
6 if (function_exists('iconv')) {
7 $text = iconv('utf-8', 'us - ascii//
TRANSLIT', $text);
8 }
9
10 $text = preg_replace('~[^-w.]+~', '', $text);
11
12 return $text;
13 }

Full migration took 1 year
from april 2015 to march 2016

This talk is not about blaming RCS…

…simply it wasn’t suitable anymore
for our needs :-)

macro services FTW!
HTTP cache helped us a lot!
measure measure measure

Rationalizing api: less call, less $$$
Reorganize frontend stuff
Get rid of sf1
Upgrading to php7

Michele Orselli
CTO@Ideato
_orso_
micheleorselli / ideatosrl
mo@ideato.it
Thank you!
https://joind.in/talk/1cf3c
Thanks to @dennais, @paolo, @alemazz, @ricfrank

Hopping in clouds: a tale of migration from one cloud provider to another

More Related Content

What's hot

Viewers also liked

Similar to Hopping in clouds: a tale of migration from one cloud provider to another

More from Michele Orselli

Recently uploaded

Hopping in clouds: a tale of migration from one cloud provider to another