Scaling with Symfony - PHP UK
36 likes12,971 views
The document is a presentation about scaling applications with Symfony. It discusses concepts related to scalability like load balancing and sharding. It provides advice on profiling code to diagnose bottlenecks and optimizing aspects like caching, databases, and front-end performance. Specific technologies discussed include APC, Zend Opcache, Memcached, Redis, MySQL, and NoSQL databases. Real-world examples of large applications built with Symfony like a social game with millions of daily users are also presented.
Scaling with Symfony - PHP UK
- 1. SCALING WITH SYMFONY + = PHP UK CONFERENCE 2014
- 2. HELLO PHPUK! • Ricard Clau, born and grown up in Barcelona, Spain • Software engineer at Hailo London • Symfony2 lover and PHP believer • Open-source contributor, sometimes I give talks • Twitter @ricardclau / Gmail [email protected]
- 3. WHY THIS TALK? • Lots of PHP haters these days. I just don´t get it. • Some misconceptions about full-stack PHP frameworks • PHP is absolutely fine for most of your applications • But at some point you may need to replace some bits • DragonCity - Social game - 7.5M DAU - Symfony
- 4. AGENDA • Performance and scalability • Profiling and optimization • Storage: SQL vs NoSQL • Symfony2 is fast enough! Seriously! • Some war stories! • Almost everything applies to all PHP projects
- 5. Millions of Users Worldwide 24x7x365 CONCEPTS Scalability, Availability, Load Balancing, Sharding
- 6. SCALABILITY Which model would you choose?
- 7. AVAILABLE 24X7X365 We need to recover fast from any failure
- 8. LOAD BALANCING It allows you to make changes without stopping your app
- 9. SHARDING Luckily, you will get to a point when you need to do it... And it can hurt if you have never considered it!
- 10. DON´T IF NOT NEEDED • Avoid premature optimizations • Microoptimizations are completely useless • Focus on impacting things • Incremental improvements • If the business is not working, it will be useless
- 11. BUT ACTIVELY PREVENT RISKS • Monitor servers: Ram, CPUs, Disk, network, idle processes... • Try to engage stakeholders • When something surpasses half capacity, start creating a B plan • Death of success is terrible
- 12. HANDS ON! Where do we start from?
- 13. LOVE YOUR FRONTEND Reduce # of requests and their size
- 14. SOME QUICK ADVICES • Fastest request -> No actual request • Icons in base64 inside your CSS • Minify CSS and JS, image size (jpegoptim, pngcrush, ...) • Use HTTP headers to cache at several layers • Network latency matters a lot • User experience takes benefit from all of these
- 15. SCALING PHP Facebook, Wikipedia, Yahoo, Wordpress, Etsy, Zynga, YouPorn, Hailo, Softonic, Emagister, Privalia, SocialPoint, Tuenti, ...
- 16. IS SCALING ACTUALLY EASY? • PHP bootstraps on every Request, so scaling is as “easy” as adding machines behind a Load Balancer • But it makes it much slower than many other languages • Some things are shared among requests (sessions, storage, ...) but frameworks like Symfony make it easier • PHP has been proof-tested in several BIG projects • Most of the times, problems are caused by the DB
- 17. LOOK BEYOND LAMP • Light-weight HTTP servers / CDNs for assets • Introduce reverse proxy cache like Varnish to cache pages (or ESI blocks) • Cache all the things (APC, Memcached, Redis, ...). 10 seconds cache -> Only 6 hits per minute! • RDBMS can deal with A LOT of data. But some problems are much better solved with NoSQL databases • Sometimes other languages are more suitable for our problem
- 18. PHP PERFORMANCE Opcode caches, composer, HHVM, quick wins Upgrading from PHP5.3 will boost ~20-40%
- 19. APC: OUR OLD BESTIE • If you have ever worried about performance... you know it! • apc.stat -> Off (Apache reload to see changes) • apc.serializer -> igbinary (compact_strings -> Off) • apc.shm_size -> Check if 32Mb is enough • apc.write_lock -> Avoids cold cache issues • Check http://www.php.net/manual/en/apc.configuration.php
- 21. ZEND OPCACHE • Included in PHP5.5 standard distribution (valid with 5.2) will avoid weird random issues like PHP5.4+APC • Better performance than APC in all tests • README https://github.com/zendtech/ZendOptimizerPlus • APC Userland Cache + Upload Hooks now live in APCU (https://github.com/krakjoe/apcu) • Lots of companies are already using it!
- 22. OPCACHE STATUS 1 full-stack file, Rasmus style :) (https://github.com/rlerdorf/opcache-status)
- 23. COMPOSER AUTOLOAD • You all already use Composer, right? • Default PSR-0 implementation is slow because we access the FileSystem a lot • composer.phar install --optimize-autoloader (-o) generates ClassMap • Same performance apc.stat = Off than ApcUniversalClassLoader and less keys stored in APC
- 24. APC VS ZEND OPCACHE http://www.ricardclau.com/2013/03/apc-vs-zend-optimizer-benchmarks-withsymfony2/
- 25. APC VS ZEND OPCACHE http://www.ricardclau.com/2013/03/apc-vs-zend-optimizer-benchmarks-withsymfony2/
- 26. HHVM: THE FUTURE ENGINE? • Impressive progress in HHVM these past months • Most frameworks and libraries ~100% compatibility • Fewer memory, interesting speed improvements • We might finally have a language specification for PHP! • Specific HHVM features? -> Dangerous! • It will still not be comparable to compiled languages
- 27. FINE TUNING SYMFONY2 • Disable unused bundles • SwiftMailer is quite slow, delegate to queue processes • If you can, use SubRequests with ESI and Varnish • Also check Twig C extension • You can implement a Cache Warmer for special needs • If using Apache, try app/console router:dump-apache
- 29. PROFILING TOOLS • How often do you profile your code? • xDebug (Derick Rethans) and XHProf (Facebook) • Very convenient to install them in live servers and activate to grab real data • Many things can also be discovered locally • Some tragedies only happen with live traffic
- 30. XHPROF (DEV VS PROD) https://github.com/jonaswouters/XhprofBundle
- 31. HELLO WORLD PHP and its frameworks... Benchmarks comparing completely different things...
- 32. ¿FRAMEWORKS == SLOW? • Some people think that writing their own framework from scratch is a good idea • Also, that big frameworks use a lot of memory. (ZF1 and Symfony2 work with memory_limit 32M) • If this is your problem, PHP is not the solution • Don´t trust Hello World Performance Tests! • Nobody is better than a big community
- 33. TECHEMPOWER BENCHMARK • Symfony2 appears at the bottom • Comparison with light-weight frameworks is unfair. Some incoherences (frameworks PHP >> PHP) • PHP will never be faster than a compiled language • In real apps, caching and DB optimizations equal things • Symfony2 roadmap improving performance
- 34. STORAGE Use the right tool for the job. Silver bullets do not exist
- 35. RELATIONAL DBS • Mature technologies, good performance • Transactions: Atomicity, Consistency, Isolation, Durability • Foreign Keys, Joins, complex reporting queries • Millions of records without issues • We can use a NOSQLs key-value approach using BLOB fields with serialized objects
- 36. NOSQL SOLUTIONS • Solutions to some concrete problems • CAP theorem (Consistency, Availability, Partition tolerance). You can only get 2 of them... theoretically • Inmaturity in some of them. Complex to scale. • ¿Big Data? 100M of records is NOT BIG DATA. • Redis, Cassandra, Riak and Solr work really well
- 37. QUEUE SYSTEMS Read the docs, they are all completely different!
- 38. REAL-TIME DATA? • Try to delegate as much as possible to batch processes • Sending mails, external API requests, image resizing, non-critical stats, ... • 99% of stats don´t need to be real-time • 60 seconds delay is mostly real-time! • Batch processes can be coded in different languages!
- 39. PHP NEEDS FRIENDS Sometimes it is just not the best tool for our needs
- 40. PHP BAD SCENARIOS • 24x7 running CLI daemon apps • Heavy math calculations, massive data processing, programming contests • High concurrency apps with non-cachable requests • Threading, Forking, concurrent programming... • Writing DSLs
- 41. LOOK BEYOND PHP • JavaScript will be the king in the client side • In the server, Erlang and Go are growing adopters. And time will say about Node.JS • Compiled languages like Go, C, C++ or Java will always be used in our stack. • JVM languages like Scala and Clojure are sexy now! • Learning other languages makes you a better engineer!
- 42. SYMFONY2 It is fast enough for most of what you want to do
- 43. WHY SYMFONY2? • Symfony2 is NOT the fastest “Hello world” framework... but it is fast enough for most APIs and web applications • Big-community frameworks allow you to test any technology in less than 5 minutes • A custom framework has usually higher costs and risks • LTS releases roadmap, stability, big projects are using it • Mature community, # of bundles
- 44. SOCIAL VIDEOGAME https://apps.facebook.com/dragoncity/ (~7.5M DAU Facebook, iOS & Android Game)
- 45. DRAGONCITY: NUMBERS • ~7.5 million daily users • ~300 millions of registered users • Hundreds of millions of records for analytics generated daily • MySQL (32x2 + Analytics), Redis (~30), Cassandra (3) • HTTP requests are your progress -> No cacheable • Yes, most of the code is Symfony2! :)
- 46. POWERED BY SYMFONY2 @adanlobato showing happiness and pride!
- 47. 6 DIGITS IN ANALYTICS Yeah, still Symfony2! o/
- 48. WRAP-UP • PHP and Symfony2 work for most projects • Clever caching strategy is helpful • Profiling can make performance improve • The storage is almost always the bottleneck • Proper architecture design is crucial • PHP is not the best solution to some problems
- 49. QUESTIONS? • Twitter: @ricardclau • E-mail: [email protected] • Github: https://github.com/ricardclau • Blog about PHP and Symfony2: http://www.ricardclau.com • Please, rate this talk at https://joind.in/talk/view/10697