SparxUp - Growth VS Scalability
Download as KEY, PDF0 likes495 views
Urbanesia is a lifestyle city directory with over 220,000 points of interest in Jakarta and over 160 million search results. It focuses on reviews and faces challenges with scalability due to its growth. The document discusses separating applications, databases, storage and caching to optimize performance. It also recommends virtualization, optimization of MySQL queries, memcaching, and using Nginx as a reverse proxy to improve scalability as the site continues to grow.
SparxUp - Growth VS Scalability
- 1. Urbanesia Growth VS Scalability Batista R. Harahap [email protected]
- 2. URBANESIA? • Lifestyle City Directory • Over than 220.000 POIs in Jakarta • Over than 160 millions search results • Over than 6 millions “what & where” search keywords • It’s all about Reviews (More Data to handle)
- 3. FACTS & STATS • The keywords with the highest number of search results is “toko baju di itc mangga dua” >>> OF COURSE lol • More than any other keywords, people search frequently for “Restoran” and “Hotel” • “Grand Indonesia” and “Senayan City” is the two most searched malls
- 5. Growth VS Scalability Kernel/Core Application Application/DB/Storage Separation Caching Urbanesia Optimization Virtualization
- 6. Growth VS Scalability Kernel/Core Application • KISS (Keep It SIMPLE & SLIM) • Extend REUSABILITY through modules and object oriented programming techniques • Centralized API approach to serve data • Use Database only for Data • Use or create an MVC (Models, Views & Controllers) where applicable • Best practices everywhere, better to have headaches at the beginning then throughout the lifetime
- 7. Growth VS Scalability Kernel/Core Application • Reusable UrSpot module in every part of Urbanesia • Load or Unload submodules dynamically if needed
- 8. Growth VS Scalability Kernel/Core Application • Helpers really helps :) • Speak plainly with CSS classes and id’s, makes debugging a lot easier
- 9. Growth VS Scalability Application/DB/Storage Separation • Partition main areas of the Infrastructure • Decide based on needs and predicted growth of each partition • Isolate problems if any to each partitions • Prepare partitions to be separated easily from a single server to multiple servers • Optimize optimize and optimize based on each partition’s performance
- 10. Meet Davinci & Sofvia
- 11. Meet Davinci & Sofvia
- 12. Meet Davinci & Sofvia Davinci
- 13. Meet Davinci & Sofvia Davinci
- 14. Meet Davinci & Sofvia Davinci Sofvia
- 15. Growth VS Scalability Caching • Design Asynchronous architecture to handle multiple instances easily • Don’t let your application do the same thing over and over, cache everywhere if applicable • Design access to apache/httpd to be the last resort, save cpu cycles • Memcache and Squid is used for caching on app servers • Nginx is used as a Reverse Proxy Load Balancer • Static Files from dynamic contents
- 16. Growth VS Scalability Caching • A redundant application Memcache #1 Memcache #2 feature turned into GOLD • Increase redundancy and App Server let memcache requests be served by multiple instances • Different parts of the Memcache #3 application uses different Memcache server
- 17. Growth VS Scalability Caching Nginx • Nginx handle request asynchronously by design Static Content Dynamic Content • Cache Cache Static and Dynamic Content cached by Nginx • Apache/httpd Minimize apache cpu cycles, use dynamically generated static files Application and flag renewals by application triggers Memcache Static Files MySQL
- 18. Growth VS Scalability Optimization - Application • Access database only as a last resort, use Memcache and dynamically generated static files instead • Keep Models, Views and Controller exactly why they’re named MVC • Minify static text contents (CSS, JavaScript & HTML) • Optimize MySQL queries • Properly index MySQL tables • Use MySQL Slow Query Log to pin point potential optimizations • Realtime serving is overrated - cron where applicable
- 19. Growth VS Scalability Optimization - Nginx Brought almost 300% increase in Urbanesia static files serving
- 20. Growth VS Scalability Optimization - GlusterFS • Multi storage server filesystem • Increase speed with more servers • Serves up to petabytes level • Striping is by default
- 21. Growth VS Scalability Virtualization • Virtualization is NOT expensive, hardware is • Citrix XenServer is an Entry Level Virtualization Platform • Virtualize utility servers • Use templates for multiple server types • Instant deployment of VMs • Save rack space!
- 22. THE RESULT
- 23. Growth VS Scalability RESULT
- 24. Growth VS Scalability RESULT
- 25. Growth VS Scalability Batista R. Harahap [email protected] Twitter: @tista www.bango29.com