Job Queue - web is more than request and response
Download as PPT, PDF4 likes1,947 views
The document discusses using job queues for asynchronous background processing in web applications. It compares using cron jobs versus job queues, and recommends job queues because they allow processing to be asynchronous, reliable, and scalable. It then describes some popular job queue options like Gearman and Beanstalkd, noting advantages like speed, parallelism, scalability, and integrating different programming languages. The document emphasizes that job queues offload long-running tasks from blocking the web server and help applications handle more requests.
Job Queue - web is more than request and response
- 1. JOB QUEUE web is more than request and response Abhinav Lal
- 2. WHO AM I CTO at Naabo TurboDoc.in Cooking PHP for 4 years
- 3. Job Queue In computer science, message queues and mailboxes are software-engineering components used for interprocess communication, or for inter-thread communication within the same process. http://en.wikipedia.org/wiki/Message_queue Any long running work which does not require user feedback should be done in background.
- 4. Order processing without job queue
- 5. Order processing with job queue
- 6. Jobs to run in background Sending email Fetching RSS feed periodically Interacting with other application API Complex report generation Log analysis cleaning file system periodically or after certain action.
- 7. But I can use cron for that Asynchronous Low latency Reliable Scalable
- 10. Gearman = Manager Developed by Danga (People behind memcache) Open source C daemon Native PHP extension Persistence – drizzle, postgres, memcache, sqllite Distributed No single point of failure
- 11. Beanstalkd Open source C daemon Fast Persistence – bin log file Distributed PHP library – pheanstalk, beanstalk Simple protocol
- 12. Simple setup Client script Job Server Worker script
- 13. Beanstalkd
- 16. Beanstalkd Goodness Named Tubes – max 200 bytes Priority Delayed job Bury Kick
- 17. Advantages Speed up work Parallel and asynchronous work Scales easily Call functionality in other programming language Does not block your apache process.
- 18. Running worker script Use supervisord Gearman Manager Gear up Check for memory leaks in worker scripts Restart your worker scripts after certain number of runs.
- 19. Questions Abhinav Lal Blog: abhinavlal.wordpress.com Twitter: @abhinavlal Slideshare: slideshare.net/abhinavlal