Scaling application with RabbitMQ
4 likes3,601 views
The document discusses RabbitMQ, a widely used messaging broker, emphasizing its features, use cases, and implementation details. It covers topics such as scaling, clustering, high availability, and integration with various programming languages and tools. The document also highlights practical examples, installation steps, and performance considerations, illustrating RabbitMQ's suitability for enterprise messaging needs.
![var factory = new ConnectionFactory() { HostName = "localhost" };
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
channel.QueueDeclare("hello", false, false, false, null);
string message = "Hello World!";
var body = Encoding.UTF8.GetBytes(message);
channel.BasicPublish("", "hello", null, body);
Console.WriteLine(" [x] Sent {0}", message);
}
}
Queue Declare](https://image.slidesharecdn.com/scalingapplicationwithrabbitmq-150210024959-conversion-gate02/85/Scaling-application-with-RabbitMQ-35-320.jpg)
![var factory = new ConnectionFactory() { HostName = "localhost" };
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
channel.QueueDeclare("hello", false, false, false, null);
var consumer = new QueueingBasicConsumer(channel);
channel.BasicConsume("hello", true, consumer);
Console.WriteLine(" [*] Waiting for messages." +
"To exit press CTRL+C");
while (true)
{
var ea = (BasicDeliverEventArgs)consumer.Queue.Dequeue();
var body = ea.Body;
var message = Encoding.UTF8.GetString(body);
Console.WriteLine(" [x] Received {0}", message);
}
}
}](https://image.slidesharecdn.com/scalingapplicationwithrabbitmq-150210024959-conversion-gate02/85/Scaling-application-with-RabbitMQ-36-320.jpg)
Scaling application with RabbitMQ
- 3. Nahidul Kibria Co-Leader, OWASP Bangladesh, Principal Software Engineer, Orbitax Bangladesh Ltd. @nahidupa
- 4. What is RabbitMQ? We See Code Clustering and High Availability Docker+RabbitMQ I will stop on some checkpoint and take question. Birds eye view.
- 6. What is RabbitMQ ?
- 8. Polyglot Java and JVM Ruby Python .NET PHP Perl C / C++ Node.js Go •Erlang •Haskell •Provisioning (Chef, Puppet, etc) and Operations •Mobile •Database Integration •Web Messaging •CLI •3rd party plugins •Ocaml •Common Lisp •COBOL
- 9. Scaling Scaling UP vs S c a leO UT
- 10. Why do we need messaging?
- 11. We are about to get a new project give me a prototype
- 12. will be simple
- 14. We need to post in twitter
- 15. First delivery
- 16. It take too much bandwidth…lablabla.. I actually need that yesterday
- 17. We are going to third- party integration PHP Python Java
- 18. UX guys, Customer feedback we don’t want to wait
- 19. We need to build instagram!
- 20. Poor you (:
- 21. You play boss killer some time and get back to work
- 22. All of this the simple solution is messaging
- 24. Who are using RabbitMQ? It is used in the Ocean Observatories Initiative—an architecture that collects 8 terabytes of data per day.
- 25. Who are using RabbitMQ? Instagram Indeed.com Telefonica Mercado Libre NHS Mozilla
- 26. Install RabbitMQ
- 29. After Install
- 30. Install Management portal for RabbitMQ
- 31. Demo time
- 34. Hello word
- 35. var factory = new ConnectionFactory() { HostName = "localhost" }; using (var connection = factory.CreateConnection()) { using (var channel = connection.CreateModel()) { channel.QueueDeclare("hello", false, false, false, null); string message = "Hello World!"; var body = Encoding.UTF8.GetBytes(message); channel.BasicPublish("", "hello", null, body); Console.WriteLine(" [x] Sent {0}", message); } } Queue Declare
- 36. var factory = new ConnectionFactory() { HostName = "localhost" }; using (var connection = factory.CreateConnection()) { using (var channel = connection.CreateModel()) { channel.QueueDeclare("hello", false, false, false, null); var consumer = new QueueingBasicConsumer(channel); channel.BasicConsume("hello", true, consumer); Console.WriteLine(" [*] Waiting for messages." + "To exit press CTRL+C"); while (true) { var ea = (BasicDeliverEventArgs)consumer.Queue.Dequeue(); var body = ea.Body; var message = Encoding.UTF8.GetString(body); Console.WriteLine(" [x] Received {0}", message); } } }
- 37. Exchange
- 42. Coding
- 43. Work Queues
- 44. Exchanges
- 46. Log
- 47. Topic exchange
- 49. easynetq https://www.rabbitmq.com/devtools.html
- 51. Easynetq save your time //many simplification var bus = RabbitHutch.CreateBus("host=ubuntu:5672,ubuntu:5673"); //also give freedom var advancedBus = RabbitHutch.CreateBus("host=localhost").Advanced
- 54. Large Message Messaging in general should be small AMPQ support very large messages RabbitMQ you can use large messages but you need to be careful MSMQ-Message Queuing is limited in the size of documents that can be processed. The maximum size document that Message Queuing supports is 4 MB if the document is in ASCII. The message queue can handle a maximum of 2 MB if the document is in Unicode.
- 56. Virtual Hosts
- 57. Error Handling
- 58. //no retry var ea = (BasicDeliverEventArgs) consumer.Queue.Dequeue(); channel.BasicReject(ea.DeliveryTag,false); //retry var ea = (BasicDeliverEventArgs) consumer.Queue.Dequeue(); channel.BasicReject(ea.DeliveryTag,true);
- 59. We cannot retry forever!
- 62. feelsleepy
- 63. Clustering and High Availability
- 64. Demo Environment
- 67. If erlang version mismatch Join cluster
- 69. Demo in management Details http://goo.gl/BqQHjp ???
- 70. Breaking up a cluster rabbit3$ rabbitmqctl stop_app Stopping node rabbit@rabbit3 ...done. rabbit3$ rabbitmqctl reset Resetting node rabbit@rabbit3 ...done. rabbit3$ rabbitmqctl start_app Starting node rabbit@rabbit3 ...done.
- 72. Shovel plugin Loose coupling they may have different users and virtual hosts; they may run on different versions of RabbitMQ and Erlang.
- 73. A process that replicates data to the remote server
- 76. rabbitmq-plugins enable rabbitmq_shovel rabbitmq-plugins enable rabbitmq_shovel_management Demo time
- 80. RabbitMQ Federation •Supports replication across different administrative domains •Supports mix of Erlang and RabbitMQ versions •Supports Network Partitions •Specificity - not everything has to be federated
- 83. Topologies
- 86. Topologies Ring
- 87. Federated Queues
- 88. rabbitmqctl set_parameter federation-upstream my-upstream '{"uri":"amqp://admin:[email protected]","expires":36000 00}‘ rabbitmqctl set_policy federate-me '^DigitalWorld.' '{"federation-upstream-set":"all"}' --priority 0 --apply-to exchanges
- 89. Enterprise Integration Patterns for RabbitMQ
- 90. Connecting with SSL Authentication and Authorization Security
- 94. Load Test jMeter Apache JMeter and the AMQP plugin to run load tests: https://github.com/jlavallee/JMeter-Rabbit-AMQP
- 97. SignalR http://goo.gl/xumPOG A client that is connected to one server will not receive messages sent from another server.
- 99. RabbitMQ is extremely efficient, widely deployed & tested message broker but can hit performance, if tuned for more durability and reliability. It’s more targeted towards serving enterprise messaging with advanced routing requirements. Redis is quick to start, lightweight & fast broker but does Not support reliable delivery Hence can be chosen for applications where in case system terminates, losing the information about tasks for a few minutes is Not critical. http://goo.gl/J5ak0j
- 101. Questions @nahidupa
Editor's Notes
- #5: Ask audience Any bus lover
- #8: Advanced Message Queuing Protocol Message Queue Telemetry Transport—IBM-- binary packet payload--low bandwidth, high latency networks such as dial up lines and satellite links Simple/Streaming Text Oriented Messaging Protocol--browser, mobile app, or machine in real-time Publish from a protocol consume from another protocol
- #10: Single point of failure
- #24: Do not start showing other pattern
- #28: sysv-rc-conf
- #29: nano /etc/apt/sources.list deb http://www.rabbitmq.com/debian/ testing main wget https://www.rabbitmq.com/rabbitmq-signing-key-public.asc sudo apt-key add rabbitmq-signing-key-public.asc apt-get update sudo apt-get install rabbitmq-server
- #30: update-rc.d -f rabbitmq-server remove update-rc.d -f rabbitmq-server defaults http://localhost:15672/ Sudo invoke-rc.d rabbitmq-server stop/start/etc apt-get install chkconfig chkconfig –list chkconfig rabbitmq-server on
- #31: C:\Program Files (x86)\RabbitMQ Server\rabbitmq_server-3.4.3\sbin rabbitmq-plugins enable rabbitmq_management "C:\Program Files (x86)\RabbitMQ Server\rabbitmq_server-3.4.3\sbin" "rabbitmq-plugins enable rabbitmq_management“
- #32: Show management console
- #39: Again go to simulator
- #45: routekey
- #53: mnesia
- #55: Message Queuing is limited in the size of documents that can be processed. The maximum size document that Message Queuing supports is 4 MB if the document is in ASCII. The message queue can handle a maximum of 2 MB if the document is in Unicode.
- #57: /debug /production demo
- #64: C:\Program Files (x86)\RabbitMQ Server\rabbitmq_server-3.4.3\sbin>rabbitmqctl jo in_cluster --ram rabbit@exdev Clustering node rabbit@JINOM with rabbit@exdev ... Error: {inconsistent_cluster,"OTP version mismatch: local node is 17.4, remote n ode R15B01"} Check erlang verssion erl -eval 'erlang:display(erlang:system_info(otp_release)), halt().' –noshell in_cluster --ram rabbit@exdev Clustering node rabbit@JINOM with rabbit@exdev ...
- #66: On Unix systems, the cookie will be typically located in /var/lib/rabbitmq/.erlang.cookie or$HOME/.erlang.cookie. On Windows, the locations are C:\Users\Current User\.erlang.cookie(%HOMEDRIVE% + %HOMEPATH%\.erlang.cookie) orC:\Documents and Settings\Current User\.erlang.cookie, and C:\Windows\.erlang.cookie for RabbitMQ Windows service. If Windows service is used, the cookie should be placed in both places. As an alternative, you can insert the option "-setcookie cookie" in the erl call in the rabbitmq-serverand rabbitmqctl scripts.
- #77: demo
- #80: https://www.rabbitmq.com/federation.html rabbitmq-plugins enable rabbitmq_federation rabbitmq-plugins enable rabbitmq_federation_management
- #89: demo
- #92: https://www.youtube.com/watch?v=CAak2ayFcV0 https://github.com/zenoss/ZenPacks.zenoss.RabbitMQ http://looselycoupledlabs.com/2014/08/monitoring-rabbitmq/ http://www.rabbitmq.com/how.html have a section
- #94: curl -L https://github.com/docker/fig/releases/download/1.0.1/fig-`uname -s`-`uname -m` > /usr/local/bin/fig; chmod +x /usr/local/bin/fig http://java.dzone.com/articles/docker-rabbitmq-cluster
- #96: https://github.com/rajaraodv/rabbitpubsub