Building Serverless Apps with Kafka (Dale Lane, IBM) Kafka Summit London 2019
0 likes•1,102 views
The document introduces serverless architecture, detailing its definition, differences from other deployment patterns, and use cases such as event processing and scalable APIs. It explains how serverless eliminates the need for server management and is suitable for dynamic, stateless workloads. Additionally, it discusses Apache OpenWhisk as a platform for implementing serverless applications with examples and demos.
![© 2019 IBM Corporation
Definition
applications that do not require server management
applications bundled as one or more functions
[developers] no longer need to spend time and resources on server
provisioning, maintenance, updates, scaling and capacity planning](https://image.slidesharecdn.com/dalelaneibmbuildingserverlessappswithkafka-190520215728/85/Building-Serverless-Apps-with-Kafka-Dale-Lane-IBM-Kafka-Summit-London-2019-12-320.jpg)
![© 2019 IBM Corporation
wsk trigger create myKafkaTrigger
-f /whisk.system/messaging/kafkaFeed
-p brokers '["9.174.23.208:3092"]'
-p topic MY.EVENTS](https://image.slidesharecdn.com/dalelaneibmbuildingserverlessappswithkafka-190520215728/85/Building-Serverless-Apps-with-Kafka-Dale-Lane-IBM-Kafka-Summit-London-2019-81-320.jpg)
Building Serverless Apps with Kafka (Dale Lane, IBM) Kafka Summit London 2019
- 1. © 2019 IBM Corporation Building serverless apps with Kafka Dale Lane An introduction to serverless with Apache OpenWhisk IBM Event StreamsApache Kafka @dalelane
- 2. © 2019 IBM Corporation Agenda § What is “serverless”? § Implementations § Apache OpenWhisk § Demos
- 3. © 2019 IBM Corporation What is “serverless”?
- 4. © 2019 IBM Corporation Serverless §An architectural pattern where your code is run: – in response to events – only on-demand on a per-request basis
- 5. © 2019 IBM Corporation Evolution of deployment patterns Monolith
- 6. © 2019 IBM Corporation Evolution of deployment patterns Microservice Microservice Microservice Monolith
- 7. © 2019 IBM Corporation Evolution of deployment patterns Function Function Function Function Function Function Function Function Microservice Microservice Microservice Monolith
- 8. © 2019 IBM Corporation Evolution of deployment patterns Bare metal Virtual machines Containers Orchestration Serverless
- 9. © 2019 IBM Corporation Function FaaS Platform PaaS Infrastructure IaaS Container CaaS Bare metal Evolution of deployment patterns
- 10. © 2019 IBM Corporation Definition github.com/cncf/wg-serverless
- 11. © 2019 IBM Corporation Definition github.com/cncf/wg-serverless
- 12. © 2019 IBM Corporation Definition applications that do not require server management applications bundled as one or more functions [developers] no longer need to spend time and resources on server provisioning, maintenance, updates, scaling and capacity planning
- 13. © 2019 IBM Corporation How is serverless different? Monolith Microservice Microservice Microservice Function Function Function Function Function Function Function Function
- 14. © 2019 IBM Corporation How is serverless different?
- 15. © 2019 IBM Corporation How is serverless different?
- 16. © 2019 IBM Corporation How is serverless different? £ $ € kr Fr ¥
- 17. © 2019 IBM Corporation Example use cases § Event processing § Data processing § Scheduled tasks § Scalable REST APIs
- 18. © 2019 IBM Corporation The Twelve Factor App
- 19. © 2019 IBM Corporation Serverless is good for: § Infrequent or sporadic demand § Focused, short duration processing § Highly dynamic workloads § Stateless workloads § Inconsistent demand § Scheduled infrequent tasks § Flexibility
- 20. © 2019 IBM Corporation Implementations
- 21. © 2019 IBM Corporation Implementations – platform
- 22. © 2019 IBM Corporation Implementations – platform
- 23. © 2019 IBM Corporation Implementations – platform
- 24. © 2019 IBM Corporation Implementations – platform
- 25. © 2019 IBM Corporation Implementations – platform
- 26. © 2019 IBM Corporation Implementations – hosted
- 27. © 2019 IBM Corporation Implementations – private cloud
- 28. © 2019 IBM Corporation Implementations – installable
- 29. © 2019 IBM Corporation Implementations s.cncf.io
- 30. © 2019 IBM Corporation Serverless framework serverless.com/framework
- 31. © 2019 IBM Corporation Apache OpenWhisk
- 32. © 2019 IBM Corporation OpenWhisk openwhisk.apache.org
- 33. © 2019 IBM Corporation OpenWhisk – who
- 34. © 2019 IBM Corporation OpenWhisk – where
- 35. © 2019 IBM Corporation OpenWhisk concepts
- 36. © 2019 IBM Corporation OpenWhisk concepts event
- 37. © 2019 IBM Corporation OpenWhisk concepts event action
- 38. © 2019 IBM Corporation OpenWhisk concepts event action function (eventobj) { // do something return result; }
- 39. © 2019 IBM Corporation OpenWhisk languages
- 40. © 2019 IBM Corporation OpenWhisk languages
- 41. © 2019 IBM Corporation OpenWhisk concepts event action
- 42. © 2019 IBM Corporation OpenWhisk concepts event actiontrigger
- 43. © 2019 IBM Corporation OpenWhisk concepts actiontrigger ruleevent
- 44. © 2019 IBM Corporation OpenWhisk concepts actionrule event actionrule actionrule actionrule trigger
- 45. © 2019 IBM Corporation OpenWhisk concepts rule event rule rule rule trigger trigger trigger trigger action event event event
- 46. © 2019 IBM Corporation OpenWhisk concepts actiontrigger ruleevent function main(params) { return { hello : 'world' }; } hello.js
- 47. © 2019 IBM Corporation OpenWhisk concepts actiontrigger ruleevent function main(params) { return { hello : 'world' }; } hello.js
- 48. © 2019 IBM Corporation OpenWhisk concepts actiontrigger ruleevent function main(params) { return { hello : 'world' }; } $ wsk action create helloworld hello.js ok: created action helloworld hello.js
- 49. © 2019 IBM Corporation OpenWhisk concepts actiontrigger ruleevent function main(params) { return { hello : 'world' }; } $ wsk action create helloworld hello.js ok: created action helloworld $ wsk action invoke helloworld ok: invoked /guest/helloworld with id 7df9c9dfbb hello.js
- 50. © 2019 IBM Corporation OpenWhisk actions blocking non-blocking
- 51. © 2019 IBM Corporation OpenWhisk actions blocking non-blocking periodic
- 52. © 2019 IBM Corporation OpenWhisk actions action action action
- 53. © 2019 IBM Corporation OpenWhisk concepts actiontrigger ruleevent
- 54. © 2019 IBM Corporation OpenWhisk implementation proxy
- 55. © 2019 IBM Corporation OpenWhisk implementation proxy controller
- 56. © 2019 IBM Corporation OpenWhisk implementation proxy controller store
- 57. © 2019 IBM Corporation OpenWhisk implementation proxy controller store service registry invoker
- 58. © 2019 IBM Corporation OpenWhisk implementation proxy controller store service registry invoker invoker invoker invoker invoker invoker
- 59. © 2019 IBM Corporation OpenWhisk implementation controller invoker invoker invoker invoker invoker invoker
- 60. © 2019 IBM Corporation OpenWhisk implementation controller invoker invoker invoker invoker invoker invoker
- 61. © 2019 IBM Corporation OpenWhisk implementation controller invoker invoker invoker invoker invoker invoker
- 62. © 2019 IBM Corporation OpenWhisk implementation controller invoker invoker invoker invoker invoker invoker
- 63. © 2019 IBM Corporation OpenWhisk implementation controller invoker invoker invoker invoker invoker invoker
- 64. © 2019 IBM Corporation OpenWhisk implementation controller invoker invoker invoker invoker invoker invoker
- 65. © 2019 IBM Corporation OpenWhisk implementation controller invoker invoker invoker invoker invoker invoker
- 66. © 2019 IBM Corporation OpenWhisk implementation controller invoker invoker invoker invoker invoker invoker
- 67. © 2019 IBM Corporation Container management start X init X X run X X X medium.com/openwhisk/squeezing-the-milliseconds-how-to-make-serverless-platforms-blazing-fast-aea0e9951bd0
- 68. © 2019 IBM Corporation Container management start X init X X run X X X medium.com/openwhisk/squeezing-the-milliseconds-how-to-make-serverless-platforms-blazing-fast-aea0e9951bd0
- 69. © 2019 IBM Corporation Container management start X init X X run X X X medium.com/openwhisk/squeezing-the-milliseconds-how-to-make-serverless-platforms-blazing-fast-aea0e9951bd0
- 70. © 2019 IBM Corporation Resource limits github.com/apache/incubator-openwhisk
- 71. © 2019 IBM Corporation Demos – installable
- 72. © 2019 IBM Corporation Deployment options
- 73. © 2019 IBM Corporation git clone https://github.com/apache/incubator-openwhisk-devtools.git cd incubator-openwhisk-devtools/docker-compose make quick-start
- 74. © 2019 IBM Corporation OpenWhisk event providers trigger ruleevent
- 75. © 2019 IBM Corporation cd incubator-openwhisk-devtools/docker-compose make create-provider-cloudant make create-provider-kafka
- 76. © 2019 IBM Corporation git clone https://github.com/apache/incubator-openwhisk-package-kafka.git cd incubator-openwhisk-package-kafka export OPENWHISK_AUTH=$OPENWHISK_HOME/ansible/files/auth.whisk.system ./installKafka.sh $OPENWHISK_AUTH $SYSTEM_IP_ADDR http://$SYSTEM_IP_ADDR:5984 local_ $SYSTEM_IP_ADDR
- 77. © 2019 IBM Corporation Demo
- 78. © 2019 IBM Corporation Demo event Message produced to Kafka topic MY.EVENTS
- 79. © 2019 IBM Corporation kafka-topics.sh --create --topic MY.EVENTS --replication-factor 1 --partitions 1 --zookeeper $SYSTEM_IP_ADDR:3181
- 80. © 2019 IBM Corporation Demo event Message produced to Kafka topic MY.EVENTS myKafkaTrigger trigger
- 81. © 2019 IBM Corporation wsk trigger create myKafkaTrigger -f /whisk.system/messaging/kafkaFeed -p brokers '["9.174.23.208:3092"]' -p topic MY.EVENTS
- 82. © 2019 IBM Corporation Demo event Message produced to Kafka topic MY.EVENTS myKafkaTrigger trigger
- 83. © 2019 IBM Corporation Demo event Message produced to Kafka topic MY.EVENTS myKafkaTrigger trigger sendToSlack action action
- 84. © 2019 IBM Corporation Demo event Message produced to Kafka topic MY.EVENTS myKafkaTrigger trigger sendToSlack action action
- 85. © 2019 IBM Corporation wsk action create sendToSlack slack-action.js
- 86. © 2019 IBM Corporation Demo event Message produced to Kafka topic MY.EVENTS myKafkaTrigger trigger sendToSlack action action
- 87. © 2019 IBM Corporation Demo ruleevent Message produced to Kafka topic MY.EVENTS myKafkaTrigger trigger slackOnKafkaMessages sendToSlack action action
- 88. © 2019 IBM Corporation wsk rule create slackOnKafkaMessages myKafkaTrigger sendToSlack
- 89. © 2019 IBM Corporation Demo ruleevent Message produced to Kafka topic MY.EVENTS myKafkaTrigger trigger slackOnKafkaMessages sendToSlack action action
- 90. © 2019 IBM Corporation
- 91. © 2019 IBM Corporation Reminder
- 92. © 2019 IBM Corporation Demos – using a hosted service
- 93. © 2019 IBM Corporation Demo Orders Voyages Ships Containers Alerts VoyagesOrder Order Management Microservice Fleet/Ships Simulator Microservice Voyages Management Microservice Kafka Streams Kafka
- 94. © 2019 IBM Corporation
- 95. © 2019 IBM Corporation Demo
- 96. © 2019 IBM Corporation Demo ruleevent Message produced to Kafka topic “alerts” kafkaAlert trigger processAlert action action
- 97. © 2019 IBM Corporation
- 98. © 2019 IBM Corporation Demo ruleevent Message produced to Kafka topic “alerts” kafkaAlert trigger processAlert action action
- 99. © 2019 IBM Corporation Orders Voyages Ships Containers Alerts VoyagesOrder Order Management Microservice Fleet/Ships Simulator Microservice Voyages Management Microservice Kafka Kafka Streams
- 100. © 2019 IBM Corporation Summary
- 101. © 2019 IBM Corporation Summary / roundup §An architectural pattern where your code is run: – in response to events – only on-demand on a per-request basis
- 102. © 2019 IBM Corporation Getting started twitter.com/openwhisk medium.com/openwhisk slack.openwhisk.org youtube.com/channel/UCbzgShnQk8F43NKsvEYA1SA #openwhisk developer.ibm.com/code/patterns/category/serverless serverlessconf.io
- 103. © 2019 IBM Corporation Thank you Dale Lane Software Engineer, IBM Event Streams IBM Event Streams - ibm.com/cloud/event-streams https://slack-invite-ibm-cloud-tech.mybluemix.net/