apidays LIVE Jakarta - REST the events: REST APIs for Event-Driven Architecture by Mark Teehan, Confluent APAC
Download as PPTX, PDF•0 likes•178 views
This document discusses using REST APIs with event-driven architectures and Kafka. It describes three REST servers for Kafka: the Confluent REST Proxy, the Confluent Broker REST API, and the Confluent Cloud REST API. The REST Proxy provides a RESTful interface for producing, consuming, and administering a Kafka cluster. The Broker REST API allows administration of brokers, topics, and consumer groups directly on the brokers. The Confluent Cloud REST API manages connectors, users, and environments for the Confluent Cloud hosted service. The document also discusses when and why REST may be used with event-driven systems using Kafka.
apidays LIVE Jakarta - REST the events: REST APIs for Event-Driven Architecture by Mark Teehan, Confluent APAC
- 1. REST the Events - REST APIs for Event- Driven Architecture Mark Teehan - Principal Engineer - Confluent Singapore API Days - Jakarta
- 2. REST the Events - REST APIs for Event- Driven Architecture
- 3. Win OVO Vouchers @ 2 Million Rupiah Complete a 2 minute survey 1/ Scan this barcode / click the link or 2/https://www.surveymonkey.com/r/CONFLUENT_APIDAYS or 3/ visit our virtual booth
- 4. What is Kafka A modern distributed platform for data streams
- 6. Java Framework etc App() App() Event streaming etc Kafka Connect HTTP Sink REST Proxy AppHttp() AppHttp() CDC HTTP This session will focus on options REST for Kafka clients request / response 1 2 3 4
- 7. Request-Response - http Client sends request and waits for response Image: The most useless Machine (imgur) Low Latency Synchronous (usually) Point to point Pre-defined API
- 8. Event Streaming - kafka Messages received in a queue: drop and process later Image: reddit - oddly satisfying Continuous Processing Synchronous (often) Event-driven General Purpose Events
- 9. Request-Response - http Challenges ● Difficult to enforce standards across services ● Does not scale if servers are synchronous ● Risky inter-service dependencies ● Services required to maintain state ● Complex management and version compatibility—slows development ● Requires load balancing
- 10. Event Streaming - kafka Challenges ● Complex Infrastructure ● Event thinking is hard ● Adopting the Kafka Way
- 11. So when do we want to Use HTTP with Kafka?
- 12. So when do we want to Use HTTP with Kafka? ● Management Plane ● Data Plane ● Other Reasons
- 13. ● Management Plane ● Data Plane ● Other Reasons Cluster configuration: Management of Kafka topics, consumer groups, ACLs, etc. CI/CD and DevOps integration: Build delivery pipelines and to automate administration HTTP APIs - Yes Python/Others - no
- 14. ● Management Plane ● Data Plane ● Other Reasons Legacy apps: use of offbeat middleware Usually good ETL options, but poor streaming options HTTP/REST is sometimes the only protocol in common Mobile apps: Natural fit for request response Use of websockets and Server Sent Events API Gateway: Most API management tools do not have native event streaming support Common for integration with partner systems and service monetization HTTP/REST is sometimes the only protocol in common Other Cobol,Erlang, Kotlin, Node.js, PHP, Ruby, Rust; 0% ~ some% support Languages:
- 15. ● Management Plane ● Data Plane ● Other Reasons Familiarity: Quicker and easier than learning a new API Technology Lock In Avoid all Client dependencies Security: Securing HTTP ports is easier (F5 proxies owned by Infosec) Securing TCP is harder Domain Use both: Driven HTTP/REST for synchronous client/server Design: Kafka protocol for decoupling A service mesh to combine both is a common architecture.
- 16. Three REST servers : Confluent REST Proxy Confluent Broker REST Confluent Cloud REST API
- 18. Confluent REST Proxy a RESTful interface to an Apache Kafka® cluster making it easy to ● produce and consume messages ● view the metadata of the cluster ● perform administrative actions
- 19. Management Plane ● Brokers ● Topics ● Consumer Groups ● ACLS Data Plane ● Produce ● Consume Low throughput Limited API calls Continuous consumption Clustering for throughput
- 20. Confluent Platform REST API
- 21. Confluent Platform REST API Brokers: Describe, List, Configure Topics: create, delete, describe, list, configure Consumer Groups: delete, describe, list ACLs: create, delete, describe, list Partition Reassignments: list Runs on the broker - not a separate service
- 22. Confluent Cloud REST API
- 23. Confluent Cloud REST API Launched (early-access) on 17-Feb-2021 Connectors | Users | service accounts | environments (coming soon: topics | ACLs | Consumer Lag)
- 25. What about REST Gateways? Mulesoft Apigee Kong Anypoint Tibco M ...
- 26. Mark Teehan - Principal Engineer - Confluent Singapore [email protected] / LinkedIn Attend Naveen Nandan’s workshop to receive a Certificate of Attendance from Confluent
- 27. REST the Events - REST APIs for Event- Driven Architecture Mark Teehan - Principal Engineer - Confluent Singapore [email protected] / LinkedIn