Next Generation Client APIs in Envoy Mobile
3 likes887 views
The document discusses the evolution of client APIs using Envoy Mobile, highlighting the importance of data shape and transport in API management. It emphasizes the need for consistency, performance, extensibility, resilience, security, and observability in mobile client APIs, while sharing insights on how Envoy can help standardize infrastructure and improve API behavior. Additionally, it outlines the benefits of using Protobuf for generating strongly-typed APIs and reducing payload sizes, ultimately aiming for a unified approach to enhance software development.
Next Generation Client APIs in Envoy Mobile
- 1. Jose Nino @junr03 Next Generation Client APIs with Envoy Mobile QCon London - March 2020 Some slides built in collaboration with: @goaway & @rebello95
- 2. InfoQ.com: News & Community Site Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ api-envoy/ • Over 1,000,000 software developers, architects and CTOs read the site world- wide every month • 250,000 senior developers subscribe to our weekly newsletter • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • 2 dedicated podcast channels: The InfoQ Podcast, with a focus on Architecture and The Engineering Culture Podcast, with a focus on building • 96 deep dives on innovative topics packed as downloadable emags and minibooks • Over 40 new content items per week
- 3. Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide Presented at QCon London www.qconlondon.com
- 7. Service Networking Networking Server @junr03 API Server Networking Service Networking Server Networking Service Networking Service Networking Service Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Service Networking Server Networking Networking Server Networking Server Networking Server Networking
- 8. Service Networking Networking Server @junr03 API Server Networking Service Networking Server Networking Service Networking Service Networking Service Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Service Networking Server Networking Networking Server Networking Server Networking Server Networking �� �� �� �� �� ��
- 9. Jose Nino @junr03 Infrastructure Engineer ~4 years @
- 10. Service Networking Networking Server @junr03 API Server Networking Service Networking Server Networking Service Networking Service Networking Service Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Service Networking Server Networking Networking Server Networking Server Networking Server Networking �� �� �� �� �� ��
- 12. We recognized two important dimensions in API management: what data we send (shape), and how we send it (transport). @junr03 ~3 years in Server Networking
- 13. Server-side Interface Definition Language (IDL) Service A “server” @junr03 #service_a.proto service ServiceA { rpc UpdateA(ARequest) returns (AResponse) { option (http.http_options).path = "/service_a"; option (http.http_options).method = "POST"; } } Service B “client”
- 14. ● Network Proxy driven by Open-source (OSS) community ● Common substrate for network behavior What is ? @junr03 Sidecar Service B “client” Sidecar Service A “server”
- 15. Storage EdgeMobile Client Sidecar Service B Sidecar Service A 3rd-party Strongly-typed APIs and a Universal Network Primitive @junr03
- 16. (circa early 2019) The End @junr03
- 17. Storage EdgeMobile Client Sidecar Service B Sidecar Service A 3rd-party�� Strongly-typed APIs and a Universal Network Primitive @junr03
- 18. Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? @junr03
- 19. Consistency ? ✓ Performance ? ✓ Extensibility ? ✓ Resilience ? ✓ Security ? ✓ Observability ? ✓ Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? @junr03
- 20. Storage Edge Sidecar Service B Sidecar Service A 3rd-party Clients as another node in the mesh Mobile Client @goaway @junr03 API
- 21. Next generation Client APIs with Envoy Mobile Mobile Client @junr03 API �� We recognized two important dimensions in API management: what data we send (shape), and how we send it (transport). ~1 year in Client Networking
- 22. We recognized two important dimensions in API management: what data we send (shape), and how we send it (transport). @junr03 API Shape
- 23. Early API workflow at Lyft Tech spec handwritten.swift handwritten.kt handwritten.py/.go @rebello95
- 24. iOS Server { “key_a”: “hello” } Android { “kye_a”: “hello” } Programming errors @rebello95
- 25. iOS Server { “key_a”: “hello” } Android { “kye_a”: “hello” } Programming errors @rebello95
- 26. iOS Server { “key_a”: “hello” } Android { “kye_a”: “hello” } Programming errors @rebello95 @junr03 ��
- 27. iOS Server { “key_a”: “hello” } Android { “kye_a”: “hello” } Programming errors @rebello95 @junr03
- 28. iOS Server { “key_a”: “hello” } Android { “kye_a”: “hello” } if (android): else: Programming errors @rebello95
- 29. ● Handwritten code led to mistakes/typos ● No visibility into deserialization failures at compile time Problems @rebello95 @junr03
- 31. paths: /foo: post: parameters: - in: body schema: $ref: '#/definitions/FooRequest' responses: 200: schema: $ref: '#/definitions/FooResponse' YAML definitions @rebello95
- 32. ● Manual code generation for Android ● No integration on iOS ● No single source of truth YAML results @rebello95
- 33. Consistency @rebello95 @junr03 We needed a source of truth that provided guarantees about the shape of any given API.
- 34. Envisioned workflow Tech spec IDL repo foo_api.swift foo_api.kt foo_api.py/.go foo_api.proto @rebello95
- 35. ● Platform-neutral IDL for defining and serializing strongly-typed APIs. ● JSON support (migration) ● Already used server-side (consistency) Why protobuf? @rebello95 @junr03
- 37. IDL pipeline /api/v1/foo.proto Generators +7 others foo.go foo.py foo.kt foo.swift FooV1API library foo-v1-api library @rebello95
- 38. Using generated APIs in Swift @rebello95
- 39. // Modules/FooV1API/FooAPI.swift public struct FooRequest: Codable, Equatable { … } public struct FooResponse: Codable, Equatable { public let id: Int64 public let firstName: String } Generated models @rebello95
- 40. // Modules/FooV1API/FooAPI.swift import RxSwift public protocol FooAPI { func updateFoo(_ request: FooRequest) -> Single<Result<FooResponse, FooAPIError>> } public struct FooClientAPI: FooAPI { // Abstracted implementation } Generated APIs @rebello95
- 41. // Modules/FooV1API/FooAPI.swift import RxSwift public protocol FooAPI { func updateFoo(_ request: FooRequest) -> Single<Result<FooResponse, FooAPIError>> } public struct FooClientAPI: FooAPI { 🌟 // Abstracted implementation 🌟 (hint hint wink wink!) } Generated APIs @rebello95 @junr03
- 42. // Modules/FooV1APIMock/FooAPI.swift open class FooMockAPI: FooAPI { open lazy var mockUpdateFoo: (FooRequest) -> Result<FooResponse, FooAPIError> = { … } open func updateFoo(_ request: FooRequest) -> Single<Result<FooResponse, FooAPIError>> { … } } Generated mocks @rebello95
- 43. ● Single source of truth for all APIs ● Consistent, generated code on all platforms ● Highly testable ● Underlying transport fully abstracted 🌟 565958 Results @rebello95
- 44. Performance @rebello95 @junr03 Experimenting with payload encoding
- 45. Client Service JSON request Existing system JSON response @rebello95
- 46. Client Service JSON request Content-Type: json Accept: x-protobuf,json Content negotiation Middleware JSON > Protobuf Protobuf request Content-Type: x-protobuf Protobuf response Content-Type: x-protobuf Middleware No change Protobuf response Content-Type: x-protobuf @rebello95
- 48. ● 40%+ reduction in payload sizes ● Faster response times ● Higher success rates ● Transparent to engineers Wins from protobuf @rebello95
- 49. Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? @junr03 Consistency ✓ ✓ Performance ✓ ✓ Extensibility ? ✓ Resilience ? ✓ Security ? ✓ Observability ? ✓
- 50. Extensibility @junr03 Using protobuf annotations to declare API behavior
- 51. service Foo { rpc GetFoo(FooRequest) returns (FooResponse) { option (http.http_options).path = "/foo/receive"; option (http.http_options).method = "GET"; option (http.http_options).client_poll_ms = 5000; option (http.http_options).gzip_enabled = true; } } Declarative APIs @rebello95 @junr03
- 53. Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? @junr03 Consistency ✓ ✓ Performance ✓ ✓ Extensibility 🤔 ? ✓ Resilience ? ✓ Security ? ✓ Observability ? ✓
- 54. We recognized two important dimensions in API management: what data we send (shape), and how we send it (transport). @junr03 API Transport
- 56. Clients as another node in the mesh Storage Edge Sidecar Service B Sidecar Service A 3rd-party Client @goaway @junr03
- 58. Why is world domination transport standardization useful? ● Write once, deploy everywhere ● Common tooling for common problems ● Reduce cognitive load @goaway @junr03
- 59. Envoy Mobile to the rescue! ● v0.1: proof-of-concept ● v0.2: laying the foundation ● v0.3: first production-ready release @goaway @junr03
- 60. Envoy as a Library @junr03 How did we turn a network proxy into a networking library?
- 61. Build System: bazel @goaway @junr03 ● Cross-platform support ● Used by Envoy
- 62. //library/common:c_types //library/kotlin:android_framework //library/swift:ios_framework Layered Architecture //:ios_dist //:android_dist //library/common:main_interface @goaway @junr03 //library/common/... @envoy//source/... Platform (iOS/Android) Bridge (C) Native (C++/Envoy)
- 63. How to run a process in an app? picture of an engine (a very fast one) @goaway @junr03
- 64. Threading contexts Application Threads Envoy Main Thread Callback Threads @goaway @junr03
- 65. Library Matrix Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy) Application Threads Envoy Main Thread Callback Threads @goaway @junr03
- 66. Library Lifecycle - Running Envoy Application Threads Envoy Main Thread Callback Threads EnvoyEngine @goaway @junr03 Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)
- 67. Library Lifecycle - Running Envoy Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Engine @goaway @junr03 Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)
- 68. Server Envoy Envoy Main Thread Worker Threads Envoy Dispatcher Http Manager Requests @goaway @junr03
- 69. Platform (iOS/Android) Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Dispatcher Http Manager @goaway @junr03 Bridge (C types/bindings) Native (C++/Envoy) Library Lifecycle - using Envoy Constructs
- 70. Platform (iOS/Android) Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Dispatcher Http Manager HttpStream @goaway @junr03 Bridge (C types/bindings) Native (C++/Envoy) Library Lifecycle - starting a stream
- 71. Library Lifecycle - dispatching a stream Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Dispatcher Http Manager HttpStream @goaway @junr03 Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)
- 72. Http Manager Http Filters @junr03 L7 Filter L7 FilterHttp Filter Dispatcher ● Foundational extension point ● Enact behavior ● Provide the guarantees we want!
- 73. Callbacks Library Lifecycle - callbacks Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Dispatcher Http Manager HttpStream @goaway @junr03 Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)
- 74. Library Lifecycle - platform callbacks Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Dispatcher Http Manager HttpStream Callbacks Lambdas Dispatcher @goaway @junr03 Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)
- 75. Platform (iOS/Android) Overall View Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Dispatcher Http Manager HttpStream Callbacks Lambdas Dispatcher Canceled @goaway @junr03 Bridge (C types/bindings) Native (C++/Envoy) Canceled
- 76. Platform Code let envoy = try EnvoyClientBuilder(domain: "api.envoyproxy.io") .addLogLevel(.warn) .addStatsFlushSeconds(60) .build() val envoy = EnvoyClientBuilder( Domain("api.envoyproxy.io")) .addLogLevel(LogLevel.WARN) .addStatsFlushSeconds(60) ... .build() @goaway @junr03
- 77. Build an Engine let envoy = try EnvoyClientBuilder(domain: "api.envoyproxy.io") .addLogLevel(.warn) .addStatsFlushSeconds(60) .build() @goaway @junr03
- 78. Build an Engine let envoy = try EnvoyClientBuilder(domain: "api.envoyproxy.io") .addLogLevel(.warn) .addStatsFlushSeconds(60) .build() @goaway @junr03
- 79. Build a Request let request = RequestBuilder(path: "/pb.api.v1.Foo/GetBar") .addHeader(name: "x-custom-header", value: "foobar") .build() @goaway @junr03
- 80. Build a Request let request = RequestBuilder(path: "/pb.api.v1.Foo/GetBar") .addHeader(name: "x-custom-header", value: "foobar") .build() @goaway @junr03
- 81. Build a Response Handler let handler = ResponseHandler() .onHeaders { headers, status, _ -> ... } .onData { data -> // Deserialize message data here } ... @goaway @junr03
- 82. Build a Response Handler let handler = ResponseHandler() .onHeaders { headers, status, _ -> // Product logic } .onData { data -> // Product logic } ... @goaway @junr03
- 83. Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? @junr03 Consistency ✓ ✓ Performance ✓ ✓ Extensibility 🤔 ? ✓ Resilience ? ✓ Security ? ✓ Observability ? ✓
- 84. IDL pipeline /api/v1/foo.proto Generators +7 others foo.go foo.py foo.kt foo.swift FooV1API library foo-v1-api library @goaway @junr03 NSUrl OkHttp
- 85. IDL pipeline /api/v1/foo.proto Generators +7 others foo.go foo.py foo.kt foo.swift FooV1API library foo-v1-api library Client @goaway @junr03
- 86. Ecosystem @junr03 Shape + Transport = Ecosystem!
- 87. Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? Consistency ✓ ✓ Performance ✓ ✓ Extensibility ? ✓ Resilience ? ✓ Security ? ✓ Observability ? ✓ @junr03
- 88. Extensibility @junr03 Using protobuf annotations to declare API behavior and Envoy Mobile to enact consistent behavior.
- 89. service Foo { rpc GetFoo(FooRequest) returns (FooResponse) { option (http.http_options).path = "/foo/receive"; option (http.http_options).method = "GET"; option (http.http_options).client_poll_ms = 5000; option (http.http_options).gzip_enabled = true; } } Declarative APIs @rebello95
- 91. Extensibility @junr03 Using protobuf annotations to declare API behavior and Envoy Mobile to enact consistent behavior.
- 92. Http Manager Http Filters @junr03 L7 Filter L7 FilterHttp Filter Dispatcher ● Foundational extension point ● Enact behavior ● Provide the guarantees we want!
- 93. Envoy filters Compression Filter gzip request Mobile Proxy Compression Filter Unzip payload 🥳 EdgeClient @junr03
- 94. Advanced transport @junr03 ● Dynamic Forward Proxy ✅ ● Compression 🚧 ● Deferred Requests 🕒 (Resilience) ● OAuth 🕒 (Security) ● Request Signing 🕒 (Security) ● Network condition 🕒 (Resilience)
- 95. Deferred Requests @junr03 Deferred Request Filter Mobile Proxy Client ❌ Service ❌ Deferred Request Filter Mobile Proxy Client ✅ Service ✅ 🥳 �� ��
- 96. Request Signing @junr03 Client Request Signature Filter sign request Mobile Proxy Request Signature Filter Verify signature Edge 🕵♀
- 97. Advanced transport @junr03 ● Dynamic Forward Proxy ✅ ● Compression 🚧 ● Deferred Requests 🕒 (Resilience) ● OAuth 🕒 (Security) ● Request Signing 🕒 (Security) ● Network condition 🕒 (Resilience)
- 102. Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? @junr03 Consistency ✓ ✓ Extensibility ✓ ✓ Performance ✓ ✓ Resilience ✓ ✓ Security ✓ ✓ Observability ✓ ✓
- 103. Onwards! ● Additional functionality in filters ● Protocol Experimentation with QUIC ● OSS Code Generators: complete OSS ecosystem for model-based APIs. @junr03
- 104. Next Generation Client APIs @junr03 Model-based APIs defined in a strongly-typed IDL so platform engineers can iterate on behavior using a common transport layer.
- 105. Alpha Beta Production Experiment @Lyft! @goaway @junr03
- 107. Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ api-envoy/