Building Event Driven API Services Using Webhooks
5 likes5,139 views
The document discusses building event-driven API services using webhooks, highlighting their utility in asynchronous integrations and user experience improvements. It details various use cases, potential pitfalls, and testing methodologies for webhooks during development, such as TCP tunneling and dynamic DNS. Additionally, it describes a webhook workflow example and mentions security considerations when exposing public API endpoints.
Building Event Driven API Services Using Webhooks
- 1. Building Event Driven API Services Using Webhooks Travis McChesney - Cloud Elements
- 2. Confidential & ProprietaryConfidential & Proprietary Applications are InherentlyDecoupled APIs glue applications together Most APIs can be used synchronously
- 3. Confidential & ProprietaryConfidential & Proprietary Application should provide EventDrivenAPIs Very useful for some integration use cases Improves UX of application APIs
- 4. Confidential & ProprietaryConfidential & Proprietary UseCasesforEventDrivenAPIs Integrations that need to consume large volumes of data Can’t keep a synchronous connection open for 15 minutes Long running, intensive operations such as complicated searches Services that cannot provide a definitive response synchronously
- 5. Confidential & ProprietaryConfidential & Proprietary Asynchronous APIOptions Polling Uses unnecessary compute cycles Frequency is application specific Concurrent polling is complicated and introduces instability
- 6. Confidential & ProprietaryConfidential & Proprietary LettheAPIProviderNotifyYourApplication Register a callback API with Event Handler logic Registered Event Handler is invoked when the data is ready for consumption This is basically a Webhook Trigger action for Callback API, and respond immediately to your users
- 7. Confidential & ProprietaryConfidential & Proprietary Example WebhookWorkflow Your app send an email using a Cloud Email Service Create an email message and send it via the Cloud Email Service Respond to your user that the message has been queued Persist the Email Message ID Your app wants to know when the email is processed, sent and opened Register the Event Handler or Callback API for the Webhook
- 8. Confidential & ProprietaryConfidential & Proprietary Example WebhookWorkflow The Cloud Email Service Provider invokes the Callback API for each event Your app receives the event and message ID via the Callback API request Your app handles each event and notifies the user of each event
- 9. Confidential & ProprietaryConfidential & Proprietary WebhookPitfalls Watch for circular references Updating Field 1 invokes a callback with a specific event Your app processes the event and invokes the API to update Field 2 The update of Field 2 invokes a callback, which in turn updates Field 1 again Security concerns Public API endpoint is now exposed Temp tokens or consumable message IDs
- 10. Confidential & ProprietaryConfidential & Proprietary Using WebhooksinDevelopment A Webhook needs a public callback URL to invoke Most developers use localhost So, how can I test a webhook while developing?
- 11. Confidential & ProprietaryConfidential & Proprietary Testing WebhooksinDevelopment API Capture TCP Tunneling OpenSSH Tunneling Dynamic DNS Remote Server
- 12. Confidential & ProprietaryConfidential & Proprietary API Capture forWebhooksinDevelopment API Capture is one mechanism to test webhooks in development Leverage services such a Runscope’s requestb.in Register this URL as the application callback URL for the Webhook Great for inspecting the posted callback data When the callback is invoked, use this URL in a browser Create a webhook callback URL at requestb.in Inspect the posted data Not as easy for debugging when something’s not quite right
- 13. Confidential & ProprietaryConfidential & Proprietary TCP Tunneling forWebhooksinDevelopment Services such as localtunnel.me, localtunnel.com, ProxyLocal or ForwardHQ > npm install -g localtunnel Need to reset tunnel when internet connection changes > lt —port <port> localtunnel.me example Need to install client on localhost
- 14. Confidential & ProprietaryConfidential & Proprietary OpenSSH Tunneling forWebhooksinDevelopment Open and keep open socket from localhost to remote server Configure remote server to pass requests down the connection Setup OpenSSH reverse proxy to a public server Need to reset tunnel when internet connection changes Needs GatewayPorts and AllowTcpForwarding enabled on remote server ssh -N -g -R :8080:localhost:8080 user@remote_server & Needs remote server with root access or system administrator access
- 15. Confidential & ProprietaryConfidential & Proprietary Dynamic DNS forWebhooksinDevelopment Use DynDNS or No-IP based DNS to route to localhost Needs router or firewall configuration Get a free DNS from one of the providers Configure DNS to forward to external IP of home office Configure home office router to route requests to private IP Works only from a single location
- 16. Confidential & ProprietaryConfidential & Proprietary Remote Server forWebhooksinDevelopment Use a public remote server for development Deploy application to publicly available host Need a public host, although easier now with public cloud providers Connect your IDE’s debugger to public host Debugging over the internet is typically slow
- 17. Confidential & ProprietaryConfidential & Proprietary TCP Tunneling DemowithSendGridElement Get localtunnel.me based application callback URL Register localtunnel.me application callback URL with SendGrid Send email using Cloud Elements’ Messaging Hub Upon email reception, handle event callback
- 18. Cloud Elements 2014 - Confidential & Proprietary CLOUD ELEMENTS One-to-Many One API connects to Many of the leading services in a category (e.g. CRM, Documents, Marketing) Multi-Tenant Connectors Each Element automatically authenticates to an unlimited number of customer accounts Embed Seamlessly in Your App Keeps your app all about your brand as our Elements work behind the scenes through RESTful APIs Monthly, Pay As You Go Charged only when your customers use your application 18 Less work, less cost, and more connections for your application.
- 19. Confidential & ProprietaryConfidential & Proprietary References [email protected] @travisj37 References http://www.cloud-elements.com/4-options-developing-webhooks/ http://progrium.com/blog/2012/11/19/from-webhooks-to-the-evented-web/
Editor's Notes
- #3: Set the stage for use cases for asynchronous APIs. As APIs get more and more prevalent and start doing more and more tasks, an asynchronous approach is going to be necessary
- #5: Bulk uploads and downloads from CRM systems. Don’t want to do that synchronously. Long search operations
- #6: Using compute cycles if there is no data to be pulled. Cases where the frequency of polling is application specific. Some cases every 5 seconds, other times every hour. Could depend on the data. Example: Buying Rockies tickets. Rather than refreshing the page, you could get notified. Multiple threads trying to get the same data.
- #7: Providing an event handler for your callback API. Ex: SFDC. Notified when an Opportunity object probability goes above 60%.
- #9: There are things to consider with web hooks as well. Some service providers only allow one callback URL, and your app has to handle the event based on the payload received.