Data normalization across API interactions
4 likes5,657 views
The document discusses the importance of data normalization when integrating APIs. It notes that APIs often have inconsistent data formats and endpoints that are not standardized. This can make APIs difficult to reuse and maintain. The document proposes normalizing data into a common structure and configuration-based transformations to map different APIs and data formats to this normalized structure. This allows applications to have a unified view of data across APIs and makes the APIs more reusable and interchangeable. It also discusses using asynchronous interactions and message buses to link normalized data across APIs rather than application-level logic.
Data normalization across API interactions
- 1. Data Normalization for API Interactions
- 2. Importance of APIs Allows you to use software more selectively Leverage best of breed apps in your app Provide value to your users by focusing your resources on your app Confidential & Proprietary
- 3. APIs are nice and all, but… Endpoints are not standardized. What! You don’t have RESTful API? Same type of data represented in different formats with different names How can I reuse without abuse? My app’s APIs are fast becoming unusable and unmaintainable Confidential & Proprietary
- 4. Do you really want this? What, I’ve to put this all together? Why don’t I just write my own storage solution and my own CRM system? Dynamics Account SFDC Account Element Element Box Folder Confidential & Proprietary Sharepoint Document Google Drive Folder
- 5. What if you had this? Account Folder Document Integration and Normalization Element Element Confidential & Proprietary
- 6. Normalization Example Confidential & Proprietary
- 7. Use Best Practices for Data Normalization No matter the endpoint protocol, provide a uniform protocol from your app, preferably REST with JSON Transform data from an endpoint into a normalized structure. Configure your transformations and not code them Always provide direct access to the endpoint data, so your app does not become a throttle Provide customization, but not at the cost of multi-tenancy Confidential & Proprietary
- 8. Multi-tenancy via Configuration Your Application Your Customers Toyota Users Normalization Configuration SFDC 1 Toyota SFDC 2 BestBuy Users BestBuy SFDC 3 Whirlpool Users Whirlpool Sugar Confidential & Proprietary Vendor
- 9. Custom Transforms and Rules Canonical SFDC mapping CRM Hub - Create Account Transformation Result Create CRM Account 4 API JSON Input 5 Transformation 6 3 Custom SFDC mapping script 2 Custom SFDC mapping Confidential & Proprietary 1
- 10. Data Normalization works for me I have one view of an Account object, whether it’s from Salesforce.com or Microsoft Dynamics I can use a Document object in a uniform way regardless of the Storage provider And I can switch my Cloud Storage provider at any time Which should allow me to write code in my app to hook these all together! Wait a minute…that can be time consuming. Is there some other way? Confidential & Proprietary
- 11. Which leads to API Interactions I want my Account to be linked to the contractual Documents But, I don’t want to save any of this information in my application But, can I achieve these interactions without writing messy application logic If my data is already normalized, why can’t something stitch is all together! Confidential & Proprietary
- 12. Normalized Data can Interact Use an asynchronous model for interactions, the matrix for point-to-point interactions will become unmanageable Don’t write your own Message Bus, use an off the shelf system that can be extended Provide a facility for customizable interaction rules Plan for scale, especially when dealing with Social API endpoints Be agile, don’t try to replace all the app logic in one attempt Confidential & Proprietary
- 13. Embrace Standards True REST may not always make sense for your application, but try your best to adhere to the standard RAML is now open sourced, definitely something to look at XML is great, JSON is better, but try to make your APIs speak one or both languages Standardize on a format for each object across your apps Confidential & Proprietary