Serverless without Code (Lambda)
- 1. Serverless without Code (Lambda) SoFlo DevCon April 16th, 2022
- 2. Who am I? • Who am I?
- 3. Who is CloudHesive? • Professional Services • Assessment (Current environment, datacenter or cloud) • Strategy (Getting to the future state) • LandingZone (Pre-Migration) • Migration (Environment-to-cloud, Datacenter-to-cloud) • Implementation (Point solutions) • Support (Break/fix and ongoing enhancement) • DevOps Services • Assessment • Strategy • Implementation (Point solutions) • Management (Supporting infrastructure, solutions or ongoing enhancement) • Support (Break/fix and ongoing enhancement) • Managed Security Services (SecOps) • Encryption as a Service (EaaS) – encryption at rest/in flight • End Point Security as a Service • Threat Management • SOC IIType 2Validated • Next Generation Managed Services (MSP) • Leveraging our Professional, DevOps and Managed Security Services • Single payer billing • Intelligent operations and automation • AWS Audited • Cost Management
- 4. AWS User Groups of Florida • Almost 2Years ofVirtual Meetups/11 Meetups in the past 12 Months! • Doral: https://www.meetup.com/AWSUserGroupDoral • Miami: https://www.meetup.com/Miami-AWS-Users-Group • Miami Beach: https://www.meetup.com/aws-user-group-miami • Fort Lauderdale: https://www.meetup.com/South-Florida-Amazon-Web-Services-Meetup • Boca Raton: https://www.meetup.com/awsflorida • Orlando: https://www.meetup.com/Orlando-AWS-Users-Group • Tampa: https://www.meetup.com/Tampa-AWS-Users-Group • Jacksonville: https://www.meetup.com/AWS-User-Groups-of-Florida-Jacksonville
- 5. Agenda • Agenda/Introduction/Why? • Services Background • Serverless • APIs • Event Buses/Messaging/Queuing • Data Stores • Demo • Operational Considerations • General • API Gateway • Lambda • Other • Adoption Approach • Conclusion
- 6. Inspiration • “I'm wondering if there is a feature request to create something like a saved query in Athena that can be executed via a CloudWatch Event?” • The AWS Step Functions service integration with Amazon Athena enables you to use Step Functions to start and stop query execution, and get query results • AWS User Groups of Florida MeetUp - AWS APIArchitectures - Scott Hendrickson, Partner SolutionsArchitect, AWS • Data sources and resolvers are how AWS AppSync translates GraphQL requests and fetches information from your AWS resources • AWSWell Architected Framework Serverless Application Lens • If your Lambda function is not performing custom logic while integrating with other AWS services, chances are that it may be unnecessary
- 7. Background
- 8. Compute’sTransition to Serverless • Compute - EC2 Bare Metal (Intel, AMD, Graviton, M1) • Compute - EC2Virtual > Bare Metal (Xen, KVM/Nitro) • Containers - Fargate > ContainderD (was DockerD) > EC2 • Serverless - Lambda > Firecracker (MicroVM) > EC2
- 9. Serverless’ Flavors • High Level Abstractions • SaaS (Connect) • Hybrid Abstractions • PaaS (DynamoDB) • Low Level Abstractions • IaaS (Lambda)
- 10. CloudWorkload Lifecycle Management • Workload (What is it?) • Architecture (How does it run?) • Monitoring (How do I know something has gone wrong?) • Automation (If something goes wrong, do I need to fix it?) • Processes (How do I keep it alive)?
- 11. Workload + Architecture Drives Service Selection • Virtual Machines • AMI • Patching • Multi-threaded/Multi-task • Hours to Months • PerVM/Per Hour • Functions/Services • Code • Versioning • Single-threaded/Single-task • Microseconds to Seconds • Per Memory/Second/Per Request • Containers • Container File • Versioning • Multi-threaded/Single-task • Minutes to Days • PerVM/Per Hour
- 12. APITransition
- 13. Primary Services • APITier • API Gateway – API Management • AppSync – GraphQL API • Application (Execution)/CodeTier • Lambda – Serverless Compute • Data StoreTier • DynamoDB – Key/Value Data Base • ServiceTier • Event Bridge/Step Functions – Event Bus, Low Code/No Code Workflow • Athena – Interactive Query Service • S3 – Object Storage • Glue – Data Integration Service
- 14. Options for APIs • Client > API Gateway HTTP >Things • Client > API Gateway REST >Things • Client > AppSync GraphQL >Things • Client > Application Load Balancer > Lambda • Client > Lambda Function URLs • Client > CloudFront (Authorizer) > Lambda • Client > AWS IoT
- 15. Options to call AWS services w/o Lambda • APIs • API Gateway > AWS Services • AppSync > GraphQL > Resolvers > AWS Services • Event • Step Functions > AWS Services • EventBridge
- 16. API Gateway Integrations • AWS • Service • Lambda • AWS Proxy • Service • Lambda • HTTP • HTTP Proxy • Mock
- 17. AppSync Resolvers • DynamoDB • RDS • OpenSearch • Lambda • HTTP
- 18. Step Functions Optimized Integrations • Lambda • Batch • DynamoDB • ECS/Fargate • SNS • SQS • Glue, DataBrew • SageMaker • EMR • CodeBuild • Athena • EKS • API Gateway • EventBridge • Step Functions
- 19. Options for Event Buses/Messaging/Queuing • DynamoDB >Triggers • CloudWatch Logs > Metrics > Alarms / Lambda • CloudWatch Metrics > Destination • Kinesis > Lambda • Event Bridge (DLQ Support) > Lambda • SQS (DLQ Support) > Lambda • SNS (DLQ Support) > Lambda • (DLQ Support) Lambda
- 21. Serverless Data Stores -The Easy Button • S3 Query – Query objects in S3, through S3 • Athena (and S3 and Glue) – Query objects in S3, Presto • AppFlow – Data Integration Platform • Profiles • Wisdom • Tasks • HoneyCode – No Code Platform
- 22. Serverless Data Stores • DynamoDB – Key/Value • Timescale –Time Series • Keyspaces – Cassandra • QLDB – Ledger • Aurora – Relational • Prometheus – Prometheus • Grafana – Grafana • MWAA – Airflow
- 23. Demo
- 24. API Example (API Gateway) AWS Well Architected Framework Serverless Application Lens
- 25. Event Example
- 26. Demo • Simple API • Proxy (SNS) • Simple Graph API • Resolver (DynamoDB) • Simple Event/Simple Step Function
- 27. Let’s not forget operational things…
- 28. General Considerations • Multi-Region? Single-Region?Which Region(s)? • Which Services? • What will they cost? How are they metered/billed? • How far do we need to scale? • What compliance requirements do we need to meet? • What tools do we have in our reach? (Frameworks, Patterns, etc.)
- 29. API Gateway • Development (Isolation, Stages, SAM) • Client Security (Certificates, API Keys, Authorizers) • Gateway Security (WAF,Throttling) • EndpointType (Edge optimized, Regional, Private, API Cache) • Integration (Methods, Proxy, ResponseCodes) • Operationalization (CloudWatch Logs,CloudWatch Metrics, Access Logging, X-Ray • Testing (Direct, PostMan)
- 30. Lambda • Runtime • Pre-Warming • Sizing/Timeouts • Development (Isolation,Versions, SAM, Cloud9, Parameterization) • Integration (Methods, ResponseCodes) • Security (KMS, Execution Role) • Operationalization (CloudWatch Logs, CloudWatch Metrics, X-Ray) • Testing (Direct)
- 31. “The Rest” • Development (Coding Best Practices, Runtime, RDBMS, DevOps) • Data Stores that are not Serverless (Sizing,CloudWatch, Logs, Events, Backup/Recovery, Multi- AZ, Database “Stuff”) • Trade-off • VPC (Public Subnets, Private Subnets, Security Groups) • Typical of Legacy Integrations, Non-Serverless Data Stores, etc. • General (What are all of the things we need to think about when we create a new AWS account?) • “Landing Zone”
- 33. Application • Customer operates their own Electronic Medical Record (EMR) platform, which was developed by a third party, and wanted to extend it’s capabilities to support a new line of business (durable medical equipment). • This was a compelling event to change the way they introduce new capabilities in their EMR: Rather than continue to rely on a third party to extend it, why not grow their in-house talent and lessen the dependency on an outside organization, which would allow them to remain nimble as the state of their business changes. • To get started, they nominated their business intelligence team, who had experience with the data models that would be leveraged by the new line of business, to create an API to extend the EMR platform, allowing for business rule changes to be made at their pace.
- 34. Challenge • Limited Experience with Cloud, AWS, Security, Serverless, Application Development, etc. • ShortTimeline to deliver a functional solution (6Weeks). • Essentially looking for an insurance policy – to their credit they had already started going down the serverless path, and many of the solutions they selected were ultimately implemented. • To that end, we proposed an initial workshop along with bi-weekly check-ins.
- 35. Approach -Workshop • Define • Review requirements • Review technologies selected • Review work completed so far • Setup access • Design • Recommendations and selection of appropriateAWS services • Weigh pros and cons of said services • Develop a basic framework for the application • Provide primer onAWS in the areas of most importance to delivery of the project
- 36. Approach –Week byWeek • Scheduled a series of standing check-ins. • Based on progress made throughout the week, these check-ins would either be used to talk through a blocking issue, review work completed, or if time permitted, introduce new concepts. • These meetings also allowed us to keep up with material changes to the overall project, which allowed us to provide long term guidance around the customer’s overarching cloud strategy.
- 37. Outcome • Through the workshops and check-ins the customer was able to: • Build APIs leveraging API Gateway and Lambda. • Do it in a way that ensured compliance with their organizational and industrial standards while also adhering to best practices. • Successfully integrate with their EMR. • …and do this with an estimated AWS Spend of ~1,000.00 USD/Month • With 75% of costs driven by requests (that’s why we like serverless), meaning the majority of the costs will directly correlate to servicing a request and not idling. • We also learned some good lessons along the way, specifically around backlog management/prioritization and specification changes.
- 38. What’s next? • Multiple Streams ofWork (Serverless, Analytics, Landing Zone) • While the initial set of APIs allowed the EMR to integrate with a system of record and business rules maintained by the customer, there was a desire to capture data entered into the EMR in a system maintained by the customer. • From a technology perspective, additional APIs would be exposed from the EMR and the EMR would populate a series of DynamoDB tables. From a compliance perspective, this also introduces additional controls required around this data. • As such, DynamoDB and DynamoDB Streams has been introduced as system of record and to provide an audit trail of the data into a Database controlled by the customer. • These additional services will bring the AWS Spend up to ~3,000.00/Month
- 39. References
- 40. Well Architected Framework (WAF) • General • Event-Triggered • Workload-Focused • General Design Principals • Pillars • Design Principals • Best Practices • Lenses
- 41. WAF – Pillars • Operational Excellence • Security • Reliability • Performance Efficiency • Cost Optimization • Sustainability – New!
- 42. WAF – Lenses • High Performance Computing (HPC) • Serverless • Internet ofThings (IOT) • Financial Services Industry (FinServ) • FoundationalTechnical Review (FTR) • SaaS • Streaming Media • Machine Learning • SAP • DataAnalytics • Games Industry • Hybrid Networking • Management andGovernance
- 43. WAF – General Design Principals • Stop guessing your capacity needs • Test systems at production scale • Automate to make architectural experimentation easier • Allow for evolutionary architectures • Drive architectures using data • Improve through game days
- 44. WAF – Game Days • Prepare • Is the process/are the processes to be tested during the game day well defined? Is access in place? Has training been performed? • Define • Workload, Personnel, Scenario, Environment, Schedule • Execute • Start, Middle, End • Analyze • Debrief, Examine, Document, Root Cause Analysis (RCA), Correction of Error (CoE)
- 45. References • https://docs.aws.amazon.com/wellarchitected/latest/serverless-applications-lens/wellarchitected- serverless-applications-lens.pdf –Well Architected Serverless Application Lens • https://docs.aws.amazon.com/apigateway/latest/developerguide/getting-started-aws-proxy.html – API Gateway Service Proxy Example • https://docs.aws.amazon.com/apigateway/latest/developerguide/websocket-api-chat-app.html – API Gateway Websocket Example • https://docs.aws.amazon.com/appsync/latest/devguide/tutorials.html – AppSyncTutorials • https://docs.aws.amazon.com/appsync/latest/devguide/tutorial-dynamodb-resolvers.html – AppSync Tutorial DynamoDB Resolver • https://docs.aws.amazon.com/lambda/latest/dg/lambda-urls.html – Lambda URLS • https://docs.aws.amazon.com/step-functions/latest/dg/connect-supported-services.html – Step Functions Supported Services • https://docs.aws.amazon.com/step-functions/latest/dg/sample-athena-query.html – Step Functions Athena Query
- 46. Conclusion • AWS continues to increase the breadth and depth of their service offerings • I wish it did that • I didn’t know I needed that • It’s easier to get started today than it was yesterday • Simplicity • Support • Cost
- 47. Thank you!
- 48. Contact Us • Partner Solutions Finder • https://aws.amazon.com/partners/find/partnerdetails/?n=CloudHesive&id=001E000000qK5f6IAC • E-Mail • [email protected] • URL • https://cloudhesive.com • Phone • United States: 800-860-2040 x1 (Fort Lauderdale, Florida, US & Norfolk,Virginia, US) • Argentina: +54 (11) 51737475 x1 (Buenos Aires,AR & Santiago,CL) • United Kingdom: +44 (20) 37955127 x1 • Australia: +61 (2) 80742932 x1
Editor's Notes
- #13: +Websockets