Pragmatic Enterprise Application Migration to AWS
7 likes4,588 views
The document presents a case study on the pragmatic migration of enterprise applications to AWS, discussing challenges and lessons learned from a 'lift and shift' strategy. It highlights the importance of cloud adaptation, security, and operational tools while emphasizing the need for a holistic approach to application architecture, infrastructure, and system integration. Key recommendations include enhancing security, automating deployments, and adopting DevOps practices to achieve better performance and resilience in cloud environments.
Pragmatic Enterprise Application Migration to AWS
- 1. © 2015 Cloud Technology Partners, Inc. / Confidential 1 v Pragmatic Enterprise Application Migration to AWS Boston AWS Meetup 7/14/2015
- 2. © 2015 Cloud Technology Partners, Inc. / Confidential 2 Lift & Shift: Case Study• Vice President, Principal Architect at Cloud Technology Partners • AWS Certified Solutions Architect • Cloud architecture, application migration, cloud data management, DevOps / ProdOps [email protected] @kacyclarke updraft-downdraft.blogspot.com Presenter Kacy Clarke
- 3. © 2015 Cloud Technology Partners, Inc. / Confidential 3 Enterprise Applications Don’t Always Fit the Cloud
- 4. © 2015 Cloud Technology Partners, Inc. / Confidential 4 Lift & Shift: Case Study Cloud Adoption Starts with these Six Key Tenets 1 2 3 4 5 6 Strategy & Economics Executive Cloud Benefits, Objectives and Goals Cloud Readiness Assessment - 3 Year Actionable Roadmap ROI / TCO Economic Models Security & Governance CSA/ISO 2700X Reference Architecture Gap Analysis of InfoSec Policy, Procedures, and Key Controls Cloud Security Tooling Gap Analysis Application Portfolio Assessment Portfolio Assessment Readiness for Cloud Cloud Reference Architecture and Capabilities Matrix Application Cloud Roadmap & Plans Application Migration & Development (MVC) Design and Delivery of a Minimum Viable Cloud (MVC) Application Migration Factory Infrastructure Automation DevOps DevOps Maturity Assessment DevOps Certification and Training DevOps as a Service (DOaaS) CloudOps Service Management, Cloud Operational Model (MSP) Governance, Billing, Chargeback, Audit, Logging, Escalation Organizational & Skill Set Training 1 2 3 4 5 6
- 5. © 2015 Cloud Technology Partners, Inc. / Confidential 5 Objective: Rehost / Lift & Shift • Migrated to AWS quickly to address rapid growth, minimize risk with a typical outsourcer • Minimized changes to the applications and infrastructure with minimal operational tools • Used traditional data center application technology stack: Weblogic cluster, Oracle RAC, Coherence with stateful Weblogic session management • New security architecture did not mesh with security scanning protocols Client began moving customer facing products to AWS to leverage scalability and regional deployments for millions of international users. Lift and Shift Case Study: Global Media Client Lessons Learned • Applications typically relied on infrastructure supported availability, rather than being designed to be self-healing for component failure • Infrastructure oriented monitoring tools make it very difficult to troubleshoot application problems. Insufficient knowledge of application execution • Scale out matters at so many levels – e.g. resource bottlenecks, noisy neighbors, load distribution and component performance variability • Applications were designed for fixed resources, so had to over allocate reserved instances, driving up cost Result: Application Did Not Meet SLA’s
- 6. © 2015 Cloud Technology Partners, Inc. / Confidential 6 Lift & Shift: Case Study • Mission critical Service Level Agreement • Full production migration • Multi-tier architecture • Ecosystem dependencies • Security and compliance concerns • Data creation and/or update • Limited time and resources Target Application Characteristics
- 7. © 2015 Cloud Technology Partners, Inc. / Confidential 7 Lift & Shift: Case Study • Missed SLA’s • Performance and latency problems • Environment inconsistencies • Gaps in monitoring and operational tools • Data synchronization issues • SDLC environment sprawl • Lack of transparency for what’s happening with the application or the system • Development and operational skills and knowledge gaps • Manual or frozen configurations • Compliance or security findings • Cost savings objectives not met Pure Lift and Shift Can Result in Cloud Migration Debt
- 8. © 2015 Cloud Technology Partners, Inc. / Confidential 8 The Cloud is Very Different Traditional Architectures • Scale Up • Monolithic • Stateful • Infrastructure Dependent (i.e LAN,SAN, etc.) • Fixed Capacity • Latency intolerant • Consolidated / clustered DB • Commercial licenses • Manual build/deploy • Manual fault recovery • Active/Passive/DR • Perimeter Security • Allocated costs Cloud Aligned Architectures • Scale Out • Distributed • Stateless • Elastic capacity • WAN, Location transparency • Latency tolerant • Loosely coupled • Sharded / replicated / distributed DB • Mobile/thin client • Cloud PaaS / Open Source • Automation • Self healing • Active/Active • Metered cost Refactor Continuous Delivery
- 9. © 2015 Cloud Technology Partners, Inc. / Confidential 9 Lift & Shift: Case Study How Much Do You Change for Your Initial AWS Application Migration? Which operational tools should I keep and which should I replace? How much application code do I need to change? Should I replace my application technology stack with AWS services? Should I automate my deployments? Do I need to add security tools or configuration? Do I need to implement autoscaling? What do I need to do with my application integration? What if other applications need to access my database? How do I address data retention and compliance?
- 10. © 2015 Cloud Technology Partners, Inc. / Confidential 10 Lift & Shift: Case Study Cloud CLOUD WASHED CLOUD ADOPTED CLOUD OPTIMIZED CLOUD NATIVE - Force fit to run in cloud environment - Resources not optimize – no horizontal scaling - Minimal app modification to be cloud compliant - Infrastructure based availability - Reuse of data center management and monitoring technologies - Resources not optimized - No autoscaling - Failure intolerant – connections and resources may not all automatically restart - Some app modification to be cloud compliant and use cloud platforms - Over or under utilization of resources has cost/performance impact - Some cloud aligned operational tools - Resources being optimized – horizontal scaling possible - Managed elasticity– cloud management layer determines when to start/stop additional instances - Major app modification to be cloud aligned - Coarse grained cost and performance resource optimization - Cloud operational tools - Fully cloud aware – app communicates with the cloud management layer to start-up or shut down instances as needed - Designed for failure and self healing - Resource efficient - Cloud native platform and operational tools - Fine grained cost and performance resource optimization Cloud Application Maturity Business Needs Drive Cloud Maturity Target
- 11. © 2015 Cloud Technology Partners, Inc. / Confidential 11 Lift & Shift: Case Study Traditional (Anti-Pattern) Cloud Washed Cloud Adopted Cloud Optimized Cloud Native Automation Manual or limited scripts Manual or limited scripts Semi-automated Full automation Continuous delivery Scaling Fixed capacity Reserved capacity Reserved and elastic Resources on demand Autoscaling at every tier State Management Stateful session mgmt Stateful sessions, stickiness Stateless, location sensitive Stateless, multiple AZ load distribution Stateless, optimized load management Resiliency Infrastructure oriented Infrastructure oriented Redundant, retry, restart Self-healing Perf/health sensitive Integration Tightly coupled, native interfaces Mixed coupling Both synch and asynch, svc bus Loose coupling, API API, eventually consistent Availability Management Infrastructure based, clustering Infrastructure based, clustering Application aware, manual initiation Application initiated Application controlled Database Proprietary, big iron, physical Some VM DB, most on physical Sharding, distribution, eventual consistency Cloud aligned dbms, distributed, dbms elasticity Cloud native dbms Applications Targeting the Cloud
- 12. © 2015 Cloud Technology Partners, Inc. / Confidential 12 Lift & Shift: Case Study Traditional (Anti-Pattern) Cloud Washed Cloud Adopted Cloud Optimized Cloud Native Storage SAN, NAS, local storage Mounted storage, shared file systems Storage agnostic, location sensitive Storage agnostic, replication for speed/ resiliency Storage location insensitive, replication for speed/ resiliency Network App tiers and integration on LAN, latency sensitive App tiers and integration on LAN, latency sensitive, external WAN App tiers in limited AZ’s, LLB, latency sensitive App tiers across AZ’s, GLB, latency tolerant GLB, Location transparency, latency tolerant Compute Physical , some VM VM with exceptions (DB, FW, …) VM, speed, memory sensitive VM agnostic, speed/memory adaptive VM agnostic, speed/memory optimized OS/Container Proprietary Linux, Win, Commercial SW Linux, Win, mixed SW stack Open source/ Cloud PaaS Cloud native Network/Host Security Perimeter, physical Access controls, perimeter focus Service level security System Defense in Depth Application Defense in Depth Data Protection Encrypt for external transfer Encrypt for external transfer Encrypt in flight, at rest Encrypt in flight, at rest Encrypt in flight, at rest, in use Applications Targeting the Cloud
- 13. © 2015 Cloud Technology Partners, Inc. / Confidential 13 Systems Thinking: A holistic approach to analysis that focuses on the way that a system's constituent parts interrelate and how systems work and change over time and within the context of larger systems. The systems thinking approach contrasts with traditional analysis, which studies systems by breaking them down into their separate elements Cloud Migration Impacts – Application Architecture – Infrastructure Architecture – Data Lifecycle Management – Security and Compliance Application migration requires the holistic analysis of systems thinking It’s Not an Application, It’s a System – SDLC – Operations and Monitoring – Support – Cost Management and Planning
- 14. © 2015 Cloud Technology Partners, Inc. / Confidential 14 Lift & Shift: Case Study 1. Replace key operational tools 2. Enhance security infrastructure 3. Remediate application code to maintain SLA’s 4. Selectively replace platform technology 5. Automate application provisioning and deployment - automate everything else while you’re at it 6. Create dashboards to monitor the full stack 7. Adopt a DevOps approach Recommendations
- 15. © 2015 Cloud Technology Partners, Inc. / Confidential 15 Lift & Shift: Case Study Machine Image Creation – OS hardening and tool installation for base AMI’s – AMI incremental builds for common application infrastructure (database, app server) – Consider an AMI build tool like Packer Cloud Aware Monitoring – System monitoring with integration with AWS, CloudWatch (StackDriver, Data Dog, Science Logic) – Application Performance Monitoring with distributed environments (New Relic, App Dynamics) Log Management – Infrastructure for both Security and DevOps (ELK, Splunk, SumoLogic) Replace key operational tools
- 16. © 2015 Cloud Technology Partners, Inc. / Confidential 16 Lift & Shift: Case Study • Add cloud oriented security tools – Security groups replacing data center firewalls – Security group configuration scanning (Dome 9, etc.) – Instance configuration scanning (Alert Logic, CloudPassage, etc.) – Web application firewalls – Log management for audit (Alert Logic, ELK, Splunk, etc.) • Add Key and Certificate Management • Encrypt data at rest – File systems – Object stores (S3, Glacier) – Database – Backup – AMI’s • Reexamine SSL offloading Enhance security infrastructure
- 17. © 2015 Cloud Technology Partners, Inc. / Confidential 17 Minimize app changes to focus on SLA’s, dynamic environment and security Remediate application code to maintain SLA’s Resiliency – Spread application tiers across Availability Zones – Timeout, retry, reconnect for integration points – Add load balancers between tiers – Enhance error checking and alerting – Fail database over between AZ’s – Selectively add logging Abstract Configuration – Replace hard coded IP addresses, host names – Generate property/config files in automation Loose Coupling – Replace RMI, EJB calls with web services – Selectively replace synch with asynch integration – Add database caching if database is remote – Remove clustering and session replication if possible. Move session state to DynamoDB or other DB Application Security – Encrypt data in flight, at rest – Encrypt or lookup integration credentials
- 18. © 2015 Cloud Technology Partners, Inc. / Confidential 18 Target areas that would enhance SLA’s, performance, manageability Selectively replace application platform technology Component Examples AWS Options Rationale Messaging Middleware IBM MQ, Tibco SQS, RabbitMQ Cost, manageability, effort Application Server Websphere, Weblogic Elasticbeanstalk, Jboss, Tomcat Cost, horizontal scaling Caching Coherence, EHCache, Hazelcast Elasticache Cross-AZ, scalability, manageability CDN Akamai, Level 3, Limelight CloudFront Cost, integration, manageability Shared File System NetApp, Windows file server AWS EFS Scalability, manageability Database Oracle RAC, SQL Server Clusters RDS, Enterprise DB Cost, manageability Load Balancing F5, Netscaler AWS ELB Cross-AZ, horizontal scaling, manageability Static content File server S3 Availability, scalability
- 19. © 2015 Cloud Technology Partners, Inc. / Confidential 19 “Everything fails, all the time.” - Werner Vogels Automate application provisioning and deployment • Automate full stack, push button deployments, startup/shutdown • Forbid manual changes to application configuration or environments after Dev • Build full production environments (except data tier) for each release, and then cutover when ready • Bake AMI’s for rapid deployment • Log automation steps • Automate acceptance/smoke tests
- 20. © 2015 Cloud Technology Partners, Inc. / Confidential 20 • Service catalog automation – AMI builds – Service lifecycle management – Testing • Auto response to common issues • Failover/failback • Disaster recovery • Self-service user support • Patching Automate everything else while you’re at it • Capacity management • Backup/recovery • Data archive/restore • Space reclamation • Audit • Reporting
- 21. © 2015 Cloud Technology Partners, Inc. / Confidential 21 • Iterative implementation for visualization refinement and tuning data collection • Dashboards for: – Alerts – Application performance – Activity/load – Systems resources – Change activity – Historical trends • Different views for different stakeholders • Implement periodic reviews for continuous improvement Create dashboards to monitor the full stack and the processes
- 22. © 2015 Cloud Technology Partners, Inc. / Confidential 22 Adopt a DevOps approach
- 23. © 2015 Cloud Technology Partners, Inc. / Confidential 23 • Leverage DevOps best practices to change how app teams interact with operations • ITIL based practices/structure will not vanish overnight, gradually adapt to DevOps/Kanban • Treat the platform as a product Adopt a DevOps approach Agile/Lean SDLC DevOps: Agile Infrastructure Cloud Service Management ProdOps: Cloud Operations Network ServerSecurity Storage App Infra Database Operations Support Application Focus Cloud Focus
- 24. © 2015 Cloud Technology Partners, Inc. / Confidential 24 Example Minimum Viable Cloud for Enterprise Applications Applications Application/Data Infrastructure Full Stack App Automation Service Catalog Automation Tools, Log Mgmt, Monitoring, Image Mgmt, Backup, Support Network, Security, IAM, Audit Amazon Web Services Development Toolkit
- 25. © 2015 Cloud Technology Partners, Inc. / Confidential 25 Boston Headquarters 263 Summer Street Fourth Floor Boston MA, 02210 Contact 617.674.0874 [email protected] www.cloudtp.com vv Questions? [email protected]