Slides: Proven Strategies for Hybrid Cloud Computing with Mainframes — From AWS and Qlik

© 2020 QlikTech International AB. All rights reserved.
Proven Strategies
for Hybrid Cloud
Computing with
Mainframes
From AWS and Qlik
6th Oct 2020
Adam Mayer - Snr Manager Technical Product Marketing
Steve Steuart - Global Lead of Mainframe migration
and modernization

2
About your speakers
Steve Steuart
Global Lead of Mainframe Transformation, Amazon Web Services
Mr. Steuart is responsible for the global strategy for
mainframe transformation at AWS. He has been a
recognized leader in mainframe transformation for close to
30 years with a proven track record helping companies
transform their legacy assets to Hybrid IT or Cloud.
Adam Mayer
Senior Technical Product Marketing Manager, Qlik
Mr. Mayer is responsible for CDC Streaming and
mainframe product marketing, in addition to delivering
Qlik’s Internet of Things (IoT) and GDPR go-to-market
strategy. He has a strong technical background in
computing spanning over 20 years, and is an avid follower
of new technology especially IoT, particularly on the data
streaming and analytics side.

3
Modernize and Automate Data Integration
Efficiently capture large volumes of changed data from heterogeneous sources
and deliver analytics ready data in real-time to Cloud Platforms where it can be
catalogued for discovery and provisioning to Analysts and Data Scientists
TRUSTED MARKET LEADER
CHANGE DATA CAPTURE CLOUD INTEGRATION
SOLUTIONS
EASE OF USE
AND AUTOMATION

4
Modernize and Automate Data Integration
REAL-TIME HETEROGENEOUS COMPLETE
&
AUTOMATED
SCALE
&
STABILITY

5
Trends Driving Integration Modernization & Automation
CLOUD APPLICATION DEVELOPMENT NEXT GENERATION ANALYTICS & DATA
MONETIZATION
IaaS
PaaS
Micro
Services
DB
MF
EDW
FILES
DATA WAREHOUSE MODERNIZATION
DWaaS
DB
MF
EDW
FILES
IaaS
PaaS
SaaS
DB
MF
EDW
FILES
Data Lake
DATA
CONSUMPTION &
ANALYTICS

6
Common Enterprise Mainframe Objectives
Leverage Mainframe Data for Analytics and Reduce TCO
Provide greater agility and
insight to the business
Unlock mainframe data
and reduce MIPS charges
Enable cloud migrations
and analytics

7
Requirement Challenge
MIPS Cannot increase MIPS consumption
Low Latency Complexity of achieving low latency with DB replication
Platform Flexibility Managing multiple legacy and modern platforms
Time to Value Realizing analytics value and quick wins from new analytics
investments
Efficiency Manual, time consuming process of landing, staging and reconciling
data
Mainframe Needs and Challenges

8
Data Warehouse Automation
Streaming Data Pipeline Automation
Design, Manage & Monitor
Qlik Data Integration Platform
CDC Streaming
Azure
SQL DW
Amazon
Redshift
Managed Data Lake Creation
Generate
Change Data
Streams
Deliver
To Clouds,
Lakes…
Refine &
Merge
For Analytics,
AI/ML, Data
Science…
AI/ML
Analytics
Data
Science
Model
Commit
Conform
Consume
Catalog
Shop, Organize & Provision
Catalog
Shop, Organize & Provision
RDBMS
Data Warehouse
Files
Mainframe
SAAS
APPS
SAP
Amazon RDS Azure SQL DB
Google Cloud SQL

9
Qlik and AWS better together
RDS (All)
S3
EMR
Kinesis
Redshift
Snowflake
Databricks
DBaaS
STORAGE
HADOOP
STREAMING
DWaaS
OTHER DWaaS
SPARK
SOURCES
 Oracle
 SQL Server
 DB2 iSeries
 DB2 z/OS
 DB2 LUW
 MySQL
 PostrgeSQL
 Sybase ASE
 Informix
 SAP HANA
 ODBC
 DB2 z/OS
 IMS/DB
 VSAM
 COBOL Copybooks
 ERP
 CRM
 SRM
 GTS
 MDG
 SAP ECC - HANA
 SAP HANA (database)
 (on Oracle, SQL, DB2 LUW, HANA)
Database Mainframe SAP
 Exadata
 Teradata
 Netezza
 Vertica
 Pivotal
 Amazon RDS
(SQL Server, Oracle,
MySQL, Postgres)
 Amazon Aurora (MySQL)
 Amazon Redshift
 Azure SQL Server MI
 Google Cloud SQL (MySQL,
PostgreSQL)
 Salesforce
EDW
Cloud
SaaS
 Delimited
(e.g., CSV, TSV)
Flat Files

10
Scaling Data & Analytic Workloads on AWS
RDBMS FilesMainframe SAASAPPSSAPData Warehouse Data Lake ODS
Amazon
DynamoDB
Amazon
Aurora
Amazon
Redshift
Amazon
RDS
Amazon
EMR
Amazon
Kinesis
AWS CloudFormation
Data Lakes
Amazon
S3
Qlik Data Integration
Quickly GET Data to AWS
Database Replication (CDC) Data Warehouse/Lake Automation
Qlik Data Analytics
Easily USE Data on AWS
Conversational
Analytics
Mobile
Analytics
Guided
Analytics
Self-Service
Analytics Reporting
& Alerting
Embedded
Analytics

11
Steve Steuart
Global Lead of Mainframe migration
and modernization

12
the broadest and deepest cloud platform

13
Use Case 1: Mainframe Data Analytics on AWS
Relational Hierarchical Data files
Transaction Manager
Mainframe
Amazon Redshift
data warehouse
Analysis Dashboards
Amazon S3
data lake
Amazon EMR
analytics
Amazon
QuickSight
Legacy
requests
Amazon
Aurora
Replicate

14
Use Case 2: Mainframe Data-Driven Augmentation
Transaction Manager
Mainframe
Amazon Kinesis
Streams
Amazon
Aurora
Amazon ECS
containers
Amazon API
Gateway
Amazon Lambda
microservices
Amazon
Machine Learning
Alexa Skill
Mobile Voice
Legacy
requests
Amazon
Redshift
Replicate

15
Use Case 3: Mainframe Data Workload Offload
Transaction Manager
Mainframe
Amazon Kinesis
Streams
Amazon
Aurora
Amazon ECS
containers
Amazon API
Gateway
Amazon Lambda
microservices
Offloaded
workload 1
Offloaded
workload 2
Legacy
requests
Elastic Load
Balancing
Amazon EC2
instances
Amazon
RDS
Replicate

16
meets or exceeds mainframe workloads requirements
High Security AWS top priority – Encryption everywhere – AWS Key Management Service – AWS CloudHSM – AWS VPN – AWS Direct
Connect – AWS IAM central access control – AWS CloudTrail central auditing – Virtual Private Cloud (VPC) – VPC Flow Logs
– Security Groups – Network Access Control Lists – AWS Config – Organizations Service Control Policies – Amazon
Inspector – Amazon GuardDuty – AWS Web Application Firewall – Automated reasoning technology – And more
High Availability Redundancy across Availability Zones (AZ) – Replication across Regions – Inter-AZ and inter-regions load-balancing –
Health-checks and automatic restart – Managed data synchronization and replication – Amazon Aurora 6 data copies across
3 AZ – Aurora Multi-Master – Aurora Global Database – Smaller blast radius with horizontal redundancy
Scalability and
Elasticity
Horizontal scalability with virtually unlimited capacity – Dynamic elasticity with Auto Scaling Groups – Vertical scalability with
EC2 instance type wide selection – Vertical scalability up to 224 physical CPU cores – Scalability across AZ and Regions –
Caching with Amazon ElastiCache or CloudFront – Right tool for the right job with many database types (no-SQL, graph,
document…)
System Management Amazon CloudWatch monitoring – CloudWatch Logs – Compliance with AWS Config – AWS Control Tower – AWS
Organizations – AWS Systems Manager – AWS Service Catalog – AWS OpsWorks – AWS trusted Advisor – AWS Backup –
Cost Allocation Tags – AWS License Manager – AWS Budgets – AWS Marketplace – AWS X-Ray – Reduced administration
with fully managed services
Cost Optimization Optimized pricing with Spot Instances, Reserved Instances, Savings Plans – Pay-as-you-go with Per Second Billing – Right-
sizing with EC2 type wide selection – Dynamic allocation with Auto Scaling Groups – AWS Cost and Usage Report – Cost
Explorer
Agility Continuous Integration and Continuous Deployment with AWS Code* – Infrastructure-as-code – AWS CloudFormation
automation – AWS Cloud Development Kit – Amazon Machine Image (AMI) – Microservices on containers or serverless
computing – Cloud speed with managed services – Large choice of services and frameworks – Rapid growth of AWS
innovations

18
REPLICATE
IN-MEMORY
FILTER
DATA LAKE
RDBMS
DATA
WAREHOUSE
FILES
MAINFRAME
TRANSFORM
PERSISTENT
STORE
Log Based CDC
BATCH
INCREMENTAL
BATCH
RDBMS
DATA
WAREHOUSE
STREAMING
FILES
DATA LAKE
Data Replication Architecture

19
Replicate Task
Source Endpoint
Replicate Task: High Level Logical Overview
Source DB Target
Transaction Logs
Source Tables Target Tables
Change Data
Capture
Source
Unload
In-Memory Processing Target Endpoint
Stream
Apply
Target
Load
Filter
and
Transform

20
Mainframe Customer Examples
“Qlik Replicate enables us to build agile
applications and analytics in the AWS
Cloud, dealing efficiently with the volume
and velocity of our mainframe data.”
- Donovan Stockton, Platform Owner, Cloud Data as a
Service, Vanguard
Challenge:
To have efficient and real-time access data from
large mainframe systems to cloud
Solution:
Near real-time mainframe data into AWS with
Qlik Replicate
Results:
20 million rows of data moved per hour, on avg
Over 60 million rows of data per hour during
peak demand.
200% Increased cloud adoption year-on-year
30% Reduced compute & build costs

21
Non-disruptively extract MF data for analytics
Transaction log-based CDC minimizes PROD impact
Propagate live updates to target at near-zero latency
REAL TIME REPLICATION FOR MAINFRAME
Improve cost and efficiency
Eliminate disruptive full loads
Reduce MIPS consumption
Integrate real-time with all major end points
DB2 z/OS, VSAM, IMS/DB sources
Data lake, cloud and streaming targets

22
To learn more…
qlik.com/aws

Qlik.com/DataIntegration
Qlik.com/contact
mainframe@amazon.com

Slides: Proven Strategies for Hybrid Cloud Computing with Mainframes — From AWS and Qlik

More Related Content

What's hot (20)

Similar to Slides: Proven Strategies for Hybrid Cloud Computing with Mainframes — From AWS and Qlik (20)

More from DATAVERSITY (20)

Recently uploaded (20)

Slides: Proven Strategies for Hybrid Cloud Computing with Mainframes — From AWS and Qlik