Db2 tools

October 1st, 2018
The Data Console Strategy
Peter Kohlmann
Principal Offering Manager, Database Tools
kohlmann@ca.ibm.com

Safe Harbor Statement
Copyright © IBM Corporation 2018. All rights reserved.
U.S. Government Users Restricted Rights - Use, duplication, or disclosure restricted by GSA ADP Schedule Contract with IBM Corporation
THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS
WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS
PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON
CURRENT THINKING REGARDING TRENDS AND DIRECTIONS, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT
NOTICE. FUNCTION DESCRIBED HEREIN MY NEVER BE DELIVERED BY I BM. IBM SHALL NOT BE RESPONSIBLE FOR ANY
DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.
NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES
OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY
AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS AND/OR SOFTWARE.
IBM, the IBM logo, ibm.com and DB2 are trademarks or registered trademarks of International Business Machines Corporation
in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence
in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks
owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks
in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at
www.ibm.com/legal/copytrade.shtml

Db2 Tech Talk:
IBM Integrated Analytics System
and the Data Console Strategy

Agenda
• Hybrid Data Management Tools Portfolio
• Data Management Console Strategy and Direction
• What’s new and coming for the IBM Integrated Analytics System (IIAS)
console
• Time Permitting: A deep dive into monitoring and problem identification
for the DSM and the consoles with a focus on IIAS

Backup and Recovery
Improve data availability, mitigate risk
and accelerate administrative tasks
Monitor and Optimize
Workloads
Identify potential performance
issues along with optimizing
queries and workloads
Administer Databases
View and act on the database
objects as well as create and
manage database jobs.
Design Data Models
IBM provides a collaborative
data design solution
Access and Connect to Data
Provides distributed client and
application access to IBM Data
stores
Develop Data Applications
and stored procedures using
Data Studio
Run and optimize SQL
Build, format, explain and run SQL statements
Core Database tools for the entire Data Lifecycle
Data Server
Manager
and
Consoles
Advanced
Recovery
Feature
InfoSphere
Data
Architect
Db2 Connect
DS Driver
DS Gateway
for ODATA
Data Studio
& pureQuery

What can you do today?
Here and Now

Data Server Manager Enterprise: Core Capabilities
Latest: DSM 2.1.5
• Monitoring and Alerts
• SQL Tuning and Execution
• Database Administration
• Configuration Management

Simple
• Easy to access
• Easy to connect
• Easy to learn
Scalable
• Scale from one to hundreds of databases
• Scale from an hour to years of data
• Scale from one to dozens of users
Smart
• Smart monitoring
• Smart alerts
• Smart high availability and scalability
Why Data Server Manager?

One Pane of Glass
• Zero in on the servers that need
attention
• Drill down for detailed analysis
DSM Enterprise Triage

Identify what is outside
normal behavior
Understand Normal and Unusual
Behavior

Consoles Powered by Data Server Manager
Cloud Convenience with Appliance Simplicity – Convergence Goal
Walk up and use:
• Database and System Monitoring
• Database Object Management
• Data Virtualization
• Load Data
• Backup and Restore
• DR
• Security
• Connection Information

§ Data Server Manager lets you manage all
your IBM Hybrid Data Management
servers from a single pane of glass
§ Alerts and Key Performance Indicators at
a glance
§ Dive into detailed monitoring and
management for individual databases and
servers with a single click
-Db2 for LUW
-Db2 on Cloud
-Db2 Warehouse Private
-Db2 Warehouse on Cloud
-IIAS
Data Server Manager for Consoles

Download
Get the latest version of DSM from our
marketplace page - ibm.biz/IWantMyDSM
Content Catalogue
Blog: ibm.biz/DSMBlog
DSM Security: ibm.biz/DSMSecurity
DSM Knowledge Center: ibm.biz/DSMKnowledge
Unlock DSM’s full potential by
accessing our resources section
Data Server Manager resources

Data Server Manager Blog - ibm.biz/DSMBlog
New HDM Blog: ibm.biz/HDMDSMCommunity
Articles on:
• Mining the DSM History
• Access to Db2 from Python Notebooks
• Historical vs Real-time Monitoring
• What’s new in DSM
• Accessing Remote tables to build a virtual warehouse
• How DSM determines what is normal
• Individual Statement History
• Workload Monitoring
Latest Article:
• “Can DSM monitor itself?”

Were are we going
IBM Data Management Console Vision

Adaptive
Cognitive
Change how you manage your system
• Predicts so you don't have to react
• Understand normal vs abnormal systems
• Manage your systems like a
Data Scientist
• Build Apps in Minutes
An experience that changes when you
need to
• Adapts to different users and their
behavior i.e. DBA, SYS Admin, App
Dev, Data Scientist
• Move data and applications
between ground and cloud
• Control changing workloads
and competing users
Enterprise Ready
The cockpit for the IBM Hybrid Data Enterprise
• Enterprise-wide seamless experience from
ground to cloud
• See your whole enterprise on one pane of
glass
• Ensure availability, security and recoverability
2018-2019
Inclusive
One experience for:
• Db2 on Premises & Cloud
• Db2 Warehouse Private
• Db2 on Cloud & Db2 Warehouse on Cloud
• BigSQL
• IIAS
• IBM Cloud Private for Data
Virtual Data Access
• Make remote data act like local tables
• Wide data source support
Open Integration
• Open Source databases: PostgreSQL and MongoDB
• Hadoop through BigSQL
Data Management Console
Vision for Hybrid Data Management

Data Management Console Principals
• Design driven unified experience across HDM
• Composable services architecture
• Delivered to fit the offering and the user
• Upgrade from Data Server Manager like a fixpack

At the leading edge of our console evolution
IBM Integrated Analytics System

Introducing the IBM Integrated Analytics System
A Next Generation Hybrid Data Warehouse That Does Data Science Faster
Cloud-ready to support multiple workload
deployment options
Built-in IBM Data Science Experience to
collaboratively analyze data
Optimized for high performance to support
the broadest array of workload options for
structured and unstructured data in your
hybrid data management infrastructures
Reliable, elastic and flexible system
that reduces and simplifies
management resources
Real time analytics with machine learning
that accelerates decision making, bringing
new opportunities to the business – ready
for business analyst and data scientist
Leverages a Common SQL Engine for
workload portability and skill sharing across
public and private cloud
http://www.ibmbigdatahub.com/blog/dispelling-myths-about-ibm-integrated-analytics-system

Write Once, Run Anywhere with a Common SQL Engine
IBM Hybrid Data Management solutions
Anchored by a Common SQL Engine enabling true, highly scalable
hybrid data warehousing solutions with portable analytics
Application agility
Write once, run anywhere. One ISV
product certification for all platforms
Operational compatibility
Reuse operational and
housekeeping procedures
Standardized analytics
Common programming model for
in-DB analytics
Common Skills
One skill set for all deployments. Drive higher
efficiencies and portfolio rationalization
Licensing
Flexible entitlements for business
agility and cost-optimization
Integration
Common Data Virtualization capabilities
for query federation and data movement
Managed Public
Cloud DBaaS
—
Db2 (Warehouse)
on Cloud
Software
defined warehouse
on-premises or
in cloud
—
Db2
warehouse
Dedicated
analytics
appliance
—
IBM Integrated
Analytics System
Custom
deployable
database
—
Db2
Open source
Hadoop with
Hortonworks
—
Big SQL

What will be available in the near term
Coming Soon

Rapid Triage and Personalization – Today for IIAS
• tri·age (trēˈäZH) – Assign a degree of
urgency
• Each widget tells a story with several
metrics
• Is my system available? Running
short or long statements? Doing a lot
or a little work? In conflict? Running
out of resources? Spending time in
the wrong area?
• Together, the widgets tell the whole story
• Drill down to details
22

Options
• db_backup
• db_restore
• Web console
Backup levels
• full Online backup
• full Offline backup
• Online incremental backup
Location
• file system
• Spectrum storage
Ability to
schedule daily
and weekly
backups.
Backup and Restore – Today for IIAS

Protecting from Failures
• Write to logs
• Issue Alerts when a failure of any kind occurs
• Mouse-over capability to determine status and next steps
Waiting for next – Connection OK Connection is broken
Disaster Recovery – GPFS AFM – Coming Soon for IIAS

Move tables or schemas from
IIAS to Db2 Warehouse on
Cloud
• IIAS can be the hub with spokes to all your
development or data mart systems on the
Cloud
• A few simple clicks
• Automatic table mapping
• Append or Replace Data
• Repeatable
Lift to Cloud – Coming Soon for IIAS

Continuously Replicate Tables
between IIAS Systems
• Create multiple replication processes
(“Applications’)
• Easily select one or thousands of tables
to replicate
• Built in monitoring and alerting
• Future support for
Db2 Warehouse systems
Table Replication – Coming Soon for IIAS

Coming to Db2 Warehouse
on Cloud, Db2 on Cloud,
Db2 Warehouse Private and
BigSQL
Rapid Triage in IIAS

Database Widgets - Availability
Answers the questions:
• Is the system available over time?
• Are there issues that need immediate attention?
• Is the system at risk?

Database Widgets - Throughput
Answers the question:
• How much work is the system doing?

Database Widgets - Responsiveness
• How quickly is the system processing statements?

Example of Open Rest API
coming first to
Db2 Warehouse on Cloud

Database Widgets – Database Resource Usage
• Is this resource a bottleneck in the system?

Database Widgets – Contention
• Are my users and applications holding each other up?

Database Widgets – Time Spent
• Where is my database spending its CPU time?

Hardware System Monitoring Widgets – Just for IIAS
Answers the questions:
• Are all my software services running?
• Is my hardware running and healthy?
• Are any OS level resources a potential bottleneck?

Hardware System Monitoring Widgets
What happens if the hardware monitor goes down?
You may still be able to monitor your database.
You will see a message in the hardware widget letting you know that the console cannot
talk to the hardware monitor.

Drilling down and problem determination
In IIAS, DSM, and Db2 Warehouse Private and BigSQL
Selected real time monitoring for On Cloud
Detailed Monitoring

Detailed Metric Collection
What metrics are monitored?
Connections, Statements, Locking, Throughput, Memory, I/O and
Storage

Normal range and Automatic Baseline
• “Normal” is when the current value is within this range
of values.
• Maximum and minimum values are also tracked
• Automatic baselines are dynamic. There is no pruning
involved. IIAS uses “circular buffering” to keep the
latest 5 weeks for comparisons.
• The Normal range appears as a blue field behind the
monitoring graph

User-defined Baselines
• Enables the user to browse existing Automatic
Baselines and save copies of their favorites
• Multiple saved baselines supported
• User names each saved baseline
• Saved baselines are only deleted by user, not by
pruning
• User can specify a saved baseline to show as
“normal” and used for Smart Alerts
40

Time Spent Metrics
• Explore where the database is spending time:
• SQL Execution
• IO
• Lock Waits
• Other Processing and Waits
• Interactive Drill down on each aspect

Monitoring With Workloads
• The database engine classifies every new statement by workload
• You can define workloads by Client IP, Application Name or user name
• DSM_WORKLOAD captures all the monitoring statements from the console
• Select Database to see a single view independent of workload
• Select All workloads to see a combined view of all workloads
• Or select an individual workload
• Define Workloads using
• Administer->Workloads

Historical Reports
• From the Overview Monitoring Page select History
• You can then select from a number of HTML or PDF or EXCEL reports
• Select the time period for the report

Database Partitions
• Review the current CPU, Sorts and I/O across all the partitions in your database
• The normal range for each partition is displayed using the indicator bars

Table Performance
• Identify HOT tables
• Top 500 tables access since the database was activated
• Not comprehensive as a report to identify unused tables

Monitoring SQL
In IIAS, Db2 Warehouse
Private, and DSM
SQL Statements

Statements (real time mode)
• In-flight executions
• Shows “currently” executing statements
• Package Cache
• Shows the contents of the Package Cache as of the last in-memory collection.
• Stored Procedures
• Monitor statements within a stored procedure
• Requires database configuration parameter MON_RTN_DATA to be set to BASE

Package Cache Monitor
• The Package Cache stores the most commonly used SQL Statements
• Easily identify statements that use a lot of resources or take long to run
• Review by the Average or Total values for each statement
• Viewable in current real time or as a view across a historical time period

Considerations for processing the package cache
statements
• Top N Entries
• Only the top N entries from the package cache are displayed.
• Averages versus Totals
• The statistics maintained for each statement by the package cache are accumulated
as totals.
• Column displays
• Can customize what metrics are displayed
• Can sort by a specific metric.

Statements > Package Cache (history mode)
• The big difference between real-time mode and historical
mode
• Real time mode shows the current values stored in the package
cache
• For historical mode, the delta is calculated between the metric
values of the current sampling and the sampling before it.
• Parameter Markers Statements with parameters are
grouped together using parameter markers “?” and can be
used in either mode.

Statements (History mode)
• Individual Executions
• Show statements obtained from the ACTIVITY and ACTIVITYSTMT event monitors.
Preconfigured in IIAS
• Package Cache*
• Contains the “most interesting” SQL statements obtained from the package cache in
the duration or time period specified
• Stored Procedures
• Not available in history mode
* IIAS captures to top 20 “interesting” statements to capture. A SQL statement is
deemed interesting if the statement is:
• Executing a large number of times
• Running a long time
• Consuming large amounts of resources (CPU, IO, memory)
• Waiting a long time
• Reading a lot of data
• Returning a lot of rows
• Escalating a lot of locks

Individual Statement History Monitoring
• Drill down from the Database Responsiveness Widget
• The history of individual statements is automatically collected
• Which statements are collected is controlled through workload definitions
• Group statements by different columns to get totals and average values

Controlling Collection through Workloads
• Workloads define when to collect individual statement details
• IIAS turns on monitoring by default for everything excepts statements generated by the console
itself
You can add your own workload definitions by selecting
Admin->Workloads->Add

Direct SQL Access to Statement History
• IBMOTS.SQL_DIM: Contains the complete SQL statement for both the Package Cache
and Individual Statement monitoring
• IBMOTS.SQL_FACT: Contains a record for each time a specific statement was found in
the Package Cache
• IBM_RTMON_EVMON.EVENT_ACTIVITY: Contains a record for each statement
where records were found in the Activity Event Monitor tables

Only for IIAS
System Monitoring

System Monitoring - Hardware
Review the overall hardware status and the details for each component.

System Monitoring - Software
Review the status for each software service

System Monitoring - Storage
Current File system status for the OS

System Monitoring – Data Slices
Review the status of each data slice in the system

System Monitoring - Notifications
• See the full history of all software and hardware events
• Choose to only see current issue
• And call home status

How IIAS can ask for help
Call Home

Call Home – System Configuration
System and Customer Information – Who am I?

Call Home – Contacts
Add Telephone, Cell or Email Contact Information

Call Home – Mail Server Configuration
Mail Server

Call Home – Mail Server Configuration
Communication Rules

It slices and dices
For all consoles and DSM
But wait, there’s more

IIAS Object Administration
• Create, Alert and Drop Objects
• Explore Object Dependencies
• Understand Distribution of Data by Table
• Compare Tables in different schema
• Manage object user and group privileges
• Revalidate indexes
• Create MQT to cache query results
• Create connects to remote data sources and make remote data look like local tables
• Define Aliases and Schema
• Manage Application Objects
• Define Workloads for more precise monitoring
All Objects for
DSM
Column oriented
table objects for
DB2 Warehouse

Federation - ibm.biz/DSMRemoteTables
Federated Object Management
Statement Optimization and Monitoring
Simplify application development
Supported Data Sources in IIAS
Make remote tables
look, act and feel like local tables
• Provide access to application developers and
data analysts to federated remote data sources
that look and act like they are local tables or
views.
• A guided interface to walk through the creation of
Wrappers, Servers, User Mappings
and Nicknames
Manage all Federation objects
• Create, Drop and Alter Wrappers Servers, User
Mappings and Nicknames
• Manage privileges to access federated objects
Monitor and optimize federated statements
• Monitor the federated stages of queries,
including time spent metrics
• Federation details in access plan graphs
• MQTs to
accelerate
performance
Coming to Db2
Warehouse on Cloud

SQL Editor
• Live Parsing Editor with error detection
• Save Favorite SQL fragments and statements as templates
• Share with other users
• Save scripts and results on server or to desktop
• One button download results to CSV or Excel
• Search and Replace
• Statement Formatting

Hint: Show and Hide System Objects and Statements
• You can use the SYS icon to hide and show information that is part of the database
engine
• Statements generated by the monitor can be excluded
• Catalog Objects can be hidden

Learn more
IBM Hybrid Cloud / 2018 / © 2018 IBM Corporation
Hybrid Data Management DSM Blog and Community:
ibm.biz/HDMDSMCommunity
Download DSM: ibm.biz/IWantMyDSM
Older Blog: ibm.biz/DSMBlog
DSM Security: ibm.biz/DSMSecurity
This presentation and recording: ibm.biz/Db2TT_Aug2018
IIAS: ibm.com/marketplace/integrated-analytics-system

2018 IDUG Db2 Tech Conferences
India
Bengalore, March 27, 2018
Chennai, March 29, 2018
Philadelphia, Pennsylvania, USA
April 29 – May 03, 2018
Brazil
August 13 & 15, 2018
Sydney, Australia
September 11-12, 2018 www.IDUG.org
Experience Data With IDUG
The Worldwide Db2 User Community
@IDUGDB2
A Community Focused on and
Driven by Users
• Network & learn from nearly 7,000 members
• Top product developers, industry experts,
renowned consultants, fellow users & vendors
• Find your IDUG community close to home – we have
regional user groups across 40 countries
• ¿hablas español? Tenemos contenido para ti
Your Go-To Resource for Online
Education About Data
• 100s of blog posts covering the latest topics such
as Db2 on all platforms, analytics and SQL
• On-demand webinars available 24/7 on the Db2
Tech Channel
All by and for DBAs, IT architects,
developers and data scientists
We Are…
Malta
November 04-08, 2018

Db2 tools

More Related Content

What's hot (20)

Similar to Db2 tools (20)

More from ModusOptimum (12)

Recently uploaded (20)

Db2 tools