© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
Enterprise InformationEnterprise Information
Management [EIM]Management [EIM] ––
Proven Techniques ForProven Techniques For
SuccessSuccess
Robert J. Abate, CBIP
Global Managing Principal, RCG ITGlobal Managing Principal, RCG IT
Monday, March 17th, 2008 5PM – 6PM
AgendaAgenda
Presentation TopicsPresentation Topics
EndEnd--ToTo--End Data Management [EIM]End Data Management [EIM]
Requires A Integration ArchitectureRequires A Integration Architecture
Integrating Governance & QualityIntegrating Governance & Quality
Proven Techniques For SuccessProven Techniques For Success
Key TakeKey Take--AwaysAways
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
EndEnd--ToTo--End DataEnd Data
Management [EIM]Management [EIM]
Robert J. Abate, CBIP
Global Managing Principal, RCG ITGlobal Managing Principal, RCG IT
Monday, March 17th, 2008 5PM – 6PM
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
4
What Is Enterprise Information Management?
Intersection Of Strategy, Architecture & Delivery Of InfoIntersection Of Strategy, Architecture & Delivery Of Info
It Is A Managed Information Environment (with End – To – End Data
Management) Incorporating:
Where Data Is Known To Be Of High Quality
Information Is Wrapped With (Or Having) Governance
Contain A Single Source Of Truth
Flexible Structure or “Architecture”
Responsive To Change
Supportive Of Integration And Delivery Of Information
A flexible structure is important because:A flexible structure is important because:
Business needs are changing at an accelerating pace
It is the core feature that allows IT to be responsive in meeting new
business requirements
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
5
The Information Foundation: Data Management
This requires:This requires:
Metadata managementMetadata management for compliance enforcement, audit support,for compliance enforcement, audit support,
analysis, and reportinganalysis, and reporting
Master dataMaster data integration and controlintegration and control
NearNear--real timereal time business informationbusiness information
Source data managementSource data management for controlling data quality at thefor controlling data quality at the
transaction leveltransaction level
Effective governanceEffective governance for a successful managed data environmentfor a successful managed data environment
IntegrationIntegration of analytics, reporting, and transaction controlof analytics, reporting, and transaction control
ControlControl of business processes and information usageof business processes and information usage
Unstructured dataUnstructured data needs to be addressed as wellneeds to be addressed as well
The role of data management in the business is evolving from
“A Single Source Of Truth” to becoming
“The Information Assurance And Delivery Mechanism”
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
6
How Do You Build One?
Flexible Reference / Data ArchitectureFlexible Reference / Data Architecture
Controlled Data LifecycleControlled Data Lifecycle
Metadata / Information TaxonomyMetadata / Information Taxonomy
Master Data ManagementMaster Data Management
ModelModel--Driven DevelopmentDriven Development
Data Integration ToolsData Integration Tools
Extraction, Transformation & Loading [ETL]
Data Delivery Services
Information DeliveryInformation Delivery
Reporting & Analytics
Security
Dimensional Models (Data Warehouse)Dimensional Models (Data Warehouse)
Conformed Dimensions
Data GovernanceData Governance
Data QualityData Quality
Data Profiling
Exception Handling
Action Time
ValueLost
Action Time
ValueLost
Time
Value
Action
Business Event
Taken
Business Event
Taken
Capture
Latency
Analysis
Latency
Decision
Latency
Data Ready For Analysis
Information Delivered
Capture
Latency
Analysis
Latency
Decision
Latency
Data Ready For Analysis
Information Delivered
© Chart: Richard Hackathorn, Bolder Technology, Inc., 2007. All Rights Reserved. Used with Permission.
Data
Sources
Finance
XLS
POS
ERP
SCM
ERP
Transaction Level Detail
Granular
Denormalized
Integrated
Current & History
Reporting
Transaction Level Detail
Granular
Denormalized
Integrated
Current & History
Reporting
Metadata
(Extract/Transform/Load)
-- Apply business rules
-- Manage metadata
-- Consolidate data
ETL / Data
Integration Hub
Transaction Level Detail
‘Atomic’-level Data
Normalized (3NF)
Integrated
Temporary
Integration
Aggregates & KPIs
Dimensional
Drilling
Conformed
Analytics
ETL
Server
Dashboards
Predictive &
Behavioral
Analytics
Reporting &
Ad hoc Queries
Database
Server Reporting
Server /
Portal
Scoring
Algorithms
Metadata Inquiries
Others
A “permanent” Integration Data Store can
facilitate data conversions and system
upgrades because it contains data already
standardized, cleansed, and integrated.
*
*
Data DeliveryEnterprise Data
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
7
What Is A Flexible Architecture?
Definition of a flexible architectureDefinition of a flexible architecture (from Dictionary.com)(from Dictionary.com)::
Architecture:Architecture: “Orderly arrangement of parts; structure”
Flexibility:Flexibility: “Responsive to change; adaptable”
Responsive:Responsive: “readily reacting to events or stimuli”
Therefore, a flexible architecture is one that is:Therefore, a flexible architecture is one that is:
Readily adaptable to events or stimuli that require it to change
Designed to provide the key capabilities necessary for BI
today
And an inflexible architecture is one that is:And an inflexible architecture is one that is:
Hard or time-consuming to change
Unable to incorporate new capabilities when needed
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
8
Partner with the Business: Focus on Impacting the Bottom Line
Operational BI
Strategic BI
Simple metrics:
• Customer satisfaction
• Inventory turns
• Quality measurement
• Financial reporting
Dashboard with metrics:
• Customer profitability
• Product profitability
• Operational performance
Performance
Management
Performance
Management
Performance
Optimization
Performance
Optimization
Complex metrics:
• Market success
• Customer relationship
• Information innovation
• Value creation
Value to the
Corporate
Bottom Line
These inform different kinds of decisions at
different levels of the organization
Operational
Reporting
& Analysis
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
9
Best Practices & Trends…
Separation of Architectural Layers usingSeparation of Architectural Layers using ReferenceReference
ArchitecturesArchitectures ((““akaaka”” Architectural FrameworksArchitectural Frameworks))
Model Driven Development [MDD]Model Driven Development [MDD] with a focus on thewith a focus on the
business functions / processes to drive alignment withbusiness functions / processes to drive alignment with
business goals and objectivesbusiness goals and objectives –– ““akaaka”” BusinessBusiness
ArchitectureArchitecture
Providing requirements traceability
“Glue” or integration points for iterative development
Convergence OfConvergence Of Applications (SOA)Applications (SOA) && Managed DataManaged Data
Environments (Business Intelligence)Environments (Business Intelligence)
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
Requires An IntegrationRequires An Integration
ArchitectureArchitecture
Robert J. Abate, CBIP
Global Managing Principal, RCG ITGlobal Managing Principal, RCG IT
Monday, March 17th, 2008 5PM – 6PM
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
11
The Problem – Aligning Systems with the Business
1. As proposed by the project sponsor
2. As defined by the requirements doc’s
3. As designed by the Senior Analyst
4. As produced by the programmers
5. As installed at the user’s site
6. What The User Really Wanted!
This Cartoon Has Been Circulating In IT For Over 20+ Years In Various Forms…
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
12
The Importance of Requirements
““Analysts report that as many as 71% of projects thatAnalysts report that as many as 71% of projects that
fail do so because of poor requirements management,fail do so because of poor requirements management,
making it the single biggest reason for project failuremaking it the single biggest reason for project failure””
CIO magazine, November 15, 2005
Failure can also be defined as:Failure can also be defined as:
““Analysis ParalysisAnalysis Paralysis”” oror ““Scope CreepScope Creep””
A never ending test cycleA never ending test cycle
Redevelopment of systems after implementationRedevelopment of systems after implementation
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
13
Relationship Between Business Architecture & IT
Enables
Dictates
Enables
Dictates
Supplements
Dictates
Dictates
Business /
SBU Strategy
Business
Architecture
Business
Projects &
Programs
Implies
Dictates
Technology
Strategy
App. & Info.
Technology
Architecture
Technology
Projects &
Programs
Implies
Contributes&Supports
Contributes&Supports
Contributes&Supports
There Is An “Implied” Relationship Between Business & Technology Building Blocks
Business Architecture is a comprehensive framework for the representation of the business processes,
information, rules, and organizational designs that result directly from the business strategy. It provides a
coherent graphical means for the capture, management, and communication of business knowledge and
needs to support business solutions in the form of actionable requirements.
IT Confined By
Project Budgets
And Define App/Info
Architecture
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
14
Define business requirements in aDefine business requirements in a
manner thatmanner that aligns IT and Businessaligns IT and Business
Requirements areRequirements are graphicalgraphical
(pictures) and(pictures) and easy to understandeasy to understand
Supports both application andSupports both application and
information architecturesinformation architectures
Vendor agnosticVendor agnostic
Supports rules established inSupports rules established in
Business ArchitectureBusiness Architecture
Provides forProvides for checking consistencychecking consistency
and completeness throughoutand completeness throughout
Supports management withSupports management with qualityquality
control and governancecontrol and governance
ModelModel--driven approachdriven approach should use ashould use a
repositoryrepository--based toolbased tool
The Many Benefits Of Model Driven Development
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
15
MDM Has Full Traceability - Through Design, Build and Test
Design
&Build
Require
ments
Testing
Requirements and
Acceptance Criteria
Design Component and/or
Use Case that
Satisfies each Requirement
Test Groups, Test Cases
Of Each Design Component
Requirements are satisfied by a design component and each componRequirements are satisfied by a design component and each component isent is
assigned to a test group with test cases. Thus test cases testassigned to a test group with test cases. Thus test cases test requirements.requirements.
Test results can then be measured against acceptance criteria.Test results can then be measured against acceptance criteria.
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
16
Reference Architecture And It’s Views
BUSINESS ARCHITECTURE
Mission / Vision, Business Requirements, Business Rules, Organizational
Structure, Critical Success Factors, Business Process Design
APPLICATIONS ARCHITECTURE
Process Alignment, Services Definition, Services/Event Architectures, Custom
Application Development, Enterprise Applications Integration Components
INFORMATION ARCHITECTURE
Data Integration, Data Architecture, Master Data Management, Metadata
Management, Data Delivery Architecture, Dashboards, Performance
Measurement, Analytics , Business Intelligence, Enterprise Reporting
INFRASTRUCTURE ARCHITECTURE
Legacy (Non-Service Enabled) Applications
Networks, Servers / Hardware, Operating Systems
INTEGRATEDGOVERNANCE
Security,Data,SLAs,ServicesandITPortfolioManagement
QUALITYCONTROL
QualityAssurance,Testing,DataQualityManagement
Reference ArchitectureReference Architecture is a proven template for a solution domain thatis a proven template for a solution domain that
provides a common vocabulary with which to discuss designs andprovides a common vocabulary with which to discuss designs and
implementationsimplementations –– it frames solution delivery.it frames solution delivery.
Enterprise Architecture FrameworkEnterprise Architecture Framework describes a structure anddescribes a structure and
behavior for an organization's processes, information systems,behavior for an organization's processes, information systems,
personnel, and organizational subpersonnel, and organizational sub--units, aligning them with theunits, aligning them with the
organization's core goals and strategic direction.organization's core goals and strategic direction.
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
17
Reference Architecture – Artifacts / Techniques
StrategyStrategyProcessProcessFunctionFunctionTimeTimeLocationLocationDataDataOrganizationOrganization
WHYWHYHOWHOWWHENWHENWHEREWHEREWHATWHATWHOWHO
EnterpriseEnterprise
Infrastructure VisionInfrastructure Vision
Strategic NetworkStrategic Network
PlanPlan
System OperationSystem Operation
Diagram (SODDiagram (SOD--P)P)
Security ManagementSecurity Management
PlanPlan
Systems ArchitectureSystems Architecture
DiagramDiagram
Network ArchitectureNetwork Architecture
DiagramDiagram
InfrastructureInfrastructure
Calendar PlanCalendar Plan
Location Support PlanLocation Support PlanData ManagementData Management
PlanPlan
Data Capacity PlanData Capacity Plan
Data Archive PlanData Archive Plan
InfrastructureInfrastructure
Resource PlanResource Plan
CSFCSF’’ss
Business Goals /Business Goals /
DirectionDirection
Business PlanBusiness Plan
– BPM / BPI
– Continuity Plan
Policy CharterPolicy Charter
Business ProcessBusiness Process
Flow (SODFlow (SOD--L)L)
Use CaseUse Case
ScreensScreens
ReportsReports
Business Logic (ASD)Business Logic (ASD)
Business FunctionsBusiness Functions
(FSD)(FSD)
Business ProcessBusiness Process
(FSD)(FSD)
Business PrioritiesBusiness Priorities
Business TimeBusiness Time
DefinitionsDefinitions
Master ScheduleMaster Schedule
(Events & Triggers)(Events & Triggers)
Business LocationsBusiness Locations
LocationLocation--specificspecific
BusinessBusiness
Requirements &Requirements &
DefinitionsDefinitions
Entity RelationshipEntity Relationship
Diagram (EDD)Diagram (EDD)
Attribute DependencyAttribute Dependency
Diagram (ADD)Diagram (ADD)
KPI Structure DiagramKPI Structure Diagram
(KSD)(KSD)
Organization ChartOrganization Chart
(OSD)(OSD)
RolesRoles &&
ResponsibilitiesResponsibilities
System Operation Diagram (SOD-L)
System Operations Documentation
Disaster Recovery Plan
BusinessRequirementsdriveEnterpriseArchitectureandBusinessContext
Business
Architecture
Business PlanBusiness Plan
AlignmentAlignment
ApplicationApplication
PrioritizationPrioritization
ApplicationApplication
Replacement PlanReplacement Plan
Workflow ProcessWorkflow Process
ModelModel
ApplicationApplication
Architecture DesignArchitecture Design
Systems InterfaceSystems Interface
DesignDesign
ApplicationApplication
Architecture DesignArchitecture Design
Services Map ScreenServices Map Screen
DiagramsDiagrams
Application / ProcessApplication / Process
Event CorrelationEvent Correlation
DiagramsDiagrams
Location ProcessingLocation Processing
Workflow ModelsWorkflow Models
Semantic ModelSemantic Model
(Metadata)(Metadata)
Messaging DiagramMessaging Diagram
XML / XSLT DiagramsXML / XSLT Diagrams
SOAP / MessagesSOAP / Messages
Organization ProcessOrganization Process
Mapping RulesMapping Rules
User InterfaceUser Interface
DiagramsDiagrams
METADATA (repository-based): Application / Services Metaprocesses
Applications
Architecture
Enterprise ManagedEnterprise Managed
Data EnvironmentData Environment
Master DataMaster Data
Management PlanManagement Plan
BusinessBusiness
PerformancePerformance
ManagementManagement
Data QualityData Quality
MonitoringMonitoring
Data ExceptionData Exception
ProcessingProcessing
Data PerformanceData Performance
ManagementManagement
Enterprise DataEnterprise Data
Structures AuditStructures Audit
Enterprise ReportingEnterprise Reporting
& Analytics& Analytics
ArchitectureArchitecture
ETL / Data IntegrationETL / Data Integration
ArchitectureArchitecture
Data QualityData Quality
ManagementManagement
MetadataMetadata
ManagementManagement
Event DataEvent Data
Correlation DiagramsCorrelation Diagrams
Enterprise DataEnterprise Data
SynchronizationSynchronization
ScheduleSchedule
Data ControlData Control
StructuresStructures
Data Location &Data Location &
PhysicalPhysical
InterdependencyInterdependency
MapsMaps
Data Source OfData Source Of
Record (SourceRecord (Source--toto--
Target for EnterpriseTarget for Enterprise
Data Structures)Data Structures)
DimensionalDimensional
Hierarchies /Hierarchies /
TaxonomyTaxonomy
KPI / MetricsKPI / Metrics
DefinitionDefinition
Master Data ModelMaster Data Model
Logical / PhysicalLogical / Physical
Data ModelsData Models
Physical StructuresPhysical Structures
User AccessUser Access
AuthorizationAuthorization
Data Visualization &Data Visualization &
DeliveryDelivery
RequirementsRequirements
METADATA (repository-based): Business (Definitions, Access Rights…), Operational (Load Timings, Controls…) & Technical (Environmental…)
Information
Architecture
Infrastructure
Architecture
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
18
Reference Architecture – Logical Managed ComponentsINTEGRATEDGOVERNANCE
Security,Data,SLAs,ServicesandITPortfolioManagement
QUALITYCONTROL
QualityAssurance,Testing,DataQualityManagement
INFORMATION ARCHITECTURE APPLICATIONS ARCHITECTURE
Business Rules Engine
App / Svcs
Integration
Broker
Data
Rules
App
Rules
Services Mgmt
Event Correlation
Process Choreography
Svcs Orchestration
Services Mgmt
Event Correlation
Process Choreography
Svcs Orchestration
BUSINESS ARCHITECTURE
INFRASTRUCTURE ARCHITECTURE
Existing Application Backbone for Transaction Processing (includes Legacy Applications)
INFRASTRUCTURE ARCHITECTURE
Existing Application Backbone for Transaction Processing (includes Legacy Applications)
Central Metadata & Metaprocess Library
Data & Data Audit Metadata
Enterprise
Reporting
Integrated KPIs &
Dashboards
Data Quality,
Profiling & Cleansing
Enterprise
Reporting
Integrated KPIs &
Dashboards
Data Quality,
Profiling & Cleansing
ETL / Data
Integration
Hub
Enterprise Data Structures
Sync
Sync Sync
Integrated Trx Data Store
Trx Processing
Recent Data, Normalized, Fine-
grained, Unit-of-Work (UoW)
Processing
EDW & Data Marts
Reporting
Granular,
Denormalized,
Current / History
Analytics
Aggregates,
Dimensional,
Conformed
Master Data
Customer
Value, Control
Data, ID X-ref
Product
Value, Control
Data, ID X-ref
Orgn Chart
Value, Control
Data, ID X-ref
Enterprise Data Structures
Sync
Sync Sync
Integrated Trx Data Store
Trx Processing
Recent Data, Normalized, Fine-
grained, Unit-of-Work (UoW)
Processing
EDW & Data Marts
Reporting
Granular,
Denormalized,
Current / History
Analytics
Aggregates,
Dimensional,
Conformed
Master Data
Customer
Value, Control
Data, ID X-ref
Product
Value, Control
Data, ID X-ref
Orgn Chart
Value, Control
Data, ID X-ref
Applications / Services
Metaprocesses
Services Library
Application Svcs
Data Delivery Svcs
Messaging Svcs
Services Library
Application Svcs
Data Delivery Svcs
Messaging Svcs
StrategyProcessFunctionTimeLocationDataOrganization
WHYHOWWHENWHEREWHATWHO
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
Integrating GovernanceIntegrating Governance
& Quality& Quality
Robert J. Abate, CBIP
Global Managing Principal, RCG ITGlobal Managing Principal, RCG IT
Monday, March 17th, 2008 5PM – 6PM
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
20
Dedicated Management of Process and CommunicationsDedicated Management of Process and Communications
Data Governance – An Overview
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
21
What Is Data Governance Anyway…
Data Governance is what an organization does that yields:Data Governance is what an organization does that yields:
The highest quality decisions made at the lowest cost.
Increased efficiency in conducting/completing IT projects for the business.
Data Governance is NOT:Data Governance is NOT:
A project
Someone else’s responsibility
Intuitive
Something that happens on it’s own
Drivers For Data Governance:Drivers For Data Governance:
Lack of trust in the data
Reports utilizing same data do not report same totals or computations
Data not available in an easy manner
Multiple sources of data have to be gathered and manipulated
Data timeliness
Manual process to collect, analyze and provide results
Data integrity
Unknown filters, varying calculation/computations, fields used for data not indicative of field
names, data passed along from one person to another to another to another…..
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
22
Data Governance – Master Data Management Included…
A Comparison of Data Governance & MDMA Comparison of Data Governance & MDM
Master Data ManagementMaster Data Management
Brings together single, consistentBrings together single, consistent
view of data that may reside inview of data that may reside in
multiple source applications (CRM,multiple source applications (CRM,
ERP, financial applications, WebERP, financial applications, Web
servers) by maintaining a singleservers) by maintaining a single
source of critical data, then crosssource of critical data, then cross--
referencing and synchronizing datareferencing and synchronizing data
across all other missionacross all other mission--criticalcritical
applicationsapplications
Critical business data flows in realCritical business data flows in real--
time to multiple applications fortime to multiple applications for
single view of customers, supplierssingle view of customers, suppliers
and other dataand other data
Data GovernanceData Governance
StrategyStrategy –– Why? Goals andWhy? Goals and
objectives of organization (treatobjectives of organization (treat
data as an asset of organization)data as an asset of organization)
–– MDM is a goalMDM is a goal
Policies, Procedures, ProcessPolicies, Procedures, Process ––
What? Who? When? Where?What? Who? When? Where?
Provides structure,Provides structure,
communications, andcommunications, and
accountabilityaccountability
Implementation / ResolutionImplementation / Resolution ––
How? RealHow? Real--world application toworld application to
make it all happenmake it all happen
Beyond MDM: Regulatory Risk, Compliance, and SOARegulatory Risk, Compliance, and SOA
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
Proven Techniques ForProven Techniques For
SuccessSuccess
Robert J. Abate, CBIP
Global Managing Principal, RCG ITGlobal Managing Principal, RCG IT
Monday, March 17th, 2008 5PM – 6PM
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
24
Partner With The Business: Help Them Understand Themselves
Stage 4: ‘Perform’ (business results)
Actions: Structure KPIs & metrics
Focus: Dashboarding processes
Stage 3: ‘Structure’ (enterprise data)
Actions: Conform & Standardize Data
Focus: Standardizing data & terms
Stage 2: ‘Verify’ (EDW data)
Actions: Ensure Data Quality
Focus: Governing data & metadata
Stage 1: ‘Trust’ (in application data)
Actions: Collect & Organize
Focus: Gathering data for reporting
Stage 5: ‘Optimize’ (the business)
Actions: Establish Strategy & Goals
Focus: Improving market success
Departmental Usage
Enterprise Ready
Executive Driven
Reality
Perception
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
25
Develop Iteratively – It’s All About The Repeatable Process
A repeatable process implies an iterative methodologyA repeatable process implies an iterative methodology
Utilize open-ended architecture/design principles to facilitate reuse in future
iterations
Stagger iterations as shown to save time and money
Leverage best practices already available
Requirements must be thoroughly understood
Data profiling must be conducted in parallel with Requirements
Activities should take progressively lesser time during subsequent iterations
due to leveraging reusable components
Program Management and Governance will ensure that iterations meet
business requirements, minimize rework between iterations and proper
management of master data through its entire lifecycle
Requirements Architecture Development Deployment
Iteration 1
Iteration 2
Iteration …
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
26
Proven Techniques: Define Your Components
■ Data Management
– Master data
– Data standardization
– Data quality
– Metadata
– Data security, audit, and compliance
■ Data and delivery architectures
– Modeling and structuring data
– ETL data integration and data rules
– Data delivery services (SOA-compliant)
– Data technologies compatibility
■ Performance management
– KPI relationships and metrics structure
– Dashboard development and integration
■ Analytics management
– Enterprise reporting
– Analytics in transaction processing
– “Real-time” analytics
– Business activity monitoring (BAM)
INFORMATION ARCHITECTURE
Data Rules
Data & Data Audit Metadata
Enterprise
Reporting
Integrated KPIs &
Dashboards
Data Quality,
Profiling & Cleansing
ETL / Data
Integration
Hub
EnterpriseDataStructures
Sync
Sync Sync
IntegratedTrxDataStore
TrxProcessing
Recent Data,Normalized,Fine-
grained,Unit-of-Work(UoW)
Processing
EDW&DataMarts
Reporting
Granular,
Denormalized,
Current/History
Analytics
Aggregates,
Dimensional,
Conformed
MasterData
Customer
Value,Control
Data,IDX-ref
Product
Value,Control
Data,IDX-ref
OrgnChart
Value,Control
Data,IDX-ref
EnterpriseDataStructures
Sync
Sync Sync
IntegratedTrxDataStore
TrxProcessing
Recent Data,Normalized,Fine-
grained,Unit-of-Work(UoW)
Processing
EDW&DataMarts
Reporting
Granular,
Denormalized,
Current/History
Analytics
Aggregates,
Dimensional,
Conformed
MasterData
Customer
Value,Control
Data,IDX-ref
Product
Value,Control
Data,IDX-ref
OrgnChart
Value,Control
Data,IDX-ref
A Managed Data Environment
Data Delivery Svcs
Implies… Managed Data Components
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
27
Proven Techniques: Define Your Data Lifecycle…
© Richard Hackathorn, Bolder Technology, Inc., 2007. All Rights Reserved. Used with Permission.
Managing The Time-Value Curve
Action Time
ValueLost
Time
Value
Action
Business Event
Taken
Capture
Latency
Analysis
Latency
Decision
Latency
Data Ready For Analysis
Information Delivered
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
28
Proven Data Architecture – Data Validation & Verification
Data
Sources
Finance
XLS
POS
ERP
SCM
ERP
Transaction Level Detail
Granular
Denormalized
Integrated
Current & History
Reporting
Metadata
(Extract/Transform/Load)
-- Apply business rules
-- Manage metadata
-- Consolidate data
ETL / Data
Integration Hub
Transaction Level Detail
‘Atomic’-level Data
Normalized (3NF)
Integrated
Temporary
Integration
Aggregates & KPIs
Dimensional
Drilling
Conformed
Analytics
ETL
Server
Dashboards
Predictive &
Behavioral
Analytics
Reporting &
Ad hoc Queries
Database
Server Reporting
Server /
Portal
Scoring
Algorithms
Metadata Inquiries
Others
Because there are several
technologies covered by this
graphic, it is important that
they “play well together”
Data DeliveryEnterprise Data
1
2
3
4 5
6
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
29
Proven Techniques: Vision and Direction
Establish A Vision of theEstablish A Vision of the
FutureFuture
The Mission & Vision
The Stakeholders
BI and Reporting Differentiators
Operating Principles
Goals and Objectives for BI and
Reporting
Prepare a Roadmap for thePrepare a Roadmap for the
FutureFuture
Recommended Initiatives & Impact
Action Plans for High-Priority
Initiatives
Transition Strategy
Critical Success Factors for the
Transition
•DataQualityManagement,Audit&Control
•MasterDataManagement
•ScorecardDevelopment
•ETLManagementforSourceSystemChanges
•EDWArchitectureandEnhancement
•BusinessPerformanceManagement
• Marketing
• Advertising
• Customers
• Finance
• Employee
• Organization
• Transportation
• Distribution Centers
• Inventory
• Orders
• Vendors
• Sales
• Sales Transactions
• Products
• Stores Phase 1:
Sales &
Stores
CAPABILITY
Phase 2:
Inventory &
Orders
Phase 3:
Finance &
Employee
Phase 4:
Marketing &
Customer
Implementation, Phase by Phase, progresses over time
Ongoing,
Centralized,
EDW
Capabilities
and Program
Management
• Store and Product Master Data
• Product Hierarchy Reference Data
• Sales Scorecard Development
• Vendor and Order Master Data
• Vendor Scorecard Development
• Inventory Scorecard Development
• Employee Master Data
• Organization Reference Data
• Expenses and Margin Scorecard Development
• Complete Store Scorecard Development
• Customer Master Data
• Business Performance Analysis
• Profitability Analysis (stores, products, customers)
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
30
Proven Techniques: Project Selection Planning
Widely used dimensions
Low Complexity
Considerable overall value
MEDIUM PRIORITY – HIGH IMPACT
Scarcely used dimensions
Low complexity
Easy to implement
QUICK WINS – LIMITED IMPACT
Scarcely used dimensions
High complexity
Low overall value
LOW PRIORITY – LIMITED IMPACT
Widely used dimensions
High complexity
High overall value
HIGH PRIORITY – HIGH IMPACT
VIPS
Broader usage
HIGH VISIBILITY
Narrower usage
LOW VISIBILITY
“Enterprise Bus Matrix”
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
31
Proven Techniques: Implementation Planning
A sequence of dimensions and marts that takes allA sequence of dimensions and marts that takes all
criteria into accountcriteria into account
Demonstrates dimension reuse over timeDemonstrates dimension reuse over time
Stage 1 – VIPS
2
2 2 2 2 2 2 2 2 2 2 2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2 2 2
2
2
2
2
2
2
2
2
2
2 2
2 2 2
2 2
2 2
2 2
2 2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2 2
2 2
2
2
2
2
2 2
2 2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3 3 3
3 3 3
3 3 3
3 3 3 33
3
3
3
3
3
4
4
4 4
4
4
4
4
4
4
4
4
4
4 4
4
4
4 4
4
4
4
4
4
4
4
4
4
4
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
32
Flexible Architecture: A Managed Data Environment Is Needed
Universe of
Applications
Universe of
Application
Data
Specialized Data
Marts
Built as needed from
the EDW
Enterprise Data
Warehouse (EDW)
Data extracted from
source systems of record Data Delivery Services
ExistingEnvironment
Dashboards
High-End
Analytics
End-to-End Data Management
(Data Consolidated, Aggregated, Standardized, Dimensions Conformed)
ManagedDataEnvironment
Enterprise
Reporting
“O.K.,
Where do I
get the
information
I need?”
“Cubes”
Premium
Customer
Customer
Address
Claim
Employee
Cash Value
Policy
Location
Property
Reporting
DB
Reporting
DB
App DB
App DB
App DB
Department
Data MartDepartment
Data MartDepartment
Data MartDepartment
Data Mart
Enterprise
Data Warehouse
ODS
Granular
Relational
History
Metrics
Aggregates
Dimensional
Drill-Thru
Metadata Repository
Master Data
RepositoryETL
Hub
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
33
Flexible Architecture: ETL, Metadata, and OLAP as Well
■ Define ETL Reference Architecture
– Independent of implementation technologies
– Flexible process-managed delta extraction
– Initial assembly into generalized business
objects
– Comprehensive audits and controls
supporting error detection, staging, and
notification
`
BOMM
User Interface
(Standard & Custom Reports)
BO XI R2 Universes
Business Objects Metadata Manager (BOMM XI Release 2 Repository
BO Report
Objects
CA AllFusion ERwin
(EDW Models)
IBM Websphere DataStage
(ETL Jobs)
EDW Server
EDW
Database
Staging
Database
Operational Source Systems
Advance 1
Peoplesoft
BO Data Insight XI Data
Profiling Repository
Future
EDW Server
Schema
Objects
DataStage
Transformation
Objects
Source
Schema
Objects
Profiling
Statistics Log
EDW Data
Defintion
Objects
Current
Future
Current
Current
Current
Current
Current
Current
ETL Audit
Reports
■ Metadata Management Architecture
– Leverage solutions that interconnect using
the Metadata
– Proactively leverage vendor partnerships to
formulate solution
– Custom integration of ETL Audits and
Controls reporting into framework
– Position for future integration of profiling
metadata
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
34
Proven Techniques: Deploying ETL Architecture
Provide the ability to perform/handle CDC and full refreshProvide the ability to perform/handle CDC and full refresh
Provide granular data load dependenciesProvide granular data load dependencies
Provide centralized, reusable audit and controlProvide centralized, reusable audit and control
componentscomponents
Provide centralized, reusable errors and exceptionsProvide centralized, reusable errors and exceptions
componentscomponents
Make the ETL process selfMake the ETL process self--healing / selfhealing / self--cleaningcleaning
Provide technical and applicationProvide technical and application--level metadatalevel metadata
integrationintegration
Provide security managementProvide security management
Provide nearProvide near--real time supportreal time support
Provide replicated data source supportProvide replicated data source support
Design for ETL performanceDesign for ETL performance
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
35
Proven Architecture: Measures and KPIs for Analysis
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
36
Governance: Conceptual Model For Governance Metadata
Implement Governance
Incrementally via a
Seven Stage Maturity Model
Author: Martha Dember, RCG IT
Implement GovernanceImplement Governance
Incrementally via aIncrementally via a
Seven Stage Maturity ModelSeven Stage Maturity Model
Author: Martha Dember, RCG ITAuthor: Martha Dember, RCG IT
Strategy & Framework
Scenarios & Validation
Formalized Organization & Responsive Process Rollout
Proactive Process Rollout
Expanded Business Involvement
Stewardship Culture
Strategic Governance
© 2005-2009. Martha Dember, All Rights Reserved.
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
37
Issues
Constraints
ConceptualConceptual
Data Model(s)Data Model(s)
LogicalLogical
Data ModelData Model
DataData
Life CycleLife Cycle
EntityEntity
ComplexityComplexity
Actor
Roles
Activities
Objects
Sources
Locations
Volume
Frequency
Sub-Act Sub-Act
Elements
Definitions Attributes
Facts &
Measures
Reports
Events
Business
Rules
Governance: Conceptual Model For Governance Metadata Repository
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
38
MDM: A Process For Management Of Master Data
2. Persist
Integrate with other master data as needed
Store in MDR
Store DQ score/rating
6. Maintain
Use Master Data Maintenance UI
Capture changes to structure/content
Store history
1. Define
Define source system of record for master data element
Design data structure and process for creation/capture
Perform QC prior to importing
3. Secure
(Create new roles/privileges)
Apply security policy
4. Consume
Distribute to relevant application data stores
Consumption governed by security policy
Application performs QA prior to acceptance
5. Monitor
Capture operational metadata on usage of master data
Perform capacity management of MDM infrastructure
Perform periodic QA in MDR to verify DQ score/rating
MASTER DATA
LIFECYCLE
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
Key TakeKey Take –– AwaysAways
(Remember These(Remember These……))
Robert J. Abate, CBIP
Global Managing Principal, RCG ITGlobal Managing Principal, RCG IT
Monday, March 17th, 2008 5PM – 6PM
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
40
Key Take-Aways: Apply Proven Principles
FoundationFoundation:: Separation between data, source systems, and theirSeparation between data, source systems, and their
implementation technologiesimplementation technologies –– focus on BI & Data Delivery in this way tofocus on BI & Data Delivery in this way to
achieveachieve ““Information IndependenceInformation Independence””
Core PrincipleCore Principle:: Information design and managementInformation design and management –– an architecture foran architecture for
delivering correct, complete, reliable, and timely data quicklydelivering correct, complete, reliable, and timely data quickly andand
effectivelyeffectively
Keys to SuccessKeys to Success:: Information design and management are essential asInformation design and management are essential as
data delivery becomes critical to the businessdata delivery becomes critical to the business
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
41
Bottom Line BenefitsBottom Line Benefits ((essential for high business valueessential for high business value):):
Target BI analytics on improving business metrics (what would be the
value of increasing inventory turns by 5%?)
Learn from Lowe’s, 3M, and others
Engage the business in the core business process of focused
improvement by turning information into action
Cost SavingsCost Savings ((necessary, but not sufficientnecessary, but not sufficient):):
Eliminate redundant data and the associated storage and servers
Standardize on a small set of BI & Data Delivery technologies, reducing
ongoing license and maintenance fees
Match BI & Data Delivery capabilities to the users and manage license
fees for products
Reduce the time taken managing Excel and Access data manipulation,
analysis, and reporting
Key Take-Aways: Focus On Results
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
42
Every company can use data to deliver bottomEvery company can use data to deliver bottom--lineline
business valuebusiness value
Apply the core corporate performance managementApply the core corporate performance management
(CPM) business process(CPM) business process
Focus on bottomFocus on bottom--line benefits for real ROI; donline benefits for real ROI; don’’t get lostt get lost
in typical IT cost justificationsin typical IT cost justifications
While there are business and technical obstacles toWhile there are business and technical obstacles to
overall success, these are known and can be overcomeoverall success, these are known and can be overcome
Deliver through an architectural and data managementDeliver through an architectural and data management
frameworkframework
Use a combination of business, IT, and outside expertiseUse a combination of business, IT, and outside expertise
to develop a core competencyto develop a core competency
Key Take-Aways: Deliver Value
RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
43
Key Take-Aways: Industry Trends
Much more enterpriseMuch more enterprise--level thinkinglevel thinking
BI selfBI self--service for easeservice for ease--ofof--use by business usersuse by business users
ReRe--architecting existing BI and data deliveryarchitecting existing BI and data delivery
environmentsenvironments
Starting to get anal about audit and control over dataStarting to get anal about audit and control over data
movementmovement
Growing awareness:Growing awareness:
Data quality issues, especially as they impact quality of analytics
Need for performance management
Competency centers (or center of excellence) performance is
different than skilled use of technology
Attacking problems with BI and data delivery strategiesAttacking problems with BI and data delivery strategies
and roadmapsand roadmaps
© 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
Thank You For YourThank You For Your
ParticipationParticipation……
Robert J. Abate, CBIP
Global Managing Principal, RCG ITGlobal Managing Principal, RCG IT
rabate@rcgit.comrabate@rcgit.com
(732) 744(732) 744--36223622

More Related Content

PDF
Digital Transformation, Analytics, and the Modern C-Suite
PDF
A Practical Guide to Implementing Effective BI Governance
DOCX
Lindsay Scott Resume 2016 Final
PDF
Holistic data governance frame work whitepaper
PDF
RWDG Slides: Using Tools to Advance Your Data Governance Program
PDF
DAS Slides: Data Governance - Combining Data Management with Organizational ...
PDF
The Evolving Role of the Data Architect – What Does It Mean for Your Career?
PDF
Using metrics to measure data quality
Digital Transformation, Analytics, and the Modern C-Suite
A Practical Guide to Implementing Effective BI Governance
Lindsay Scott Resume 2016 Final
Holistic data governance frame work whitepaper
RWDG Slides: Using Tools to Advance Your Data Governance Program
DAS Slides: Data Governance - Combining Data Management with Organizational ...
The Evolving Role of the Data Architect – What Does It Mean for Your Career?
Using metrics to measure data quality

What's hot (20)

PDF
Master Data Management - Aligning Data, Process, and Governance
PDF
DAS Slides: Building a Data Strategy - Practical Steps for Aligning with Busi...
PDF
Data quality metrics infographic
PPTX
Эволюция Big Data и Information Management. Reference Architecture.
PDF
NTXISSACSC3 - Why Enterprise Information Management is the Key to GRC by Mika...
PDF
Enterprise Data Management Framework Overview
PDF
Aug 2017 damaga-peter-vennel
PDF
Modern Metadata Strategies
PDF
Master Data Management's Place in the Data Governance Landscape
 
PDF
When and How Data Lakes Fit into a Modern Data Architecture
PDF
How to Strengthen Enterprise Data Governance with Data Quality
PDF
Data strategy in a Big Data world
PDF
Webinar: Decoding the Mystery - How to Know if You Need a Data Catalog, a Dat...
PDF
ADV Slides: What the Aspiring or New Data Scientist Needs to Know About the E...
PDF
Data Architecture Strategy Brochure
PDF
Building a Data Strategy – Practical Steps for Aligning with Business Goals
PDF
Data Architecture Strategies: Building an Enterprise Data Strategy – Where to...
PDF
Introduction to Data Governance
PDF
DMBOK 2.0 and other frameworks including TOGAF & COBIT - keynote from DAMA Au...
PDF
Slides: Bridging the Data Disconnect – Trends in Global Data Management
Master Data Management - Aligning Data, Process, and Governance
DAS Slides: Building a Data Strategy - Practical Steps for Aligning with Busi...
Data quality metrics infographic
Эволюция Big Data и Information Management. Reference Architecture.
NTXISSACSC3 - Why Enterprise Information Management is the Key to GRC by Mika...
Enterprise Data Management Framework Overview
Aug 2017 damaga-peter-vennel
Modern Metadata Strategies
Master Data Management's Place in the Data Governance Landscape
 
When and How Data Lakes Fit into a Modern Data Architecture
How to Strengthen Enterprise Data Governance with Data Quality
Data strategy in a Big Data world
Webinar: Decoding the Mystery - How to Know if You Need a Data Catalog, a Dat...
ADV Slides: What the Aspiring or New Data Scientist Needs to Know About the E...
Data Architecture Strategy Brochure
Building a Data Strategy – Practical Steps for Aligning with Business Goals
Data Architecture Strategies: Building an Enterprise Data Strategy – Where to...
Introduction to Data Governance
DMBOK 2.0 and other frameworks including TOGAF & COBIT - keynote from DAMA Au...
Slides: Bridging the Data Disconnect – Trends in Global Data Management
Ad

Viewers also liked (20)

PPTX
FAIR Data and Model Management for Systems Biology (and SOPs too!)
PDF
The Chief Data Officer's Agenda: What a CDO Needs to Know about Data Quality
PDF
Real-World Data Governance: Build Your Own Data Governance Tools
PDF
A Data Management Maturity Model Case Study
PPT
SOA for Data Management
PDF
Maven Application Lifecycle Management for Alfresco
PDF
Agile & ALM tools
PPT
PPSX
PMI Vs SDLC
PPT
Software Development Life Cycle (SDLC)
PPTX
PDF
DMBOK and Data Governance
PPT
SOA Unit I
PDF
The Importance of MDM - Eternal Management of the Data Mind
PDF
Software Development Life Cycle (SDLC)
PPT
ALM (Application Lifecycle Management)
PDF
What Is Agile Management?
PPTX
Geek Sync I The Importance of Data Model Change Management
PPT
Data Architecture for Data Governance
PPTX
System development life cycle (sdlc)
FAIR Data and Model Management for Systems Biology (and SOPs too!)
The Chief Data Officer's Agenda: What a CDO Needs to Know about Data Quality
Real-World Data Governance: Build Your Own Data Governance Tools
A Data Management Maturity Model Case Study
SOA for Data Management
Maven Application Lifecycle Management for Alfresco
Agile & ALM tools
PMI Vs SDLC
Software Development Life Cycle (SDLC)
DMBOK and Data Governance
SOA Unit I
The Importance of MDM - Eternal Management of the Data Mind
Software Development Life Cycle (SDLC)
ALM (Application Lifecycle Management)
What Is Agile Management?
Geek Sync I The Importance of Data Model Change Management
Data Architecture for Data Governance
System development life cycle (sdlc)
Ad

Similar to DAMA International Symposium San Diego CA 03-17-2008 (20)

PDF
Bitrock manufacturing
PDF
The Enabling Power of Distributed SQL for Enterprise Digital Transformation I...
PPTX
IT Management Introduction pdf to BLZ (1).pptx
PDF
Inventory and Discovery: How to Take Charge of “What’s Out There”
PPT
The Business Case for SaaS Analytics for Salesforce.com
PDF
ADV Slides: How to Improve Your Analytic Data Architecture Maturity
PDF
What Is My Enterprise Data Maturity 2021
DOCX
Implement Data Ware House
PPTX
SDI MRO Connected
PDF
IP&A109 Next-Generation Analytics Architecture for the Year 2020
PPTX
Webinar: Maximizing the ROI of IT by Simplifying Technology Complexity
PDF
6 Principles of Modern Change Data Capture: How to Build Fast, Agile, Reliabl...
PDF
Modern data integration expert sessions
PPTX
Modern Data Integration Expert Session Webinar
 
PPT
PDF
Leveraging Streaming Data through Automation
PDF
Event-driven Business: How Leading Companies are Adopting Streaming Strategies
PDF
Ensure a Successful SAP Hybris Implementation – Part 2: Architecture and Buil...
PPTX
There are 250 Database products, are you running the right one?
PDF
When Downtime Isn’t an Option: Performance Optimization Analytics in the Era ...
Bitrock manufacturing
The Enabling Power of Distributed SQL for Enterprise Digital Transformation I...
IT Management Introduction pdf to BLZ (1).pptx
Inventory and Discovery: How to Take Charge of “What’s Out There”
The Business Case for SaaS Analytics for Salesforce.com
ADV Slides: How to Improve Your Analytic Data Architecture Maturity
What Is My Enterprise Data Maturity 2021
Implement Data Ware House
SDI MRO Connected
IP&A109 Next-Generation Analytics Architecture for the Year 2020
Webinar: Maximizing the ROI of IT by Simplifying Technology Complexity
6 Principles of Modern Change Data Capture: How to Build Fast, Agile, Reliabl...
Modern data integration expert sessions
Modern Data Integration Expert Session Webinar
 
Leveraging Streaming Data through Automation
Event-driven Business: How Leading Companies are Adopting Streaming Strategies
Ensure a Successful SAP Hybris Implementation – Part 2: Architecture and Buil...
There are 250 Database products, are you running the right one?
When Downtime Isn’t an Option: Performance Optimization Analytics in the Era ...

More from Robert J. Abate, CBIP, CDMP (8)

PDF
The 25 Predictions About The Future Of Big Data
PDF
DAMA Big Data & The Cloud 2012-01-19
PDF
Convergence SOA & BI Presentation June 2010
PDF
CTAM Making Analytics Actionable RJA FINAL
PDF
Business Rules For Metadata Governance & Stewardship
PDF
Services Based Architectures - Defining The Issues For Data Professionals
PDF
Presentation - FIRST 2001 Conference
PDF
CDOVision - RJA Presentation FINAL
The 25 Predictions About The Future Of Big Data
DAMA Big Data & The Cloud 2012-01-19
Convergence SOA & BI Presentation June 2010
CTAM Making Analytics Actionable RJA FINAL
Business Rules For Metadata Governance & Stewardship
Services Based Architectures - Defining The Issues For Data Professionals
Presentation - FIRST 2001 Conference
CDOVision - RJA Presentation FINAL

DAMA International Symposium San Diego CA 03-17-2008

  • 1. © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. Enterprise InformationEnterprise Information Management [EIM]Management [EIM] –– Proven Techniques ForProven Techniques For SuccessSuccess Robert J. Abate, CBIP Global Managing Principal, RCG ITGlobal Managing Principal, RCG IT Monday, March 17th, 2008 5PM – 6PM
  • 2. AgendaAgenda Presentation TopicsPresentation Topics EndEnd--ToTo--End Data Management [EIM]End Data Management [EIM] Requires A Integration ArchitectureRequires A Integration Architecture Integrating Governance & QualityIntegrating Governance & Quality Proven Techniques For SuccessProven Techniques For Success Key TakeKey Take--AwaysAways © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved.
  • 3. © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. EndEnd--ToTo--End DataEnd Data Management [EIM]Management [EIM] Robert J. Abate, CBIP Global Managing Principal, RCG ITGlobal Managing Principal, RCG IT Monday, March 17th, 2008 5PM – 6PM
  • 4. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 4 What Is Enterprise Information Management? Intersection Of Strategy, Architecture & Delivery Of InfoIntersection Of Strategy, Architecture & Delivery Of Info It Is A Managed Information Environment (with End – To – End Data Management) Incorporating: Where Data Is Known To Be Of High Quality Information Is Wrapped With (Or Having) Governance Contain A Single Source Of Truth Flexible Structure or “Architecture” Responsive To Change Supportive Of Integration And Delivery Of Information A flexible structure is important because:A flexible structure is important because: Business needs are changing at an accelerating pace It is the core feature that allows IT to be responsive in meeting new business requirements
  • 5. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 5 The Information Foundation: Data Management This requires:This requires: Metadata managementMetadata management for compliance enforcement, audit support,for compliance enforcement, audit support, analysis, and reportinganalysis, and reporting Master dataMaster data integration and controlintegration and control NearNear--real timereal time business informationbusiness information Source data managementSource data management for controlling data quality at thefor controlling data quality at the transaction leveltransaction level Effective governanceEffective governance for a successful managed data environmentfor a successful managed data environment IntegrationIntegration of analytics, reporting, and transaction controlof analytics, reporting, and transaction control ControlControl of business processes and information usageof business processes and information usage Unstructured dataUnstructured data needs to be addressed as wellneeds to be addressed as well The role of data management in the business is evolving from “A Single Source Of Truth” to becoming “The Information Assurance And Delivery Mechanism”
  • 6. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 6 How Do You Build One? Flexible Reference / Data ArchitectureFlexible Reference / Data Architecture Controlled Data LifecycleControlled Data Lifecycle Metadata / Information TaxonomyMetadata / Information Taxonomy Master Data ManagementMaster Data Management ModelModel--Driven DevelopmentDriven Development Data Integration ToolsData Integration Tools Extraction, Transformation & Loading [ETL] Data Delivery Services Information DeliveryInformation Delivery Reporting & Analytics Security Dimensional Models (Data Warehouse)Dimensional Models (Data Warehouse) Conformed Dimensions Data GovernanceData Governance Data QualityData Quality Data Profiling Exception Handling Action Time ValueLost Action Time ValueLost Time Value Action Business Event Taken Business Event Taken Capture Latency Analysis Latency Decision Latency Data Ready For Analysis Information Delivered Capture Latency Analysis Latency Decision Latency Data Ready For Analysis Information Delivered © Chart: Richard Hackathorn, Bolder Technology, Inc., 2007. All Rights Reserved. Used with Permission. Data Sources Finance XLS POS ERP SCM ERP Transaction Level Detail Granular Denormalized Integrated Current & History Reporting Transaction Level Detail Granular Denormalized Integrated Current & History Reporting Metadata (Extract/Transform/Load) -- Apply business rules -- Manage metadata -- Consolidate data ETL / Data Integration Hub Transaction Level Detail ‘Atomic’-level Data Normalized (3NF) Integrated Temporary Integration Aggregates & KPIs Dimensional Drilling Conformed Analytics ETL Server Dashboards Predictive & Behavioral Analytics Reporting & Ad hoc Queries Database Server Reporting Server / Portal Scoring Algorithms Metadata Inquiries Others A “permanent” Integration Data Store can facilitate data conversions and system upgrades because it contains data already standardized, cleansed, and integrated. * * Data DeliveryEnterprise Data
  • 7. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 7 What Is A Flexible Architecture? Definition of a flexible architectureDefinition of a flexible architecture (from Dictionary.com)(from Dictionary.com):: Architecture:Architecture: “Orderly arrangement of parts; structure” Flexibility:Flexibility: “Responsive to change; adaptable” Responsive:Responsive: “readily reacting to events or stimuli” Therefore, a flexible architecture is one that is:Therefore, a flexible architecture is one that is: Readily adaptable to events or stimuli that require it to change Designed to provide the key capabilities necessary for BI today And an inflexible architecture is one that is:And an inflexible architecture is one that is: Hard or time-consuming to change Unable to incorporate new capabilities when needed
  • 8. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 8 Partner with the Business: Focus on Impacting the Bottom Line Operational BI Strategic BI Simple metrics: • Customer satisfaction • Inventory turns • Quality measurement • Financial reporting Dashboard with metrics: • Customer profitability • Product profitability • Operational performance Performance Management Performance Management Performance Optimization Performance Optimization Complex metrics: • Market success • Customer relationship • Information innovation • Value creation Value to the Corporate Bottom Line These inform different kinds of decisions at different levels of the organization Operational Reporting & Analysis
  • 9. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 9 Best Practices & Trends… Separation of Architectural Layers usingSeparation of Architectural Layers using ReferenceReference ArchitecturesArchitectures ((““akaaka”” Architectural FrameworksArchitectural Frameworks)) Model Driven Development [MDD]Model Driven Development [MDD] with a focus on thewith a focus on the business functions / processes to drive alignment withbusiness functions / processes to drive alignment with business goals and objectivesbusiness goals and objectives –– ““akaaka”” BusinessBusiness ArchitectureArchitecture Providing requirements traceability “Glue” or integration points for iterative development Convergence OfConvergence Of Applications (SOA)Applications (SOA) && Managed DataManaged Data Environments (Business Intelligence)Environments (Business Intelligence)
  • 10. © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. Requires An IntegrationRequires An Integration ArchitectureArchitecture Robert J. Abate, CBIP Global Managing Principal, RCG ITGlobal Managing Principal, RCG IT Monday, March 17th, 2008 5PM – 6PM
  • 11. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 11 The Problem – Aligning Systems with the Business 1. As proposed by the project sponsor 2. As defined by the requirements doc’s 3. As designed by the Senior Analyst 4. As produced by the programmers 5. As installed at the user’s site 6. What The User Really Wanted! This Cartoon Has Been Circulating In IT For Over 20+ Years In Various Forms…
  • 12. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 12 The Importance of Requirements ““Analysts report that as many as 71% of projects thatAnalysts report that as many as 71% of projects that fail do so because of poor requirements management,fail do so because of poor requirements management, making it the single biggest reason for project failuremaking it the single biggest reason for project failure”” CIO magazine, November 15, 2005 Failure can also be defined as:Failure can also be defined as: ““Analysis ParalysisAnalysis Paralysis”” oror ““Scope CreepScope Creep”” A never ending test cycleA never ending test cycle Redevelopment of systems after implementationRedevelopment of systems after implementation
  • 13. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 13 Relationship Between Business Architecture & IT Enables Dictates Enables Dictates Supplements Dictates Dictates Business / SBU Strategy Business Architecture Business Projects & Programs Implies Dictates Technology Strategy App. & Info. Technology Architecture Technology Projects & Programs Implies Contributes&Supports Contributes&Supports Contributes&Supports There Is An “Implied” Relationship Between Business & Technology Building Blocks Business Architecture is a comprehensive framework for the representation of the business processes, information, rules, and organizational designs that result directly from the business strategy. It provides a coherent graphical means for the capture, management, and communication of business knowledge and needs to support business solutions in the form of actionable requirements. IT Confined By Project Budgets And Define App/Info Architecture
  • 14. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 14 Define business requirements in aDefine business requirements in a manner thatmanner that aligns IT and Businessaligns IT and Business Requirements areRequirements are graphicalgraphical (pictures) and(pictures) and easy to understandeasy to understand Supports both application andSupports both application and information architecturesinformation architectures Vendor agnosticVendor agnostic Supports rules established inSupports rules established in Business ArchitectureBusiness Architecture Provides forProvides for checking consistencychecking consistency and completeness throughoutand completeness throughout Supports management withSupports management with qualityquality control and governancecontrol and governance ModelModel--driven approachdriven approach should use ashould use a repositoryrepository--based toolbased tool The Many Benefits Of Model Driven Development
  • 15. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 15 MDM Has Full Traceability - Through Design, Build and Test Design &Build Require ments Testing Requirements and Acceptance Criteria Design Component and/or Use Case that Satisfies each Requirement Test Groups, Test Cases Of Each Design Component Requirements are satisfied by a design component and each componRequirements are satisfied by a design component and each component isent is assigned to a test group with test cases. Thus test cases testassigned to a test group with test cases. Thus test cases test requirements.requirements. Test results can then be measured against acceptance criteria.Test results can then be measured against acceptance criteria.
  • 16. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 16 Reference Architecture And It’s Views BUSINESS ARCHITECTURE Mission / Vision, Business Requirements, Business Rules, Organizational Structure, Critical Success Factors, Business Process Design APPLICATIONS ARCHITECTURE Process Alignment, Services Definition, Services/Event Architectures, Custom Application Development, Enterprise Applications Integration Components INFORMATION ARCHITECTURE Data Integration, Data Architecture, Master Data Management, Metadata Management, Data Delivery Architecture, Dashboards, Performance Measurement, Analytics , Business Intelligence, Enterprise Reporting INFRASTRUCTURE ARCHITECTURE Legacy (Non-Service Enabled) Applications Networks, Servers / Hardware, Operating Systems INTEGRATEDGOVERNANCE Security,Data,SLAs,ServicesandITPortfolioManagement QUALITYCONTROL QualityAssurance,Testing,DataQualityManagement Reference ArchitectureReference Architecture is a proven template for a solution domain thatis a proven template for a solution domain that provides a common vocabulary with which to discuss designs andprovides a common vocabulary with which to discuss designs and implementationsimplementations –– it frames solution delivery.it frames solution delivery. Enterprise Architecture FrameworkEnterprise Architecture Framework describes a structure anddescribes a structure and behavior for an organization's processes, information systems,behavior for an organization's processes, information systems, personnel, and organizational subpersonnel, and organizational sub--units, aligning them with theunits, aligning them with the organization's core goals and strategic direction.organization's core goals and strategic direction.
  • 17. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 17 Reference Architecture – Artifacts / Techniques StrategyStrategyProcessProcessFunctionFunctionTimeTimeLocationLocationDataDataOrganizationOrganization WHYWHYHOWHOWWHENWHENWHEREWHEREWHATWHATWHOWHO EnterpriseEnterprise Infrastructure VisionInfrastructure Vision Strategic NetworkStrategic Network PlanPlan System OperationSystem Operation Diagram (SODDiagram (SOD--P)P) Security ManagementSecurity Management PlanPlan Systems ArchitectureSystems Architecture DiagramDiagram Network ArchitectureNetwork Architecture DiagramDiagram InfrastructureInfrastructure Calendar PlanCalendar Plan Location Support PlanLocation Support PlanData ManagementData Management PlanPlan Data Capacity PlanData Capacity Plan Data Archive PlanData Archive Plan InfrastructureInfrastructure Resource PlanResource Plan CSFCSF’’ss Business Goals /Business Goals / DirectionDirection Business PlanBusiness Plan – BPM / BPI – Continuity Plan Policy CharterPolicy Charter Business ProcessBusiness Process Flow (SODFlow (SOD--L)L) Use CaseUse Case ScreensScreens ReportsReports Business Logic (ASD)Business Logic (ASD) Business FunctionsBusiness Functions (FSD)(FSD) Business ProcessBusiness Process (FSD)(FSD) Business PrioritiesBusiness Priorities Business TimeBusiness Time DefinitionsDefinitions Master ScheduleMaster Schedule (Events & Triggers)(Events & Triggers) Business LocationsBusiness Locations LocationLocation--specificspecific BusinessBusiness Requirements &Requirements & DefinitionsDefinitions Entity RelationshipEntity Relationship Diagram (EDD)Diagram (EDD) Attribute DependencyAttribute Dependency Diagram (ADD)Diagram (ADD) KPI Structure DiagramKPI Structure Diagram (KSD)(KSD) Organization ChartOrganization Chart (OSD)(OSD) RolesRoles && ResponsibilitiesResponsibilities System Operation Diagram (SOD-L) System Operations Documentation Disaster Recovery Plan BusinessRequirementsdriveEnterpriseArchitectureandBusinessContext Business Architecture Business PlanBusiness Plan AlignmentAlignment ApplicationApplication PrioritizationPrioritization ApplicationApplication Replacement PlanReplacement Plan Workflow ProcessWorkflow Process ModelModel ApplicationApplication Architecture DesignArchitecture Design Systems InterfaceSystems Interface DesignDesign ApplicationApplication Architecture DesignArchitecture Design Services Map ScreenServices Map Screen DiagramsDiagrams Application / ProcessApplication / Process Event CorrelationEvent Correlation DiagramsDiagrams Location ProcessingLocation Processing Workflow ModelsWorkflow Models Semantic ModelSemantic Model (Metadata)(Metadata) Messaging DiagramMessaging Diagram XML / XSLT DiagramsXML / XSLT Diagrams SOAP / MessagesSOAP / Messages Organization ProcessOrganization Process Mapping RulesMapping Rules User InterfaceUser Interface DiagramsDiagrams METADATA (repository-based): Application / Services Metaprocesses Applications Architecture Enterprise ManagedEnterprise Managed Data EnvironmentData Environment Master DataMaster Data Management PlanManagement Plan BusinessBusiness PerformancePerformance ManagementManagement Data QualityData Quality MonitoringMonitoring Data ExceptionData Exception ProcessingProcessing Data PerformanceData Performance ManagementManagement Enterprise DataEnterprise Data Structures AuditStructures Audit Enterprise ReportingEnterprise Reporting & Analytics& Analytics ArchitectureArchitecture ETL / Data IntegrationETL / Data Integration ArchitectureArchitecture Data QualityData Quality ManagementManagement MetadataMetadata ManagementManagement Event DataEvent Data Correlation DiagramsCorrelation Diagrams Enterprise DataEnterprise Data SynchronizationSynchronization ScheduleSchedule Data ControlData Control StructuresStructures Data Location &Data Location & PhysicalPhysical InterdependencyInterdependency MapsMaps Data Source OfData Source Of Record (SourceRecord (Source--toto-- Target for EnterpriseTarget for Enterprise Data Structures)Data Structures) DimensionalDimensional Hierarchies /Hierarchies / TaxonomyTaxonomy KPI / MetricsKPI / Metrics DefinitionDefinition Master Data ModelMaster Data Model Logical / PhysicalLogical / Physical Data ModelsData Models Physical StructuresPhysical Structures User AccessUser Access AuthorizationAuthorization Data Visualization &Data Visualization & DeliveryDelivery RequirementsRequirements METADATA (repository-based): Business (Definitions, Access Rights…), Operational (Load Timings, Controls…) & Technical (Environmental…) Information Architecture Infrastructure Architecture
  • 18. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 18 Reference Architecture – Logical Managed ComponentsINTEGRATEDGOVERNANCE Security,Data,SLAs,ServicesandITPortfolioManagement QUALITYCONTROL QualityAssurance,Testing,DataQualityManagement INFORMATION ARCHITECTURE APPLICATIONS ARCHITECTURE Business Rules Engine App / Svcs Integration Broker Data Rules App Rules Services Mgmt Event Correlation Process Choreography Svcs Orchestration Services Mgmt Event Correlation Process Choreography Svcs Orchestration BUSINESS ARCHITECTURE INFRASTRUCTURE ARCHITECTURE Existing Application Backbone for Transaction Processing (includes Legacy Applications) INFRASTRUCTURE ARCHITECTURE Existing Application Backbone for Transaction Processing (includes Legacy Applications) Central Metadata & Metaprocess Library Data & Data Audit Metadata Enterprise Reporting Integrated KPIs & Dashboards Data Quality, Profiling & Cleansing Enterprise Reporting Integrated KPIs & Dashboards Data Quality, Profiling & Cleansing ETL / Data Integration Hub Enterprise Data Structures Sync Sync Sync Integrated Trx Data Store Trx Processing Recent Data, Normalized, Fine- grained, Unit-of-Work (UoW) Processing EDW & Data Marts Reporting Granular, Denormalized, Current / History Analytics Aggregates, Dimensional, Conformed Master Data Customer Value, Control Data, ID X-ref Product Value, Control Data, ID X-ref Orgn Chart Value, Control Data, ID X-ref Enterprise Data Structures Sync Sync Sync Integrated Trx Data Store Trx Processing Recent Data, Normalized, Fine- grained, Unit-of-Work (UoW) Processing EDW & Data Marts Reporting Granular, Denormalized, Current / History Analytics Aggregates, Dimensional, Conformed Master Data Customer Value, Control Data, ID X-ref Product Value, Control Data, ID X-ref Orgn Chart Value, Control Data, ID X-ref Applications / Services Metaprocesses Services Library Application Svcs Data Delivery Svcs Messaging Svcs Services Library Application Svcs Data Delivery Svcs Messaging Svcs StrategyProcessFunctionTimeLocationDataOrganization WHYHOWWHENWHEREWHATWHO
  • 19. © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. Integrating GovernanceIntegrating Governance & Quality& Quality Robert J. Abate, CBIP Global Managing Principal, RCG ITGlobal Managing Principal, RCG IT Monday, March 17th, 2008 5PM – 6PM
  • 20. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 20 Dedicated Management of Process and CommunicationsDedicated Management of Process and Communications Data Governance – An Overview
  • 21. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 21 What Is Data Governance Anyway… Data Governance is what an organization does that yields:Data Governance is what an organization does that yields: The highest quality decisions made at the lowest cost. Increased efficiency in conducting/completing IT projects for the business. Data Governance is NOT:Data Governance is NOT: A project Someone else’s responsibility Intuitive Something that happens on it’s own Drivers For Data Governance:Drivers For Data Governance: Lack of trust in the data Reports utilizing same data do not report same totals or computations Data not available in an easy manner Multiple sources of data have to be gathered and manipulated Data timeliness Manual process to collect, analyze and provide results Data integrity Unknown filters, varying calculation/computations, fields used for data not indicative of field names, data passed along from one person to another to another to another…..
  • 22. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 22 Data Governance – Master Data Management Included… A Comparison of Data Governance & MDMA Comparison of Data Governance & MDM Master Data ManagementMaster Data Management Brings together single, consistentBrings together single, consistent view of data that may reside inview of data that may reside in multiple source applications (CRM,multiple source applications (CRM, ERP, financial applications, WebERP, financial applications, Web servers) by maintaining a singleservers) by maintaining a single source of critical data, then crosssource of critical data, then cross-- referencing and synchronizing datareferencing and synchronizing data across all other missionacross all other mission--criticalcritical applicationsapplications Critical business data flows in realCritical business data flows in real-- time to multiple applications fortime to multiple applications for single view of customers, supplierssingle view of customers, suppliers and other dataand other data Data GovernanceData Governance StrategyStrategy –– Why? Goals andWhy? Goals and objectives of organization (treatobjectives of organization (treat data as an asset of organization)data as an asset of organization) –– MDM is a goalMDM is a goal Policies, Procedures, ProcessPolicies, Procedures, Process –– What? Who? When? Where?What? Who? When? Where? Provides structure,Provides structure, communications, andcommunications, and accountabilityaccountability Implementation / ResolutionImplementation / Resolution –– How? RealHow? Real--world application toworld application to make it all happenmake it all happen Beyond MDM: Regulatory Risk, Compliance, and SOARegulatory Risk, Compliance, and SOA
  • 23. © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. Proven Techniques ForProven Techniques For SuccessSuccess Robert J. Abate, CBIP Global Managing Principal, RCG ITGlobal Managing Principal, RCG IT Monday, March 17th, 2008 5PM – 6PM
  • 24. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 24 Partner With The Business: Help Them Understand Themselves Stage 4: ‘Perform’ (business results) Actions: Structure KPIs & metrics Focus: Dashboarding processes Stage 3: ‘Structure’ (enterprise data) Actions: Conform & Standardize Data Focus: Standardizing data & terms Stage 2: ‘Verify’ (EDW data) Actions: Ensure Data Quality Focus: Governing data & metadata Stage 1: ‘Trust’ (in application data) Actions: Collect & Organize Focus: Gathering data for reporting Stage 5: ‘Optimize’ (the business) Actions: Establish Strategy & Goals Focus: Improving market success Departmental Usage Enterprise Ready Executive Driven Reality Perception
  • 25. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 25 Develop Iteratively – It’s All About The Repeatable Process A repeatable process implies an iterative methodologyA repeatable process implies an iterative methodology Utilize open-ended architecture/design principles to facilitate reuse in future iterations Stagger iterations as shown to save time and money Leverage best practices already available Requirements must be thoroughly understood Data profiling must be conducted in parallel with Requirements Activities should take progressively lesser time during subsequent iterations due to leveraging reusable components Program Management and Governance will ensure that iterations meet business requirements, minimize rework between iterations and proper management of master data through its entire lifecycle Requirements Architecture Development Deployment Iteration 1 Iteration 2 Iteration …
  • 26. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 26 Proven Techniques: Define Your Components ■ Data Management – Master data – Data standardization – Data quality – Metadata – Data security, audit, and compliance ■ Data and delivery architectures – Modeling and structuring data – ETL data integration and data rules – Data delivery services (SOA-compliant) – Data technologies compatibility ■ Performance management – KPI relationships and metrics structure – Dashboard development and integration ■ Analytics management – Enterprise reporting – Analytics in transaction processing – “Real-time” analytics – Business activity monitoring (BAM) INFORMATION ARCHITECTURE Data Rules Data & Data Audit Metadata Enterprise Reporting Integrated KPIs & Dashboards Data Quality, Profiling & Cleansing ETL / Data Integration Hub EnterpriseDataStructures Sync Sync Sync IntegratedTrxDataStore TrxProcessing Recent Data,Normalized,Fine- grained,Unit-of-Work(UoW) Processing EDW&DataMarts Reporting Granular, Denormalized, Current/History Analytics Aggregates, Dimensional, Conformed MasterData Customer Value,Control Data,IDX-ref Product Value,Control Data,IDX-ref OrgnChart Value,Control Data,IDX-ref EnterpriseDataStructures Sync Sync Sync IntegratedTrxDataStore TrxProcessing Recent Data,Normalized,Fine- grained,Unit-of-Work(UoW) Processing EDW&DataMarts Reporting Granular, Denormalized, Current/History Analytics Aggregates, Dimensional, Conformed MasterData Customer Value,Control Data,IDX-ref Product Value,Control Data,IDX-ref OrgnChart Value,Control Data,IDX-ref A Managed Data Environment Data Delivery Svcs Implies… Managed Data Components
  • 27. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 27 Proven Techniques: Define Your Data Lifecycle… © Richard Hackathorn, Bolder Technology, Inc., 2007. All Rights Reserved. Used with Permission. Managing The Time-Value Curve Action Time ValueLost Time Value Action Business Event Taken Capture Latency Analysis Latency Decision Latency Data Ready For Analysis Information Delivered
  • 28. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 28 Proven Data Architecture – Data Validation & Verification Data Sources Finance XLS POS ERP SCM ERP Transaction Level Detail Granular Denormalized Integrated Current & History Reporting Metadata (Extract/Transform/Load) -- Apply business rules -- Manage metadata -- Consolidate data ETL / Data Integration Hub Transaction Level Detail ‘Atomic’-level Data Normalized (3NF) Integrated Temporary Integration Aggregates & KPIs Dimensional Drilling Conformed Analytics ETL Server Dashboards Predictive & Behavioral Analytics Reporting & Ad hoc Queries Database Server Reporting Server / Portal Scoring Algorithms Metadata Inquiries Others Because there are several technologies covered by this graphic, it is important that they “play well together” Data DeliveryEnterprise Data 1 2 3 4 5 6
  • 29. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 29 Proven Techniques: Vision and Direction Establish A Vision of theEstablish A Vision of the FutureFuture The Mission & Vision The Stakeholders BI and Reporting Differentiators Operating Principles Goals and Objectives for BI and Reporting Prepare a Roadmap for thePrepare a Roadmap for the FutureFuture Recommended Initiatives & Impact Action Plans for High-Priority Initiatives Transition Strategy Critical Success Factors for the Transition •DataQualityManagement,Audit&Control •MasterDataManagement •ScorecardDevelopment •ETLManagementforSourceSystemChanges •EDWArchitectureandEnhancement •BusinessPerformanceManagement • Marketing • Advertising • Customers • Finance • Employee • Organization • Transportation • Distribution Centers • Inventory • Orders • Vendors • Sales • Sales Transactions • Products • Stores Phase 1: Sales & Stores CAPABILITY Phase 2: Inventory & Orders Phase 3: Finance & Employee Phase 4: Marketing & Customer Implementation, Phase by Phase, progresses over time Ongoing, Centralized, EDW Capabilities and Program Management • Store and Product Master Data • Product Hierarchy Reference Data • Sales Scorecard Development • Vendor and Order Master Data • Vendor Scorecard Development • Inventory Scorecard Development • Employee Master Data • Organization Reference Data • Expenses and Margin Scorecard Development • Complete Store Scorecard Development • Customer Master Data • Business Performance Analysis • Profitability Analysis (stores, products, customers)
  • 30. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 30 Proven Techniques: Project Selection Planning Widely used dimensions Low Complexity Considerable overall value MEDIUM PRIORITY – HIGH IMPACT Scarcely used dimensions Low complexity Easy to implement QUICK WINS – LIMITED IMPACT Scarcely used dimensions High complexity Low overall value LOW PRIORITY – LIMITED IMPACT Widely used dimensions High complexity High overall value HIGH PRIORITY – HIGH IMPACT VIPS Broader usage HIGH VISIBILITY Narrower usage LOW VISIBILITY “Enterprise Bus Matrix”
  • 31. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 31 Proven Techniques: Implementation Planning A sequence of dimensions and marts that takes allA sequence of dimensions and marts that takes all criteria into accountcriteria into account Demonstrates dimension reuse over timeDemonstrates dimension reuse over time Stage 1 – VIPS 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 33 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
  • 32. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 32 Flexible Architecture: A Managed Data Environment Is Needed Universe of Applications Universe of Application Data Specialized Data Marts Built as needed from the EDW Enterprise Data Warehouse (EDW) Data extracted from source systems of record Data Delivery Services ExistingEnvironment Dashboards High-End Analytics End-to-End Data Management (Data Consolidated, Aggregated, Standardized, Dimensions Conformed) ManagedDataEnvironment Enterprise Reporting “O.K., Where do I get the information I need?” “Cubes” Premium Customer Customer Address Claim Employee Cash Value Policy Location Property Reporting DB Reporting DB App DB App DB App DB Department Data MartDepartment Data MartDepartment Data MartDepartment Data Mart Enterprise Data Warehouse ODS Granular Relational History Metrics Aggregates Dimensional Drill-Thru Metadata Repository Master Data RepositoryETL Hub
  • 33. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 33 Flexible Architecture: ETL, Metadata, and OLAP as Well ■ Define ETL Reference Architecture – Independent of implementation technologies – Flexible process-managed delta extraction – Initial assembly into generalized business objects – Comprehensive audits and controls supporting error detection, staging, and notification ` BOMM User Interface (Standard & Custom Reports) BO XI R2 Universes Business Objects Metadata Manager (BOMM XI Release 2 Repository BO Report Objects CA AllFusion ERwin (EDW Models) IBM Websphere DataStage (ETL Jobs) EDW Server EDW Database Staging Database Operational Source Systems Advance 1 Peoplesoft BO Data Insight XI Data Profiling Repository Future EDW Server Schema Objects DataStage Transformation Objects Source Schema Objects Profiling Statistics Log EDW Data Defintion Objects Current Future Current Current Current Current Current Current ETL Audit Reports ■ Metadata Management Architecture – Leverage solutions that interconnect using the Metadata – Proactively leverage vendor partnerships to formulate solution – Custom integration of ETL Audits and Controls reporting into framework – Position for future integration of profiling metadata
  • 34. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 34 Proven Techniques: Deploying ETL Architecture Provide the ability to perform/handle CDC and full refreshProvide the ability to perform/handle CDC and full refresh Provide granular data load dependenciesProvide granular data load dependencies Provide centralized, reusable audit and controlProvide centralized, reusable audit and control componentscomponents Provide centralized, reusable errors and exceptionsProvide centralized, reusable errors and exceptions componentscomponents Make the ETL process selfMake the ETL process self--healing / selfhealing / self--cleaningcleaning Provide technical and applicationProvide technical and application--level metadatalevel metadata integrationintegration Provide security managementProvide security management Provide nearProvide near--real time supportreal time support Provide replicated data source supportProvide replicated data source support Design for ETL performanceDesign for ETL performance
  • 35. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 35 Proven Architecture: Measures and KPIs for Analysis
  • 36. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 36 Governance: Conceptual Model For Governance Metadata Implement Governance Incrementally via a Seven Stage Maturity Model Author: Martha Dember, RCG IT Implement GovernanceImplement Governance Incrementally via aIncrementally via a Seven Stage Maturity ModelSeven Stage Maturity Model Author: Martha Dember, RCG ITAuthor: Martha Dember, RCG IT Strategy & Framework Scenarios & Validation Formalized Organization & Responsive Process Rollout Proactive Process Rollout Expanded Business Involvement Stewardship Culture Strategic Governance © 2005-2009. Martha Dember, All Rights Reserved.
  • 37. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 37 Issues Constraints ConceptualConceptual Data Model(s)Data Model(s) LogicalLogical Data ModelData Model DataData Life CycleLife Cycle EntityEntity ComplexityComplexity Actor Roles Activities Objects Sources Locations Volume Frequency Sub-Act Sub-Act Elements Definitions Attributes Facts & Measures Reports Events Business Rules Governance: Conceptual Model For Governance Metadata Repository
  • 38. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 38 MDM: A Process For Management Of Master Data 2. Persist Integrate with other master data as needed Store in MDR Store DQ score/rating 6. Maintain Use Master Data Maintenance UI Capture changes to structure/content Store history 1. Define Define source system of record for master data element Design data structure and process for creation/capture Perform QC prior to importing 3. Secure (Create new roles/privileges) Apply security policy 4. Consume Distribute to relevant application data stores Consumption governed by security policy Application performs QA prior to acceptance 5. Monitor Capture operational metadata on usage of master data Perform capacity management of MDM infrastructure Perform periodic QA in MDR to verify DQ score/rating MASTER DATA LIFECYCLE
  • 39. © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. Key TakeKey Take –– AwaysAways (Remember These(Remember These……)) Robert J. Abate, CBIP Global Managing Principal, RCG ITGlobal Managing Principal, RCG IT Monday, March 17th, 2008 5PM – 6PM
  • 40. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 40 Key Take-Aways: Apply Proven Principles FoundationFoundation:: Separation between data, source systems, and theirSeparation between data, source systems, and their implementation technologiesimplementation technologies –– focus on BI & Data Delivery in this way tofocus on BI & Data Delivery in this way to achieveachieve ““Information IndependenceInformation Independence”” Core PrincipleCore Principle:: Information design and managementInformation design and management –– an architecture foran architecture for delivering correct, complete, reliable, and timely data quicklydelivering correct, complete, reliable, and timely data quickly andand effectivelyeffectively Keys to SuccessKeys to Success:: Information design and management are essential asInformation design and management are essential as data delivery becomes critical to the businessdata delivery becomes critical to the business
  • 41. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 41 Bottom Line BenefitsBottom Line Benefits ((essential for high business valueessential for high business value):): Target BI analytics on improving business metrics (what would be the value of increasing inventory turns by 5%?) Learn from Lowe’s, 3M, and others Engage the business in the core business process of focused improvement by turning information into action Cost SavingsCost Savings ((necessary, but not sufficientnecessary, but not sufficient):): Eliminate redundant data and the associated storage and servers Standardize on a small set of BI & Data Delivery technologies, reducing ongoing license and maintenance fees Match BI & Data Delivery capabilities to the users and manage license fees for products Reduce the time taken managing Excel and Access data manipulation, analysis, and reporting Key Take-Aways: Focus On Results
  • 42. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 42 Every company can use data to deliver bottomEvery company can use data to deliver bottom--lineline business valuebusiness value Apply the core corporate performance managementApply the core corporate performance management (CPM) business process(CPM) business process Focus on bottomFocus on bottom--line benefits for real ROI; donline benefits for real ROI; don’’t get lostt get lost in typical IT cost justificationsin typical IT cost justifications While there are business and technical obstacles toWhile there are business and technical obstacles to overall success, these are known and can be overcomeoverall success, these are known and can be overcome Deliver through an architectural and data managementDeliver through an architectural and data management frameworkframework Use a combination of business, IT, and outside expertiseUse a combination of business, IT, and outside expertise to develop a core competencyto develop a core competency Key Take-Aways: Deliver Value
  • 43. RCG INFORMATION TECHNOLOGYRCG INFORMATION TECHNOLOGY © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. 43 Key Take-Aways: Industry Trends Much more enterpriseMuch more enterprise--level thinkinglevel thinking BI selfBI self--service for easeservice for ease--ofof--use by business usersuse by business users ReRe--architecting existing BI and data deliveryarchitecting existing BI and data delivery environmentsenvironments Starting to get anal about audit and control over dataStarting to get anal about audit and control over data movementmovement Growing awareness:Growing awareness: Data quality issues, especially as they impact quality of analytics Need for performance management Competency centers (or center of excellence) performance is different than skilled use of technology Attacking problems with BI and data delivery strategiesAttacking problems with BI and data delivery strategies and roadmapsand roadmaps
  • 44. © 2008 RCG IT, DAMA & Wilshire Conferences. All rights reserved. Thank You For YourThank You For Your ParticipationParticipation…… Robert J. Abate, CBIP Global Managing Principal, RCG ITGlobal Managing Principal, RCG IT [email protected]@rcgit.com (732) 744(732) 744--36223622