Enterprise search and unstructured data with CGI and Elastic
0 likes829 views
The document presents a case study on an elastic integration framework for the UK government aimed at better managing data risks and compliance. It highlights challenges such as aging systems, technical debt, and the need for operational improvements while proposing solutions like data classification and transformation processes. The approach emphasizes treating data as an asset and utilizing advanced tools for monitoring and remediation to mitigate risks effectively.
Enterprise search and unstructured data with CGI and Elastic
- 1. Case Study: An Elastic Integration for UK Government to Control Risk April 2021
- 2. © 2021 CGI Inc. Public • Maintain IT availability • Maintain the data position • Minimal reconfiguration • Appetite for new services • Need to collaborate • Need to perform UK Government agency • Began in the 1980s • Multiple pre-cursor organisations • Multiple technology approaches Aggregation • Corporate responsibilities • People and infrastructure • Information Information Technology (IT) Organisational Change The Case Study
- 3. © 2021 CGI Inc. Public • Systems are ageing • Change is difficult to enact • Transformation is constrained • You cannot manage what you cannot measure • Compromised data migration • Increasing compliance Information Assets • Poorly defined • Impact of loss is unclear • Limited mapping to systems Information Management • Limited lifecycle enablement • High reliance on file share • High reliance on email Technical Debt Risk Unstructured Data The Reality and the Problem
- 4. © 2021 CGI Inc. Public Key Risks 01 02 04 03 Data Protection. Data Obligation & Right of Access. Regulatory Data should be treated as an asset. 3rd party audit is likely to lead to penalty. Financial Interoperability. Likelihood of intelligence failure. Practical threat and risk to life. Operational What data have we got? What do we need? Where does the data need to go? How do we protect data proportionately? Cyber & Transformation
- 5. © 2021 CGI Inc. Public Integrated Approach 01 02 04 03 Classify new data. Who are you? What do you do? Where are you? Control the Business Treat information as an Asset. Label data to enable transformation. Treat the risk. Act on Decisions Crawl the data. Index the data. Identify easy wins. Industrialise the Data Apply current intelligence, to legacy data. Apply policy thresholds. Identify key trends over time. Analyse at Scale
- 7. © 2021 CGI Inc. Public The User and Plug Ins • Extract • Transform • Tag/Write • Migrate • Delete Data Modelling Elastic Indexes FS Crawler MVP Classify File Share 7 Kibana Structured Data Sources Classification DB Modeller INSIGHT Outcome: Modelled Data Alignment and Risk Remediation METADATA + CONTENT Business Decisions VISUALISE DIRECTS Industrial Classification & ETL Services INSTRUCTS EXECUTES/SCHEDULES ANALYSE • The current business position • Changing data • Emerging risk DISCOVER & CONTAIN • Stop unstructured data growth • Provide insight • Measure the “now” BUSINESS ENGAGEMENT • Develop rules to apply at scale • Refine remediation proposals • Obtain risk acceptance of change RECONCILIATION & TRACKING • Measuring progress • Is the problem stabilising? Reducing? Growing? 1 2 3 Export Decisions 4 6 EXECUTE • Extract • Transform • Load • Migrate • Delete 5
- 8. © 2021 CGI Inc. Public The User and Plug Ins • Extract • Transform • Tag/Write • Migrate • Delete Data Modelling Elastic Indexes FS Crawler MVP Classify File Share Kibana Structured Data Sources Classification DB Modeller INSIGHT Outcome: Modelled Data Alignment and Risk Remediation METADATA + CONTENT Business Decisions VISUALISE DIRECTS Industrial Classification & ETL Services INSTRUCTS EXECUTES/SCHEDULES 1 2 3 4 6 5
- 9. © 2021 CGI Inc. Public Reporting and Insight 9 Data and Risk Ownership Overview
- 10. © 2021 CGI Inc. Public 10 Accessible Views of Ownership
- 11. © 2021 CGI Inc. Public 11 Tracking Data and Change Over Time
- 12. © 2021 CGI Inc. Public File Creation, Modification and Access Over Time
- 13. © 2021 CGI Inc. Public 13 File Type, Size and Count
- 14. © 2021 CGI Inc. Public 14 Understanding Sensitivity and Size
- 15. © 2021 CGI Inc. Public 15 Supporting Analysis and Making it Easy Filter / Navigate / Help
- 16. © 2021 CGI Inc. Public 16 Tagging and Aggregating Data to Model Decisions
- 17. © 2021 CGI Inc. Public 17 Tracking Progress
- 18. © 2021 CGI Inc. Public • 1 decision applies to millions of files • The approach can be re-used • GUIs can be developed for other purposes • Data can be identified for: • What it is • Where it is • Where it needs to go next 1. Tracking Data • Growth can be measured • Ownership can be applied • Sensitivity can be monitored 2. Risk Remediation • Decisions can be modelled • Meaningful reports for business are provided • Performance can be tracked 3. Scalability & Extensibility 4. Organisational Change Results and Outcomes
- 19. Questions Visit: www.cgi.com Adam Burr: [email protected] Andy Burston: [email protected] Insights you can act on Founded in 1976, CGI is among the largest IT and business consulting services firms in the world. We are insights-driven and outcomes-based to help accelerate returns on your investments. Across 21 industry sectors in 400 locations worldwide, our 76,000 professionals provide comprehensive, scalable and sustainable IT and business consulting services that are informed globally and delivered locally.