DAMA Webinar: The Theory of Everything - Is it Time to Rethink Data Management?
10 likes•2,242 views
The document discusses the distinction and relationship between data, information, knowledge, and decision-making, emphasizing that data is raw input that becomes useful information through processing. It critiques the artificial separation of structured and unstructured data and advocates for a unified approach to data governance. The document suggests that effective data management requires clear definitions, governance, and a strategic vision that aligns with business objectives.
DAMA Webinar: The Theory of Everything - Is it Time to Rethink Data Management?
- 1. The Theory of Everything Is it Time to Rethink Image © Thomas Leuthard The best source of knowledge is experience Data Management?
- 2. Your Presenter • Founder of Equillian • Experts in Enterprise Information Management • Three founding values – Independence – Passion – Knowledge • Jon Evans • Information Strategist • Self-confessed Data Quality geek • @MadAboutData
- 4. Image © Thomas Leuthard The best source of knowledge is experience Data or information?
- 5. W W W. E Q U I L L I A N . C O M 5! In the blue corner… In the red corner… The “data-philes” The “info-holics”
- 6. Data Asset Data Quality Data Governance Data Architecture ….. ….. ….. And it doesn’t stop there… W W W. E Q U I L L I A N . C O M 6! Information Asset Information Quality Information Governance Information Architecture ….. ….. ….. Is it any wonder our stakeholders are confused?
- 7. The quality of our data ultimately affects the quality of our decisions The Data Value Chain • Data is the digital representation of objects and events (“raw input”) • Information is data that has been collated and organised (“data in context”) • Knowledge is the understanding we derive through interpreting information (“insight”) • Decisions are the judgements we make based on our acquired knowledge (“outcomes”) W W W. E Q U I L L I A N . C O M 7! Data Information Knowledge Decision is collated from is derived from is based on Raw Refined
- 8. The quality of our data ultimately affects the quality of our decisions An Example • Data is the digital representation of objects and events (“raw input”) • Information is data that has been collated and organised (“data in context”) • Knowledge is the understanding we derive through interpreting information (“insight”) • Decisions are the judgements we make based on our acquired knowledge (“outcomes”) W W W. E Q U I L L I A N . C O M 8! Data Information Knowledge Decision is collated from is derived from is based on 21, 6, 2016 Today’s date is 21st June 2016 My summer break is only 4 weeks away I’d better book my flights and hotel
- 9. Data versus Information W W W. E Q U I L L I A N . C O M 9! Data Information is collated from • Raw • Limited context • Implicit relationships • Processed • Meaningful context • Explicit relationships That’s settled then – information is simply data that’s been through a degree of processing to make it more useful So at what point does data become information?
- 10. W W W. E Q U I L L I A N . C O M 10! A file containing raw sales figures A formatted report that lists sales figures ordered by territory A formatted report that compares sales performance across different territories Data Information
- 11. W W W. E Q U I L L I A N . C O M 11! The Data Continuum Raw Refined Data Information A file containing raw sales figures A formatted report that compares sales performance across different territories A formatted report that lists sales figures ordered by territory
- 12. Two Different Perspectives W W W. E Q U I L L I A N . C O M 12! A file containing raw sales figures A formatted report that compares sales performance across different territories A formatted report that lists sales figures ordered by territory It’s all just data, but some is more processed It’s all information, but some is just more usable The Machine Perspective The Human Perspective
- 13. • Humans operate in the information world • When we see data, our brains sub-consciously convert it to information It’s time to think differently… We have two parallel and co-existent worlds – the data world and the information world • Machines operate in the data world • We use them to store data so we can readily access it as information • Information is simply the human interpretation of data • To a machine, your sophisticated sales report is just data • To a human, a set of raw sales figures is still information
- 14. If a tree falls in a forest and no one is around to hear it, does it make a sound? If data is stored on a computer and no one ever sees it, is it information?
- 15. W W W. E Q U I L L I A N . C O M 15! Data Information Knowledge Decision Raw Refined Information (Continuum) Knowledge Decision Data (Continuum) Knowledge Decision Raw Refined
- 16. W W W. E Q U I L L I A N . C O M 16! So where does that leave us?
- 17. Do we need to change our vocabulary? W W W. E Q U I L L I A N . C O M 17! INFOVERSITY Data ?No, let’s just agree that we’re talking about exactly the same thing from slightly different perspectives
- 18. Image © D. Sharon Pruitt The best source of knowledge is experience What about unstructured data?
- 19. What is unstructured data? • Data that has no structure? • Documents & reports? • Audio & video? W W W. E Q U I L L I A N . C O M 19! OK, so what is structured data? • Data that’s stored in a relational database? But surely all data has some structure But all documents are structured to some degree and even language follows the structural rules of grammar But don’t media files conform to defined standards and include structured meta-data? But there lots of alternative approaches to storing data nowadays, and I wouldn’t describe them as unstructured… Oh dear, have we confused matters again?
- 20. W W W. E Q U I L L I A N . C O M 20! A database table containing monthly sales figures An extract of the monthly sales figures saved as an Excel file The monthly board report, which shows a table of monthly sales figures on page 6 Structured Unstructured
- 21. W W W. E Q U I L L I A N . C O M 21! The Data Continuum Firm structure Fluid structure Structured Unstructured A database table containing monthly sales figures The monthly board report, which shows a table of monthly sales figures on page 6 An extract of the monthly sales figures saved as an Excel file
- 22. The Data Continuum W W W. E Q U I L L I A N . C O M 22! RawRefined Firm Fluid A database table containing monthly sales figures The monthly board report, which shows a table of monthly sales figures on page 6 An extract of the monthly sales figures saved as an Excel file A file containing raw sales figures A formatted report that compares sales performance across different territories
- 23. W W W. E Q U I L L I A N . C O M 23! RawRefined Firm Fluid A database table containing monthly sales figures The monthly board report, which shows a table of monthly sales figures on page 6 An extract of the monthly sales figures saved as an Excel file A file containing raw sales figures A formatted report that compares sales performance across different territories Which of these would you want to avoid falling into the hands of a competitor?
- 24. So is there such a thing as unstructured data? Or are we just managing a continuum of data? Maybe we should ask a couple of our employees…
- 25. Bill (Database Administrator)Betty (Records Manager) I look after my company’s documents and files – I don’t really get involved with data When I joined, we kept lots of our information on paper…and still do… …so I quickly learned the importance of only keeping the stuff we really need I look after my company’s databases – I don’t really get involved with documents When I joined, we only had one server and I kept a close watch on disk space… …but now hardware is so cheap, if we run out of storage, we just buy more! Images © Thomas Leuthard
- 26. It’s time to think differently… Images © Thomas Leuthard
- 27. We’ve created an artificial and unhelpful separation of structured and unstructured data We need to cross- pollinate our skills and knowledge to bring them back together again Just because storage is cheap doesn’t mean we can do without a retention and archiving strategy Just because we use SharePoint doesn’t mean we can avoid defining our document standards Images © Thomas Leuthard
- 28. W W W. E Q U I L L I A N . C O M 28! Data Information Knowledge Decision Raw Refined Information (Continuum) Knowledge Decision What is knowledge? • “the understanding we derive through interpreting information” • “information and skills acquired through experience or education” • It’s the documented (and undocumented) “learnings” that tell us how to run our business • Includes methods, procedures, best practice, patents, trade secrets…
- 29. W W W. E Q U I L L I A N . C O M 29! Data Information Knowledge Decision Raw Refined Information (Continuum) Knowledge Decision Knowledge Management is so often the poor relation of Data Management • “it’s all in people’s heads so there’s nothing to manage” • “we have a folder somewhere that contains all the procedures” • “we don’t really have a formal approach – everyone looks after their own stuff” How well do you manage the knowledge in your organisation?
- 30. W W W. E Q U I L L I A N . C O M 30! Data Information Knowledge Decision Raw Refined Information (Continuum) Knowledge DecisionDecisionInformation (Continuum) Data (Continuum) Knowledge Decision Raw Refined DecisionData (Continuum) Solution: just manage it in the same way as your other information assets
- 31. Image © Jenny Downing The best source of knowledge is experience Where does big data fit in?
- 32. n. data big data Image © Eric Fischer
- 33. The Data Continuum W W W. E Q U I L L I A N . C O M 33! RawRefined Firm Fluid The data continuum isn’t limited to 2 dimensions – we could include volume, velocity, variety…
- 34. It’s time to think differently… Ultimately, it doesn’t matter whether our data is big or small, firm or fluid, raw or refined In all cases we need… • Good definitions so we understand what we’re dealing with • Clear processes for managing it through its lifecycle • Sophisticated techniques for exploiting its value • Strong governance to ensure it’s being treated as an asset • A robust technical infrastructure to support all of the above
- 35. W W W. E Q U I L L I A N . C O M 35! Data Architecture Management Data Development Data Operations Management Data Security Management Reference & Master Data Management Data Warehousing & Business Intelligence Management Document & Content Management Meta-Data Management Data Quality Management Data Governance DMBoK wheel courtesy of DAMA International
- 36. W W W. E Q U I L L I A N . C O M 36! Data Architecture Management Data Development Data Operations Management Data Security Management Reference & Master Data Management Data Warehousing & Business Intelligence Management Document & Content Management Meta-Data Management Data Quality Management Data Governance Governance Exploitation Management Definition Infrastructure DMBoK wheel courtesy of DAMA International
- 37. Round is the perfect shape for many things… W W W. E Q U I L L I A N . C O M 37! but unfortunately, data management isn’t perfect
- 38. The DAMA Square? W W W. E Q U I L L I A N . C O M 38! Data Governance Data Architecture Management DataOperationsManagement DataSecurityManagement DataDevelopment Reference & Master Data Management Data Warehousing & Business Intelligence Document & Content Management Meta-Data Management Data Exploitation Data Management Data Definition Data Governance Data Infrastructure Data Quality Management I like the symmetry of the wheel, but prefer the transparency of the square
- 39. W W W. E Q U I L L I A N . C O M 39! Data Governance Data Architecture Management DataOperationsManagement DataSecurityManagement DataDevelopment Reference & Master Data Management Data Warehousing & Business Intelligence Document & Content Management Meta-Data Management Data Exploitation Data Management Data Definition Data Governance Data Infrastructure Data Quality Management We need to formalise our data quality policies, standards, roles and responsibilities We need to ensure the data definitions and data quality rules are documented We need to identify the structure, location and flow of data to guide our data quality efforts Enablers Improved data quality leads to more accurate insight and better decision making Beneficiaries High quality master and reference data helps to improve consistency and simplify integration Data Quality in Context
- 40. Image © D. Sharon Pruitt The best source of knowledge is experience Is our strategy working?
- 41. Military Strategy “The planning, coordination, and general direction of military activities to meet overall political and military objectives” W W W. E Q U I L L I A N . C O M 41!
- 42. Data Strategy “The planning, coordination, and general direction of data activities to meet overall business objectives” W W W. E Q U I L L I A N . C O M 42!
- 43. Developing a Data Strategy • Step 1 – Conduct a data management maturity assessment • Step 2 – Determine that the overall maturity is extremely low (just like everyone else) • Step 3 – Endure a painful meeting with senior execs where the full enormity of the challenge is laid bare • Step 4 – Agree some short term tactical fixes, because anything else is deemed too difficult • Step 5 – Return to your desk, cry into your coffee and carry on as before W W W. E Q U I L L I A N . C O M 43! Sound familiar?
- 44. It’s time to think differently… A strategy… • is a plan of activities to get us from the current state to a desired future state • is often clouded by negative perceptions of our starting point • can sometimes turn into nothing more than a series of short-term tactical fixes • ideally needs to be driven by a high-level business vision A vision… • is a picture of our desired future state created by the business • focuses on our future aspirations, rather than our current issues • takes a long-term view, which can be broken down into interim milestones • provides the basis for a more detailed strategy aligned with our business Don’t start your strategy until you’ve developed your vision
- 45. Developing a Data Vision • Step 1 – Engage with key stakeholders to understand the future role of data from a business perspective • Step 2 – Analyse the business drivers, opportunities and challenges to identify a set of underlying themes • Step 3 – Switch negatives to positives and craft a description of a brighter future that aligns with each of themes • Step 4 – Present your themes to senior execs using imagery and metaphors to bring your vision alive • Step 5 – Agree that it will take sustained effort to realise the vision and carve each theme into interim milestones • Step 6 – Keep the vision at the forefront as you develop it into a fully fledged strategy W W W. E Q U I L L I A N . C O M 45!
- 46. Images © Jenny Downing
- 47. Image © D. Sharon Pruitt The best source of knowledge is experience Is it time for a rethink?
- 48. Information is simply our interpretation of the complex world around us But in our rush to push the limits of achievement and embrace the possibilities, we’ve confused our thinking and created artificial barriers where none should exist At the end of the day, it doesn’t matter whether data is big or small, firm or fluid, raw or refined, created by man or created by machine – data is just data Our technology might have advanced, but our goal remains the same – a unified approach to governing, defining, managing and exploiting our data Digitising information and using computers to store it as data have helped us achieve things we never thought were possible even 10 years ago
- 50. Training changes the way people act Education changes the way people think W W W. E Q U I L L I A N . C O M [email protected] @MadAboutData