Big Data & the importance of Data Science
66 likes15,373 views
The document discusses the concept of big data, emphasizing its characteristics defined by volume, velocity, variety, and veracity, and highlights the importance of data science. It outlines the potential benefits of big data, including improved product development and organizational insights, while also addressing challenges such as hype, skills shortages, and the need for strategic problem-solving. A call to action is made for collaboration in building systems that harness big data effectively.
Big Data & the importance of Data Science
- 1. Big Data & the importance of Data Science 18 december 2014 @wimvanleuven [email protected] 1
- 3. –Edd Dumbill “Big data is data that exceeds the processing capacity of conventional database systems. The data is too big, moves too fast, or doesn’t fit the strictures of your database architectures.” 3 http://radar.oreilly.com/2012/01/what-is-big-data.html What is Big Data?
- 4. The 3 V’s of Big Data 4 • Volume • Velocity • Variety • (Veracity)
- 6. … moves to fast … 6
- 7. … doesn’t fit … 7
- 8. … what? 8
- 9. New tools and technologies to store and process all data on a cluster of commodity hardware so that the system acts as one, is resilient and scales linearly. 9 What is Big Data? — revisited
- 10. So what? 10 the data lake is a large data pool in which the schema and data requirements are not defined until the data is queried, processed, analysed or delivered as information to the end-user
- 11. –??? “We don’t do Hadoop because we have Big Data; we do Big Data because we have Hadoop.” 11 So what?
- 12. –Matt Ehrlichman “In the years ahead, the same power that big data awards enterprise companies will be the norm for small business.” 12 So what? http://blogs.wsj.com/accelerators/2014/10/31/matt-ehrlichman-big-data-for-small-firms/
- 13. 13 What does Big Data enable? • Combine data from within and without your organisation • Build new products and services • Analyse all data (e.g. 5TB historic event data at rest in Oracle db)
- 14. Big Data is no panacea 14 • First decide what problem you want to solve; pick a real business problem to add immediate value • Start small, the technology is made for linear scalability (a 3-node cluster is a cluster!) • Then become lean: learn through experimentation
- 15. Big Data challenges • Beware of hype, Big Data - washing and fad • Tech infancy • IT | Biz • Data is hard • Lack of skills! shameless self plug: BigBoards! 15
- 16. Big Data opportunity • Big Data is here to stay • Vendor market is HUGE and will grow massively as Big Data will blend in within the datacenter • However, the Practitioner market can deliver EXPONENTIALLY more value 16
- 17. 17 It is time to band together and build these systems that deliver this kind of value for fun for profit for good for Belgium? Call for Action
- 18. https://www.ted.com/talks/susan_etlinger_what_do_we_do_with_all_this_big_data “Data doesn't create meaning. We do.” –Susan Etlinger 18 Data Science FTW