Cloud Computing y Big Data, próxima frontera de la innovación
2 likes•390 views
The document discusses the intersection of cloud computing and big data as a critical frontier for innovation, emphasizing their capacity to handle vast amounts of data generated by modern scientific endeavors. It outlines the challenges posed by big data, particularly in storage, management, and analysis, necessitating new technologies and methodologies such as NoSQL systems and enhanced data processing techniques. Additionally, it highlights the necessity for the scientific community to adapt and innovate in order to fully leverage the potential of big data and cloud computing.
Cloud Computing y Big Data, próxima frontera de la innovación
- 1. Cloud Computing y Big Data, próxima frontera de la innovación Cloud Computing and Big Data, the next frontier of innovation Jordi Torres, UPC-BSC Madrid, 21 Marzo 2013
- 2. HOW DID SCIENCE START?
- 3. Source: Prof. Mateo Valero, BSC-CNS 2010
- 4. Source: Prof. Mateo Valero, BSC-CNS 2010
- 5. HOW IS SCIENCE ADVANCING TODAY?
- 6. Source: Prof. Mateo Valero, BSC-CNS 2010
- 7. Source: Prof. Mateo Valero, BSC-CNS 2010
- 8. MATHEMATICAL CALCULATIONS? WHERE?
- 10. MN3 Cores/chip 8 Chip/node 2 Compute Cores/node 16 Nodes 3028 Total cores 48448 Freq. 2,6 Gflops/core 20,8 Performance Gflops/node 332,8 Total Tflops 1000,0 GB/core (GB) 2 Memory GB/node (GB) 32 Total (TB) 96,89 Latency (μs) 0,7 Network Bandwidth (Gb/s) 40 Storage (TB) 2000 Consumption (KW) 1080
- 11. FOR SOME SPANISH RESEARCH GROUPS!
- 12. AND… FOR THE REST OF THE WORLD?
- 14. CLOUD COMPUTING?
- 16. Source: http://www.facebook.com/media/ set/?set=a.190842620965185.47008.140375289345252 40 Mw 28.000 m2
- 17. Foto: Google
- 18. HUGE DATA CENTERS Foto: Google > football pitch x 4
- 22. Different IT production Foto: J.T.
- 23. CLOUD COMPUTING: IT as a service On-demand self-service Pay per use Rapid elasticity Ubiquitous access .... Source: http://www.telegraph.co.uk/technology /reviews/9241719/Power-Ethernet-Sockets-review.html
- 24. Example of benefits (IaaS): 1 computer in a rack for 120 hours 120 computers in three racks for 1 hour Idea : Tutorial SC2011 - Robert Grossman
- 25. AND DATA?
- 26. Source: http://www.docuciencia.es/2009/05/lhc-el-acelerador-de-particulas/ “… the LHC produces 1PetaByte of data every second, big data and lack of computing resources were becoming the European Organization for Nuclear Research’s biggest IT challenges…” Source: computerweekly.com/news/2240173897/CERN-adopts -OpenStack-private-cloud-to-solve-big-data-challenges
- 27. 1 Gigabyte (GB) = 1.000.000.000 byte 1 Terabyte (TB) = 1.000 Gigabyte (GB) 1 Petabyte (PB) = 1.000.000 Gigabyte (GB) 1 Exabyte (EB) = 1.000.000.000 Gigabyte (GB) 1 Zettabyte (ZB) = 1.000.000.000.000 (GB)
- 28. Deluge of data created daily Source: Economist , Feb 25th, 2010 http://www.economist.com/node/15579717
- 30. BIG DATA? Big Data is data that exceeds the storing, processing and managing capacity of conventional systems.
- 31. BIG DATA? The reason is that the data is too big, moves too fast, or doesn’t fit the structures of our current systems’ architectures.
- 32. BIG DATA? Moreover, to gain value from this data, we must change the way to analyze it.
- 33. BIG DATA? Big Data is data that exceeds the storing, processing and managing capacity of conventional systems. The reason is that the data is too big, moves too fast, or doesn’t fit the structures of our current systems’ architectures. Moreover, to gain value from this data, we must change the way to analyze it.
- 34. NEW CHALLENGES that must be addressed urgently, in order to respond to the needs of the advancement of science 1. Storing 2. Managing 3. Processing 4. Analyzing
- 36. But scanning disks… assume 100MB/sec
- 37. But scanning disks… assume 100MB/sec more than 5 hours
- 38. approach: massive parallelism assume 20.000 disks: scanning 2 TB takes 1 second Source: http://www.google.com/about/datacenters/gallery/images/_2000/IDI_018.jpg
- 39. 1 Data processing challenges Rethinking data processing is required: MapReduce, Storm, S4,… Source: http://www.google.com/about/datacenters/gallery/images/_2000/IDI_018.jpg
- 40. 2 Data storage challenges New Storage technologies are required HHD 100 cheaper than RAM But 1000 times slower RAM vs HHD Solid- state drive (SSD) Not volatile Present solutions: Storage Class Memory (SCM) Research:
- 41. 3 Data management challenges Relational DB can’t support everything Example: eventual consistency Solution: “NoSQL systems” Research: New management systems Source: gigaom.com/cloud/big-data- and-nosql-march-to-the-enterprise/ 43
- 42. 4 Obtaining value from data The information is non actionable knowledge - Data prediction using data mining & + machine learning techniques Value Volume Information Research: The majority of algorithms function well in thousands of + registers, however at the moment they Knowledge - are impractical for thousands of milions.
- 43. Cloud Computing and Big Data: the next frontier of science and innovation
- 44. Thank you for your attention www.JordiTorres.org - @JordiTorresBCN www.smartcityexpo.com www.bsc.es/eBusiness Autonomic Systems and e-Business Platforms research line at BSC/UPC
- 46. Thank you for your attention www.JordiTorres.org - @JordiTorresBCN www.smartcityexpo.com www.bsc.es/eBusiness Autonomic Systems and e-Business Platforms research line at BSC/UPC