Proactive Data Containers (PDC): An Object-centric Data Store for Large-scale Computing Systems
1 like201 views
The document discusses Proactive Data Containers (PDC), an advanced object-centric data storage solution aimed at addressing challenges in large-scale computing systems due to extreme parallelism and massive scientific data. It highlights the features of PDC, such as transparent data management and efficient I/O performance, which are crucial for scalable data management and automation across various domains including simulations and experimental data. Additionally, the document emphasizes the importance of rich metadata and a seamless interface for data handling to optimize performance in high-performance computing environments.
Proactive Data Containers (PDC): An Object-centric Data Store for Large-scale Computing Systems
- 1. Proactive Data Containers (PDC): An Object-centric Data Store for Large-scale Computing Systems Suren Byna Lawrence Berkeley National Lab (LBNL), Berkeley Co-authors Quincey Koziol (LBNL), Venkat Vishwanath (ANL), Jerome Soumagne (THG), Houjun Tang (LBNL), Kimmy Mu (THG), Bin Dong (LBNL), Richard Warren (THG), François Tessier (ANL, now @ CSCS), Teng Wang (LBNL), and Jialin Liu (LBNL)
- 2. ▪ Extreme parallelism ▪ Massive Data ▪ Hierarchical storage Scalable data management – Three disrupting trends 2
- 3. 3 Extreme parallelism Summit, ORNL Sierra, LLNL Sunway Taihulight NSC Wuxi, China Trinity, LANL Cori, LBNL Summit - ~2.4M cores - ~143 PFlops - 9.7 MW Sierra - ~1.5M cores - ~94 PFlops - 7.4 MW Taihulight - ~10.6M cores - ~93 PFlops - 15 MW
- 4. § Simulations – Multi-physics (FLASH) – 10 PB – Cosmology (NyX) – 10 PB – Plasma physics (VPIC) – 1 PB § Experimental and observational data (EOD) – LHC (100 PB), – LSST (60 PB), – Genomics (100 TB to 1 PB) Massive scientific data FLASH NyX VPIC 4 LHC LSST Genomics
- 5. Hierarchical and heterogeneous storage 5 IO Gap Memory Parallel file system (Lustre, GPFS) Archival Storage (HPSS tape) IO Gap Shared burst buffer Memory Parallel file system (Lustre, GPFS) Archival Storage (HPSS tape) Memory Parallel file system Archival storage (HPSS tape) Shared burst buffer Node-local storage Conventional Current Eg. Cori @ NERSC Upcoming Campaign storage
- 6. Reading and writing data on scalable systems 6 ▪ Types of parallel I/O • 1 writer/reader, 1 file • N writers/readers, N files (File-per-process) • N writers/readers, 1 file • M writers/readers, 1 file – Aggregators – Two-phase I/O • M aggregators, M files (file-per-aggregator) – Variations of this mode P0 P1 Pn-1 Pn … file.0 1 Writer/Reader, 1 File P0 P1 Pn-1 Pn … file.0 n Writers/Readers, n Files file.1 file.n-1 file.n P0 P1 Pn-1 Pn … n Writers/Readers, 1 File File.1 P0 P1 Pn-1 Pn … file.0 M Writers/Readers, M Files file.m P0 P1 Pn-1 Pn … M Writers/Readers, 1 File File.1
- 7. Scalable Storage Systems: Challenges 7 Memory Disk-based storage Archival storage (HPSS tape) Shared burst buffer Hardware Node-local storage Campaign storage Software High-level I/O lib (netCDF, HDF5, etc.) IO middleware (POSIX, MPI-IO) IO forwarding Parallel file systems Applications Usage … Data (in memory) IO software … Files in file system • Challenges – POSIX-IO semantics hinder scalability and performance of file systems and IO software – Multi-level hierarchy complicates data movement, especially if user has to be involved Tune middleware Tune file systems
- 8. Scalable data management requirements Use case Domain Sim/EOD/ analysis Data size I/O Requirements FLASH High-energy density physics Simulation ~1PB Data transformations, scalable I/O interfaces, correlation among simulation and experimental data CMB / Planck Cosmology Simulation, EOD/ Analysis 10PB Automatic data movement optimizations DECam & LSST Cosmology EOD/Analysis ~10TB Easy interfaces, data transformations E3SM Climate Simulation ~10PB Async I/O, derived variables, automatic data movement TECA Climate Analysis ~10PB Data organization and efficient data movement HipMer Genomics EOD/Analysis ~100TB Scalable I/O interfaces, efficient and automatic data movement 8 Easy interfaces and superior performance Transparent data management Information capture and management 8
- 9. Next Gen Storage – Proactive Data Containers (PDC) Memory Disk-based storage Archival storage (HPSS tape) Shared burst buffer Hardware Node-local storage Campaign storage Software High-level API Applications Usage … Data (in memory) 9
- 10. ▪ Object-centric data access interface § Simple put, get interface § Array-based variable access ▪ Transparent data management § Data placement in storage hierarchy § Automatic data movement ▪ Information capture and management § Rich metadata § Connection of results and raw data with relationships Persistent Storage API BB FS Lustre DAOS … PDC System – High-level Architecture 10
- 11. ▪ Object-level interface – Create – containers and objects – Add attributes – Put object – Get object – Delete object ▪ Array-specific interface – Create regions – Map regions in PDC objects – Lock – Release 11 Object-centric PDC Interface J. Mu, J. Soumagne, et al., “A Transparent Server-managed Object Storage System for HPC”, IEEE Cluster 2018 Proactive Data Container Container Dataset KV-Store Group <root> A B C D E F PDC Locus Dataset KV-Store Group Container Collection Locus Container: X <root> A B C D E F Container: W <root> A B C D E F Container: Z <root> A B C D E F Collection: P Collection: Q PDC Collection Container: X <root> A B C D E F Container: W <root> A B C D E F Container: Z <root> A B C D E F Container: Y <root> A B C D E F Proactive Data Container Container <root> A B C D E F Key
- 12. ▪ Object-level interface – Create – containers and objects – Add attributes – Put object – Get object – Delete object ▪ Array-specific interface – Create regions – Map regions in PDC objects – Lock – Release 12 Object-centric PDC Interface J. Mu, J. Soumagne, et al., “A Transparent Server-managed Object Storage System for HPC”, IEEE Cluster 2018
- 13. ▪ Usage of compute resources for I/O – Shared mode – Compute nodes are shared between applications and I/O services – Dedicated mode – I/O services on separate nodes ▪ Transparent data movement by PDC servers – Apps map data buffers to objects and PDC servers place and manage data – Apps query for data objects using attributes ▪ Superior I/O performance 13 Transparent data movement in storage hierarchy H. Tang, S. Byna, et al., “Toward Scalable and Asynchronous Object-centric Data Management for HPC”, IEEE/ACM CCGrid 2018 0 350 700 1050 124 248 496 992 1984 3968 7936 15872 Time in seconds Number of processes HDF5 read (Lustre) PLFS read (Lustre) PDC read (Lustre) HDF5 read (BB) PDC read (BB) 0 250 500 750 124 248 496 992 1984 3968 7936 15872 Time in seconds Number of processes HDF5 write (Lustre) PLFS write (Lustre) PDC write (Lustre) HDF5 write (BB) PDC write (BB)
- 14. ▪ Flat name space ▪ Rich metadata – Pre-defined tags that includes provenance – User-defined tags for capturing relationships between data objects ▪ Distributed in memory metadata management – Distributed hash table and bloom filters used for faster access 14 Metadata management H. Tang, S. Byna, et al., “SoMeta: Scalable Object-centric Metadata Management for High Performance Computing”, to be presented at IEEE Cluster 2017
- 15. ▪ Take home message – Scalable storage systems impacted by: • Extreme level of parallelism • Massive amounts of scientific data • Transforming storage architectures – Proactive data containers • Object-centric interfaces • Transparent data movement in storage hierarchies • Scalable management of extensive metadata 15 Conclusions