YDAL Barcelona
0 likesā¢452 views
YADL is a software defined deduplication library that provides a stream-based API for integrating dedupe capabilities into storage systems like file systems, block storage, and object storage. It offers configurable chunking options and can store chunks on disk or in memory. Future plans for YADL include distributed dedupe across multiple sites, locality-sensitive hashing, and wrappers to provide direct data access without the API.
YDAL Barcelona
- 1. 1 YADL āYet another dedup libraryā Ideators Dan Lambright (Principal Software Engineer) Luis Pabon (Principal Software Engineer) Joseph Fernandes (Senior Software Engineer) Red Hat
- 2. 2 What is YADL ? ā Software Defined Dedupe Solution ā User-Land Library for Storage Systems ā Stream based API's : inbound dedupe ā Integrates with ā File (eg: GlusterFS) ā Block (eg: Ceph, pblcache) ā Object (eg: Ceph , Open Stack Swift )
- 3. 3 What is YADL ? ā Configurable ā Fixed Chunk ā Variable Chunk ā Storage options ā On disk ā In memory
- 4. 4 File Block Object What is YADL ? YADL API YADL Engine Stream Stream Stream . . . . YADL Stores YADL Library
- 5. 5 Future ā Distributed Dedupe: YADL-Eco ā Optimal data distribution ā Network traffic sensitive ā SuperChunks : Aggregated Chunks ā Locality Sensitive Hashes ā YADL Shells ā Direct Data Access (without api) ā File, Object, iscsi etc ā Wrappers over YADL
- 6. 6 Developers Karthik, Sukumar, Sonal, Srinivas & Ewen Master of Computer Application, NMAMIT, Nitte ā India Dan Lambright Principal Software Engineer Red Hat Joseph Elwin Fernandes Senior Software Engineer Red Hat Luis Pabon Principal Software Engineer Red Hat
- 7. 7 NMAMIT, Nitte ? ā¦ with connections to EMC and Intel.. and now Red Hat! Gluster Conference 2015
- 8. 8 Put the thank you side here
Editor's Notes
- #5: Apply tools to new environments My pov: systems software and educator
- #6: Apply tools to new environments My pov: systems software and educator
- #8: Triplication is 200% overhead Isillon Raid5 ā 1 failures Raid6 - 2 Controller is local to server, need one for each
- #9: Triplication is 200% overhead Isillon Raid5 ā 1 failures Raid6 - 2 Controller is local to server, need one for each