Sharding with MongoDB (Eliot Horowitz)

Sharding with MongoDB (Eliot Horowitz)

• 1.
  open-source, high-performance, document-oriented database
• 2.
  Why Sharding • Verticalscaling is limited by physics and cost • Hard to scale vertically in the cloud • Can scale wider than higher
• 3.
  What is Sharding •ad-hoc partitioning • consistent hashing (Dynamo) • range based partitioning (BigTable/PNUTS)
• 4.
  Overview • Automatic partitioningand management • range based • Can convert from single master to sharded system with 0 downtime • Almost no functionality lost over single master • Fully consistent
• 5.
  Range Based • collectionis broken into chunks by range
• 6.
  Auto-sharding Shards mongo mongo mongo .. Config mongo mongo mongo Servers mongod mongod mongod mongos mongos .. client
• 7.
  Config Servers • 3of them • changes are made with 2 phase commit • if any are down, meta data goes read only • system is online as long as 1/3 is up
• 8.
  Shards • Can bemaster, master/slave or replica sets • Replica sets gives sharding + full auto- failover • Regular mongod processes
• 9.
  mongos • Sharding Router •Acts just like a mongod to clients • Can have 1 or as many as you want • Can run on appserver so no extra network traffic
• 10.
  Writes • Inserts :require shard key, routed • Removes: routed and/or scattered • Updates: routed or scattered
• 11.
  Queries • By shardkey: routed • sorted by shard key: routed in order • by non shard key: scatter gather • sorted by non shard key: distributed merge sort
• 12.
  Operations • split: breakinga chunk into 2 • migrate: move a chunk from 1 shard to another
• 13.
  Balancing • distributes chunksautomatically • can look at: disk ops, cpu, data size
• 14.
  Multi DataCenter • intelligentgeo honing • auto failover
• 15.
  Sharding Limitations • Uniqueindexes not based on shard key • current version limited to ~ 20 petabytes
• 16.
  After Party! Download MongoDB http://www.mongodb.org and let us know what you think @eliothorowitz @mongodb