MongoDB: An Introduction - june-2011
12 likes1,529 views
The document is an introduction to MongoDB and non-relational databases, discussing the limitations of relational databases in a web environment, such as schema evolution and write rate issues. It outlines different non-relational data models, including key-value stores, document stores, and graph databases, along with their functionalities and consistency models. Additionally, it explains how MongoDB addresses these challenges with its architecture, features like sharding and replica sets, and its support for advanced queries and various programming languages.
MongoDB: An Introduction - june-2011
- 1. MongoDB: An IntroductionChris WestinSoftware Engineer, 10gen© Copyright 2010 10gen Inc.
- 2. OutlineThe Whys of Non-Relational DatabasesVocabulary of the Non-Relational WorldMongoDB
- 3. Why did non-relational databases arise?Problems with relational databases in the web worldThe Whys of Non-Relational Databases
- 4. Problem - Schema EvolutionApplications are evolving all the timeApplications need new fieldsApplications need new indexesData is growing – sometimes very fastUsers need to be able to alter their schemas without making their data unavailableThe web world expects 24x7 serviceRDBMSs can have a hard time doing this
- 5. Problem – Write RatesReplication is a solution for high read loadsSooner or later, writing becomes a bottleneckSharding – partitioning a logical database across multiple database instancesJoins and aggregation become a problemDistributed transactions are too slow for the webManual management of shardsChoosing shard partitionsRebalancing shards
- 6. An introduction to terminology you’re going to be seeing a lotVocabulary of the Non-Relational World
- 7. Data ModelsA non-relational database’s data model determines the kinds of items it can contain and how they can be retrievedWhat can the system store, and what does it know about what it contains?The relational data model is about storing records made up of named, scalar-valued fields, as specified by a schema, or type definitionWhat kind of queries can you do?SQL is a manifestation of the kinds of queries that fall out of relational algebra
- 8. Non-Relational Data ModelsKey-value storesDocument storesColumn-oriented databasesGraph databases
- 9. Key-Value StoresA mapping from a key to a valueThe store doesn’t know anything about the the key or valueThe store doesn’t know anything about the insides of the valueOperationsSet, get, or delete a key-value pair
- 10. Document StoresThe store is a container for documentsDocuments are made up of named fieldsFields may or may not have type definitionse.g. XSDs for XML stores, vs. schema-less JSON storesCan create “secondary indexes”These provide the ability to query on any document field(s)Operations:Insert and delete documentsUpdate fields within documents
- 11. Column-Oriented StoresLike a relational store, but flipped around: all data for a column is kept togetherAn index provides a means to get a column value for a recordOperations:Get, insert, delete records; updating fieldsStreaming column data in and out of Hadoop
- 12. Graph DatabasesStores vertex-to-vertex edgesOperations:Getting and setting edgesSometimes possible to annotate vertices or edgesQuery languages support finding paths between vertices, subject to various constraints
- 13. Consistency ModelsRelational databases support transactionsCan only see committed changesCommit/abort span multiple changesRead-only transaction flavorsRead committed, repeatable read, etcClassic assumption: “I’m querying the one-and-only database”Scaling reads and writes introduce different problems
- 14. Replication - The 1st Breakdown of Consistency
- 15. Limitations of a Single MasterReplication can provide arbitrary read scalabilitySubject to coping with read-consistency issuesSooner or later, writing becomes a bottleneckPhysical limitations (seek time)Throughput of a single I/O subsystem
- 16. ShardingParitition the primary key space via hashingSet up a duplicate system for each shardThe write-rate limitation now applies to each shardJoins or aggregation across shards are problematicCan the data be re-sharded on a live system?Can shards be re-balanced on a live system?
- 17. Multi-Site OperationFailure of a single-master system’s masterA new master can be chosenBut what if there’s a network partition?Can the application continue in read-only mode?
- 18. DynamoNow a generic term for multi-master systemsWrites can occur to any nodeThe same record can be updated on different nodes by different clientsAll writes are replicated everywhere
- 19. Dynamo – the 2nd breakdown of consistencyCollisions can occurWho wins?A collision resolution strategy is requiredVector clockshttp://en.wikipedia.org/wiki/Vector_clockApplication access must be aware of this
- 21. Key Client Implementation ConcernsMonotonic readsCan my reads go back in time?Read-your-own-writesIf I issue a query immediately after an insert or update, will I see my changes?Uninterrupted writesAm I always guaranteed the ability to write?Conflict ResolutionDo I need to have a conflict resolution strategy?
- 22. Using a Single-Master SystemWhat does the intermediate agent or system do for…Monotonic reads?Read-your-own-writes?Uninterrupted writes?Conflict Resolution?
- 23. Using a Multi-Master SystemWhat does the intermediate agent or system do for…Monotonic reads?Read-your-own-writes?Uninterrupted writes?Conflict Resolution?
- 24. Where MongoDB fits in the non-relational worldMongoDB’s architecture and featuresSome real-world usersMongoDB
- 25. MongoDB is a Document StoreMongoDB stores JSON objects as BSON{ LastName: ‘Flintstone’, FirstName: ‘Fred’, …}Secondary Indexesdb.collection.ensureIndex({LastName : 1, FirstName : 1});Simple QBE-like query syntaxdb.collection.find({LastName : ‘Flintstone’});db.collection.find({LastName : { $gte : ‘Flintstone’});
- 26. MongoDB – Advanced QueriesGeo-spatial queriesCreate a geo indexFind points near a given point, sorted by radial distanceCan be planar or sphericalFind points within a certain radial distance, within a bounding box, or a polygonBuilt-in Map-ReduceThe caller provides map and reduce functions written in JavaScript
- 27. MongoDB is a Single-Master SystemA database is served by members of a “replica set”The system elects a primary (master)Failure of the master is detected, and a new master is electedApplication writes get an error if there is no quorum to elect a new masterReads continue to be fulfilled
- 29. MongoDB Supports ShardingA collection can be shardedEach shard is served by its own replica setNew shards (each a replica set) can be added at any timeShard key ranges are automatically balanced
- 30. MongoDB – Sharded Deployment
- 31. MongoDB Storage ManagementData is kept in memory-mapped filesServers should have a lot of memoryFiles are allocated as neededDocuments in a collection are kept on a list using a geographical addressing schemeIndexes (B*-trees) point to documents using geographical addresses
- 32. MongoDB Server ManagementReplica set members are aware of each otherA majority of votes is required to elect a new primaryMembers can be assigned priorities to affect the electione.g., an “invisible” replica can be created with zero priority for backup purposes
- 33. MongoDB AccessDrivers are available in many languages10gen supportedC, C# (.Net), C++, Erlang, Haskell, Java, JavaScript, Perl, PHP, Python, Ruby, ScalaCommunity supportedClojure, ColdFusion, F#, Go, Groovy, Lua, Rhttp://www.mongodb.org/display/DOCS/Overview+-+Writing+Drivers+and+Tools
- 34. MongoDB AvailabilitySourcehttps://github.com/mongodb/mongoServerLicense: AGPLhttp://www.mongodb.org/downloadsDriversLicense: Apachehttp://www.mongodb.org/display/DOCS/Drivers
- 35. MongoDB – Hosted Serviceshttp://www.mongodb.org/display/DOCS/Hosting+CenterMongoHQ, Mongo Machine, MongoLabRESTful access to collections
- 36. MongoDB SupportPaid Supporthttp://www.10gen.com/client-portal10gen Hosted MonitoringConsulting, trainingFree Supporthttp://groups.google.com/group/mongodb-userhttp://stackoverflow.com/questions/tagged/mongodb
- 37. MongoDB Usershttp://www.10gen.com/customershttp://www.10gen.com/presentationscraigslist: http://www.10gen.com/presentation/mongosf2011/craigslistbit.ly: http://blip.tv/mongodb/bit-ly-user-history-auto-sharded-3723147shutterfly: http://www.10gen.com/presentation/mongosv2010/shutterfly