MongoDB Aggregation MongoSF May 2011
12 likes2,200 views
This document discusses MongoDB's new aggregation framework, which provides a more performant and declarative way to perform data aggregation tasks compared to MapReduce. The framework includes pipeline operations like $match, $project, and $group that allow filtering, reshaping, and grouping documents. It also features an expression language for computed fields. The initial release will support aggregation pipelines and sharding, with future plans to add more operations and expressions.
![Computed ExpressionsAvailable in $project operationsPrefix expression languageAdd two fields: $add:[“$field1”, “$field2”]Provide a value for a missing field: $ifnull:[“$field1”, “$field2”]Nesting: $add:[“$field1”, $ifnull:[“$field2”, “$field3”]]Other functions….And we can easily add more as required](https://image.slidesharecdn.com/aggregation-may-2011-110526134052-phpapp02/85/MongoDB-Aggregation-MongoSF-May-2011-6-320.jpg)
MongoDB Aggregation MongoSF May 2011
- 1. MongoDB’s New Aggregation FeaturesChris Westin© Copyright 2010 10gen Inc.
- 2. What problem are we solving?Map/Reduce can be used for aggregation…Currently being used for totaling, averaging, etcMap/Reduce is a big hammerSimpler tasks should be easierShouldn’t need to write JavaScriptAvoid the overhead of JavaScript engineWe’re seeing requests for help in handling complex documentsSelect only subdocuments or arrays
- 3. How will we solve the problem?Our new aggregation frameworkDeclarative frameworkNo JavaScript requiredDescribe a chain of operations to applyExpression evaluationReturn computed valuesFramework: we can add new operations easilyC++ implementationHigher performance than JavaScript
- 4. Aggregation - PipelinesAggregation requests specify a pipelineA pipeline is a series of operationsConceptually, the members of a collection are passed through a pipeline to produce a resultSimilar to a command-line pipe
- 5. Pipeline Operations$matchUses a query predicate (like .find({…})) as a filter$projectUses a sample document to determine the shape of the result (similar to .find()’s optional argument)This can include computed values$groupAggregates items into buckets defined by a key
- 6. Computed ExpressionsAvailable in $project operationsPrefix expression languageAdd two fields: $add:[“$field1”, “$field2”]Provide a value for a missing field: $ifnull:[“$field1”, “$field2”]Nesting: $add:[“$field1”, $ifnull:[“$field2”, “$field3”]]Other functions….And we can easily add more as required
- 7. Projections$project can reshape results$unwind expression doles out array values one at a timePull fields from nested documents to the topPush fields from the top down into new virtual documents
- 8. Grouping$group aggregation expressionsTotal of column values: $sumAverage of column values: $avgCollect column values in an array: $push
- 9. Demo(See script at https://gist.github.com/993733)
- 10. Usage TipsUse $match in a pipeline as early as possibleThe query optimizer can then be used to choose an index and avoid scanning the entire collection
- 11. Driver SupportInitial version is a commandFor any language, build a JSON database object, and execute the command{ aggregate : <collection>, pipeline : {…} }Beware of command result size limit
- 12. When is this being released?In final development nowExpect to see this in the near future
- 13. Sharding supportInitial release will support shardingMongos analyzes pipeline, and forwards operations up to $group to shards; combines shard server results and continues
- 14. Pipeline Operations – Future Plans$sortSorts the document stream according to a key$outSaves the document stream to a collectionSimilar to M/R $out, but with sharded output
- 15. Expressions – Future PlansDate field extractionGet year, month, day, hour, etc, from DateDate arithmetic