MongoDB Introduction talk at Dr Dobbs Conference, MongoDB Evenings at Bangalore and Delhi
12 likes7,613 views
The document introduces MongoDB as an open-source, high-performance document database that is horizontally scalable and supports flexible schema design. It covers MongoDB's features such as automated replication, failover, real-time aggregation, and its ability to work with multiple programming languages, making it suitable for building applications like a blog. The document also provides steps for getting started with MongoDB, including its installation and creating a RESTful API with Python using the Flask microframework.
![MongoDB ERD
User
·Name
·Email address
Article
·Name
·Slug
·Publish date
·Text
·Author
Comment[]
·Comment
·Date
·Author
Tag[]
·Value
Category[]
·Value](https://image.slidesharecdn.com/mongodb-140412093504-phpapp02/85/MongoDB-Introduction-talk-at-Dr-Dobbs-Conference-MongoDB-Evenings-at-Bangalore-and-Delhi-29-320.jpg)
![# Python dictionary (or object)
>>> article = { ‘title’ : ‘Schema design in MongoDB’,
‘author’ : ‘prasoonk’,
‘section’ : ‘schema’,
‘slug’ : ‘schema-design-in-mongodb’,
‘text’ : ‘Data in MongoDB has a flexible schema.
So, 2 documents needn’t have same structure.
It allows implicit schema to evolve.’,
‘date’ : datetime.utcnow(),
‘tags’ : [‘MongoDB’, ‘schema’] }
>>> db[‘articles’].insert(article)
Design schema.. In application code](https://image.slidesharecdn.com/mongodb-140412093504-phpapp02/85/MongoDB-Introduction-talk-at-Dr-Dobbs-Conference-MongoDB-Evenings-at-Bangalore-and-Delhi-39-320.jpg)
![>>> img_data = Binary(open(‘article_img.jpg’).read())
>>> article = { ‘title’ : ‘Schema evolutionin MongoDB’,
‘author’ : ‘mattbates’,
‘section’ : ‘schema’,
‘slug’ : ‘schema-evolution-in-mongodb’,
‘text’ : ‘MongoDb has dynamic schema. For good
performance, you would need an implicit
structure and indexes’,
‘date’ : datetime.utcnow(),
‘tags’ : [‘MongoDB’, ‘schema’, ‘migration’],
‘headline_img’ : {
‘img’ : img_data,
‘caption’ : ‘A sample document at the shell’
}}
Let’s add a headline image](https://image.slidesharecdn.com/mongodb-140412093504-phpapp02/85/MongoDB-Introduction-talk-at-Dr-Dobbs-Conference-MongoDB-Evenings-at-Bangalore-and-Delhi-40-320.jpg)
![>>> article = { ‘title’ : ‘Favourite web application framework’,
‘author’ : ‘prasoonk’,
‘section’ : ‘web-dev’,
‘slug’ : ‘web-app-frameworks’,
‘gallery’ : [
{ ‘img_url’ : ‘http://x.com/45rty’, ‘caption’ : ‘Flask’, ..},
..
]
‘date’ : datetime.utcnow(),
‘tags’ : [‘Python’, ‘web’],
}
>>> db[‘articles’].insert(article)
And different types of article](https://image.slidesharecdn.com/mongodb-140412093504-phpapp02/85/MongoDB-Introduction-talk-at-Dr-Dobbs-Conference-MongoDB-Evenings-at-Bangalore-and-Delhi-41-320.jpg)
![>>> user = {
'user' : 'prasoonk',
'email' : 'prasoon.kumar@mongodb.com',
'password' : 'prasoon101',
'joined' : datetime.utcnow(),
'location' : { 'city' : 'Mumbai' },
}
} >>> db[‘users’].insert(user)
Users and profiles](https://image.slidesharecdn.com/mongodb-140412093504-phpapp02/85/MongoDB-Introduction-talk-at-Dr-Dobbs-Conference-MongoDB-Evenings-at-Bangalore-and-Delhi-42-320.jpg)
![Modelling comments (1)
• Two collections–articles and comments
• Use a reference (i.e. foreign key) to link together
• But.. N+1 queries to retrieve article and comments
{
‘_id’: ObjectId(..),
‘title’:‘Schema design in MongoDB’,
‘author’:‘mattbates’,
‘date’: ISODate(..),
‘tags’: [‘MongoDB’, ‘schema’],
‘section’:‘schema’,
‘slug’:‘schema-design-in-mongodb’,
‘comments’:[ObjectId(..),…]
}
{ ‘_id’: ObjectId(..),
‘article_id’: 1,
‘text’: ‘A great article,helped me
understand schema design’,
‘date’: ISODate(..),,
‘author’:‘johnsmith’
}](https://image.slidesharecdn.com/mongodb-140412093504-phpapp02/85/MongoDB-Introduction-talk-at-Dr-Dobbs-Conference-MongoDB-Evenings-at-Bangalore-and-Delhi-43-320.jpg)
![Modelling comments (2)
• Single articles collection–
embed comments in article
documents
• Pros
• Single query, document designed
for the access pattern
• Locality (disk, shard)
• Cons
• Comments array is unbounded;
documents will grow in size
(remember 16MB document
limit)
{
‘_id’: ObjectId(..),
‘title’:‘Schema design in MongoDB’,
‘author’:‘mattbates’,
‘date’: ISODate(..),
‘tags’: [‘MongoDB’,‘schema’],
…
‘comments’:[
{
‘text’: ‘Agreatarticle,helpedme
understandschemadesign’,
‘date’:ISODate(..),
‘author’:‘johnsmith’
},
…
]
}](https://image.slidesharecdn.com/mongodb-140412093504-phpapp02/85/MongoDB-Introduction-talk-at-Dr-Dobbs-Conference-MongoDB-Evenings-at-Bangalore-and-Delhi-44-320.jpg)
![Modelling comments (3)
{
‘_id’:ObjectId(..),
‘title’:‘SchemadesigninMongoDB’,
‘author’:‘mattbates’,
‘date’:ISODate(..),
‘tags’:[‘MongoDB’, ‘schema’],
…
‘comments_count’:45,
‘comments_pages’:1
‘comments’:[
{
‘text’: ‘Agreatarticle,helpedme
understandschemadesign’,
‘date’:ISODate(..),
‘author’:‘johnsmith’
},
…
]
}
Total number of comments
• Integer counter updated by
update operation as
comments added/removed
Number of pages
• Page is a bucket of 100
comments (see next slide..)
Fixed-size comments array
• 10 most recent
• Sorted by date on insertion](https://image.slidesharecdn.com/mongodb-140412093504-phpapp02/85/MongoDB-Introduction-talk-at-Dr-Dobbs-Conference-MongoDB-Evenings-at-Bangalore-and-Delhi-46-320.jpg)
![myCMS REST endpoints
Method URI Action
GET /articles Retrieve all articles
GET /articles-by-tag/[tag] Retrieve all articles by tag
GET /articles/[article_id] Retrieve a specific article by article_id
POST /articles Add a new article
GET /articles/[article_id]/comments Retrieve all article comments by
article_id
POST /articles/[article_id]/comments Add a new comment to an article.
POST /users Register a user user
GET /users/[username] Retrieve user’s profile
PUT /users/[username] Update a user’s profile](https://image.slidesharecdn.com/mongodb-140412093504-phpapp02/85/MongoDB-Introduction-talk-at-Dr-Dobbs-Conference-MongoDB-Evenings-at-Bangalore-and-Delhi-51-320.jpg)
![$ git clone http://www.github.com/mattbates/mycms_mongodb
$ cd mycms-mongodb
$ virtualenv venv
$ source venv/bin/activate
$ pip install –r requirements.txt
$ mkdir –p data/db
$ mongod --dbpath=data/db --fork --logpath=mongod.log
$ python web.py
[$ deactivate]
Getting started with the skeleton code](https://image.slidesharecdn.com/mongodb-140412093504-phpapp02/85/MongoDB-Introduction-talk-at-Dr-Dobbs-Conference-MongoDB-Evenings-at-Bangalore-and-Delhi-52-320.jpg)
![@app.route('/cms/api/v1.0/articles', methods=['GET'])
def get_articles():
"""Retrieves all articles in the collection
sorted by date
"""
# query all articles and return a cursor sorted by date
cur = db['articles'].find().sort('date’)
if not cur:
abort(400)
# iterate the cursor and add docs to a dict
articles = [article for article in cur]
return jsonify({'articles' : json.dumps(articles, default=json_util.default)})
RESTful API methods in Python + Flask](https://image.slidesharecdn.com/mongodb-140412093504-phpapp02/85/MongoDB-Introduction-talk-at-Dr-Dobbs-Conference-MongoDB-Evenings-at-Bangalore-and-Delhi-53-320.jpg)
![@app.route('/cms/api/v1.0/articles/<string:article_id>/comments', methods = ['POST'])
def add_comment(article_id):
"""Adds a comment to the specified article and a
bucket, as well as updating a view counter
"””
…
page_id = article['last_comment_id'] // 100
…
# push the comment to the latest bucket and $inc the count
page = db['comments'].find_and_modify(
{ 'article_id' : ObjectId(article_id),
'page' : page_id},
{ '$inc' : { 'count' : 1 },
'$push' : {
'comments' : comment } },
fields= {'count' : 1},
upsert=True,
new=True)
RESTful API methods in Python + Flask](https://image.slidesharecdn.com/mongodb-140412093504-phpapp02/85/MongoDB-Introduction-talk-at-Dr-Dobbs-Conference-MongoDB-Evenings-at-Bangalore-and-Delhi-54-320.jpg)
![# $inc the page count if bucket size (100) is exceeded
if page['count'] > 100:
db.articles.update(
{ '_id' : article_id,
'comments_pages': article['comments_pages'] },
{ '$inc': { 'comments_pages': 1 } } )
# let's also add to the article itself
# most recent 10 comments only
res = db['articles'].update(
{'_id' : ObjectId(article_id)},
{'$push' : {'comments' : { '$each' : [comment],
'$sort' : {’date' : 1 },
'$slice' : -10}},
'$inc' : {'comment_count' : 1}})
…
RESTful API methods in Python + Flask](https://image.slidesharecdn.com/mongodb-140412093504-phpapp02/85/MongoDB-Introduction-talk-at-Dr-Dobbs-Conference-MongoDB-Evenings-at-Bangalore-and-Delhi-55-320.jpg)
![def add_interaction(article_id, type):
"""Record the interaction (view/comment) for the
specified article into the daily bucket and
update an hourly counter
"""
ts = datetime.datetime.utcnow()
# $inc daily and hourly view counters in day/article stats bucket
# note the unacknowledged w=0 write concern for performance
db['interactions'].update(
{ 'article_id' : ObjectId(article_id),
'date' : datetime.datetime(ts.year, ts.month, ts.day)},
{ '$inc' : {
'daily.{}’.format(type) : 1,
'hourly.{}.{}'.format(ts.hour, type) : 1
}},
upsert=True,
w=0)
RESTful API methods in Python + Flask](https://image.slidesharecdn.com/mongodb-140412093504-phpapp02/85/MongoDB-Introduction-talk-at-Dr-Dobbs-Conference-MongoDB-Evenings-at-Bangalore-and-Delhi-56-320.jpg)
![$ curl -i http://localhost:5000/cms/api/v1.0/articles
HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 335
Server: Werkzeug/0.9.4 Python/2.7.5
Date: Thu, 10 Apr 2014 16:00:51 GMT
{
"articles": "[{"title": "Schema design in MongoDB", "text": "Data in MongoDB
has a flexible schema..", "section": "schema", "author": "prasoonk", "date":
{"$date": 1397145312505}, "_id": {"$oid": "5346bef5f2610c064a36a793"},
"slug": "schema-design-in-mongodb", "tags": ["MongoDB", "schema"]}]"}
Testing the API – retrieve articles](https://image.slidesharecdn.com/mongodb-140412093504-phpapp02/85/MongoDB-Introduction-talk-at-Dr-Dobbs-Conference-MongoDB-Evenings-at-Bangalore-and-Delhi-57-320.jpg)
![Schema Design @
User
·Name
·Email address
Article
·Name
·Slug
·Publish date
·Text
·Author
Comment[]
·Comment
·Date
·Author
Tag[]
·Value
Category[]
·Value](https://image.slidesharecdn.com/mongodb-140412093504-phpapp02/85/MongoDB-Introduction-talk-at-Dr-Dobbs-Conference-MongoDB-Evenings-at-Bangalore-and-Delhi-64-320.jpg)
MongoDB Introduction talk at Dr Dobbs Conference, MongoDB Evenings at Bangalore and Delhi
- 1. Consulting Engineer, MongoDB Prasoon Kumar #DDBIndia Building your first app; an introduction to MongoDB
- 3. MongoDB is a ___________ database 1. Document 2. Open source 3. High performance 4. Horizontally scalable 5. Full featured
- 4. 1. Document Database • Not for .PDF & .DOC files • A document is essentially an associative array • Document = JSON object • Document = PHP Array • Document = Python Dict • Document = Ruby Hash • etc
- 5. 1. Database Landscape Depth of Functionality Scalability&Performance Memcached MongoDB RDBMS
- 6. 1. NoSQL Data Model Key-Value Store Riak Memcache Project Voldemort Redis BerkeleyDB Document Database MongoDB CouchDB OrientDB Column-Family Stores Amazon SimpleDB Cassandra Hbase Hypertable Graph Databases Neo4J FlockDB OrientDB
- 8. 2. Open Source • MongoDB is an open source project • On GitHub • Licensed under the AGPL • Started & sponsored by MongoDB Inc (formerly known as 10gen) • Commercial licenses available • Contributions welcome
- 9. 7,000,000+ MongoDB Downloads 150,000+ Online Education Registrants 35,000+ MongoDB Management Service (MMS) Users 30,000+ MongoDB User Group Members 20,000+ MongoDB Days Attendees 2. Global Community
- 10. 3. High Performance • Written in C++ • Extensive use of memory-mapped files i.e.read-through write-through memory caching. • Runs nearly everywhere • Data serialized as BSON (fast parsing) • Full support for primary & secondary indexes • Document model = less work
- 11. Better Data Locality 3. Performance In-Memory Caching In-Place Updates
- 12. 4. Scalability Auto-Sharding • Increase capacity as you go • Commodity and cloud architectures • Improved operational simplicity and cost visibility
- 13. 4. High Availability • Automated replication and failover • Multi-data center support • Improved operational simplicity (e.g., HW swaps) • Data durability and consistency
- 14. 4. Scalability: MongoDB Architecture
- 15. 5. Full Featured • Ad Hoc queries • Real time aggregation • Rich query capabilities • Strongly consistent • Geospatial features • Support for most programming languages • Flexible schema
- 17. $ tar –zxvf mongodb-osx-x86_64-2.6.0.tgz $ cd mongodb-osx-i386-2.6.0/bin $ mkdir –p /data/db $ ./mongod Running MongoDB
- 18. MacBook-Pro-:~ $ mongo MongoDB shell version: 2.6.0 connecting to: test > db.test.insert({text: 'Welcome to MongoDB'}) > db.test.find().pretty() { "_id" : ObjectId("51c34130fbd5d7261b4cdb55"), "text" : "Welcome to MongoDB" } Mongo Shell
- 19. _id • _id is the primary key in MongoDB • Automatically indexed • Automatically created as an ObjectId if not provided • Any unique immutable value could be used
- 20. ObjectId • ObjectId is a special 12 byte value • Guaranteed to be unique across your cluster • ObjectId("50804d0bd94ccab2da652599") |----ts-----||---mac---||-pid-||----inc-----| 4 3 2 3
- 22. Terminology RDBMS MongoDB Table, View ➜ Collection Row ➜ Document Index ➜ Index Join ➜ Embedded Document Foreign Key ➜ Reference Partition ➜ Shard
- 23. Let’s Build a Blog
- 24. First step in any application is Determine your entities
- 25. Entities in our Blogging System • Users (post authors) • Article • Comments • Tags,Category • Interactions (views,clicks)
- 26. In a relational base app We would start by doing schema design
- 27. Typical (relational) ERD User ·Name ·Email address Category ·Name ·URL Comment ·Comment ·Date ·Author Article ·Name ·Slug ·Publish date ·Text Tag ·Name ·URL
- 28. In a MongoDB based app We start building our app and let the schema evolve
- 29. MongoDB ERD User ·Name ·Email address Article ·Name ·Slug ·Publish date ·Text ·Author Comment[] ·Comment ·Date ·Author Tag[] ·Value Category[] ·Value
- 30. Seek = 5+ ms Read = really really fast Post Author Comment Disk seeks and data locality
- 31. Post Author Comment Comment Comment Comment Comment Disk seeks and data locality
- 33. Real applications are not built in the shell
- 34. MongoDB has native bindings for over 12 languages
- 37. MongoDB Drivers • Official Support for 12 languages • Community drivers for tons more • Drivers connect to mongo servers • Drivers translate BSON into native types • mongo shell is not a driver,but works like one in some ways • Installed using typical means (maven,npm,pecl,gem, pip)
- 39. # Python dictionary (or object) >>> article = { ‘title’ : ‘Schema design in MongoDB’, ‘author’ : ‘prasoonk’, ‘section’ : ‘schema’, ‘slug’ : ‘schema-design-in-mongodb’, ‘text’ : ‘Data in MongoDB has a flexible schema. So, 2 documents needn’t have same structure. It allows implicit schema to evolve.’, ‘date’ : datetime.utcnow(), ‘tags’ : [‘MongoDB’, ‘schema’] } >>> db[‘articles’].insert(article) Design schema.. In application code
- 40. >>> img_data = Binary(open(‘article_img.jpg’).read()) >>> article = { ‘title’ : ‘Schema evolutionin MongoDB’, ‘author’ : ‘mattbates’, ‘section’ : ‘schema’, ‘slug’ : ‘schema-evolution-in-mongodb’, ‘text’ : ‘MongoDb has dynamic schema. For good performance, you would need an implicit structure and indexes’, ‘date’ : datetime.utcnow(), ‘tags’ : [‘MongoDB’, ‘schema’, ‘migration’], ‘headline_img’ : { ‘img’ : img_data, ‘caption’ : ‘A sample document at the shell’ }} Let’s add a headline image
- 41. >>> article = { ‘title’ : ‘Favourite web application framework’, ‘author’ : ‘prasoonk’, ‘section’ : ‘web-dev’, ‘slug’ : ‘web-app-frameworks’, ‘gallery’ : [ { ‘img_url’ : ‘http://x.com/45rty’, ‘caption’ : ‘Flask’, ..}, .. ] ‘date’ : datetime.utcnow(), ‘tags’ : [‘Python’, ‘web’], } >>> db[‘articles’].insert(article) And different types of article
- 42. >>> user = { 'user' : 'prasoonk', 'email' : '[email protected]', 'password' : 'prasoon101', 'joined' : datetime.utcnow(), 'location' : { 'city' : 'Mumbai' }, } } >>> db[‘users’].insert(user) Users and profiles
- 43. Modelling comments (1) • Two collections–articles and comments • Use a reference (i.e. foreign key) to link together • But.. N+1 queries to retrieve article and comments { ‘_id’: ObjectId(..), ‘title’:‘Schema design in MongoDB’, ‘author’:‘mattbates’, ‘date’: ISODate(..), ‘tags’: [‘MongoDB’, ‘schema’], ‘section’:‘schema’, ‘slug’:‘schema-design-in-mongodb’, ‘comments’:[ObjectId(..),…] } { ‘_id’: ObjectId(..), ‘article_id’: 1, ‘text’: ‘A great article,helped me understand schema design’, ‘date’: ISODate(..),, ‘author’:‘johnsmith’ }
- 44. Modelling comments (2) • Single articles collection– embed comments in article documents • Pros • Single query, document designed for the access pattern • Locality (disk, shard) • Cons • Comments array is unbounded; documents will grow in size (remember 16MB document limit) { ‘_id’: ObjectId(..), ‘title’:‘Schema design in MongoDB’, ‘author’:‘mattbates’, ‘date’: ISODate(..), ‘tags’: [‘MongoDB’,‘schema’], … ‘comments’:[ { ‘text’: ‘Agreatarticle,helpedme understandschemadesign’, ‘date’:ISODate(..), ‘author’:‘johnsmith’ }, … ] }
- 45. Modelling comments (3) • Another option: hybrid of (2) and (3),embed top x comments (e.g.by date,popularity) into the article document • Fixed-size (2.4 feature) comments array • All other comments ‘overflow’ into a comments collection (double write) in buckets • Pros – Document size is more fixed – fewer moves – Single query built – Full comment history with rich query/aggregation
- 46. Modelling comments (3) { ‘_id’:ObjectId(..), ‘title’:‘SchemadesigninMongoDB’, ‘author’:‘mattbates’, ‘date’:ISODate(..), ‘tags’:[‘MongoDB’, ‘schema’], … ‘comments_count’:45, ‘comments_pages’:1 ‘comments’:[ { ‘text’: ‘Agreatarticle,helpedme understandschemadesign’, ‘date’:ISODate(..), ‘author’:‘johnsmith’ }, … ] } Total number of comments • Integer counter updated by update operation as comments added/removed Number of pages • Page is a bucket of 100 comments (see next slide..) Fixed-size comments array • 10 most recent • Sorted by date on insertion
- 47. Modelling comments (3) { ‘_id’: ObjectId(..), ‘article_id’: ObjectId(..), ‘page’: 1, ‘count’: 42 ‘comments’: [ { ‘text’: ‘A great article,helped me understand schema design’, ‘date’: ISODate(..), ‘author’:‘johnsmith’ }, … } One comment bucket (page) document containing up to about 100 comments Array of 100 comment sub- documents
- 48. Modelling interactions • Interactions – Article views – Comments – (Social media sharing) • Requirements – Time series – Pre-aggregated in preparation for analytics
- 49. Modelling interactions • Document per article per day– ‘bucketing’ • Daily counter and hourly sub- document counters for interactions • Bounded array (24 hours) • Single query to retrieve daily article interactions; ready-made for graphing and further aggregation { ‘_id’: ObjectId(..), ‘article_id’: ObjectId(..), ‘section’:‘schema’, ‘date’: ISODate(..), ‘daily’: {‘views’: 45,‘comments’: 150 } ‘hours’: { 0 : {‘views’: 10 }, 1 : {‘views’: 2 }, … 23 : {‘comments’: 14,‘views’: 10 } } }
- 50. JSON and RESTful API Client-side JSON (eg AngularJS, (BSON) Real applications are not built at a shell–let’s build a RESTful API. Pymongo driver Python web app HTTP(S) REST Examples to follow: Python RESTful API using Flask microframework
- 51. myCMS REST endpoints Method URI Action GET /articles Retrieve all articles GET /articles-by-tag/[tag] Retrieve all articles by tag GET /articles/[article_id] Retrieve a specific article by article_id POST /articles Add a new article GET /articles/[article_id]/comments Retrieve all article comments by article_id POST /articles/[article_id]/comments Add a new comment to an article. POST /users Register a user user GET /users/[username] Retrieve user’s profile PUT /users/[username] Update a user’s profile
- 52. $ git clone http://www.github.com/mattbates/mycms_mongodb $ cd mycms-mongodb $ virtualenv venv $ source venv/bin/activate $ pip install –r requirements.txt $ mkdir –p data/db $ mongod --dbpath=data/db --fork --logpath=mongod.log $ python web.py [$ deactivate] Getting started with the skeleton code
- 53. @app.route('/cms/api/v1.0/articles', methods=['GET']) def get_articles(): """Retrieves all articles in the collection sorted by date """ # query all articles and return a cursor sorted by date cur = db['articles'].find().sort('date’) if not cur: abort(400) # iterate the cursor and add docs to a dict articles = [article for article in cur] return jsonify({'articles' : json.dumps(articles, default=json_util.default)}) RESTful API methods in Python + Flask
- 54. @app.route('/cms/api/v1.0/articles/<string:article_id>/comments', methods = ['POST']) def add_comment(article_id): """Adds a comment to the specified article and a bucket, as well as updating a view counter "”” … page_id = article['last_comment_id'] // 100 … # push the comment to the latest bucket and $inc the count page = db['comments'].find_and_modify( { 'article_id' : ObjectId(article_id), 'page' : page_id}, { '$inc' : { 'count' : 1 }, '$push' : { 'comments' : comment } }, fields= {'count' : 1}, upsert=True, new=True) RESTful API methods in Python + Flask
- 55. # $inc the page count if bucket size (100) is exceeded if page['count'] > 100: db.articles.update( { '_id' : article_id, 'comments_pages': article['comments_pages'] }, { '$inc': { 'comments_pages': 1 } } ) # let's also add to the article itself # most recent 10 comments only res = db['articles'].update( {'_id' : ObjectId(article_id)}, {'$push' : {'comments' : { '$each' : [comment], '$sort' : {’date' : 1 }, '$slice' : -10}}, '$inc' : {'comment_count' : 1}}) … RESTful API methods in Python + Flask
- 56. def add_interaction(article_id, type): """Record the interaction (view/comment) for the specified article into the daily bucket and update an hourly counter """ ts = datetime.datetime.utcnow() # $inc daily and hourly view counters in day/article stats bucket # note the unacknowledged w=0 write concern for performance db['interactions'].update( { 'article_id' : ObjectId(article_id), 'date' : datetime.datetime(ts.year, ts.month, ts.day)}, { '$inc' : { 'daily.{}’.format(type) : 1, 'hourly.{}.{}'.format(ts.hour, type) : 1 }}, upsert=True, w=0) RESTful API methods in Python + Flask
- 57. $ curl -i http://localhost:5000/cms/api/v1.0/articles HTTP/1.0 200 OK Content-Type: application/json Content-Length: 335 Server: Werkzeug/0.9.4 Python/2.7.5 Date: Thu, 10 Apr 2014 16:00:51 GMT { "articles": "[{"title": "Schema design in MongoDB", "text": "Data in MongoDB has a flexible schema..", "section": "schema", "author": "prasoonk", "date": {"$date": 1397145312505}, "_id": {"$oid": "5346bef5f2610c064a36a793"}, "slug": "schema-design-in-mongodb", "tags": ["MongoDB", "schema"]}]"} Testing the API – retrieve articles
- 58. $ curl -H "Content-Type: application/json" -X POST -d '{"text":"An interesting article and a great read."}' http://localhost:5000/cms/api/v1.0/articles/52ed73a30bd031362b3c6bb3/ comments { "comment": "{"date": {"$date": 1391639269724}, "text": "An interesting article and a great read."}” } Testing the API – comment on an article
- 59. Schema iteration New feature in the backlog? Documents have dynamic schema so we just iterate the object schema. >>> user = {‘username’:‘matt’, ‘first’:‘Matt’, ‘last’:‘Bates’, ‘preferences’: {‘opt_out’: True } } >>> user.save(user)
- 60. docs.mongodb.org
- 61. Online Training at MongoDB University
- 62. For More Information Resource Location MongoDB Downloads mongodb.com/download Free Online Training education.mongodb.com Webinars and Events mongodb.com/events White Papers mongodb.com/white-papers Case Studies mongodb.com/customers Presentations mongodb.com/presentations Documentation docs.mongodb.org Additional Info [email protected] Resource Location
- 63. We've introduced a lot of concepts here
- 64. Schema Design @ User ·Name ·Email address Article ·Name ·Slug ·Publish date ·Text ·Author Comment[] ·Comment ·Date ·Author Tag[] ·Value Category[] ·Value
- 65. Replication @ Secondary Secondary Primary Client Application Driver Write Read Read
- 66. Indexing @ 7 16 1 2 5 6 9 12 18 21
- 68. Questions?
- 69. Consulting Engineer, MongoDB Prasoon Kumar #DDJIndia @prasoonk Thank You