SlideShare a Scribd company logo
  
MongoDB + Pylons FTW: 
Scalabale Web apps with Python & NoSQL
 Niall O'Higgins
  
MongoDB ??
 Non­relational (NoSQL) document­oriented 
database
 Rich query language
 Flexible data model (JSON)
 Sharding and replication with automatic­failover
 CAP write quorum support
  
MongoDB vs MySQL vs BDB
 Feature Comparison
MongoDB MySQL BDB
Master-Slave
Replication
Yes Yes No
Automatic
master
failover
Yes No No
Ad-hoc
Schemas
Yes No No
Manual index
specification
Yes Yes Yes
Rich query
language
Yes Yes No
Joins No Yes No
  
MongoDB: NoSQL?
 Closer to MySQL than to BDB/Tyrant/Redis
 Less emphasis on Google­scale scalability at this 
point compared to e.g. Cassandra/HBase
 Today, MongoDB scales slightly better than 
MySQL IMHO.
 Main advantage over RDBMS is ease of 
development!
  
NoORM
 MongoDB document is a JSON object.
 PyMongo driver exposes these as dictionaries
E.g. db.mycollection.insert(dict(myprop='foo!'))
 You can even embed objects within a document:
E.g. db.mycollection.insert(
dict(emails=
      [dict(email_lower='niallo@pywebsf.org')]
))
  
NoORM II
 You can query across properties, embedded or 
otherwise, easily.
E.g. 
r = db.mycollection.find_one(
{emails.email_lower:'niallo@pywebsf.org'}
)
print r['emails'][0]['email_lower']
  
NoORM III
 SQL queries can get very complicated (huge joins, 
aggregate queries, etc)
 Marshalling SQL to/from  objects is painful, 
tedious.
 Cross­RDBMS portability issues
  → ORMs can make sense with RDBMS
 At least in Python, I see absolutely no need for 
ORM with MongoDB.
  
More on querying MongoDB
 I alluded to a rich query language:
 $lt, $lte, $gt, $gte == <, <=, >, =>
 $ne == <>, !=
 $in == IN
 $or == OR
 Sort() == ORDER BY
 Limit() == LIMIT
 Skip() == OFFSET
 Group() == GROUP BY
  
Geo Features
Geospatial querying out of the box:
 $near (x, y): Return documents sorted by distance 
from point.
 $within, $box/$center: Return documents within 
bounds of a rectangle/circle. Earth is flat in 1.6.x 
and lower. Spherical model in 1.7.0 and up.
 Other limits: Only one geo index per collection. No 
sharding support.
  
More on querying MongoDB
 Sample rich query:
db.foo.find({'$or': [{'owner_account_id': Id}, 
{'shared_to': Id}], 'location': {'$near': [37.783, 
­122.393]}}.skip(offset).limit(PAGE_SIZE)
  
MongoDB Gotchas
 Obviously not as mature as any RDBMS on the 
market. You can find bugs, but developers are 
extremely responsive.
 4MB limit per document.
 Cap on number of indexes per collection.
 No transactions, although some imperfect support 
for atomic operations. Race with e.g. Deleting 
single embedded object from list property.
  → Not the right fit for every system! 
  → Think through your schema!
  
Pylons
 Elegant, lightweight but fully­featured framework.
 Perfect fit for mixed API and Web rendering 
environment we have at Catch.com
 Threaded design, easy to program. No callbacks 
needed. Works fine with synchronous libraries.
 Threads in Python suck due to the GIL.
 Multi­core systems necessitate multi­process 
operation.
 Options: Apache+mod_wsgi or multiple Pasters.
  
mod_wsgi vs multiple Pasters
 Apache+mod_wsgi is kind of a pain to setup, 
configure.
 Past production experiences with mod_wsgi include 
memory leaks, CPU spinning.
 Paster is simple, built into Pylons. One Paster 
server maps cleanly to one Python interpreter 
process.
  LOLapps serve high­volume production traffic 
with Paster just fine.
  
Catch.com Paster
 Multiple Pasters per machine, each listening on a 
separate port.
 Nginx load balances between them all.
 Nginx also terminates SSL.
 No issues whatsoever handling the load today.
  
Paster Debugging Tip
 Configure smtp_server
 Point it at a Google Mail list
 Set up a Gmail label to filter messages into
 Now you have nicely searchable, real­time stack 
traces aggregated from each Paster. Very useful 
for debugging distributed systems!

More Related Content

What's hot (20)

PPTX
NoSQL and MongoDB Introdction
Brian Enochson
 
PDF
MongoDB World 2016: Poster Sessions eBook
MongoDB
 
PDF
guacamole: an Object Document Mapper for ArangoDB
Max Neunhöffer
 
PDF
Mongo DB: Operational Big Data Database
Xpand IT
 
PDF
Introduction to ArangoDB (nosql matters Barcelona 2012)
ArangoDB Database
 
PDF
Introduction to Web Designing
Kailaash Balachandran
 
KEY
MongoDB SF Ruby
Mike Dirolf
 
PPTX
Using MongoDB For BigData in 20 Minutes
András Fehér
 
PDF
From RDBMS to MongoDB
MongoDB
 
PPTX
JS App Architecture
Corey Butler
 
PPT
5 Things You Didn't Know You Could do with CouchDB
Mark Headd
 
KEY
MongoDB Hadoop DC
Mike Dirolf
 
PPTX
NYT Web Archive
Justin Heideman
 
PPTX
MongoDB Introduction - Document Oriented Nosql Database
Sudhir Patil
 
PPTX
The Hive Think Tank: Rocking the Database World with RocksDB
The Hive
 
PPTX
MongoDB
Muhammad zubair
 
PPTX
Introduction to NoSQL Database
Mohammad Alghanem
 
PDF
Lean and mean MongoDB
Oleg Podsechin
 
PPTX
Benefits of Using MongoDB Over RDBMSs
MongoDB
 
NoSQL and MongoDB Introdction
Brian Enochson
 
MongoDB World 2016: Poster Sessions eBook
MongoDB
 
guacamole: an Object Document Mapper for ArangoDB
Max Neunhöffer
 
Mongo DB: Operational Big Data Database
Xpand IT
 
Introduction to ArangoDB (nosql matters Barcelona 2012)
ArangoDB Database
 
Introduction to Web Designing
Kailaash Balachandran
 
MongoDB SF Ruby
Mike Dirolf
 
Using MongoDB For BigData in 20 Minutes
András Fehér
 
From RDBMS to MongoDB
MongoDB
 
JS App Architecture
Corey Butler
 
5 Things You Didn't Know You Could do with CouchDB
Mark Headd
 
MongoDB Hadoop DC
Mike Dirolf
 
NYT Web Archive
Justin Heideman
 
MongoDB Introduction - Document Oriented Nosql Database
Sudhir Patil
 
The Hive Think Tank: Rocking the Database World with RocksDB
The Hive
 
Introduction to NoSQL Database
Mohammad Alghanem
 
Lean and mean MongoDB
Oleg Podsechin
 
Benefits of Using MongoDB Over RDBMSs
MongoDB
 

Similar to MongoDB + Pylons FTW: Scalable Web apps with Python & NoSQL (20)

ODP
Python and MongoDB
Christiano Anderson
 
PDF
MongoDB Basics
Sarang Shravagi
 
PPTX
Mongodb
ASEEMSRIVASTAVA22
 
KEY
MongoDB EuroPython 2009
Mike Dirolf
 
PDF
No sql database and python -- Presentation done at Python Developer's meetup ...
Roshan Bhandari
 
KEY
MongoDB NYC Python
Mike Dirolf
 
KEY
MongoDB at ZPUGDC
Mike Dirolf
 
PDF
Drupal Day 2011 - Drupal and the rise of the documents
DrupalDay
 
PPTX
Silicon Valley Code Camp: 2011 Introduction to MongoDB
Manish Pandit
 
PDF
SFScon14: Schrödinger’s elephant: why PostgreSQL can solve all your database ...
South Tyrol Free Software Conference
 
PPTX
ГАННА КАПЛУН «noSQL vs SQL: порівняння використання реляційних та нереляційни...
QADay
 
PPT
Mongo Bb - NoSQL tutorial
Mohan Rathour
 
PPT
MongoDB Pros and Cons
johnrjenson
 
PDF
MongoDB: a gentle, friendly overview
Antonio Pintus
 
PDF
SQL vs NoSQL, an experiment with MongoDB
Marco Segato
 
KEY
MongoDB at RuPy
Mike Dirolf
 
PPTX
Mongo db operations_v2
Thanabalan Sathneeganandan
 
PPTX
MongoDB vs Scylla: Production Experience from Both Dev & Ops Standpoint at Nu...
ScyllaDB
 
PDF
1428393873 mhkx3 ln
WilfredodelaCernaJr
 
PPT
MongoDb - Details on the POC
Amardeep Vishwakarma
 
Python and MongoDB
Christiano Anderson
 
MongoDB Basics
Sarang Shravagi
 
MongoDB EuroPython 2009
Mike Dirolf
 
No sql database and python -- Presentation done at Python Developer's meetup ...
Roshan Bhandari
 
MongoDB NYC Python
Mike Dirolf
 
MongoDB at ZPUGDC
Mike Dirolf
 
Drupal Day 2011 - Drupal and the rise of the documents
DrupalDay
 
Silicon Valley Code Camp: 2011 Introduction to MongoDB
Manish Pandit
 
SFScon14: Schrödinger’s elephant: why PostgreSQL can solve all your database ...
South Tyrol Free Software Conference
 
ГАННА КАПЛУН «noSQL vs SQL: порівняння використання реляційних та нереляційни...
QADay
 
Mongo Bb - NoSQL tutorial
Mohan Rathour
 
MongoDB Pros and Cons
johnrjenson
 
MongoDB: a gentle, friendly overview
Antonio Pintus
 
SQL vs NoSQL, an experiment with MongoDB
Marco Segato
 
MongoDB at RuPy
Mike Dirolf
 
Mongo db operations_v2
Thanabalan Sathneeganandan
 
MongoDB vs Scylla: Production Experience from Both Dev & Ops Standpoint at Nu...
ScyllaDB
 
1428393873 mhkx3 ln
WilfredodelaCernaJr
 
MongoDb - Details on the POC
Amardeep Vishwakarma
 
Ad

Recently uploaded (20)

PPTX
From Sci-Fi to Reality: Exploring AI Evolution
Svetlana Meissner
 
PDF
Newgen Beyond Frankenstein_Build vs Buy_Digital_version.pdf
darshakparmar
 
PDF
SIZING YOUR AIR CONDITIONER---A PRACTICAL GUIDE.pdf
Muhammad Rizwan Akram
 
PDF
Newgen 2022-Forrester Newgen TEI_13 05 2022-The-Total-Economic-Impact-Newgen-...
darshakparmar
 
PPTX
Digital Circuits, important subject in CS
contactparinay1
 
PPTX
New ThousandEyes Product Innovations: Cisco Live June 2025
ThousandEyes
 
PPTX
Designing_the_Future_AI_Driven_Product_Experiences_Across_Devices.pptx
presentifyai
 
PPTX
The Project Compass - GDG on Campus MSIT
dscmsitkol
 
PDF
UiPath DevConnect 2025: Agentic Automation Community User Group Meeting
DianaGray10
 
PDF
Book industry state of the nation 2025 - Tech Forum 2025
BookNet Canada
 
PDF
LOOPS in C Programming Language - Technology
RishabhDwivedi43
 
PDF
What’s my job again? Slides from Mark Simos talk at 2025 Tampa BSides
Mark Simos
 
PDF
CIFDAQ Market Wrap for the week of 4th July 2025
CIFDAQ
 
PDF
“Squinting Vision Pipelines: Detecting and Correcting Errors in Vision Models...
Edge AI and Vision Alliance
 
PDF
Future-Proof or Fall Behind? 10 Tech Trends You Can’t Afford to Ignore in 2025
DIGITALCONFEX
 
PDF
AI Agents in the Cloud: The Rise of Agentic Cloud Architecture
Lilly Gracia
 
PDF
Kit-Works Team Study_20250627_한달만에만든사내서비스키링(양다윗).pdf
Wonjun Hwang
 
PDF
Transcript: Book industry state of the nation 2025 - Tech Forum 2025
BookNet Canada
 
PDF
Automating Feature Enrichment and Station Creation in Natural Gas Utility Net...
Safe Software
 
PDF
“Computer Vision at Sea: Automated Fish Tracking for Sustainable Fishing,” a ...
Edge AI and Vision Alliance
 
From Sci-Fi to Reality: Exploring AI Evolution
Svetlana Meissner
 
Newgen Beyond Frankenstein_Build vs Buy_Digital_version.pdf
darshakparmar
 
SIZING YOUR AIR CONDITIONER---A PRACTICAL GUIDE.pdf
Muhammad Rizwan Akram
 
Newgen 2022-Forrester Newgen TEI_13 05 2022-The-Total-Economic-Impact-Newgen-...
darshakparmar
 
Digital Circuits, important subject in CS
contactparinay1
 
New ThousandEyes Product Innovations: Cisco Live June 2025
ThousandEyes
 
Designing_the_Future_AI_Driven_Product_Experiences_Across_Devices.pptx
presentifyai
 
The Project Compass - GDG on Campus MSIT
dscmsitkol
 
UiPath DevConnect 2025: Agentic Automation Community User Group Meeting
DianaGray10
 
Book industry state of the nation 2025 - Tech Forum 2025
BookNet Canada
 
LOOPS in C Programming Language - Technology
RishabhDwivedi43
 
What’s my job again? Slides from Mark Simos talk at 2025 Tampa BSides
Mark Simos
 
CIFDAQ Market Wrap for the week of 4th July 2025
CIFDAQ
 
“Squinting Vision Pipelines: Detecting and Correcting Errors in Vision Models...
Edge AI and Vision Alliance
 
Future-Proof or Fall Behind? 10 Tech Trends You Can’t Afford to Ignore in 2025
DIGITALCONFEX
 
AI Agents in the Cloud: The Rise of Agentic Cloud Architecture
Lilly Gracia
 
Kit-Works Team Study_20250627_한달만에만든사내서비스키링(양다윗).pdf
Wonjun Hwang
 
Transcript: Book industry state of the nation 2025 - Tech Forum 2025
BookNet Canada
 
Automating Feature Enrichment and Station Creation in Natural Gas Utility Net...
Safe Software
 
“Computer Vision at Sea: Automated Fish Tracking for Sustainable Fishing,” a ...
Edge AI and Vision Alliance
 
Ad

MongoDB + Pylons FTW: Scalable Web apps with Python & NoSQL