Implementing Schema Validation in MongoDB with Pydantic
0 likes187 views
The document discusses the integration of MongoDB schema validation with Pydantic for data governance and integrity. It outlines how MongoDB enforces validation rules during data insertion, demonstrates the setup of a FastAPI application with routes for submitting user data, and includes examples of handling validation errors and displaying user information. Pydantic enhances data validation by defining models and constraints, making it suitable for robust API applications.
![Configuring Schema Validation
Can be performed at collection
creation time or to an already existing
collection
db.createCollection ("users", {
validator: {
$jsonSchema : {
bsonType: "object",
required: ["name", "email"],
properties: {
name: {
bsonType: "string",
description: "must be a string and is required" ,
maxLength: 50
},
email: {
bsonType: "string",
description: "must be a string and is required" ,
pattern: "^S+@S+.S+$" // Enforces valid
email pattern
},
age: {
bsonType: "int",
description: "must be an integer and optional" ,
minimum: 18,
maximum: 120
}
}
}
},
validationAction: "error"
})](https://image.slidesharecdn.com/implementingschemavalidationinmongodbwithpydantic-241011100807-aa47533d/85/Implementing-Schema-Validation-in-MongoDB-with-Pydantic-5-320.jpg)
![Apply Schema to Existing Collection
Use collMod command to update
schema validation rules if collection
exists
db.runCommand({
collMod: "users",
validator: {
$jsonSchema: {
bsonType: "object",
required: ["name", "email"],
properties: {
name: {
bsonType: "string",
description: "must be a string and is required",
maxLength: 50
},
email: {
bsonType: "string",
description: "must be a string and is required",
pattern: "^S+@S+.S+$"
},
age: {
bsonType: "int",
description: "must be an integer and optional",
minimum: 18,
maximum: 120
}
}
}
},
validationAction: "error"
})](https://image.slidesharecdn.com/implementingschemavalidationinmongodbwithpydantic-241011100807-aa47533d/85/Implementing-Schema-Validation-in-MongoDB-with-Pydantic-6-320.jpg)
![Define the structure and validation rules for incoming data
Example for a user model:
Defining Pydantic Models
class UserModel(BaseModel):
name: str = Field(..., max_length=50)
email: EmailStr
age: Optional[int]= Field(None, ge=18, le=120)](https://image.slidesharecdn.com/implementingschemavalidationinmongodbwithpydantic-241011100807-aa47533d/85/Implementing-Schema-Validation-in-MongoDB-with-Pydantic-10-320.jpg)
![post route to handle form submission,
Validate the data using the Pydantic model
Save validated data to MongoDB
Building Routes
@app.post("/submit")
async def submit_form(
request: Request,
name: str = Form(...),
email: str = Form(...),
age: Optional[int] = Form(None)
):
try:
# Validate the data using the Pydantic model
user = UserModel(name=name, email=email, age=age)
# Insert into MongoDB
await users_collection.insert_one(user.dict())
# Success response
return templates.TemplateResponse("success.html", {"request":
request, "user": user.dict()})
except Exception as e:
raise HTTPException(status_code=400, detail=str(e))](https://image.slidesharecdn.com/implementingschemavalidationinmongodbwithpydantic-241011100807-aa47533d/85/Implementing-Schema-Validation-in-MongoDB-with-Pydantic-13-320.jpg)
![Handling Validation Errors in Code
from fastapi import FastAPI, HTTPException
from motor.motor_asyncio import AsyncIOMotorClient
from pydantic import BaseModel, EmailStr, Field
app = FastAPI()
client =
AsyncIOMotorClient("mongodb://localhost:27017")
db = client.mydatabase
users_collection = db.users
class UserModel(BaseModel):
name: str = Field(..., max_length=50)
email: EmailStr
age: Optional[int] = Field(None, ge=18, le=120)
@app.post("/submit")
async def submit_user(user: UserModel):
try:
await users_collection.insert_one(user.dict())
return {"message": "User inserted successfully"}
except Exception as e:
raise HTTPException(status_code=400,
detail=str(e))](https://image.slidesharecdn.com/implementingschemavalidationinmongodbwithpydantic-241011100807-aa47533d/85/Implementing-Schema-Validation-in-MongoDB-with-Pydantic-14-320.jpg)
Implementing Schema Validation in MongoDB with Pydantic
- 1. Samuel Folasayo MongoDB Schema Validation with Pydantic Data Governance, Integrity and Consistency with Pydantic & MongoDB Joe Nyirenda
- 2. What is MongoDB Schema Validation? Ensures data complies with a predefined set of rules(data types etc.) Maintains data integrity Helps enforces data governance Can help optimise schema size It can be applied when creating or updating collections.
- 3. What is Pydantic? Python library used for data validation and parsing Enforces data types and constraints Defining models based on Python type annotations Provide error handling Provides conversion between different data formats, like JSON or dictionaries
- 4. Pros and Cons of Pydantic + Schema Validation Pros Enforces data validity before storing Ensures code reliability Double-layer validation Greater security Data integrity Robust validation Cons Potential overhead Complexity in setup Inflexible schema
- 5. Configuring Schema Validation Can be performed at collection creation time or to an already existing collection db.createCollection ("users", { validator: { $jsonSchema : { bsonType: "object", required: ["name", "email"], properties: { name: { bsonType: "string", description: "must be a string and is required" , maxLength: 50 }, email: { bsonType: "string", description: "must be a string and is required" , pattern: "^S+@S+.S+$" // Enforces valid email pattern }, age: { bsonType: "int", description: "must be an integer and optional" , minimum: 18, maximum: 120 } } } }, validationAction: "error" })
- 6. Apply Schema to Existing Collection Use collMod command to update schema validation rules if collection exists db.runCommand({ collMod: "users", validator: { $jsonSchema: { bsonType: "object", required: ["name", "email"], properties: { name: { bsonType: "string", description: "must be a string and is required", maxLength: 50 }, email: { bsonType: "string", description: "must be a string and is required", pattern: "^S+@S+.S+$" }, age: { bsonType: "int", description: "must be an integer and optional", minimum: 18, maximum: 120 } } } }, validationAction: "error" })
- 7. After applying the schema, try inserting documents to see how MongoDB enforces the validation rules. Testing the Schema db.users.insertOne({ name: "John Doe", email: "[email protected]", age: 30 }) This will work because it complies with the schema.
- 8. MongoDB enforces the validation rules Testing the Schema: invalid document Invalid Document Insertion: db.users.insertOne({ name: "John Doe", email: "[email protected]", age: 17 }) This will fail because age is less than 18.
- 9. Demo: minimalist web form Tools: FastAPI: Python web framework for building APIs Motor: Asynchronous MongoDB driver. Pydantic: Validating and parsing data. Jinja2: HTML templating engine
- 10. Define the structure and validation rules for incoming data Example for a user model: Defining Pydantic Models class UserModel(BaseModel): name: str = Field(..., max_length=50) email: EmailStr age: Optional[int]= Field(None, ge=18, le=120)
- 11. Motor connects asynchronously to MongoDB: Connecting to MongoDB from motor.motor_asyncio import AsyncIOMotorClient from fastapi import FastAPI, HTTPException app = FastAPI() # MongoDB connection client = AsyncIOMotorClient("mongodb://localhost:27017") db = client.mydatabase users_collection = db.users
- 12. Here, we’ll create two main routes: GET route to serve the form HTML. Building Routes from fastapi import Request, Form from fastapi.responses import HTMLResponse from fastapi.templating import Jinja2Templates # Initialize Jinja2 template rendering templates = Jinja2Templates(directory="templates") @app.get("/", response_class=HTMLResponse) async def get_form(request: Request): return templates.TemplateResponse("form.html", {"request": request})
- 13. post route to handle form submission, Validate the data using the Pydantic model Save validated data to MongoDB Building Routes @app.post("/submit") async def submit_form( request: Request, name: str = Form(...), email: str = Form(...), age: Optional[int] = Form(None) ): try: # Validate the data using the Pydantic model user = UserModel(name=name, email=email, age=age) # Insert into MongoDB await users_collection.insert_one(user.dict()) # Success response return templates.TemplateResponse("success.html", {"request": request, "user": user.dict()}) except Exception as e: raise HTTPException(status_code=400, detail=str(e))
- 14. Handling Validation Errors in Code from fastapi import FastAPI, HTTPException from motor.motor_asyncio import AsyncIOMotorClient from pydantic import BaseModel, EmailStr, Field app = FastAPI() client = AsyncIOMotorClient("mongodb://localhost:27017") db = client.mydatabase users_collection = db.users class UserModel(BaseModel): name: str = Field(..., max_length=50) email: EmailStr age: Optional[int] = Field(None, ge=18, le=120) @app.post("/submit") async def submit_user(user: UserModel): try: await users_collection.insert_one(user.dict()) return {"message": "User inserted successfully"} except Exception as e: raise HTTPException(status_code=400, detail=str(e))
- 15. Form Template (templates/form.html): Displaying HTML Form <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0" > <title>User Form</title> </head> <body> <h1>Enter User Information </h1> <form action="/submit" method="post"> <label for="name">Name:</label> <input type="text" id="name" name="name" required><br><br> <label for="email">Email:</label> <input type="email" id="email" name="email" required><br><br> <label for="age">Age:</label> <input type="number" id="age" name="age"><br><br> <input type="submit" value="Submit"> </form> </body> </html>
- 16. Success Template (templates/success.html): Displaying HTML Form <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>Form Submitted</title> </head> <body> <h1>Submission Successful</h1> <p>User Details:</p> <ul> <li><strong>Name:</strong> {{ user.name }}</li> <li><strong>Email:</strong> {{ user.email }}</li> <li><strong>Age:</strong> {{ user.age }}</li> </ul> </body> </html>
- 17. Results Displayed on Browser
- 18. Conclusion Pydantic provides an extra layer of validation Schema validation enforces data type and domain specific rules for data model Schema validation great for mature applications