Django for IoT: From hackathon to production (DjangoCon US)
6 likes3,222 views
This document presents a talk by Anna Schneider on using Django for Internet of Things (IoT) applications, highlighting the difference between hackathons and production environments. It covers essential design patterns, tasks, and production strategies for deploying Django projects to monitor and control IoT devices, including the usage of Celery for task management. The session aims to equip participants with practical tools and knowledge to effectively develop and deploy IoT solutions using Django.
![tasks.py
def do_something_awesome(device_id):
# get device from pk
device = Device.objects.get(
pk=device_id
)
# decide to turn on or off
is_on = run_decision_criterion()
# set status using device API
set_status(device, is_on)
# create status in db
status = device.status_set.create(
is_on=is_on,
valid_at=timezone.now(),
)
# return
return [status.pk]
pass pks
pro: don’t rely on database state
con: may be extra queries
🚢](https://image.slidesharecdn.com/djangoconusiotstd-160718173411/85/Django-for-IoT-From-hackathon-to-production-DjangoCon-US-24-320.jpg)
![from celery.schedules import crontab
SCHEDULE = {
'run_task': {
# use the actual path
'task': ‘do_something_awesome’,
# every 5 minutes
'schedule': crontab(minute='*/5'),
# args and kwargs
'args': [],
'kwargs': {},
},
}
very close to crontab
once per minute - once per year
schedule.py
🚢
(when)](https://image.slidesharecdn.com/djangoconusiotstd-160718173411/85/Django-for-IoT-From-hackathon-to-production-DjangoCon-US-37-320.jpg)
![from celery.schedules import crontab
SCHEDULE = {
'run_task': {
# use the actual path
'task': ‘do_something_awesome’,
# every 5 minutes
'schedule': crontab(minute='*/5'),
# args and kwargs
'args': [],
'kwargs': {'device_id': d.pk},
} for d in Device.objects.all()
}
schedule.py
🚢
static arguments only
code is only evaluated once at run time,
better to have one task spawn daughters
(when)
😱](https://image.slidesharecdn.com/djangoconusiotstd-160718173411/85/Django-for-IoT-From-hackathon-to-production-DjangoCon-US-38-320.jpg)
![from celery.schedules import crontab
SCHEDULE = {
'run_task': {
# use the actual path
'task': ‘run_all',
# every 5 minutes
'schedule': crontab(minute='*/5'),
# args and kwargs
'args': [],
'kwargs': {},
}
}
@shared_task
def run_all():
for d in Device.objects.all():
do_something_awesome(d.pk)
schedule.py
🚢
static arguments only
code is only evaluated once at run time,
better to have one task spawn daughters
(when)](https://image.slidesharecdn.com/djangoconusiotstd-160718173411/85/Django-for-IoT-From-hackathon-to-production-DjangoCon-US-39-320.jpg)
Django for IoT: From hackathon to production (DjangoCon US)
- 1. Django for Internet of Things: from hackathon to production Anna Schneider DjangoCon US, 18 July 2016 @windupanna
- 2. I’m Anna :) Co-founder and CTO at WattTime WattTime.org @wattTime
- 4. what?? Django for Internet of Things: from hackathon to production
- 6. Internet of Things (IoT) when a thing you don’t normally think of as a computer can transmit data and respond to controls in real time 🏡🚗 📡🎮⏰
- 7. some people like to write the code on the thing, I like to talk to things through their APIs Internet of Things (IoT) ☁️ 🚗☁️you them the thing
- 8. a really fun way to write really fragile code really quickly hackathon 💻☕️ 👏⚠️
- 9. when you can ship your code and trust it while you’re having fun at DjangoCon production 💻 🍎 🚢 😎
- 10. design patterns (and anti-patterns) for writing and deploying Django projects to monitor and control an IoT device using its API so you can get started fast then build well what you’ll learn ☕️ 🚢 😱
- 11. python manage.py startproject awesome_iot_hackathon 💻☕️ 👏⚠️
- 12. models!
- 14. the books IoT app Observation value device timestamp Device name location Device name location Device name location Observation value device timestamp Observation value device timestamp Observation value device timestamp
- 15. the IoT app Device name location vendor ID Device name location vendor ID Device name location vendor ID ☕️ 🚢 vendor’s unique ID(s) whatever data you need to send to their API Observation value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp
- 16. Observation value device timestamp the IoT app plan for big(gish) time series data eg use db_index 🚢 Device name location vendor ID Observation value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp Device name location vendor ID Device name location vendor ID
- 17. the IoT app Attribute vs Status numerical vs str/bool values Observation value device timestamp Device name location vendor ID Observation value device timestamp Observation value device timestamp Attribute value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp Attribute value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp Attribute value device timestamp Device name location vendor ID Device name location vendor ID Observation value device timestamp Observation value device timestamp Observation value device timestamp Status is_on device timestamp ☕️ 🚢
- 18. views? models
- 19. views? tasks!
- 20. ☁️ 🚗☁️you them the thing request response ☁️ 💻you them request response normal views IoT tasks
- 21. what tasks do we need to do? pull new attributes pull new on/off statuses set attributes set on/off statuses monitor control
- 22. tasks.py def do_something_awesome(device): # decide to turn on or off is_on = run_decision_criterion() # set status using device API set_status(device, is_on) # create status in db status = device.status_set.create( is_on=is_on, valid_at=timezone.now(), ) # return return status ☕️ the awesome part
- 23. tasks.py def do_something_awesome(device): # decide to turn on or off is_on = run_decision_criterion() # set status using device API set_status(device, is_on) # create status in db status = device.status_set.create( is_on=is_on, valid_at=timezone.now(), ) # return return status 🚢 bad for asynchronous 😱
- 24. tasks.py def do_something_awesome(device_id): # get device from pk device = Device.objects.get( pk=device_id ) # decide to turn on or off is_on = run_decision_criterion() # set status using device API set_status(device, is_on) # create status in db status = device.status_set.create( is_on=is_on, valid_at=timezone.now(), ) # return return [status.pk] pass pks pro: don’t rely on database state con: may be extra queries 🚢
- 27. the production apps devices models.py admin.py views.py tests.py interactions tasks.py views.py tests.py vendor client.py tests.py observations models.py admin.py views.py tests.py views for adding/removing devices analytics, DRF for dashboards models for logging, views for clickable tasks swappable vendors 🚢
- 29. deploy tasks? goals: • run any task (control or monitor) • at frequent, deterministic times • outside of request/response cycle cron in the cloud
- 30. 1) the hackathon way two ways to automate management commands + Heroku Scheduler ☕️ python manage.py do_something_awesome --device_id=1
- 31. Heroku Scheduler pros: • easy to set up cons: • limited frequencies (daily, hourly, 10 min) • “best effort” service (may skip some) ☕️
- 32. 2) the production way task functions + celery periodic task scheduler 🚢 two ways to automate
- 33. Celery distributed message queuing system for asynchronous stuff slow event-driven tasks send the user sign-up email scheduled periodic tasks run the daily report 🚢
- 34. web servers Celery architecture (how) tasks tasks worker servers messages results result store message broker transport queue messages tasks scheduler 🚢
- 35. Celery architecture (how) scheduler do_awesome do_awesome worker status.pk ☁️ turn on on! Status(is_on=True) 🚢
- 36. from celery import shared_task @shared_task def set_status(device_id): ... @shared_task def do_something_awesome(device_id): ... just add decorator :) tasks.py 🚢 (what)
- 37. from celery.schedules import crontab SCHEDULE = { 'run_task': { # use the actual path 'task': ‘do_something_awesome’, # every 5 minutes 'schedule': crontab(minute='*/5'), # args and kwargs 'args': [], 'kwargs': {}, }, } very close to crontab once per minute - once per year schedule.py 🚢 (when)
- 38. from celery.schedules import crontab SCHEDULE = { 'run_task': { # use the actual path 'task': ‘do_something_awesome’, # every 5 minutes 'schedule': crontab(minute='*/5'), # args and kwargs 'args': [], 'kwargs': {'device_id': d.pk}, } for d in Device.objects.all() } schedule.py 🚢 static arguments only code is only evaluated once at run time, better to have one task spawn daughters (when) 😱
- 39. from celery.schedules import crontab SCHEDULE = { 'run_task': { # use the actual path 'task': ‘run_all', # every 5 minutes 'schedule': crontab(minute='*/5'), # args and kwargs 'args': [], 'kwargs': {}, } } @shared_task def run_all(): for d in Device.objects.all(): do_something_awesome(d.pk) schedule.py 🚢 static arguments only code is only evaluated once at run time, better to have one task spawn daughters (when)
- 42. • think time series data: mind your models • async not request/response: tasks not views • ☕️ easy but rigid: Heroku Scheduler • 🚢 flexible but complex: celery periodic tasks • hack better+faster: cookiecutter-django-iot • IoT can be for good, not just for fun and profit what have we learned?