SFScon 2020 - Nikola Milisavljevic - BASE - Python REST API framework
0 likes56 views
Base3 is a Python web framework developed by Digital Cube, designed for asynchronous networking and aimed at reducing boilerplate code for application development. It focuses on code reusability, easy onboarding, scalability, and includes tools for REST API creation, user authentication, and microservices architecture. The framework supports various databases and emphasizes test-driven development and concurrent processing.
SFScon 2020 - Nikola Milisavljevic - BASE - Python REST API framework
- 1. BASE3 https://base3.dev Nikola Milisavljevic Digital CUBE, Belgrade
- 2. What is BASE? - BASE3 is a Python web framework based on Tornado asynchronous networking library, developed by Digital Cube team for internal use. It is also available to the open source community to use and contribute to the project. - We noticed we needed to write a lot of boilerplate code to start working on our applications - Solution: create a BASE framework - BASE principles: - Implementation of code reusability for common operations (user management, registration, email sending, PDF generation) - Simple to use and get going - Great for onboarding new programmers (with and without Python experience) - Scalability and microservices
- 3. BASE / main features - Methods and recipes for making REST API based application - Bundled Python tools for a seamless start - Python decorators for code reusability and common functionalities - Authentication using different identity providers (Google, Facebook…) - Tool for generating source code templates - Auto-generation of API specification - Flexibility in the scale and purpose of the application (small, template driven websites to complex async microservice application) - Education
- 4. Main goals - Making of REST APIs with focus on business and underlying logic rather than the tools themselves - Reduction of the amount of effort needed to bootstrap - Microservice organized architecture with common microservices already written and maintained for current and future projects - Automatic parameter validation with using Models as the schema - Test Driven Development - Use of template engines for simple server-side rendered websites
- 5. Common services - Implementation and maintenance of common microservices to encourage code reuse: - Users / Authentication https://github.com/digital-cube/base3service-users.git - Sending emails https://github.com/digital-cube/base3service-mailer.git - PDF generator - Multitenancy - Blog
- 6. Technologies - Python 3.x - tornadoweb (web framework and asynchronous networking library) - Database: - Currently implemented PostgreSQL using SQLAlchemy ORM, but also other popular open source RDBMS will be supported soon - CRUD implementation in API calls (with the use of Models as types) - Reduction of the amount of duplicated code for simple CRUD operations - Redis - Docker
- 7. Implemented decorators - @route - decorate Handler class and define route where methods can be reachable - @api - decorate a method to process input arguments, fetching models - @auth - decorate a method to allow only authorized users to access
- 8. Handling input parameters - Input parameters can be passed through URI or Request Body, both of them are read as parameters in the Handler class - With Python typing, we can automatically check and validate parameters - Currently supported types are str, int, bool, float - Also any defined database Model can be considered as a type - For storing Models in Database, use the Model type - For fetching Models from database use Id attribute of Model - Parameters can be mandatory or optional - For optional parameters set the default value for this parameter - None can be used as default value as well
- 9. Models as Types import base import models @base.route("/") class RegisterUserHandler(base.Base): @base.api() async def post(self, user: models.User): self.orm_session.add(user) self.orm_session.commit() return {"id": user.id}, base.http.status.CREATED if __name__ == "__main__": base.run()
- 10. Models as Types import base import models import lookup.permissions as perm @base.route("/:id_user") class SingleUserHandler(base.Base): @base.auth(permissions=perm.ADMIN) @base.api() async def get(self, user: models.User.id): return {"user": user.serialize()} if __name__ == "__main__": base.run()
- 11. Handling responses - Since the main goal of BASE is to build a REST API, default response headers are application/json, but also any other output can be sent (HTML, text or binary) - To provide the result, just return an object which can be serialized into JSON (dict, list, string) and this object will be returned to the caller - Default status code is HTTP code 200 OK, but any other code can be specified as a second parameter, for example, if you want to return 201 (Created) write: - return {“id”: obj.id}, http.status.CREATED - If the method returns None, or if there is no return statement, the default status code will be 204 (No Content)
- 12. Handling exceptions and errors - Errors, like any other response, can be sent back using a return statement with one of 4XX response code (NOT_FOUND, UNAUTHORIZED, BAD_REQUEST, …) - We can throw an exception which will be caught in the @api decorator and immediately the API call will return an error with an error message and id_message (for translation purposes) - These errors cascade all the way back to the initial service in the case of calls requiring multiple services
- 13. Inter Process Communication (IPC) - Services communicate with each other by using our IPC method - Complex request chains are handled by cascading requests onto different services (if needed) - Allows services to have separation of concerns and communicate between themselves for sharing data and logic - In BASE we share business logic and data between services through using REST APIs of the services themselves - Currently AsyncHTTP request is the only supported method for IPC, and we are planning to introduce other methods and ways
- 14. Concurrency and bottleneck optimization - We recommend the use of async/await methods to avoid blocking service by multiple requests - BASE supports threading coroutine mechanism which will process API calls in multiple threads - External workers using message queuing - Creating multiple instance of the microservice which is a bottleneck
- 15. Testing - Test Driven Development is one of the key concepts of using this framework - Services can be tested independently by mocking results from other services - Services can be tested all together in a single integration test - For this feature, developer needs to follow additional rules (routes throughout all services should be unique, as well as class name of handler) and additional configuration setup, documented in example
- 16. Try it (source code) https://base3.dev/examples/hello_world/hello_world.py import base @base.route("/") class IndexHandler(base.Base): @base.api() async def get(self): return {"message": "Hello World"} if __name__ == "__main__": base.run()
- 17. Try it (using local python3 installation and venv) Open browser and visit http://localhost:9000 mkdir base-demo && cd base-demo python3 -m venv .venv source .venv/bin/activate pip install wheel base3 curl https://base3.dev/examples/hello_world/hello_world.py -o hello_world.py python hello_world.py
- 18. Try it (using docker / docker-compose) mkdir base-demo && cd base-demo curl https://base3.dev/examples/hello_world/hello_world.py -o hello_world.py curl https://base3.dev/examples/hello_world/docker-compose.yaml -o docker-compose.yaml curl https://base3.dev/examples/hello_world/Dockerfile -o Dockerfile docker-compose up Open browser and visit http://localhost:9000
- 19. Microservice based example (using docker / docker-compose) git clone https://github.com/digital-cube/base3example-contacts.git cd base3example-contacts docker-compose up Open browser or postman and visit http://localhost:9000 Todo:.. Add more links..
- 20. app.sfscon.it
- 21. Next steps: - CLI - Project configuration - Managing services - Creating service from a pre-existing template - Support for other databases except PostgreSQL - Write more common services - Automatic documentation generator
- 22. Digital Cube - We have been in business for 5 years and delivered over 30 projects for clients across Europe and North America. - We have collaborated with start-ups, mid-markets, and large enterprises. - Some of our clients include: - Telmekom Networks (South Tyrol, Italy), OOB (Denmark), 3DSystem (USA/Switzerland), Serbian Academy of Sciences and Arts (Serbia)... - We employ experienced developers, designers and engineers with the main goal of providing the best possible products and services - The experience we have gained has enabled us to offer a wide range of software solutions and services that meet the needs of our clients - Our focus is constantly on our clients' business challenges and we provide innovative ideas for them - Our abundant technical expertise is at their disposal around the clock.
- 23. Digital Cube provide services in: - Advisory: - Product Design - Business Process, Optimisation and Analysis - Engineering: - Web Application Development - Mobile Application Development - Desktop Application Development - Assurance: - Software Testing and Quality Assurance - Technical Support