PyData - Consumindo e publicando web APIs com Python
10 gostaram6,768 visualizações
A palestra aborda as APIs web, discutindo sua definição, tipos e evolução ao longo dos anos, incluindo conceitos como REST e GraphQL. O palestrante enfatiza a importância de boas práticas na criação de APIs, como consistência, escalabilidade e segurança, utilizando frameworks como Flask e ferramentas para documentação como Swagger. O conteúdo é inteiramente baseado na experiência pessoal do palestrante e não reflete a posição oficial da Red Hat.
![$ pip install requests
>>> r = requests.get('https://api.github.com/user', auth=('user', 'pass'))
>>> r.status_code
200
>>> r.headers['content-type']
'application/json; charset=utf8'
>>> r.encoding
'utf-8'
>>> r.text
u'{"type":"User"...'
>>> r.json()
{u'private_gists': 419, u'total_private_repos': 77, ...}](https://image.slidesharecdn.com/pydata-webapis-170329034224/85/PyData-Consumindo-e-publicando-web-APIs-com-Python-15-320.jpg)
![Converter, serializar e transformar?
import requests
import rows
# Get data from Portuguese Wikipedia
city_list_url = 'https://pt.wikipedia.org/wiki/Lista_de_munic%C3%ADpios_do_Brasil'
response = requests.get(city_list_url)
html = response.content
# Extract desired data using XPath
cities = rows.import_from_xpath(
BytesIO(html),
rows_xpath='//table/tr/td/ul/li',
fields_xpath=OrderedDict([('name', './/text()'),
('link', './/a/@href')]))
# Rows supports other data sources
- csv, txt, json, html, parquet, xpath, xls, xlsx, SQlite, ODS +
github.com/turicas/rows](https://image.slidesharecdn.com/pydata-webapis-170329034224/85/PyData-Consumindo-e-publicando-web-APIs-com-Python-26-320.jpg)
![@app.route('/product', methods=['POST'])
def post_product():
data = request.json
product.create(**data)
result = {'msg': 'Produto inserido com sucesso!',
‘product’: url_for(‘get_product’, product.id)}
….
return jsonify(**result), 201
$ curl -XPOST http://localhost:5000/product -d “{‘id’: 5, ‘name’: ‘foo’, ‘category’: ‘bar’}”
….
Response 201 Created
{
“msg”: “Produto inserido com sucesso”,
“product”: “/product/5”
}](https://image.slidesharecdn.com/pydata-webapis-170329034224/85/PyData-Consumindo-e-publicando-web-APIs-com-Python-49-320.jpg)
![from flasgger import Swagger, validate
Swagger(app)
@app.route('/product', methods=['POST'])
def post_product():
“””Create a new product
parameters:
- in: body
schema:
required: [name, category, id]
properties:
name:
type: string description: The name of the product
“””
data = request.json
validate(data)
return jsonify(result={'msg': 'Deu Certo!'}), 201](https://image.slidesharecdn.com/pydata-webapis-170329034224/85/PyData-Consumindo-e-publicando-web-APIs-com-Python-51-320.jpg)
![from flasgger import Swagger, validate, swag_from
Swagger(app)
@app.route('/product', methods=['POST'])
@swag_from(‘post_product.yml’)
def post_product():
data = request.json
validate(data)
return jsonify(result={'msg': 'Deu Certo!'}), 201](https://image.slidesharecdn.com/pydata-webapis-170329034224/85/PyData-Consumindo-e-publicando-web-APIs-com-Python-52-320.jpg)
![from flasgger import Swagger, validate
Swagger(app)
@app.route('/product', methods=['POST'])
@swag_from({‘parameters’: …, ‘definitions’: ...})
def post_product():
data = request.json
validate(data)
return jsonify(result={'msg': 'Deu Certo!'}), 201](https://image.slidesharecdn.com/pydata-webapis-170329034224/85/PyData-Consumindo-e-publicando-web-APIs-com-Python-53-320.jpg)
PyData - Consumindo e publicando web APIs com Python
- 1. http://brunorocha.org *@rochaCbruno Disclaimer: O conteúdo dessa palestra representa a opinião pessoal do palestrante e não está de nenhuma forma relacionado oficialmente com a Red Hat Inc. Bruno Rocha Quality Engineer
- 2. Episódio toda segunda-feira às 11 horas! http://castalio.info
- 3. web APIs consumindo e publicando Com Python
- 4. 1. O que são APIs? 2. Consumindo web APIs 3. O que fazer com os dados? 4. Publicando web APIs
- 5. API? Application Programming Interface GUI CLI API MUI App A App B
- 6. API é só web? API App A App B def function_a(argument1, argument2): ... def function_b(argument1, argument2): ... return function_a('value1', 'value2') API API Tipos de APIs: (baseadas em seus protocolos) - Memory Pointers - Object orientation members - Remote Procedure Call - Telnet - EDI (Eletronic Data Interchange) - Serial Com - Parallel Com - Universal Serial Bus - ftp - Ssh - Gopher, POP, SMTP, IMAP... - http - …. (muitas outras)
- 7. Especificações, padrões e protocolos - RPC / CORBA (Remote Procedure Call - Xerox 1976) - EDI (Eletronic Data Exchange - University of Minnesota in 1991) - Windows COMs (Component Object Model 1993) - HTTP ( ietf 1990) - SOA (Service-Oriented Architecture - Gartner Group 1996) - XML + XML-RPC (Microsoft 1998) - SOAP (W3C - 2000) - REST (Roy Fielding 2000) - AMQP (JPMorgan 2003) - Web Services / WS protocol (W3C 2005) - JSON + JSON RPC (Douglas Crockford - 2006) - Thrift, MQ, ProtoBuf (Apache, Google, Pivotal etc…. - WIP) - Microservices - (Martin Fowler - 2014) - GraphQL - (Facebook 2015) 40 anos!
- 8. O que mudou em 20 anos? - RPC / CORBA (Remote Procedure Call - Xerox 1976) - EDI (Eletronic Data Exchange - University of Minnesota in 1991) - Windows COMs (Component Object Model 1993) - HTTP ( ietf 1990) - SOA (Service-Oriented Architecture - Gartner Group 1996) - XML + XML-RPC (Microsoft 1998) - SOAP (W3C - 2000) - REST (Roy Fielding 2000) - AMQP (JPMorgan 2003) - Web Services / WS protocol (W3C 2005) - JSON + JSON RPC (Douglas Crockford - 2006) - Thrift, MQ, ProtoBuf (Apache, Google, Pivotal etc…. - WIP) - Microservices - (Martin Fowler - 2014) - GraphQL - (Facebook 2015) 20 anos! The Internet Days
- 9. O que é uma API web (http)? Interface de comunicação baseada nos verbos HTTP com retorno de dados padronizado em formatos largamente adotados como JSON e XML etc. VERBO CRRUD Retorno esperado POST Create 201 (created), 409, 50x GET Read 200 (Ok), 204, 404, 500 PUT Replace 200, 201, 409, 404, 50x PATCH Update 200, 201, 409, 404, 50x DELETE Delete 200, 404, 50x http://www.restapitutorial.com/lessons/httpmethods.html
- 10. O que é REST? O que faz uma web API ser REST é a concordancia com o padrão HATEOAS, ou seja, é a maneira como o fluxo de navegação e estado entre os recursos é arquitetada, em uma API Restfull de verdade além dos dados em si também trafegam informações semanticas, metadados de estado e localização dos recursos. HyperMedia define o protocolo e o estado!
- 11. GraphQL - (REST vai morrer?) http://graphql.org - http://graphene-python.org/
- 12. Consumindo WEB APIs API100% RESTNão contém XML
- 13. $ curl -XPOST http://localhost:5000/product -d “{‘id’: 5, ‘name’: ‘foo’, ‘category’: ‘bar’}” …. Response 201 Created { “msg”: “Produto inserido com sucesso”, “product”: “/product/5” } Cliente HTTP *Programa capaz de se comunicar através dos verbos HTTP (GET, POST, PUT….)
- 14. https://httpie.org/ ● Colorized and formatted terminal output ● Built-in JSON support ● Forms and file uploads ● HTTPS, proxies, and authentication ● Extensions ● Linux, Mac OSX, and Windows support ● And more…
- 15. $ pip install requests >>> r = requests.get('https://api.github.com/user', auth=('user', 'pass')) >>> r.status_code 200 >>> r.headers['content-type'] 'application/json; charset=utf8' >>> r.encoding 'utf-8' >>> r.text u'{"type":"User"...' >>> r.json() {u'private_gists': 419, u'total_private_repos': 77, ...}
- 17. HTTPbin
- 18. Não reinvente a roda!
- 20. Não encontrei meu wrapper e agora?
- 22. API Wrapper Framework from tapioca import TapiocaAdapter from tapioca.serializers import SimpleSerializer class MyAPISerializer(SimpleSerializer): def serialize_datetime(self, data): return data.isoformat() class MyAPIAdapter(TapiocaAdapter): serializer_class = MyAPISerializer ... http://tapioca-wrapper.readthedocs.io
- 23. O que fazer com os dados?
- 24. Easy Data Retrieval++ - Es Engine https://github.com/catholabs/esengine - Py Mongo https://github.com/mongodb/mongo-python-driver - Monary https://monary.readthedocs.io - pyArango https://github.com/tariqdaouda/pyArango
- 25. Converter?
- 26. Converter, serializar e transformar? import requests import rows # Get data from Portuguese Wikipedia city_list_url = 'https://pt.wikipedia.org/wiki/Lista_de_munic%C3%ADpios_do_Brasil' response = requests.get(city_list_url) html = response.content # Extract desired data using XPath cities = rows.import_from_xpath( BytesIO(html), rows_xpath='//table/tr/td/ul/li', fields_xpath=OrderedDict([('name', './/text()'), ('link', './/a/@href')])) # Rows supports other data sources - csv, txt, json, html, parquet, xpath, xls, xlsx, SQlite, ODS + github.com/turicas/rows
- 29. @app.route("/hello") def hello(): return jsonify({"hello": "world"}) Flask é extremamente simples e intuitivo para criação de APIs simples. … Mas há muito mais do que JSON em boas APIs.
- 30. Caracteristicas de uma boa web API 1. CAP a. Consistencia(c) imediata é sacrificada em troca de Disponibilidade(a) e Particionamento(p)! 2. Escalável a. MicroServices (fila de processamento) b. Caching c. Async Response 3. Stateless a. REST/HATEOAS/OAuth 4. Concisa a. Versionamento b. Formato de dados c. Documentação 5. Segura a. Throtling b. Autenticação
- 31. Tem framework para fazer as mágica pra nóis? MethodViews Flask-Classy Flask RestFull Flask RestPlus Flask-Potion Eve Conexxion Flasgger
- 32. 1 from flask import MethodView, jsonify 2 3 class ProductAPI(MethodView): 4 def get(self): 5 return jsonify(....) 6 def post(self): 7 return jsonify(....) 8 def delete(self): 9 return jsonify(....) 10 11 app.add_url_rule(‘/product’, ProductAPI.as_view(‘product’))
- 33. from flask import Flask from flask_sqlalchemy import SQLAlchemy from flask_potion import Api, ModelResource app = Flask(__name__) db = SQLAlchemy(app) class Book(db.Model): id = db.Column(db.Integer, primary_key=True) title = db.Column(db.String(), nullable=False) year_published = db.Column(db.Integer) db.create_all() class BookResource(ModelResource): class Meta: model = Book api = Api(app) api.add_resource(BookResource)
- 34. from eve import Eve app = Eve() app.config.MONGO_HOST = “localhost” app.run()
- 37. Escalabilidade 1. Adotar arquitetura de Micro Serviços e Async Message Queues a. Nameko. Kafka, Celery, *MQ 2. Caching a. Redis, Memcached, Elasticsearch* 3. Distribuição a. API Gateway (ver Nginx OpenResty ;) 4. Elasticidade a. AWS, OpenShift, Kubernetes *Materialized path
- 38. from flask import Flask from flask import jsonify app = Flask() @app.route("/") def test(): return jsonify({"hello": "world"}) app.run(host="0.0.0.0", port=8000)
- 39. from sanic import Sanic from sanic.response import json app = Sanic() @app.route("/") async def test(request): return json({"hello": "world"}) app.run(host="0.0.0.0", port=8000)
- 40. from sanic import Sanic as Flask from sanic.response import json as jsonify app = Flask() @app.route("/") async def test(request): return jsonify({"hello": "world"}) app.run(host="0.0.0.0", port=8000) from flask import Flask from flask import jsonify app = Flask() @app.route("/") def test(): return jsonify({"hello": "world"}) app.run(host="0.0.0.0", port=8000) Flask (sync, blocking) Sanic (asyncio) Otimização prematura é a raiz de todo o mal! Pode ser que seu projeto não precise de AsyncIO Antes de partir para o Sanic + AsyncIO tente: - Arquitetura distribuida (+workers) - Gunicorn, twisted, gevent - Async microservices (e MQs)
- 42. import nameko class MyService(nameko.Service): @nameko.rpc def my_resource(): data = # expensive computation return data import nameko from flask import Flask from flask import jsonify app = Flask() @app.route("/") def test(): with nameko.rpc(‘my-rabbit’) as rpc: result = rpc.my_resource() return jsonify(result) # or return only the “future” as task id Microservices Nameko + rabbit MQ (microserviço) + $ gunicorn -w 4 -b 127.0.0.1:4000 myproject:app + $ twistd web --wsgi myproject.app + $ gevent.wsgi.WSGIServer(('', 5000), app) Flask API + $ nameko run my_service -c 10 + $ docker run -d --hostname my-rabbit --name my-rabbit -p 15672:15672 -p 5672:5672 rabbitmq:3-management
- 43. API é Front-End! ● Performance ● Design ● Usabilidade ● User Experience ● Java Script Compat ● ...
- 44. $ curl -XPOST http://localhost:5000/product -d “{‘id’: 5, ‘name’: ‘foo’, ‘category’: ‘bar’}” …. Response 201 Created { “msg”: “Produto inserido com sucesso”, “product”: “/product/5” }
- 45. Swagger UI? - Framework para live-documentation (API Playground) - Baseado na especificação JSONSchema e OpenAPI. - Utiliza YAML e JSON
- 46. Slate - Framework para Documentação Comercial de API - Baseado na especificação JSONSchema E Jquery. - Utiliza Markdown
- 47. Flask + Swagger = + slate
- 48. $ pip install flasgger
- 49. @app.route('/product', methods=['POST']) def post_product(): data = request.json product.create(**data) result = {'msg': 'Produto inserido com sucesso!', ‘product’: url_for(‘get_product’, product.id)} …. return jsonify(**result), 201 $ curl -XPOST http://localhost:5000/product -d “{‘id’: 5, ‘name’: ‘foo’, ‘category’: ‘bar’}” …. Response 201 Created { “msg”: “Produto inserido com sucesso”, “product”: “/product/5” }
- 51. from flasgger import Swagger, validate Swagger(app) @app.route('/product', methods=['POST']) def post_product(): “””Create a new product parameters: - in: body schema: required: [name, category, id] properties: name: type: string description: The name of the product “”” data = request.json validate(data) return jsonify(result={'msg': 'Deu Certo!'}), 201
- 52. from flasgger import Swagger, validate, swag_from Swagger(app) @app.route('/product', methods=['POST']) @swag_from(‘post_product.yml’) def post_product(): data = request.json validate(data) return jsonify(result={'msg': 'Deu Certo!'}), 201
- 53. from flasgger import Swagger, validate Swagger(app) @app.route('/product', methods=['POST']) @swag_from({‘parameters’: …, ‘definitions’: ...}) def post_product(): data = request.json validate(data) return jsonify(result={'msg': 'Deu Certo!'}), 201
- 54. from flasgger import Swagger, SwaggerView from flasgger import validate, Schema, fields class Product(Schema): name = fields.Str(required=True) category = fields.Nested(Category) id = fields.Str(required=True) class ProductAPIView(SwaggerView): parameters = Product validation = True def post(self): return jsonify(result={'msg': 'Deu Certo!'}), 201
- 56. Tutorial de Flask http://bit.ly/whattheflask DEMO do Flasgger http://flasgger-rochacbruno.rhcloud.com/ Tutorial de Flasgger http://brunorocha.org/python/flask/flasgger-api-playgrou nd-with-flask-and-swagger-ui.html API com Flasgger baseada em MicroServices http://brunorocha.org/python/microservices-with-python- rabbitmq-and-nameko.html
- 57. http://brunorocha.org @rochacbruno (everywhere) Disclaimer: O conteúdo dessa palestra representa a opinião pessoal do palestrante e não está de nenhuma forma relacionado oficialmente com a Red Hat Inc. Bruno Rocha Quality Engineer Obrigado!!! (perguntas?)