![Unit Testing
Resources (REST)
Service
Layer
Domain
Repositories
HTTPClient
Pub/SubLayer Service Boundary
“77% of the failures found in prod can be reproduced by a unit test” [1]
Run Tests in Parallel
Focus Areas
[1] Source (http://www.eecg.toronto.edu/~yuan/papers/failure_analysis_osdi14.pdf)
Data Access Layer (DAL)
Be Lazy and try:
• Lombok
• Schema & API Auto Generation
(i.e. Swagger, Avro etc)](https://image.slidesharecdn.com/shane-galvin-microservice-automated-testing-k8s-181120183218/85/Microservice-Automated-Testing-on-Kubernetes-6-320.jpg)
![Schemas like Avro
(for services that interact via Kafka Events)
{
"type" : "record",
"name" : "userInfo",
"namespace" : "my.example",
"fields" : [
{"name" : "username", "type" : "string"},
{"name" : ”address", "type" : "string"},
{"name" : ”town", "type" : "string"},
{"name" : ”county", "type" : "string"},
{"name" : "country", "type" : "string"},
{"name" : "age", "type" : "int"}
]
}
Producer ConsumerKafka
Schema
Registry
Send Schema Get Schema
Send Avro
Message
Read Avro
Message
Benefits:
• Centralized Schema Location
& Data Discovery
• Safe Schema Evolution
• Resilient Consumers
• Storage and computational
efficiency
• Encourages Collaboration](https://image.slidesharecdn.com/shane-galvin-microservice-automated-testing-k8s-181120183218/85/Microservice-Automated-Testing-on-Kubernetes-12-320.jpg)
![[QE 2017] Dawid Pacia, Tomasz Janiszewski - SQA w erze TestOps](https://cdn.slidesharecdn.com/ss_thumbnails/dawidpaciatomaszjaniszewski-170727081650-thumbnail.jpg?width=560&fit=bounds)
Microservice Automated Testing on Kubernetes
- 1. Before Test Automation Developer Tester Shane Galvin @ Devopsdays Galway 2018
- 2. E2E Tests Service Tests Unit Tests Test Automation - Anti-Patterns Manual Tests Ice Cream Cone Hour Glass/Smoke Stack Service Tests E2E Tests Unit Tests Manual Tests
- 3. Test Automation Pyramid E2E Journey Tests Component Tests Integration Tests (Contract Tests) Unit Tests Speed Scope $$$$ cents Costs Higher ROI Quantity Adapted from Mike Cohn (Succeeding with Agile: Software Development Using Scrum (2009)), and Martin Fowler https://martinfowler.com/bliki/TestPyramid.html More Time & Effort
- 4. Microservice Pull Request Pipeline (Run on every microservice code commit) Image: order-service Tag: 1.0.23artifactId: order-service version: 1.0.23 Component Tests Unit Tests Integration Tests Contract Tests Build Jar Build Image Create Helm Chart Auto Deploy Isolated Kubernetes Namespace Chart: order-service Version: 1.0.23
- 5. Resources (REST) Service Layer Domain Repositories Microservice Boundary Data Access Layer (DAL) Pub/SubLayer HTTPClient Our Standard Microservice Topic 1 •Msg1 •Msg2 Topic 2 •Msg1 Database External Microservice REST Container Boundary
- 6. Unit Testing Resources (REST) Service Layer Domain Repositories HTTPClient Pub/SubLayer Service Boundary “77% of the failures found in prod can be reproduced by a unit test” [1] Run Tests in Parallel Focus Areas [1] Source (http://www.eecg.toronto.edu/~yuan/papers/failure_analysis_osdi14.pdf) Data Access Layer (DAL) Be Lazy and try: • Lombok • Schema & API Auto Generation (i.e. Swagger, Avro etc)
- 7. Integration Testing Embedded Mock HTTP Service Network Access Embedded Database Topic1 •abc Topic2 •def Embedded Messaging Embedded HTTP Client Resources (REST) Service Layer Domain Repositories Data Access Layer (DAL) Service Boundary Pub/SubLayer HTTPClient Testing Framework Boundary Focus Areas Embedded Dependencies
- 9. Contract Testing To Prevent Microservice Integration Hell Services that interact via API’s Services that interact via Events
- 10. ConsumerConsumerConsumer(s) Pact Mock Pact File(s) Request Response “Agreement between the Consumer and Provider about the format of data that they transfer between each other.” Publishes Consumer(s) Pipeline GET: http://localhost:8080/user/1 Response: 200 OK { "id": 1, "name": ”Joe Soap", "username": ”joesoap", "email": "[email protected]" } Consumer Driven Contract Testing with Pact (for services that interact via REST & JSON)
- 11. ConsumerConsumerConsumer(s) Pact Mock Pact File(s) Request Response Provider Request Response “Agreement between the Consumer and Provider about the format of data that they transfer between each other.” Publishes Consumer(s) Pipeline Producer Pipeline GET: http://localhost:8080/user/1 Response: 200 OK { "id": 1, "name": ”Joe Soap", "username": ”joesoap", "email": "[email protected]" } Response: 200 OK { "id": 1, "name": ”Joe Soap", "username": ”joesoap", "email": "[email protected]" } GET: http://localhost:8085/user/1 Consumer Driven Contract Testing with Pact (for services that interact via REST & JSON)
- 12. Schemas like Avro (for services that interact via Kafka Events) { "type" : "record", "name" : "userInfo", "namespace" : "my.example", "fields" : [ {"name" : "username", "type" : "string"}, {"name" : ”address", "type" : "string"}, {"name" : ”town", "type" : "string"}, {"name" : ”county", "type" : "string"}, {"name" : "country", "type" : "string"}, {"name" : "age", "type" : "int"} ] } Producer ConsumerKafka Schema Registry Send Schema Get Schema Send Avro Message Read Avro Message Benefits: • Centralized Schema Location & Data Discovery • Safe Schema Evolution • Resilient Consumers • Storage and computational efficiency • Encourages Collaboration
- 13. Continuous Deployment Testing Use Prod like staging environments, and repeatable deployments So now you have completed your build tests, and have a versioned artifact….What’s next? Use Testing Framework Features or Docker Compose to create test environments
- 14. “Kubernetes is open-source system for automating deployment,scaling, and management of containerized applications” “Helm helps you manage Kubernetes applications — Helm Charts helps you define,install, and upgrade even the most complex Kubernetes application”
- 15. helm upgrade --name ServiceA --namespace dev --replicas 1 set DB.persistence.enable=false charts/ServiceA Deployment Flexibility with Helm ServiceA ServiceA ServiceC prodpull-request-1 requirements.yaml dependencies: - name: Kafka - name: Wiremock requirements.yaml dependencies: - name: Kafka - name: Service-B - name: DB-B - name: Service-C - name: DB-C - name: Service-D helm upgrade --name ServiceA --namespace prod --replicas 3 charts/ServiceA Kafka DB-B DB-CWireMock ServiceB ServiceD Kafka
- 16. Kafka Kafka ServiceA ServiceC ServiceB Kafka Database ServiceA Kafka ServiceB API ServiceC API Isolated Kubernetes Namespace (Pull Request)Production Kubernetes Component Testing (Testing Kubernetes Services in Isolation) Test Framework Database ServiceD
- 17. Kafka Kafka ServiceA ServiceC ServiceB Kafka Database ServiceA Kafka ServiceB API ServiceC API Isolated Kubernetes Namespace (Pull Request)Production Kubernetes Component Testing (Testing Kubernetes Services in Isolation) Test Framework Database ServiceD
- 18. Tools for Local Dev Testing Local Kubernetes Environments Kubernetes for Docker Automate build -> push -> deploy from Local Dev Environment Debug Locally, as if it was running Remotely
- 19. Key Takeaways (1/2) Consumer Driven Contract Testing and Schema Registries make integrations easier, and promote safe schema evolution. Helm charts facilitate rolling upgrades and flexible configurations when deploying to different kubernetes environments.
- 20. Key Takeaways (2/2) Embedded tools give you a high degree of confidence by facilitating integration and contract testing early in the pipeline (-> fast feedback). However embedded tools should never be the end goal, and you still need to test your services in isolation on Kubernetes prod like environments.