Elasticsearch Operations on K8s - Key Specificities
0 likes108 views
The document discusses the deployment of Elasticsearch on Kubernetes, highlighting best practices and challenges related to container orchestration. It emphasizes the importance of high availability, system specifications, and disaster recovery while providing guidelines on provisioning and operational strategies. The conclusion suggests using managed services to reduce operational overhead and ensure effective resource management.
Elasticsearch Operations on K8s - Key Specificities
- 1. 1 Elasticsearch Vietnam Date: Nov 17, 2022
- 2. Nov 17, 2022 8:15 - 8:45 PM (ICT) GFT Technologies Group Tyler Nguyen APAC Solution Architect Operations on Kubernetes: Key Specificities
- 4. 4 Agenda ● Introduction ● Rationale ● Best Practices ● Conclusion
- 6. 6 GFT in 60 seconds
- 7. 7
- 8. 8 Me in a nutshell ● Cloud platform evangelist ● 12+ years experience in IT industry ● APAC Solution Architect @ GFT Technologies Group ● Administrator of of Vietnam Elasticsearch Community ● Student Mentor & Sponsor @ Vietseeds Foundation ● Solo traveller / Metalhead / Coffeeholic :D<
- 9. 9 Rationale
- 10. 10 Discussion ● What is Container Orchestration Platform? Pros n Cons? ● Why do we deploy Elasticsearch on Kubernetes? ● Is Kubernetes a Good Choice for Elasticsearch? ● Challenge? ● Suggestion?
- 11. AVAILABILITY Amount of the system's uptime in a given time period RESILIENCE Ability to maintain the capability in the face of a disruption ELASTICITY Adapting to workload changes by provisioning and deprovisioning resources in an autonomic manner GRANULARITY Increasing the system's capacity and performance through incremental processor expansion. Concerns
- 13. 13 ● High availability.(*) ● Proper specifications. ● System reservations. ● Use systemd cgroups driver. ● Prefer cgroup v2.(*) ● BAU processes. ● Disaster recovery and Backups. ● Data security. DOs ● Single-node cluster. ● Same specification for all nodes. ● No system reservations. ● No limit resource allocation. ● No expandable storage management. ● No replication, backups, disaster recovery. ● No data encryption in-transit and encryption-at-rest. DONTs Key Point #1: Ensure the K8s Infrastructure Robustness
- 14. 14 ● Type-based nodes isolation. ● Resources allocation. ● Storage class of service. ● Firewall and network policies. ● Role-based Access Control (RBAC). ● Use the Elasticsearch latest version. ● Consider using Elasticsearch Operator (aka ECK) if possible. DOs ● Standalone Elasticsearch cluster. ● No dedicated master-eligible nodes. ● Use default storage class. ● Same resources allocation. ● No RBAC. ● Public accessible. ● Deprecated Elasticsearch versions (<5.x) DONTs Key Point #2: Elasticsearch Provisioning
- 15. 15 ● Update strategy. ● Graceful shutdown. ● Autoscaling(*) . ● Readiness and Liveness Probes. ● Headless services. ● Observability stack. ● Container smaller image sizes. ● Cross-cluster replication(*) . DOs ● Single-node cluster. ● Same specification for all nodes. ● Recreate deployment strategy. ● Manual scaling. ● No healthchecks. ● Communication through load balancers. ● No observability stack. DONTs Key Point #3: Elasticsearch Operations - Cluster Level
- 16. 16 ● Index organization. ● Using alias. ● Retention. ● Utilize Reindex API. ● Utilize Forcemerge(*) . ● Single-type Index. ● Index Routing. ● Sharding and sizing strategies. DOs ● No alias. ● No retention. ● Multi-type Index. ● Beware Delete Index API. ● Short refresh interval ● Dynamic string mappings DONTs Key Point #4: Elasticsearch Operations - Index Level
- 17. 17 Summary
- 18. 18 Conclusions ● High availability oriented design pattern. ● Native cloud with Elasticsearch Cloud on Kubernetes (ECK) ● Update strategies (especially in Elasticsearch/Kubernetes version upgrade). ● An essential role of observability stack. ● Consideration of using managed services to avoid operations overhead. ● Nodes/Indices/Shards strategies.
- 19. 19 Elastic is a Search Company. www.elastic.co Thank You
- 20. 20 Contact us ● tylern91.github.io Find me ● fb.com/groups/elasticsearchvn Elasticsearch Vietnam Group