Deploy Elasticsearch Cluster on Kubernetes
Download as PPTX, PDF1 like214 views
Kubernetes can be used to deploy an Elasticsearch cluster. Kubernetes runs workloads by placing containers into pods to run on nodes. Pods are the smallest deployable units that contain one or more containers with shared resources. For stateful applications like Elasticsearch, a StatefulSet should be used instead of a Deployment to ensure ordered startup and termination of pods with persistent storage. The Elasticsearch cluster can be deployed on Kubernetes using StatefulSets, ConfigMaps to store configurations, and PersistentVolumes to provide storage for data shards.
Deploy Elasticsearch Cluster on Kubernetes
- 1. Deploy Elasticsearch Cluster on Kubernetes Presenter: Elastic Saudi Arabia - Meetup Date: 23 June - 2021 - 09:00 PM
- 2. Chief Architect Elastic Community Lead in the Middle East Staff engineer | CKA and CKAD Mohammad Abusaa Ismail Anjrini
- 4. Kubernetes
- 5. What is Kubernetes ? Kubernetes, also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications. It was originally designed by Google and is now maintained by the Cloud Native Computing Foundation. Kubernetes runs your workload by placing containers into Pods to run on Nodes.
- 6. Pods Pods are the smallest deployable units of computing that you can create and manage in Kubernetes. A Pod is a group of one or more containers, with shared storage and network resources. Container vs initContainer
- 7. StatefulSets vs Deployment Item Deployment Statefulset Identity of pods Identical, it will get random Id Sticky unique Identifier for each pod Order of creation/deletion Random order Ordered, cannot be created or deleted at the same time Pods endpoints Random name, we cannot rely on it. Fixed per pod Persistent storage By nature , it does not need it, but you can use it By nature, a StatefulSet needs persistent storage so that the application saves its state and does not lose data across restarts.
- 8. Persistent volume A PersistentVolume (PV) is a piece of storage in the cluster that has been provisioned by an administrator or dynamically provisioned using Storage Classes. It have a lifecycle independent of any individual Pod that uses the PV.
- 10. Kibana
Editor's Notes
- #2: How To Set Up an Elasticsearch, Fluentd and Kibana (EFK) Logging Stack on Kubernetes: https://www.digitalocean.com/community/tutorials/how-to-set-up-an-elasticsearch-fluentd-and-kibana-efk-logging-stack-on-kubernetes Persistent Volume Claim for StatefulSet: https://zhimin-wen.medium.com/persistent-volume-claim-for-statefulset-8050e396cc51 Install Elasticsearch with Docker: https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html https://www.elastic.co/downloads/elastic-cloud-kubernetes
- #5: https://kubernetes.io/ CNCF graduated project
- #10: ELK Docker files https://github.com/elastic/dockerfiles For UserID: 1000 -> check the file https://github.com/elastic/elasticsearch/blob/master/distribution/docker/src/docker/Dockerfile -> adduser -D -u 1000 -G elasticsearch Based on the following file, you can pass elasticsearch.yml settings as Docker env variables, check after “if [[ "$ES_DISTRIBUTION_TYPE" == "docker" ]]; then” https://github.com/elastic/elasticsearch/blob/master/distribution/src/bin/elasticsearch-env
- #11: Kibana Env. variables: https://github.com/elastic/dockerfiles/blob/7.13/kibana/bin/kibana-docker