Rancher中使用promtail+loki+grafna收集k8s日志并展示

Rancher中使用promtail+loki+grafna收集k8s日志并展示

根据应用需求和日志数量级别选择对应的日志收集、过滤和展示方式，当日志量不太大，又想简单集中管理查看日志时，可使用promtail+loki+grafna的方式。本文找那个loki和grafana外置在了k8s集群之外。

1、添加Chart Repo

方式一：

helm repo add grafana https://grafana.github.io/helm-charts

方式二：

登录rancher，选择集群→应用→Chart仓库→创建，配置仓库

2、安装promtail

选择集群→应用→Chart，搜索promtail，然后安装

注意，

配置promtail

选择集群→应用→已安装应用，开始配置

修改loki地址，然后更新

更新完成后会自动弹出打印日志。

3、安装loki+grafana

loki+grafana外置，没有放到k8s集群中，我这里直接在其他机器用docker-compose方式部署

编辑docker-compose.yaml 文件如下

version: "3"

networks:
  loki:

services:
  loki:
    image: grafana/loki:2.8.3
    ports:
      - "4100:3100"
    volumes:
      - /data/loki/loki/chunks:/loki/chunks
      - /data/loki/loki/rules:/loki/rules
      - /etc/localtime:/etc/localtime
      - /data/loki/loki/loki-local-config.yaml:/etc/loki/local-config.yaml
    command: -config.file=/etc/loki/local-config.yaml
    networks:
      - loki

  grafana:
    environment:
      - GF_PATHS_PROVISIONING=/etc/grafana/provisioning
      - GF_AUTH_ANONYMOUS_ENABLED=false
      - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
    entrypoint:
      - sh
      - -euc
      - |
        mkdir -p /etc/grafana/provisioning/datasources
        cat <<EOF > /etc/grafana/provisioning/datasources/ds.yaml
        apiVersion: 1
        datasources:
        - name: Loki
          type: loki
          access: proxy 
          orgId: 1
          url: http://loki:3100
          basicAuth: false
          isDefault: true
          version: 1
          editable: false
        EOF
        /run.sh
    image: grafana/grafana:latest
    ports:
      - "3000:3000"
    volumes:
      - /etc/localtime:/etc/localtime
    networks:
      - loki

创建对应文件存储目录并编写loki配置文件

mkdir -p /data/loki/loki/{chunks,rules}
chmod 777 /data/loki/loki/{chunks,rules}

cat /data/loki/loki/loki-local-config.yaml 
auth_enabled: false

server:
  http_listen_port: 3100
  grpc_listen_port: 9096

common:
  instance_addr: 127.0.0.1
  path_prefix: /tmp/loki
  storage:
    filesystem:
      chunks_directory: /tmp/loki/chunks
      rules_directory: /tmp/loki/rules
  replication_factor: 1
  ring:
    kvstore:
      store: inmemory

query_range:
  results_cache:
    cache:
      embedded_cache:
        enabled: true
        max_size_mb: 100

schema_config:
  configs:
    - from: 2020-10-24
      store: boltdb-shipper
      object_store: filesystem
      schema: v11
      index:
        prefix: index_
        period: 24h

ruler:
  alertmanager_url: http://localhost:9093

# By default, Loki will send anonymous, but uniquely-identifiable usage and configuration
# analytics to Grafana Labs. These statistics are sent to https://stats.grafana.org/
#
# Statistics help us better understand how Loki is used, and they show us performance
# levels for most users. This helps us prioritize features and documentation.
# For more information on what's sent, look at
# https://github.com/grafana/loki/blob/main/pkg/usagestats/stats.go
# Refer to the buildReport method to see what goes into a report.
#
# If you would like to disable reporting, uncomment the following lines:
#analytics:
#  reporting_enabled: false

启动服务

docker-compose up -d

4、使用

访问http://IP:3000，登录grafana，默认账户密码:admi/admin

也可以直接进容器查看

#进入容器
docker exec -it loki_grafana_1 bash
#查看密码
cat /usr/share/grafana/conf/defaults.ini |grep -C 10 admin_user

# Controls if the UI contains any links to user feedback forms
feedback_links_enabled = true

#################################### Security ############################
[security]
# disable creation of admin user on first start of grafana
disable_initial_admin_creation = false

# default admin user, created on startup
admin_user = admin

# default admin password, can be changed before first start of grafana, or in profile settings
admin_password = admin

# default admin email, created on startup
admin_email = admin@localhost

# used for signing
secret_key = SW2YcwTIb9zpOOhoPsMm

登录后选择Explore

选择Loki，根据Lable过滤命名空间、app、容器名、文件名等条件，然后Run query 运行