使用elasticsearch+filebeat+kibana收集pod指定目錄日志，filebeat用于收集日志，es用于存儲，kibana用于展示。本例以收集部署于k8s內(nèi)的nginx日志為例子。

1、部署es+kibana

version: '3.9'
services:
  elasticsearch:
    image: elasticsearch:7.13.3
    container_name: elasticsearch
    networks:
      - net-es
    volumes:
      - ./data/elasticsearch/data:/usr/share/elasticsearch/data　　#這里將elasticsearch的數(shù)據(jù)文件映射本地，以保證下次如果刪除>了容器還有數(shù)據(jù)
    environment:
      - discovery.type=single-node
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ports:
      - "9200:9200"

  elastichd:
    image: containerize/elastichd:latest
    container_name: elasticsearch-hd
    networks:
    - net-es
    ports:
      - "9800:9800"
    depends_on:
      - "elasticsearch"
    links:
      - "elasticsearch:demo"

  kibana:
    image: kibana:7.13.3
    container_name: kibana
    depends_on:
      - elasticsearch #kibana在elasticsearch啟動之后再啟動
    environment:
      ELASTICSEARCH_HOSTS: http://ip:9200 #設(shè)置訪問elasticsearch的地址
      I18N_LOCALE: zh-CN
    ports:
      - 5601:5601
#這里要注意，es和eshd要在相同網(wǎng)絡(luò)才能被links
networks:
  net-es:
    external: false

2.創(chuàng)建filebeat配置文件（基于elasticsearch存儲）

[root@master filebeat]# cat filebeat-es-cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeatconf
data:
  filebeat.yml: |-
    filebeat.inputs:
    - input_type: log
      paths:
        - /data/log/*/*.log         # 收集日志的路徑
      tail_files: true
      fields:
        pod_name: '${podName}'
        pod_ip: '${podIp}'
        pod_deploy_name: '${podDeployName}'
        pod_namespace: '${podNamespace}'
    output.elasticsearch:     # 可以使用kafka，redis做緩存，此處直接將數(shù)據(jù)存儲在elasticsearch中
      hosts: ["ip:9200"]    # elasticsearch 的ip+port
      index: "app-%{+yyyy.MM.dd}"     # 索引名稱定義
    setup.template.name: "filebeat-sidecar"
    setup.template.pattern: "filebeat-sidecar"

3、創(chuàng)建nginx-filebeat Sidecar（基于elasticsearch的配置）

[root@k8s-master elk]# cat nginx.yml 
# lishanbin-nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: lishanbin-nginx
spec:
  selector:
    matchLabels:
      app: lishanbin-nginx
  template:
    metadata:
      labels:
        app: lishanbin-nginx
    spec:
      containers:
      - name: filebeat                        
        image: registry.cn-beijing.aliyuncs.com/dotbalo/filebeat:7.10.2 
        resources:
          requests:
            memory: "100Mi"
            cpu: "10m"
          limits:
            cpu: "200m"
            memory: "300Mi"
        imagePullPolicy: IfNotPresent
        env:
          - name: podIp
            valueFrom:
              fieldRef:
                apiVersion: v1
                fieldPath: status.podIP
          - name: podName
            valueFrom:
              fieldRef:
                apiVersion: v1
                fieldPath: metadata.name
          - name: podNamespace
            valueFrom:
              fieldRef:
                apiVersion: v1
                fieldPath: metadata.namespace
          - name: podDeployName
            value: app
          - name: TZ
            value: "Asia/Shanghai"
        securityContext:
          runAsUser: 0
        volumeMounts:
          - name: data-log
            mountPath: /data/log/app/
          - name: filebeatconf
            mountPath: /usr/share/filebeat/filebeat.yml 
            subPath: usr/share/filebeat/filebeat.yml
      - name: lishanbin-nginx
        image: nginx:1.23.1
        env:
        - name: "username"
          value: "lishanbin"
        - name: "password"
          value: "admin123"
        - name: HOST_IP
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: status.hostIP
        - name: NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        - name: POD_IP
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: status.podIP
        - name: POD_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        ports:
        - containerPort: 80
        volumeMounts:
        - name: data-log
          mountPath: /var/log/nginx
      volumes: 
      - name: data-log
        hostPath:
          path: /tmp/hostpath  
      - emptyDir: {}
        name: logpath
      - name: local-time
        hostPath:
          path: /usr/share/zoneinfo/Asia/Shanghai
      - name: filebeatconf
        configMap:
          name: filebeatconf     # 掛載filebeat配置文件
          items:
            - key: filebeat.yml
              path: usr/share/filebeat/filebeat.yml
---
apiVersion: v1
kind: Service
metadata:
  name: lishanbin-nginx
  labels:
    app: lishanbin-nginx
spec:
  ports:
  - port: 80
    protocol: TCP
    name: http
  selector:
    app: lishanbin-nginx
---
apiVersion: v1
kind: Service
metadata:
  name: ishanbin-nginx-nodeport
spec:
  selector:
    app: lishanbin-nginx
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
    nodePort: 30080
  type: NodePort

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: lishanbin-nginx
  namespace: default
spec:
  ingressClassName: lishanbin  # 使用 nginx 的 IngressClass（關(guān)聯(lián)的 ingress-nginx 控制器）
  rules:
  - host: ngdemo.qikqiak.com  # 將域名映射到 lishanbin-nginx 服務(wù)
    http:
      paths:
      - path: /
        pathType: ImplementationSpecific
        backend:
          service:  # 將所有請求發(fā)送到 lishanbin-nginx 服務(wù)的 80 端口
            name: lishanbin-nginx
            port:
              number: 80
# 不過需要注意大部分Ingress控制器都不是直接轉(zhuǎn)發(fā)到Service
# 而是只是通過Service來獲取后端的Endpoints列表，直接轉(zhuǎn)發(fā)到Pod，這樣可以減少網(wǎng)絡(luò)跳轉(zhuǎn)，提高性能

4.kibana創(chuàng)建索引，查看采集到的日志

?文章來源地址http://www.zghlxwxcb.cn/news/detail-550010.html

?

到了這里，關(guān)于k8s-EFK (filebeat)日志收集的文章就介紹完了。如果您還想了解更多內(nèi)容，請在右上角搜索TOY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！