apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd
  namespace: kube-system
spec:
  selector:
    matchLabels:
      name: fluentd
  template:
    metadata:
      labels:
        name: fluentd
    spec:
      containers:
      - name: fluentd
        image: fluent/fluentd-kubernetes-daemonset:v1-debian-elasticsearch
        env:
        - name: FLUENT_ELASTICSEARCH_HOST
          value: "elasticsearch-service"
        - name: FLUENT_ELASTICSEARCH_PORT
          value: "9200"
        volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers

Collecting Logs with Fluentd

Fluentd pods on Kubernetes nodes monitor the /var/log/containers directory on the node’s filesystem and collect all container logs found there. Fluentd then sends these logs to Elasticsearch according to its configuration.

3.Storing Logs with Elasticsearch

Fluentd sends the collected logs to Elasticsearch, which indexes and stores them. While setting up Elasticsearch, you can define a Service as shown below to enable Fluentd to connect to Elasticsearch:

apiVersion: v1
kind: Service
metadata:
  name: elasticsearch-service
  namespace: kube-system
spec:
  selector:
    app: elasticsearch
  ports:
  - port: 9200
    targetPort: 9200

4.Visualizing Logs with Kibana

Kibana allows you to visualize logs stored in Elasticsearch. After installing Kibana as a Deployment on Kubernetes, you need to configure Elasticsearch in Kibana.

Kibana Configuration:

apiVersion: v1
kind: Service
metadata:
  name: kibana-service
  namespace: kube-system
spec:
  ports:
  - port: 5601
    targetPort: 5601
  selector:
    app: kibana

5.Viewing Logs

You can analyze and visualize the collected logs using Kibana. Go to the Kibana interface, add your Elasticsearch indices, and start viewing the logs.

6.Granting RBAC Permissions

Fluentd needs appropriate permissions to collect Kubernetes metadata. For this, you should use Role-Based Access Control (RBAC) in Kubernetes to grant the necessary permissions to Fluentd.

Creating Role and RoleBinding:

Role (Defining Permissions): Use the following Role definition to grant listing permissions on namespace resources:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: fluentd-role
  namespace: kube-system
rules:
- apiGroups: [""]
  resources: ["namespaces"]
  verbs: ["list", "get", "watch"]

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: fluentd-rolebinding
  namespace: kube-system
subjects:
- kind: ServiceAccount
  name: default
  namespace: kube-system
roleRef:
  kind: Role
  name: fluentd-role
  apiGroup: rbac.authorization.k8s.io

This way, you can collect logs from all pods in the Kubernetes cluster via Fluentd, send them to Elasticsearch, and view them with Kibana. Make sure Fluentd is configured correctly and that Elasticsearch is accessible by Fluentd.