How does Filebeat collect Kubernetes logs?

To collect Kubernetes (K8s) logs, you can use Filebeat to gather and send them to a specified destination.

Here are the steps for collecting Kubernetes logs:

1. Set up Filebeat: Configure the following in the Filebeat configuration file, filebeat.yml.
2. filebeat inputs will be set to type container with paths leading to log files in the specified directory. The output will be sent to Elasticsearch at the specified host and port.
3. This configuration specifies the data type collected by Filebeat as container logs, with the path set to the log file path of Kubernetes containers. Additionally, it designates Elasticsearch as the target output.
4. Deploy Filebeat: Depending on your Kubernetes cluster environment, you can choose to deploy Filebeat using Helm Chart or manually create Kubernetes deployment files. Here is an example deployment file filebeat-kubernetes.yaml:
5. apiVersion: v1
   kind: ConfigMap
   metadata:
   name: filebeat-config
   labels:
   k8s-app: filebeat
   data:
   filebeat.yml: |-
   filebeat.inputs:
   – type: container
   paths:
   – /var/lib/docker/containers/*/*.log

   output.elasticsearch:
   hosts: [“elasticsearch:9200”]

   —
   apiVersion: apps/v1
   kind: DaemonSet
   metadata:
   name: filebeat
   labels:
   k8s-app: filebeat
   spec:
   selector:
   matchLabels:
   k8s-app: filebeat
   template:
   metadata:
   labels:
   k8s-app: filebeat
   spec:
   containers:
   – name: filebeat
   image: docker.elastic.co/beats/filebeat:7.15.1
   args: [
   “-c”, “/etc/filebeat.yml”,
   “-e”,
   ]
   env:
   – name: ELASTICSEARCH_HOST
   value: “elasticsearch:9200”
   volumeMounts:
   – name: config
   mountPath: /etc/filebeat.yml
   subPath: filebeat.yml
   readOnly: true
   …
   volumes:
   – name: config
   configMap:
   defaultMode: 0600
   name: filebeat-config

6. Apply deployment file: Utilize the kubectl command to apply the deployment file, for example:
7. Apply the configuration in the filebeat-kubernetes.yaml using kubectl.
8. This will create a DaemonSet in the Kubernetes cluster, ensuring that a Filebeat instance runs on every node.
9. View logs: Use tools like Elasticsearch or Kibana to check the collected K8s logs.

These are the general steps for collecting K8s logs using Filebeat. You can configure and adjust based on your specific requirements.