{"id":18594,"date":"2024-03-15T17:16:22","date_gmt":"2024-03-15T17:16:22","guid":{"rendered":"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/"},"modified":"2024-03-21T13:40:55","modified_gmt":"2024-03-21T13:40:55","slug":"how-does-filebeat-collect-logs-from-kubernetes","status":"publish","type":"post","link":"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/","title":{"rendered":"How does Filebeat collect logs from Kubernetes?"},"content":{"rendered":"<p>In order to collect Kubernetes logs using Filebeat, you need to configure Filebeat to monitor Kubernetes log files or container logs.<\/p>\n<p>Here is an example of a Filebeat configuration file:<\/p>\n<pre class=\"post-pre\"><code>filebeat.inputs:\r\n- type: container\r\n  paths:\r\n    - \/var\/log\/containers\/*.log\r\n  processors:\r\n    - add_kubernetes_metadata:\r\n        in_cluster: true\r\n        matchers:\r\n          - logs_path:\r\n              logs_path: \"\/var\/log\/containers\/\"\r\n\r\noutput.elasticsearch:\r\n  hosts: [\"your_elasticsearch_host:9200\"]\r\n<\/code><\/pre>\n<p>In the above configuration, the type is set to container and specifies the log path to monitor (\/var\/log\/containers\/*.log). The processors section uses the add_kubernetes_metadata processor to add Kubernetes metadata to the log events, helping to differentiate between different container logs.<\/p>\n<p>The output.elasticsearch section specifies the host address of Elasticsearch.<\/p>\n<p>After the configuration is complete, launch Filebeat and begin monitoring the log files of Kubernetes. Filebeat will collect the logs and send them to Elasticsearch.<\/p>\n<p>Please note that the above examples are based on the assumption that Filebeat is already installed in the Kubernetes cluster. If Filebeat is not in the cluster, you will need to specify the Kubernetes API address so that Filebeat can retrieve metadata about containers and Pods. This can be achieved by setting add_kubernetes_metadata.in_cluster to false in the processors section and specifying add_kubernetes_metadata.host as the Kubernetes API address.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In order to collect Kubernetes logs using Filebeat, you need to configure Filebeat to monitor Kubernetes log files or container logs. Here is an example of a Filebeat configuration file: filebeat.inputs: &#8211; type: container paths: &#8211; \/var\/log\/containers\/*.log processors: &#8211; add_kubernetes_metadata: in_cluster: true matchers: &#8211; logs_path: logs_path: &#8220;\/var\/log\/containers\/&#8221; output.elasticsearch: hosts: [&#8220;your_elasticsearch_host:9200&#8221;] In the above configuration, the [&hellip;]<\/p>\n","protected":false},"author":11,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_import_markdown_pro_load_document_selector":0,"_import_markdown_pro_submit_text_textarea":"","footnotes":""},"categories":[1],"tags":[],"class_list":["post-18594","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.5 (Yoast SEO v21.5) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>How does Filebeat collect logs from Kubernetes? - Blog - Silicon Cloud<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How does Filebeat collect logs from Kubernetes?\" \/>\n<meta property=\"og:description\" content=\"In order to collect Kubernetes logs using Filebeat, you need to configure Filebeat to monitor Kubernetes log files or container logs. Here is an example of a Filebeat configuration file: filebeat.inputs: - type: container paths: - \/var\/log\/containers\/*.log processors: - add_kubernetes_metadata: in_cluster: true matchers: - logs_path: logs_path: &quot;\/var\/log\/containers\/&quot; output.elasticsearch: hosts: [&quot;your_elasticsearch_host:9200&quot;] In the above configuration, the [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/\" \/>\n<meta property=\"og:site_name\" content=\"Blog - Silicon Cloud\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/SiliCloudGlobal\/\" \/>\n<meta property=\"article:published_time\" content=\"2024-03-15T17:16:22+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-03-21T13:40:55+00:00\" \/>\n<meta name=\"author\" content=\"Olivia Parker\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@SiliCloudGlobal\" \/>\n<meta name=\"twitter:site\" content=\"@SiliCloudGlobal\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Olivia Parker\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/\"},\"author\":{\"name\":\"Olivia Parker\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/3ff7b3da0e45ac5dbbef2502f3cea8d9\"},\"headline\":\"How does Filebeat collect logs from Kubernetes?\",\"datePublished\":\"2024-03-15T17:16:22+00:00\",\"dateModified\":\"2024-03-21T13:40:55+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/\"},\"wordCount\":191,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#organization\"},\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/\",\"url\":\"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/\",\"name\":\"How does Filebeat collect logs from Kubernetes? - Blog - Silicon Cloud\",\"isPartOf\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#website\"},\"datePublished\":\"2024-03-15T17:16:22+00:00\",\"dateModified\":\"2024-03-21T13:40:55+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.silicloud.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How does Filebeat collect logs from Kubernetes?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#website\",\"url\":\"https:\/\/www.silicloud.com\/blog\/\",\"name\":\"Silicon Cloud Blog\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#organization\"},\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#organization\",\"name\":\"Silicon Cloud Blog\",\"url\":\"https:\/\/www.silicloud.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png\",\"contentUrl\":\"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png\",\"width\":1024,\"height\":1024,\"caption\":\"Silicon Cloud Blog\"},\"image\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/SiliCloudGlobal\/\",\"https:\/\/twitter.com\/SiliCloudGlobal\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/3ff7b3da0e45ac5dbbef2502f3cea8d9\",\"name\":\"Olivia Parker\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/56c66f189ba32a6f9eb50f31a38fe774e2a725c213d4070835ccc51b8fbbc54b?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/56c66f189ba32a6f9eb50f31a38fe774e2a725c213d4070835ccc51b8fbbc54b?s=96&d=mm&r=g\",\"caption\":\"Olivia Parker\"},\"url\":\"https:\/\/www.silicloud.com\/blog\/author\/oliviaparker\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"How does Filebeat collect logs from Kubernetes? - Blog - Silicon Cloud","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/","og_locale":"en_US","og_type":"article","og_title":"How does Filebeat collect logs from Kubernetes?","og_description":"In order to collect Kubernetes logs using Filebeat, you need to configure Filebeat to monitor Kubernetes log files or container logs. Here is an example of a Filebeat configuration file: filebeat.inputs: - type: container paths: - \/var\/log\/containers\/*.log processors: - add_kubernetes_metadata: in_cluster: true matchers: - logs_path: logs_path: \"\/var\/log\/containers\/\" output.elasticsearch: hosts: [\"your_elasticsearch_host:9200\"] In the above configuration, the [&hellip;]","og_url":"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/","og_site_name":"Blog - Silicon Cloud","article_publisher":"https:\/\/www.facebook.com\/SiliCloudGlobal\/","article_published_time":"2024-03-15T17:16:22+00:00","article_modified_time":"2024-03-21T13:40:55+00:00","author":"Olivia Parker","twitter_card":"summary_large_image","twitter_creator":"@SiliCloudGlobal","twitter_site":"@SiliCloudGlobal","twitter_misc":{"Written by":"Olivia Parker","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/#article","isPartOf":{"@id":"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/"},"author":{"name":"Olivia Parker","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/3ff7b3da0e45ac5dbbef2502f3cea8d9"},"headline":"How does Filebeat collect logs from Kubernetes?","datePublished":"2024-03-15T17:16:22+00:00","dateModified":"2024-03-21T13:40:55+00:00","mainEntityOfPage":{"@id":"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/"},"wordCount":191,"commentCount":0,"publisher":{"@id":"https:\/\/www.silicloud.com\/blog\/#organization"},"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/","url":"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/","name":"How does Filebeat collect logs from Kubernetes? - Blog - Silicon Cloud","isPartOf":{"@id":"https:\/\/www.silicloud.com\/blog\/#website"},"datePublished":"2024-03-15T17:16:22+00:00","dateModified":"2024-03-21T13:40:55+00:00","breadcrumb":{"@id":"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.silicloud.com\/blog\/how-does-filebeat-collect-logs-from-kubernetes\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.silicloud.com\/blog\/"},{"@type":"ListItem","position":2,"name":"How does Filebeat collect logs from Kubernetes?"}]},{"@type":"WebSite","@id":"https:\/\/www.silicloud.com\/blog\/#website","url":"https:\/\/www.silicloud.com\/blog\/","name":"Silicon Cloud Blog","description":"","publisher":{"@id":"https:\/\/www.silicloud.com\/blog\/#organization"},"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.silicloud.com\/blog\/#organization","name":"Silicon Cloud Blog","url":"https:\/\/www.silicloud.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png","contentUrl":"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png","width":1024,"height":1024,"caption":"Silicon Cloud Blog"},"image":{"@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/SiliCloudGlobal\/","https:\/\/twitter.com\/SiliCloudGlobal"]},{"@type":"Person","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/3ff7b3da0e45ac5dbbef2502f3cea8d9","name":"Olivia Parker","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/56c66f189ba32a6f9eb50f31a38fe774e2a725c213d4070835ccc51b8fbbc54b?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/56c66f189ba32a6f9eb50f31a38fe774e2a725c213d4070835ccc51b8fbbc54b?s=96&d=mm&r=g","caption":"Olivia Parker"},"url":"https:\/\/www.silicloud.com\/blog\/author\/oliviaparker\/"}]}},"_links":{"self":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts\/18594","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/users\/11"}],"replies":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/comments?post=18594"}],"version-history":[{"count":1,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts\/18594\/revisions"}],"predecessor-version":[{"id":52285,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts\/18594\/revisions\/52285"}],"wp:attachment":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/media?parent=18594"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/categories?post=18594"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/tags?post=18594"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}