{"id":46990,"date":"2023-07-08T09:53:49","date_gmt":"2023-12-08T08:59:11","guid":{"rendered":"https:\/\/www.silicloud.com\/zh\/blog\/%e4%bd%bf%e7%94%a8oracle-cloud%e4%b8%ad%e7%9a%84fluentd%ef%bc%8c%e5%b0%9d%e8%af%95%e5%b0%86%e6%95%b0%e6%8d%ae%e4%bb%8estreaming%e4%bc%a0%e8%be%93%e5%88%b0hdfs%e8%bf%9b%e8%a1%8c%e5%8d%8f%e4%bd%9c\/"},"modified":"2024-04-30T12:35:38","modified_gmt":"2024-04-30T04:35:38","slug":"%e4%bd%bf%e7%94%a8oracle-cloud%e4%b8%ad%e7%9a%84fluentd%ef%bc%8c%e5%b0%9d%e8%af%95%e5%b0%86%e6%95%b0%e6%8d%ae%e4%bb%8estreaming%e4%bc%a0%e8%be%93%e5%88%b0hdfs%e8%bf%9b%e8%a1%8c%e5%8d%8f%e4%bd%9c","status":"publish","type":"post","link":"https:\/\/www.silicloud.com\/zh\/blog\/%e4%bd%bf%e7%94%a8oracle-cloud%e4%b8%ad%e7%9a%84fluentd%ef%bc%8c%e5%b0%9d%e8%af%95%e5%b0%86%e6%95%b0%e6%8d%ae%e4%bb%8estreaming%e4%bc%a0%e8%be%93%e5%88%b0hdfs%e8%bf%9b%e8%a1%8c%e5%8d%8f%e4%bd%9c\/","title":{"rendered":"\u4f7f\u7528Oracle Cloud\u4e2d\u7684Fluentd\uff0c\u5c1d\u8bd5\u5c06\u6570\u636e\u4eceStreaming\u4f20\u8f93\u5230HDFS\u8fdb\u884c\u534f\u4f5c"},"content":{"rendered":"<h1>\u9996\u5148<\/h1>\n<p>\u5728Oracle Cloud Infrastructure\uff08OCI\uff09\u4e2d\uff0c\u63d0\u4f9b\u4e86\u4e00\u9879\u540d\u4e3aStreaming\u7684\u670d\u52a1\uff0c\u53ef\u4ee5\u5b9e\u65f6\u6536\u96c6\u548c\u5904\u7406\u6d41\u6570\u636e\u3002Streaming\u5177\u6709\u4e0eApache Kafka\u517c\u5bb9\u7684API\uff0c\u56e0\u6b64\u53ef\u4ee5\u4eceKafka Client\u8fde\u63a5\u5e76\u8fdb\u884c\u6570\u636e\u7684\u751f\u4ea7\u548c\u6d88\u8d39\u3002<\/p>\n<p>\u6211\u5011\u5c07\u4f7f\u7528Fluentd\u4f86\u78ba\u8a8d\u5c07\u6d41\u52d5\u7684\u6578\u64da\u5b58\u5132\u5230Hadoop\u7684HDFS\u7684\u65b9\u6cd5\u3002<\/p>\n<h1>\u524d\u63d0\u662f\u8981\u6709\u4e00\u4e2a\u6761\u4ef6<\/h1>\n<p>\u524d\u63d0\u6761\u4ef6\u662f\u57fa\u4e8e\u672c\u6587\u4e2d\u6784\u5efa\u7684Hadoop\/HDFS\u3002<\/p>\n<h1>HDFS\u914d\u7f6e<\/h1>\n<p>\u4f7f\u7528Fluentd\u7684fluent-plugin-webhdfs\u5c06\u6570\u636e\u5b58\u50a8\u5230HDFS\u3002\u8be5\u63d2\u4ef6\u4f7f\u7528WebHDFS\u63a5\u53e3\u6765\u64cd\u4f5cHDFS\u7684REST API\u3002\u7531\u4e8eWebHDFS\u9ed8\u8ba4\u5904\u4e8e\u7981\u7528\u72b6\u6001\uff0c\u56e0\u6b64\u9700\u8981\u66f4\u6539Hadoop\/HDFS\u7684\u914d\u7f6e\u4ee5\u542f\u7528\u5b83\u3002<\/p>\n<p>\u7f16\u8f91hdfs-site.xml\u3002<\/p>\n<pre class=\"post-pre\"><code>vim $HADOOP_HOME\/etc\/hadoop\/hdfs-site.xml\r\n<\/code><\/pre>\n<p>\u6dfb\u52a0\u5230\u914d\u7f6e\u4e2d\u3002<\/p>\n<pre class=\"post-pre\"><code>&lt;property&gt;\r\n  &lt;name&gt;dfs.webhdfs.enabled&lt;\/name&gt;\r\n  &lt;value&gt;true&lt;\/value&gt;\r\n&lt;\/property&gt;\r\n\r\n&lt;property&gt;\r\n  &lt;name&gt;dfs.support.append&lt;\/name&gt;\r\n  &lt;value&gt;true&lt;\/value&gt;\r\n&lt;\/property&gt;\r\n\r\n&lt;property&gt;\r\n  &lt;name&gt;dfs.support.broken.append&lt;\/name&gt;\r\n  &lt;value&gt;true&lt;\/value&gt;\r\n&lt;\/property&gt;\r\n<\/code><\/pre>\n<p>\u4fee\u6539\u540e\u7684\u6587\u4ef6\u6574\u4f53\u611f\u89c9\u5982\u4e0b\u6240\u793a\u3002<\/p>\n<pre class=\"post-pre\"><code>[opc@hivenew hadoop-3.2.1]$ cat $HADOOP_HOME\/etc\/hadoop\/hdfs-site.xml\r\n&lt;?xml version=\"1.0\" encoding=\"UTF-8\"?&gt;\r\n&lt;?xml-stylesheet type=\"text\/xsl\" href=\"configuration.xsl\"?&gt;\r\n&lt;!--\r\n  Licensed under the Apache License, Version 2.0 (the \"License\");\r\n  you may not use this file except in compliance with the License.\r\n  You may obtain a copy of the License at\r\n\r\n    http:\/\/www.apache.org\/licenses\/LICENSE-2.0\r\n\r\n  Unless required by applicable law or agreed to in writing, software\r\n  distributed under the License is distributed on an \"AS IS\" BASIS,\r\n  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\n  See the License for the specific language governing permissions and\r\n  limitations under the License. See accompanying LICENSE file.\r\n--&gt;\r\n\r\n&lt;!-- Put site-specific property overrides in this file. --&gt;\r\n\r\n&lt;configuration&gt;\r\n  &lt;property&gt;\r\n    &lt;name&gt;dfs.name.dir&lt;\/name&gt;\r\n    &lt;value&gt;\/home\/opc\/hdfs\/name&lt;\/value&gt;\r\n  &lt;\/property&gt;\r\n  &lt;property&gt;\r\n    &lt;name&gt;dfs.data.dir&lt;\/name&gt;\r\n    &lt;value&gt;\/home\/opc\/hdfs\/data&lt;\/value&gt;\r\n  &lt;\/property&gt;\r\n  &lt;property&gt;\r\n    &lt;name&gt;dfs.replication&lt;\/name&gt;\r\n    &lt;value&gt;1&lt;\/value&gt;\r\n  &lt;\/property&gt;\r\n  &lt;property&gt;\r\n    &lt;name&gt;dfs.webhdfs.enabled&lt;\/name&gt;\r\n    &lt;value&gt;true&lt;\/value&gt;\r\n  &lt;\/property&gt;\r\n  &lt;property&gt;\r\n    &lt;name&gt;dfs.support.append&lt;\/name&gt;\r\n    &lt;value&gt;true&lt;\/value&gt;\r\n  &lt;\/property&gt;\r\n  &lt;property&gt;\r\n    &lt;name&gt;dfs.support.broken.append&lt;\/name&gt;\r\n    &lt;value&gt;true&lt;\/value&gt;\r\n  &lt;\/property&gt;\r\n&lt;\/configuration&gt;\r\n[opc@hivenew hadoop-3.2.1]$\r\n<\/code><\/pre>\n<p>\u542f\u52a8 Hadoop \/ HDFS\u3002<\/p>\n<pre class=\"post-pre\"><code>cd $HADOOP_HOME\r\nsbin\/start-all.sh\r\n<\/code><\/pre>\n<p>\u4f7f\u7528jps\u547d\u4ee4\u6765\u786e\u8ba4\u662f\u5426\u5df2\u542f\u52a8\u3002<\/p>\n<pre class=\"post-pre\"><code>[opc@hivenew hadoop-3.2.1]$ jps\r\n3057 NameNode\r\n4483 NodeManager\r\n4805 Jps\r\n4358 ResourceManager\r\n3210 DataNode\r\n3406 SecondaryNameNode\r\n<\/code><\/pre>\n<p>\u4e3a\u4e86\u786e\u8ba4\u64cd\u4f5c\uff0c\u5c06\u5c1d\u8bd5\u901a\u8fc7\u6d4f\u89c8\u5668\u8fde\u63a5\u5230NameNode\u6765\u786e\u8ba4\u662f\u5426\u53ef\u4ee5\u8fde\u63a5\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d7bf5913a08637a69adbe\/19-0.png\" alt=\"1589592361070.png\" \/><\/div>\n<h1>\u751f\u6210\u6388\u6743\u4ee4\u724c<\/h1>\n<p>\u8981\u5728Streaming\u4e0a\u4f7f\u7528Kafka API\u8fdb\u884c\u8fde\u63a5\uff0c\u9700\u8981\u4f7f\u7528IAM\u7528\u6237\u7684\u8eab\u4efd\u8ba4\u8bc1\u4ee4\u724c\u3002\u60a8\u53ef\u4ee5\u5728\u81ea\u5df1\u7684IAM\u7528\u6237\u8be6\u7ec6\u9875\u9762\u4e0a\u751f\u6210\u8be5\u4ee4\u724c\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d7bf5913a08637a69adbe\/22-0.2%26auto%3Dformat%26gif-q%3D60%26q%3D75%26s%3D440eb7a39c5e4d75b72a2412070b21ec\" alt=\"1588474181611.png\" \/><\/div>\n<p>\u6211\u4f1a\u968f\u610f\u5730\u8fdb\u884c\u89e3\u91ca\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d7bf5913a08637a69adbe\/24-0.2%26auto%3Dformat%26gif-q%3D60%26q%3D75%26s%3D89faeebd672917820612f9622b01c892\" alt=\"1588474211704.png\" \/><\/div>\n<p>\u7531\u4e8e\u65e0\u6cd5\u4f7f\u7528\u56fe\u50cf\uff0c\u6240\u4ee5\u6211\u4f1a\u8bb0\u4e0bToken\u7684\u4fe1\u606f\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d7bf5913a08637a69adbe\/26-0.2%26auto%3Dformat%26gif-q%3D60%26q%3D75%26s%3Dce7a9621b385f852e537d5b5dc64abff\" alt=\"1588474230416.png\" \/><\/div>\n<h1>\u521b\u5efa\u6d41 li\u00fa)<\/h1>\n<p>\u5728 OCI \u63a7\u5236\u53f0\u4e2d\uff0c\u9009\u62e9 Analytics &gt; Streaming\u3002\u7136\u540e\u9009\u62e9\u521b\u5efa\u6d41\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d7bf5913a08637a69adbe\/29-0.2%26auto%3Dformat%26gif-q%3D60%26q%3D75%26s%3Db84426187c4ac475c6b065c78c541a88\" alt=\"1588519801919.png\" \/><\/div>\n<p>\u4f7f\u7528\u9002\u5f53\u7684\u53c2\u6570\u8fdb\u884c\u521b\u5efa\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d7bf5913a08637a69adbe\/31-0.2%26auto%3Dformat%26gif-q%3D60%26q%3D75%26s%3D40aabbfc6e00eb15e6fb25ea7af419f0\" alt=\"1588519908911.png\" \/><\/div>\n<p>\u5728\u521b\u5efaStream\u65f6\uff0c\u5982\u679cStream Pool\u4e3a\u7a7a\uff0c\u5219\u9ed8\u8ba4\u7684Stream Pool\u5c06\u88ab\u81ea\u52a8\u521b\u5efa\u3002teststream01\u5c06\u88ab\u81ea\u52a8\u5206\u914d\u7ed9DefaultPool\u3002Stream Pool\u662f\u4e00\u79cd\u7528\u4e8e\u7ba1\u7406\u591a\u4e2aStream\u7684\u6982\u5ff5\uff0c\u53ef\u4ee5\u96c6\u4e2d\u7ba1\u7406Stream\u7684Endpoint\u662f\u516c\u5f00\u8fd8\u662f\u79c1\u6709\uff0c\u5e76\u51b3\u5b9a\u4f7f\u7528\u54ea\u4e2a\u5bc6\u94a5\u8fdb\u884c\u6570\u636e\u52a0\u5bc6\u3002<\/p>\n<p>\u8fd9\u6b21\u6211\u4eec\u9009\u62e9\u4e86\u81ea\u52a8\u521b\u5efa\uff0c\u6240\u4ee5 Endpoint \u662f\u516c\u5f00\u7684\uff0c\u5e76\u4e14\u4f7f\u7528\u7531Oracle\u7ba1\u7406\u7684\u52a0\u5bc6\u5bc6\u94a5\u8fdb\u884c\u8bbe\u7f6e\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d7bf5913a08637a69adbe\/34-0.2%26auto%3Dformat%26gif-q%3D60%26q%3D75%26s%3Dfd23906affa3a22a3c3b4577674e6b5b\" alt=\"1588520017422.png\" \/><\/div>\n<p>\u70b9\u51fb\u4e0a\u8ff0\u753b\u9762\u4e2d\u7684\u300c\u67e5\u770bKafka\u8fde\u63a5\u8bbe\u7f6e\u300d\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d7bf5913a08637a69adbe\/36-0.2%26auto%3Dformat%26gif-q%3D60%26q%3D75%26s%3Dd49a23af436f278045c15d5ae48fff7b\" alt=\"1588521024252.png\" \/><\/div>\n<p>\u70b9\u51fb\u201c\u590d\u5236\u5168\u90e8\u201d\uff0c\u5c06\u6240\u6709\u7684\u8bbe\u7f6e\u503c\u8bb0\u4e0b\u6765\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d7bf5913a08637a69adbe\/38-0.2%26auto%3Dformat%26gif-q%3D60%26q%3D75%26s%3D8dae892c965b4ffade83295244d47fb8\" alt=\"1588521056591.png\" \/><\/div>\n<h1>\u5b89\u88c5Fluentd<\/h1>\n<p>\u8bf7\u8bbf\u95ee\u4ee5\u4e0b\u7f51\u5740\u4ee5\u8fdb\u884c\u9884\u5148\u8bbe\u7f6e\uff1a<br \/>\nhttps:\/\/docs.fluentd.org\/installation\/before-install<\/p>\n<p>\u5b89\u88c5Fluentd\u3002<\/p>\n<pre class=\"post-pre\"><code>$ curl -L https:\/\/toolbelt.treasuredata.com\/sh\/install-redhat-td-agent3.sh | sh\r\n<\/code><\/pre>\n<p>\u7531\u4e8e\u7cfb\u7edf\u4f7f\u7528Systemd\u8fdb\u884c\u7ba1\u7406\uff0c\u56e0\u6b64\u4f1a\u542f\u52a8\u3002<\/p>\n<pre class=\"post-pre\"><code>sudo systemctl start td-agent.service\r\nsudo systemctl enable td-agent.service\r\n<\/code><\/pre>\n<h1>Fluentd\u914d\u7f6e<\/h1>\n<p>\u5c06\u6570\u636e\u4f9d\u6b21\u6d41\u5411 Streaming(Kafka\u517c\u5bb9API) \u2192 Fluentd \u2192 WebHDFS\u3002<br \/>\n\u5728\u6e90\u5934\u4e2d\uff0c\u6307\u5b9a\u4e86 kafka_group\u3002\u4f7f\u7528\u4ece Stream Pool \u83b7\u53d6\u7684\u8fde\u63a5\u4fe1\u606f\u6765\u6307\u5b9a\u3002<br \/>\n\u5728\u5b58\u50a8\u4e2d\uff0c\u6307\u5b9a\u4e86 webhdfs\u3002<\/p>\n<p>\u5982\u679c\u60a8\u60f3\u8981\u67e5\u770b\u8be6\u7ec6\u7684\u53c2\u6570\uff0c\u8bf7\u8bbf\u95ee\u4ee5\u4e0b GitHub \u94fe\u63a5\uff1a<br \/>\nhttps:\/\/github.com\/fluent\/fluent-plugin-kafka<br \/>\nhttps:\/\/github.com\/fluent\/fluent-plugin-webhdfs\/<\/p>\n<pre class=\"post-pre\"><code>cat &lt;&lt;'EOF' &gt; \/etc\/td-agent\/td-agent.conf\r\n&lt;source&gt;\r\n  @type kafka_group\r\n  @label @STREAMING\r\n\r\n  brokers cell-1.streaming.ap-tokyo-1.oci.oraclecloud.com:9092\r\n  consumer_group sugigroup\r\n  topics teststream01\r\n  format text\r\n\r\n  # sasl\r\n  username poc02\/oracleidentitycloudservice\/suguru.sugiyama@oracle.com\/ocid1.streampool.oc1.ap-tokyo-1.amaaaaaaycetm7yawtz56lnnerap4r45y4vheekgvhdaevxf3clfpuew6mla\r\n  password 8t[shwUN}I-d+{}8Nx_a\r\n  sasl_over_ssl true\r\n  ssl_ca_certs_from_system true\r\n&lt;\/source&gt;\r\n\r\n&lt;label @STREAMING&gt;\r\n  &lt;match **&gt;\r\n    @type copy\r\n    &lt;store&gt;\r\n      @type stdout\r\n    &lt;\/store&gt;\r\n    &lt;store&gt;\r\n      @type webhdfs\r\n      host 10.0.0.12\r\n      port 9870\r\n      path \"\/log\/%Y%m%d_%H\/streaming.log.#{Socket.gethostname}\"\r\n      flush_interval 10s\r\n      username opc\r\n    &lt;\/store&gt;\r\n  &lt;\/match&gt;\r\n&lt;\/label&gt;\r\nEOF\r\n<\/code><\/pre>\n<p>\u91cd\u542fFluentd\u3002<\/p>\n<pre class=\"post-pre\"><code>systemctl restart td-agent\r\n<\/code><\/pre>\n<h1>\u786e\u8ba4\u52a8\u4f5c<\/h1>\n<p>\u73b0\u5728\uff0c\u6211\u4eec\u5df2\u7ecf\u5b8c\u6210\u4e86\u8bbe\u7f6e\uff0c\u8ba9\u6211\u4eec\u5728OCI Streaming\u7684\u7ba1\u7406\u754c\u9762\u4e0a\u8fdb\u884c\u5b9e\u9645\u64cd\u4f5c\u6765\u786e\u8ba4\u4e00\u4e0b\u3002\u60a8\u53ef\u4ee5\u5728Stream\u4e0a\u8fdb\u884cProduce\u64cd\u4f5c\uff0c\u6211\u4eec\u8bd5\u7740\u5b58\u50a8\u4e00\u4e2a\u5b57\u7b26\u4e32\u201chello webhdfs\uff01\u201d\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d7bf5913a08637a69adbe\/53-0.png\" alt=\"1589582508141.png\" \/><\/div>\n<p>\u5728\u6ca1\u6709\u65bd\u52a0\u4efb\u4f55\u8d1f\u8f7d\u7684\u65e0\u98ce\u72b6\u6001\u4e0b\uff0c\u6211\u4eec\u5728\u5b58\u50a8\u4e86\u6d4b\u8bd5\u6570\u636e\u4e4b\u540e\uff0c\u5728\u5927\u7ea630\u79d2\u5185\u5c31\u53ef\u4ee5\u5728HDFS\u4e0a\u786e\u8ba4\u6570\u636e\u3002\u5f53\u7136\uff0c\u5b58\u50a8\u6240\u9700\u7684\u65f6\u95f4\u4f1a\u56e0\u8d1f\u8f7d\u60c5\u51b5\u800c\u6709\u6240\u4e0d\u540c\u3002\u6839\u636e\u914d\u7f6e\u4e86Fluentd\u7684\u5185\u5bb9\uff0c\u6570\u636e\u88ab\u5b58\u50a8\u5982\u4e0b\uff1a<\/p>\n<p>\/\u8bb0\u5f55\/&lt;\u65e5\u671f&gt;\/\u6d41\u5a92\u4f53.log.&lt;\u4e3b\u673a\u540d&gt;<\/p>\n<pre class=\"post-pre\"><code>[opc@hivenew hadoop-3.2.1]$ bin\/hadoop fs -ls \/log\/20200516_00\r\nFound 1 items\r\n-rw-r--r--   1 opc supergroup       1325 2020-05-16 00:57 \/log\/20200516_00\/streaming.log.hivenew\r\n[opc@hivenew hadoop-3.2.1]$\r\n<\/code><\/pre>\n<p>\u5f53\u4f7f\u7528cat\u547d\u4ee4\u8fdb\u884c\u786e\u8ba4\u65f6\uff0c\u53ef\u4ee5\u770b\u5230\u672b\u5c3e\u51fa\u73b0\u4e86\u201chello webhdfs!\u201d\u8fd9\u6837\u7684\u6587\u5b57\u3002\u7ecf\u8fc7\u4f20\u8f93\u7684\u6570\u636e\u5df2\u7ecf\u81ea\u52a8\u5b58\u50a8\u5230\u4e86HDFS\u4e2d\uff0c\u8fd9\u4e00\u70b9\u5df2\u7ecf\u5f97\u5230\u4e86\u786e\u8ba4\u3002<\/p>\n<pre class=\"post-pre\"><code>[opc@hivenew hadoop-3.2.1]$ bin\/hadoop fs -cat \/log\/20200516_00\/streaming.log.hivenew\r\n2020-05-16 00:57:30,350 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false\r\n2020-05-16T00:06:25Z    test.test       {\"I say\":\"hello1\"}\r\n2020-05-16T00:43:15Z    test.test       {\"I say\":\"hello1\"}\r\n2020-05-16T00:52:05Z    teststream01    {\"I say\":\"hello\"}\r\n2020-05-16T00:52:05Z    teststream01    {\"I say\":\"hello\"}\r\n2020-05-16T00:52:05Z    teststream01    {\"I say\":\"hello\"}\r\n2020-05-16T00:52:05Z    teststream01    {\"I say\":\"hello\"}\r\n2020-05-16T00:52:05Z    teststream01    {\"I say\":\"hello1\"}\r\n2020-05-16T00:52:05Z    teststream01    {\"I say\":\"hello1\"}\r\n2020-05-16T00:52:05Z    teststream01    {\"I say\":\"hello1\"}\r\n2020-05-16T00:52:05Z    teststream01    {\"I say\":\"hello\"}\r\n2020-05-16T00:52:05Z    teststream01    {\"I say\":\"hello\"}\r\n2020-05-16T00:52:05Z    teststream01    {\"I say\":\"hello1\"}\r\n2020-05-16T00:52:05Z    teststream01    {\"key\":\"a\",\"value\":\"a\"}\r\n2020-05-16T00:56:16Z    teststream01    {\"message\":\"message=1\"}\r\n2020-05-16T00:56:16Z    teststream01    {\"message\":\"message=3\"}\r\n2020-05-16T00:56:16Z    teststream01    {\"message\":\"message=5\"}\r\n2020-05-16T00:56:16Z    teststream01    {\"message\":\"message=8\"}\r\n2020-05-16T00:56:16Z    teststream01    {\"message\":\"message=4\"}\r\n2020-05-16T00:56:16Z    teststream01    {\"message\":\"message=7\"}\r\n2020-05-16T00:56:18Z    teststream01    {\"message\":\"message=0\"}\r\n2020-05-16T00:56:18Z    teststream01    {\"message\":\"message=2\"}\r\n2020-05-16T00:56:18Z    teststream01    {\"message\":\"message=6\"}\r\n2020-05-16T00:56:18Z    teststream01    {\"message\":\"message=9\"}\r\n2020-05-16T00:57:14Z    teststream01    {\"message\":\"hello webhdfs!\"}\r\n<\/code><\/pre>\n<h1>\u8bf7\u63d0\u4f9b\u76f8\u5173\u94fe\u63a5<\/h1>\n","protected":false},"excerpt":{"rendered":"<p>\u9996\u5148 \u5728Oracle Cloud Infrastructure\uff08OCI\uff09\u4e2d\uff0c\u63d0\u4f9b\u4e86\u4e00\u9879\u540d\u4e3aStreaming\u7684 [&hellip;]<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-46990","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.5 (Yoast SEO v21.5) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>\u4f7f\u7528Oracle Cloud\u4e2d\u7684Fluentd\uff0c\u5c1d\u8bd5\u5c06\u6570\u636e\u4eceStreaming\u4f20\u8f93\u5230HDFS\u8fdb\u884c\u534f\u4f5c - Blog - Silicon Cloud<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.silicloud.com\/zh\/blog\/\u4f7f\u7528oracle-cloud\u4e2d\u7684fluentd\uff0c\u5c1d\u8bd5\u5c06\u6570\u636e\u4ecestreaming\u4f20\u8f93\u5230hdfs\u8fdb\u884c\u534f\u4f5c\/\" \/>\n<meta property=\"og:locale\" content=\"zh_CN\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"\u4f7f\u7528Oracle Cloud\u4e2d\u7684Fluentd\uff0c\u5c1d\u8bd5\u5c06\u6570\u636e\u4eceStreaming\u4f20\u8f93\u5230HDFS\u8fdb\u884c\u534f\u4f5c\" \/>\n<meta property=\"og:description\" content=\"\u9996\u5148 \u5728Oracle Cloud Infrastructure\uff08OCI\uff09\u4e2d\uff0c\u63d0\u4f9b\u4e86\u4e00\u9879\u540d\u4e3aStreaming\u7684 [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.silicloud.com\/zh\/blog\/\u4f7f\u7528oracle-cloud\u4e2d\u7684fluentd\uff0c\u5c1d\u8bd5\u5c06\u6570\u636e\u4ecestreaming\u4f20\u8f93\u5230hdfs\u8fdb\u884c\u534f\u4f5c\/\" \/>\n<meta property=\"og:site_name\" content=\"Blog - Silicon Cloud\" \/>\n<meta property=\"article:published_time\" content=\"2023-12-08T08:59:11+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-04-30T04:35:38+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d7bf5913a08637a69adbe\/19-0.png\" \/>\n<meta name=\"author\" content=\"\u79d1, \u96c5\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"\u4f5c\u8005\" \/>\n\t<meta name=\"twitter:data1\" content=\"\u79d1, \u96c5\" \/>\n\t<meta name=\"twitter:label2\" content=\"\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 \u5206\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/%e4%bd%bf%e7%94%a8oracle-cloud%e4%b8%ad%e7%9a%84fluentd%ef%bc%8c%e5%b0%9d%e8%af%95%e5%b0%86%e6%95%b0%e6%8d%ae%e4%bb%8estreaming%e4%bc%a0%e8%be%93%e5%88%b0hdfs%e8%bf%9b%e8%a1%8c%e5%8d%8f%e4%bd%9c\/\",\"url\":\"https:\/\/www.silicloud.com\/zh\/blog\/%e4%bd%bf%e7%94%a8oracle-cloud%e4%b8%ad%e7%9a%84fluentd%ef%bc%8c%e5%b0%9d%e8%af%95%e5%b0%86%e6%95%b0%e6%8d%ae%e4%bb%8estreaming%e4%bc%a0%e8%be%93%e5%88%b0hdfs%e8%bf%9b%e8%a1%8c%e5%8d%8f%e4%bd%9c\/\",\"name\":\"\u4f7f\u7528Oracle Cloud\u4e2d\u7684Fluentd\uff0c\u5c1d\u8bd5\u5c06\u6570\u636e\u4eceStreaming\u4f20\u8f93\u5230HDFS\u8fdb\u884c\u534f\u4f5c - Blog - Silicon Cloud\",\"isPartOf\":{\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#website\"},\"datePublished\":\"2023-12-08T08:59:11+00:00\",\"dateModified\":\"2024-04-30T04:35:38+00:00\",\"author\":{\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/41e222757cdd2a3365361328bd79970a\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/%e4%bd%bf%e7%94%a8oracle-cloud%e4%b8%ad%e7%9a%84fluentd%ef%bc%8c%e5%b0%9d%e8%af%95%e5%b0%86%e6%95%b0%e6%8d%ae%e4%bb%8estreaming%e4%bc%a0%e8%be%93%e5%88%b0hdfs%e8%bf%9b%e8%a1%8c%e5%8d%8f%e4%bd%9c\/#breadcrumb\"},\"inLanguage\":\"zh-Hans\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.silicloud.com\/zh\/blog\/%e4%bd%bf%e7%94%a8oracle-cloud%e4%b8%ad%e7%9a%84fluentd%ef%bc%8c%e5%b0%9d%e8%af%95%e5%b0%86%e6%95%b0%e6%8d%ae%e4%bb%8estreaming%e4%bc%a0%e8%be%93%e5%88%b0hdfs%e8%bf%9b%e8%a1%8c%e5%8d%8f%e4%bd%9c\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/%e4%bd%bf%e7%94%a8oracle-cloud%e4%b8%ad%e7%9a%84fluentd%ef%bc%8c%e5%b0%9d%e8%af%95%e5%b0%86%e6%95%b0%e6%8d%ae%e4%bb%8estreaming%e4%bc%a0%e8%be%93%e5%88%b0hdfs%e8%bf%9b%e8%a1%8c%e5%8d%8f%e4%bd%9c\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"\u9996\u9875\",\"item\":\"https:\/\/www.silicloud.com\/zh\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"\u4f7f\u7528Oracle Cloud\u4e2d\u7684Fluentd\uff0c\u5c1d\u8bd5\u5c06\u6570\u636e\u4eceStreaming\u4f20\u8f93\u5230HDFS\u8fdb\u884c\u534f\u4f5c\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#website\",\"url\":\"https:\/\/www.silicloud.com\/zh\/blog\/\",\"name\":\"Blog - Silicon Cloud\",\"description\":\"\",\"inLanguage\":\"zh-Hans\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/41e222757cdd2a3365361328bd79970a\",\"name\":\"\u79d1, \u96c5\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"zh-Hans\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/1b2d3e00a7df03689797ebd4af8c5827ba5af936849a71050ec331f4cf902c5d?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/1b2d3e00a7df03689797ebd4af8c5827ba5af936849a71050ec331f4cf902c5d?s=96&d=mm&r=g\",\"caption\":\"\u79d1, \u96c5\"},\"url\":\"https:\/\/www.silicloud.com\/zh\/blog\/author\/keya\/\"},{\"@type\":\"ImageObject\",\"inLanguage\":\"zh-Hans\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/%e4%bd%bf%e7%94%a8oracle-cloud%e4%b8%ad%e7%9a%84fluentd%ef%bc%8c%e5%b0%9d%e8%af%95%e5%b0%86%e6%95%b0%e6%8d%ae%e4%bb%8estreaming%e4%bc%a0%e8%be%93%e5%88%b0hdfs%e8%bf%9b%e8%a1%8c%e5%8d%8f%e4%bd%9c\/#local-main-organization-logo\",\"url\":\"\",\"contentUrl\":\"\",\"caption\":\"Blog - Silicon Cloud\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"\u4f7f\u7528Oracle Cloud\u4e2d\u7684Fluentd\uff0c\u5c1d\u8bd5\u5c06\u6570\u636e\u4eceStreaming\u4f20\u8f93\u5230HDFS\u8fdb\u884c\u534f\u4f5c - Blog - Silicon Cloud","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.silicloud.com\/zh\/blog\/\u4f7f\u7528oracle-cloud\u4e2d\u7684fluentd\uff0c\u5c1d\u8bd5\u5c06\u6570\u636e\u4ecestreaming\u4f20\u8f93\u5230hdfs\u8fdb\u884c\u534f\u4f5c\/","og_locale":"zh_CN","og_type":"article","og_title":"\u4f7f\u7528Oracle Cloud\u4e2d\u7684Fluentd\uff0c\u5c1d\u8bd5\u5c06\u6570\u636e\u4eceStreaming\u4f20\u8f93\u5230HDFS\u8fdb\u884c\u534f\u4f5c","og_description":"\u9996\u5148 \u5728Oracle Cloud Infrastructure\uff08OCI\uff09\u4e2d\uff0c\u63d0\u4f9b\u4e86\u4e00\u9879\u540d\u4e3aStreaming\u7684 [&hellip;]","og_url":"https:\/\/www.silicloud.com\/zh\/blog\/\u4f7f\u7528oracle-cloud\u4e2d\u7684fluentd\uff0c\u5c1d\u8bd5\u5c06\u6570\u636e\u4ecestreaming\u4f20\u8f93\u5230hdfs\u8fdb\u884c\u534f\u4f5c\/","og_site_name":"Blog - Silicon Cloud","article_published_time":"2023-12-08T08:59:11+00:00","article_modified_time":"2024-04-30T04:35:38+00:00","og_image":[{"url":"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d7bf5913a08637a69adbe\/19-0.png"}],"author":"\u79d1, \u96c5","twitter_card":"summary_large_image","twitter_misc":{"\u4f5c\u8005":"\u79d1, \u96c5","\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4":"4 \u5206"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.silicloud.com\/zh\/blog\/%e4%bd%bf%e7%94%a8oracle-cloud%e4%b8%ad%e7%9a%84fluentd%ef%bc%8c%e5%b0%9d%e8%af%95%e5%b0%86%e6%95%b0%e6%8d%ae%e4%bb%8estreaming%e4%bc%a0%e8%be%93%e5%88%b0hdfs%e8%bf%9b%e8%a1%8c%e5%8d%8f%e4%bd%9c\/","url":"https:\/\/www.silicloud.com\/zh\/blog\/%e4%bd%bf%e7%94%a8oracle-cloud%e4%b8%ad%e7%9a%84fluentd%ef%bc%8c%e5%b0%9d%e8%af%95%e5%b0%86%e6%95%b0%e6%8d%ae%e4%bb%8estreaming%e4%bc%a0%e8%be%93%e5%88%b0hdfs%e8%bf%9b%e8%a1%8c%e5%8d%8f%e4%bd%9c\/","name":"\u4f7f\u7528Oracle Cloud\u4e2d\u7684Fluentd\uff0c\u5c1d\u8bd5\u5c06\u6570\u636e\u4eceStreaming\u4f20\u8f93\u5230HDFS\u8fdb\u884c\u534f\u4f5c - Blog - Silicon Cloud","isPartOf":{"@id":"https:\/\/www.silicloud.com\/zh\/blog\/#website"},"datePublished":"2023-12-08T08:59:11+00:00","dateModified":"2024-04-30T04:35:38+00:00","author":{"@id":"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/41e222757cdd2a3365361328bd79970a"},"breadcrumb":{"@id":"https:\/\/www.silicloud.com\/zh\/blog\/%e4%bd%bf%e7%94%a8oracle-cloud%e4%b8%ad%e7%9a%84fluentd%ef%bc%8c%e5%b0%9d%e8%af%95%e5%b0%86%e6%95%b0%e6%8d%ae%e4%bb%8estreaming%e4%bc%a0%e8%be%93%e5%88%b0hdfs%e8%bf%9b%e8%a1%8c%e5%8d%8f%e4%bd%9c\/#breadcrumb"},"inLanguage":"zh-Hans","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.silicloud.com\/zh\/blog\/%e4%bd%bf%e7%94%a8oracle-cloud%e4%b8%ad%e7%9a%84fluentd%ef%bc%8c%e5%b0%9d%e8%af%95%e5%b0%86%e6%95%b0%e6%8d%ae%e4%bb%8estreaming%e4%bc%a0%e8%be%93%e5%88%b0hdfs%e8%bf%9b%e8%a1%8c%e5%8d%8f%e4%bd%9c\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.silicloud.com\/zh\/blog\/%e4%bd%bf%e7%94%a8oracle-cloud%e4%b8%ad%e7%9a%84fluentd%ef%bc%8c%e5%b0%9d%e8%af%95%e5%b0%86%e6%95%b0%e6%8d%ae%e4%bb%8estreaming%e4%bc%a0%e8%be%93%e5%88%b0hdfs%e8%bf%9b%e8%a1%8c%e5%8d%8f%e4%bd%9c\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"\u9996\u9875","item":"https:\/\/www.silicloud.com\/zh\/blog\/"},{"@type":"ListItem","position":2,"name":"\u4f7f\u7528Oracle Cloud\u4e2d\u7684Fluentd\uff0c\u5c1d\u8bd5\u5c06\u6570\u636e\u4eceStreaming\u4f20\u8f93\u5230HDFS\u8fdb\u884c\u534f\u4f5c"}]},{"@type":"WebSite","@id":"https:\/\/www.silicloud.com\/zh\/blog\/#website","url":"https:\/\/www.silicloud.com\/zh\/blog\/","name":"Blog - Silicon Cloud","description":"","inLanguage":"zh-Hans"},{"@type":"Person","@id":"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/41e222757cdd2a3365361328bd79970a","name":"\u79d1, \u96c5","image":{"@type":"ImageObject","inLanguage":"zh-Hans","@id":"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/1b2d3e00a7df03689797ebd4af8c5827ba5af936849a71050ec331f4cf902c5d?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1b2d3e00a7df03689797ebd4af8c5827ba5af936849a71050ec331f4cf902c5d?s=96&d=mm&r=g","caption":"\u79d1, \u96c5"},"url":"https:\/\/www.silicloud.com\/zh\/blog\/author\/keya\/"},{"@type":"ImageObject","inLanguage":"zh-Hans","@id":"https:\/\/www.silicloud.com\/zh\/blog\/%e4%bd%bf%e7%94%a8oracle-cloud%e4%b8%ad%e7%9a%84fluentd%ef%bc%8c%e5%b0%9d%e8%af%95%e5%b0%86%e6%95%b0%e6%8d%ae%e4%bb%8estreaming%e4%bc%a0%e8%be%93%e5%88%b0hdfs%e8%bf%9b%e8%a1%8c%e5%8d%8f%e4%bd%9c\/#local-main-organization-logo","url":"","contentUrl":"","caption":"Blog - Silicon Cloud"}]}},"_links":{"self":[{"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/posts\/46990","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/comments?post=46990"}],"version-history":[{"count":2,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/posts\/46990\/revisions"}],"predecessor-version":[{"id":91944,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/posts\/46990\/revisions\/91944"}],"wp:attachment":[{"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/media?parent=46990"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/categories?post=46990"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/tags?post=46990"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}