{"id":47200,"date":"2023-03-26T00:22:39","date_gmt":"2023-04-28T07:40:18","guid":{"rendered":"https:\/\/www.silicloud.com\/zh\/blog\/logstash%e7%9a%84kinesis%e8%be%93%e5%85%a5%e6%8f%92%e4%bb%b6%e5%92%8ckinesis%e8%be%93%e5%87%ba%e6%8f%92%e4%bb%b6\/"},"modified":"2024-04-29T20:32:49","modified_gmt":"2024-04-29T12:32:49","slug":"logstash%e7%9a%84kinesis%e8%be%93%e5%85%a5%e6%8f%92%e4%bb%b6%e5%92%8ckinesis%e8%be%93%e5%87%ba%e6%8f%92%e4%bb%b6","status":"publish","type":"post","link":"https:\/\/www.silicloud.com\/zh\/blog\/logstash%e7%9a%84kinesis%e8%be%93%e5%85%a5%e6%8f%92%e4%bb%b6%e5%92%8ckinesis%e8%be%93%e5%87%ba%e6%8f%92%e4%bb%b6\/","title":{"rendered":"Logstash\u7684Kinesis\u8f93\u5165\u63d2\u4ef6\u548cKinesis\u8f93\u51fa\u63d2\u4ef6"},"content":{"rendered":"<h2>\u9996\u5148<\/h2>\n<p>\u6211\u4f7f\u7528Elastic\u63d0\u4f9b\u7684ETL\u5de5\u5177Logstash\uff0c\u5c06\u65e5\u5fd7\u6570\u636e\u5bfc\u5165\u548c\u5bfc\u51fa\u5230Amazon Kinesis Data Streams\u8fdb\u884c\u4e86\u5c1d\u8bd5\u3002<\/p>\n<p>\u3010\u53c2\u8003\u3011<br \/>\n\u30fbAmazon Kinesis\u6570\u636e\u6d41\u662f\u4ec0\u4e48<br \/>\n\u30fbAWS kinesis\u6982\u8ff0<br \/>\n\u203b\u7b80\u5355\u6765\u8bf4\uff0c\u5b83\u662fAWS\u6258\u7ba1\u7684Apache Kafka\u670d\u52a1\u3002<\/p>\n<h2>\u5229\u7528\u73af\u5883\u548c\u6784\u6210\u56fe\u3002 (Use the environment and configuration diagram.)<\/h2>\n<div>\n<div class=\"post-table\">productversionFilebeat6.6.0Logstash6.6.0Elasticsearch6.6.0Kibana6.6.0OS(EC2)Amazon Linux2<\/div>\n<\/div>\n<p>\u203b\u6211\u4eec\u6b63\u5728\u4f7f\u7528\u6700\u65b0\u7248\u672c\u7684Elastic Stack 6.6.0\u3002<br \/>\n\u203b\u6211\u4eec\u6b63\u5728\u4f7f\u7528AWS\u7684\u5f17\u5409\u5c3c\u4e9a\uff08us-east-1\uff09\u533a\u57df\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d7d73913a08637a69fda0\/6-0.png\" alt=\"image.png\" \/><\/div>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">Logstash\u304b\u3089Kinesis Data Streams\u306b\u30c7\u30fc\u30bf\u3092\u7f6e\u304f\u3068\u304d\u306fkinesis output plugin\u3092\u5229\u7528\u3057\u307e\u3059\u3002<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">Logstash\u3067Kinesis Data Streams\u304b\u3089\u30c7\u30fc\u30bf\u3092\u53d6\u308a\u51fa\u3059\u3068\u304d\u306fkinesis input plugin\u3092\u5229\u7528\u3057\u307e\u3059\u3002<\/ul>\n<h2>\u524d\u63d0<\/h2>\n<p>\u5047\u8bbe\u4e0a\u8ff0\u73af\u5883\u5df2\u7ecf\u51c6\u5907\u5c31\u7eea\uff0c\u6211\u4eec\u7701\u7565\u4e86\u5b89\u88c5 Logstash_put-kinesis \u673a\u5668\u7684 Java\u3001Logstash \u548c Filebeat\uff0c\u4ee5\u53ca\u5b89\u88c5 Logstash_get-kinesis \u673a\u5668\u7684 Java\u3001Logstash\u3001Elasticsearch \u548c Kibana\u3002<\/p>\n<h2>Logstash Kinesis output\u63d2\u4ef6\u662f\u4ec0\u4e48\uff1f<\/h2>\n<p>\u4f60\u53ef\u4ee5\u5c06Logstash\u5bfc\u5165\u7684\u65e5\u5fd7\u6570\u636e\u8f93\u51fa\u5230Amazon Kinesis Data Streams\u4f5c\u4e3a\u76ee\u6807\u3002\u5b9e\u73b0\u4e0a\u8ff0\u529f\u80fd\u7684\u662fKinesis\u8f93\u51fa\u63d2\u4ef6\u3002\u8be5\u63d2\u4ef6\u5e76\u975e\u7531Elastic\u516c\u53f8\u5b98\u65b9\u63d0\u4f9b\uff0c\u800c\u662f\u7531\u793e\u533a\u63d2\u4ef6\u63d0\u4f9b\u3002<\/p>\n<div>\n<div class=\"post-table\">\u8a2d\u5b9a\u9805\u76ee\u30c7\u30d5\u30a9\u30eb\u30c8\u5024\u8aac\u660estream_name- (\u5fc5\u9808)kinesis\u30b9\u30c8\u30ea\u30fc\u30e0\u540d\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002regionus-east-1 (\u4efb\u610f)Kinesis Data Streams\u306eAWS\u30ea\u30fc\u30b8\u30e7\u30f3\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002metrics_levelcloudwatch (\u4efb\u610f)\u30e1\u30c8\u30ea\u30c3\u30af\u7d71\u8a08\u60c5\u5831\u306eCloudWatch\u3078\u306e\u9001\u4fe1\u6709\u7121\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002access_key- (\u4efb\u610f)Kinesis\u306b\u66f8\u304d\u8fbc\u307f\u6a29\u9650\u306e\u3042\u308bIAM\u30a2\u30ab\u30a6\u30f3\u30c8\u3067\u4f5c\u6210\u3057\u305f\u30a2\u30af\u30bb\u30b9\u30ad\u30fc\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002secret_key- (\u4efb\u610f)\u30a2\u30af\u30bb\u30b9\u30ad\u30fc\u3092\u5229\u7528\u3059\u308b\u5834\u5408\u306e\u5bfe\u306b\u306a\u308b\u30b7\u30fc\u30af\u30ec\u30c3\u30c8\u30ad\u30fc\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002metrics_access_key- (\u4efb\u610f)CloudWatch\u306b\u66f8\u304d\u8fbc\u307f\u6a29\u9650\u306e\u3042\u308bIAM\u30a2\u30ab\u30a6\u30f3\u30c8\u3067\u4f5c\u6210\u3057\u305f\u30a2\u30af\u30bb\u30b9\u30ad\u30fc\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002metrics_secret_key- (\u4efb\u610f)CloudWatch\u306e\u30a2\u30af\u30bb\u30b9\u30ad\u30fc\u3092\u5229\u7528\u3059\u308b\u5834\u5408\u306e\u5bfe\u306b\u306a\u308b\u30b7\u30fc\u30af\u30ec\u30c3\u30c8\u30ad\u30fc\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002role_arn- (\u4efb\u610f)AssumeRole\u3092\u5229\u7528\u3059\u308b\u5834\u5408\u306eARN\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002metrics_role_arn- (\u4efb\u610f)CloudWatch\u3078\u306e\u66f8\u304d\u8fbc\u307f\u3067AssumeRole\u3092\u5229\u7528\u3059\u308b\u5834\u5408\u306eARN\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002event_partition_keys- (\u4efb\u610f)\u5404\u30ec\u30b3\u30fc\u30c9\u306b\u633f\u5165\u3059\u308b\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u30ad\u30fc\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002randomized_partition_keyfalse (\u4efb\u610f)\u30c7\u30fc\u30bf\u9001\u4fe1\u9806\u5e8f\u3092\u7121\u8996\u3057\u305f\u30e9\u30f3\u30c0\u30e0\u306a\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u30ad\u30fc\u306e\u5229\u7528\u6709\u7121\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002aggregation_enabledtrue (\u4efb\u610f)Kinesis\u306b\u6295\u5165\u3059\u308b\u30ec\u30b3\u30fc\u30c9\u3092\u96c6\u7d04\u3057\u3066\u9001\u308b\u304b\u6307\u5b9a\u3057\u307e\u3059\u3002max_pending_records1000 (\u4efb\u610f)Kinesis\u9ad8\u8ca0\u8377\u6642\u306eLogstash\u5074\u3067\u306e\u30d0\u30c3\u30d5\u30a1\u30ea\u30f3\u30b0\u3059\u308b\u30ec\u30b3\u30fc\u30c9\u6570\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002<\/div>\n<\/div>\n<h2>Logstash Kinesis\u8f93\u5165\u63d2\u4ef6\u662f\u4ec0\u4e48\uff1f<\/h2>\n<p>Logstash\u53ef\u4ee5\u5c06Amazon Kinesis Data Streams\u6307\u5b9a\u4e3a\u65e5\u5fd7\u6570\u636e\u7684\u83b7\u53d6\u6e90\u3002<br \/>\n\u5b9e\u73b0\u4e0a\u8ff0\u529f\u80fd\u7684\u662fkinesis input\u63d2\u4ef6\u3002<br \/>\n\u8be5\u63d2\u4ef6\u662fElastic\u5b98\u65b9\u63d0\u4f9b\u7684\u63d2\u4ef6\uff0c\u4f46\u4e0d\u53d7\u5b98\u65b9\u652f\u6301\uff08\u00b4;\u03c9;\uff40\uff09\u3002<\/p>\n<div>\n<div class=\"post-table\">\u8a2d\u5b9a\u9805\u76ee\u30c7\u30d5\u30a9\u30eb\u30c8\u5024\u8aac\u660eapplication_namelogstash (\u4efb\u610f)dynamodb\u8abf\u6574\u30c6\u30fc\u30d6\u30eb\u306b\u4f7f\u7528\u3055\u308c\u308b\u30a2\u30d7\u30ea\u30b1\u30fc\u30b7\u30e7\u30f3\u540d\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002checkpoint_interval_seconds60 (\u4efb\u610f)dynamodb\u3092\u30c1\u30a7\u30c3\u30af\u306b\u3057\u306b\u884c\u304f\u30a4\u30f3\u30bf\u30fc\u30d0\u30eb(\u79d2)\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002kinesis_stream_name- (\u5fc5\u9808)kinesis\u30b9\u30c8\u30ea\u30fc\u30e0\u540d\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002metricsnil (\u4efb\u610f)\u30e1\u30c8\u30ea\u30c3\u30af\u7d71\u8a08\u60c5\u5831\u306eCloudWatch\u3078\u306e\u9001\u4fe1\u6709\u7121\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002profile- (\u4efb\u610f)AWS\u8a8d\u8a3c\u60c5\u5831\u3092\u53c2\u7167\u3059\u308b\u5834\u5408\u306e\u30d5\u30a1\u30a4\u30eb\u30d1\u30b9\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002regionus-east-1 (\u4efb\u610f)Kinesis Data Streams\u306eAWS\u30ea\u30fc\u30b8\u30e7\u30f3\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002role_arn- (\u4efb\u610f)AssumeRole\u3092\u5229\u7528\u3059\u308b\u5834\u5408\u306eARN\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002role_session_name- (\u4efb\u610f)IAM\u30ed\u30fc\u30eb\u3092\u5f15\u304d\u53d7\u3051\u308b\u3068\u304d\u306b\u4f7f\u7528\u3059\u308b\u30bb\u30c3\u30b7\u30e7\u30f3\u540d\u3092\u6307\u5b9a\u3057\u307e\u3059\u3002<\/div>\n<\/div>\n<h2>\u8bbe\u7f6e\u6b65\u9aa4<\/h2>\n<ol>\n<li style=\"list-style-type: none;\">\n<ol>\u521b\u5efaKinesis Streams<\/ol>\n<\/li>\n<\/ol>\n<p>&nbsp;<\/p>\n<ol>\n<li style=\"list-style-type: none;\">\n<ol>\u521b\u5efaIAM\u89d2\u8272<\/ol>\n<\/li>\n<\/ol>\n<p>&nbsp;<\/p>\n<ol>\n<li style=\"list-style-type: none;\">\n<ol>\u5b89\u88c5\u63d2\u4ef6<\/ol>\n<\/li>\n<\/ol>\n<p>&nbsp;<\/p>\n<ol>\u914d\u7f6elogstash.conf<\/ol>\n<h4>1. \u521b\u5efaKinesis Streams\u3002<\/h4>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d7d73913a08637a69fda0\/19-2.png\" alt=\"image.png\" \/><\/div>\n<p>\u203b\u4ee5\u4e0b\u7684\u6587\u7ae0\u5f88\u597d\u5730\u603b\u7ed3\u4e86Kinesis Data Stream\u7684\u89c4\u6a21\u548c\u8fd0\u4f5c\u60c5\u51b5\u3002<\/p>\n<p>\u3010\u53c2\u8003\u3011<br \/>\n\u00b7 \u4ecb\u7ecd\u4e86\u901a\u8fc7\u5927\u89c4\u6a21\u6570\u636e\u9a8c\u8bc1\u548c\u4e86\u89e3AWS Kinesis Stream\u7684\u6848\u4f8b\u3002<\/p>\n<h4>2. \u521b\u5efa IAM \u89d2\u8272<\/h4>\n<ul class=\"post-ul\">Logstash_put-kinesis\u306eEC2\u30a4\u30f3\u30b9\u30bf\u30f3\u30b9\u306b\u5272\u308a\u5f53\u3066\u308bIAM Role\u306b\u5fc5\u8981\u306aIAM Policy<\/ul>\n<pre class=\"post-pre\"><code><span class=\"p\">{<\/span>\r\n    <span class=\"s2\">\"Version\"<\/span><span class=\"p\">:<\/span> <span class=\"s2\">\"2012-10-17\"<\/span><span class=\"p\">,<\/span>\r\n    <span class=\"s2\">\"Statement\"<\/span><span class=\"p\">:<\/span> <span class=\"p\">[<\/span>\r\n        <span class=\"p\">{<\/span>\r\n            <span class=\"s2\">\"Sid\"<\/span><span class=\"p\">:<\/span> <span class=\"s2\">\"VisualEditor0\"<\/span><span class=\"p\">,<\/span>\r\n            <span class=\"s2\">\"Effect\"<\/span><span class=\"p\">:<\/span> <span class=\"s2\">\"Allow\"<\/span><span class=\"p\">,<\/span>\r\n            <span class=\"s2\">\"Action\"<\/span><span class=\"p\">:<\/span> <span class=\"p\">[<\/span>\r\n                <span class=\"s2\">\"kinesis:PutRecord\"<\/span><span class=\"p\">,<\/span>\r\n                <span class=\"s2\">\"kinesis:PutRecords\"<\/span><span class=\"p\">,<\/span>\r\n                <span class=\"s2\">\"kinesis:DescribeStream\"<\/span>\r\n            <span class=\"p\">],<\/span>\r\n            <span class=\"s2\">\"Resource\"<\/span><span class=\"p\">:<\/span> <span class=\"s2\">\"arn:aws:kinesis:us-east-1:&lt;AWS\u30a2\u30ab\u30a6\u30f3\u30c8&gt;:stream\/kinesis_poc\"<\/span>\r\n        <span class=\"p\">},<\/span>\r\n        <span class=\"p\">{<\/span>\r\n            <span class=\"s2\">\"Sid\"<\/span><span class=\"p\">:<\/span> <span class=\"s2\">\"VisualEditor1\"<\/span><span class=\"p\">,<\/span>\r\n            <span class=\"s2\">\"Effect\"<\/span><span class=\"p\">:<\/span> <span class=\"s2\">\"Allow\"<\/span><span class=\"p\">,<\/span>\r\n            <span class=\"s2\">\"Action\"<\/span><span class=\"p\">:<\/span> <span class=\"p\">[<\/span>\r\n                <span class=\"s2\">\"cloudwatch:PutMetricData\"<\/span>\r\n            <span class=\"p\">],<\/span>\r\n            <span class=\"s2\">\"Resource\"<\/span><span class=\"p\">:<\/span> <span class=\"s2\">\"*\"<\/span>\r\n        <span class=\"p\">}<\/span>\r\n    <span class=\"p\">]<\/span>\r\n<span class=\"p\">}<\/span>\r\n<\/code><\/pre>\n<ul class=\"post-ul\">Logstash_get-kinesis\u306eEC2\u30a4\u30f3\u30b9\u30bf\u30f3\u30b9\u306b\u5272\u308a\u5f53\u3066\u308bIAM Role\u306b\u5fc5\u8981\u306aIAM Policy<\/ul>\n<pre class=\"post-pre\"><code><span class=\"p\">{<\/span>\r\n    <span class=\"s2\">\"Version\"<\/span><span class=\"p\">:<\/span> <span class=\"s2\">\"2012-10-17\"<\/span><span class=\"p\">,<\/span>\r\n    <span class=\"s2\">\"Statement\"<\/span><span class=\"p\">:<\/span> <span class=\"p\">[<\/span>\r\n        <span class=\"p\">{<\/span>\r\n            <span class=\"s2\">\"Sid\"<\/span><span class=\"p\">:<\/span> <span class=\"s2\">\"VisualEditor0\"<\/span><span class=\"p\">,<\/span>\r\n            <span class=\"s2\">\"Effect\"<\/span><span class=\"p\">:<\/span> <span class=\"s2\">\"Allow\"<\/span><span class=\"p\">,<\/span>\r\n            <span class=\"s2\">\"Action\"<\/span><span class=\"p\">:<\/span> <span class=\"p\">[<\/span>\r\n                <span class=\"s2\">\"kinesis:GetShardIterator\"<\/span><span class=\"p\">,<\/span>\r\n                <span class=\"s2\">\"kinesis:GetRecords\"<\/span><span class=\"p\">,<\/span>\r\n                <span class=\"s2\">\"kinesis:DescribeStream\"<\/span><span class=\"p\">,<\/span>\r\n                <span class=\"s2\">\"kinesis:ListShards\"<\/span>\r\n            <span class=\"p\">],<\/span>\r\n            <span class=\"s2\">\"Resource\"<\/span><span class=\"p\">:<\/span> <span class=\"s2\">\"arn:aws:kinesis:us-east-1:&lt;AWS\u30a2\u30ab\u30a6\u30f3\u30c8&gt;:stream\/kinesis_poc\"<\/span>\r\n        <span class=\"p\">},<\/span>\r\n        <span class=\"p\">{<\/span>\r\n            <span class=\"s2\">\"Sid\"<\/span><span class=\"p\">:<\/span> <span class=\"s2\">\"VisualEditor1\"<\/span><span class=\"p\">,<\/span>\r\n            <span class=\"s2\">\"Effect\"<\/span><span class=\"p\">:<\/span> <span class=\"s2\">\"Allow\"<\/span><span class=\"p\">,<\/span>\r\n            <span class=\"s2\">\"Action\"<\/span><span class=\"p\">:<\/span> <span class=\"p\">[<\/span>\r\n                <span class=\"s2\">\"dynamodb:CreateTable\"<\/span><span class=\"p\">,<\/span>\r\n                <span class=\"s2\">\"dynamodb:DescribeTable\"<\/span><span class=\"p\">,<\/span>\r\n                <span class=\"s2\">\"dynamodb:Scan\"<\/span><span class=\"p\">,<\/span>\r\n                <span class=\"s2\">\"dynamodb:PutItem\"<\/span><span class=\"p\">,<\/span>\r\n                <span class=\"s2\">\"dynamodb:UpdateItem\"<\/span><span class=\"p\">,<\/span>\r\n                <span class=\"s2\">\"dynamodb:GetItem\"<\/span><span class=\"p\">,<\/span>\r\n                <span class=\"s2\">\"dynamodb:DeleteItem\"<\/span><span class=\"p\">,<\/span>\r\n                <span class=\"s2\">\"cloudwatch:PutMetricData\"<\/span>\r\n            <span class=\"p\">],<\/span>\r\n            <span class=\"s2\">\"Resource\"<\/span><span class=\"p\">:<\/span> <span class=\"s2\">\"*\"<\/span>\r\n        <span class=\"p\">}<\/span>\r\n    <span class=\"p\">]<\/span>\r\n<span class=\"p\">}<\/span>\r\n<\/code><\/pre>\n<p>\u4ee5\u4e0b\u662fIAM\u7b56\u7565\u548c\u7528\u6237\u521b\u5efa\u6240\u9700\u7684IAM\u7b56\u7565\uff0c\u4f46\u8fd9\u662f\u4e0d\u591f\u7684\u3002<br \/>\n\u5982\u679c\u6743\u9650\u4e0d\u8db3\uff0c\u5728\/var\/log\/messages\u4e2d\u4f1a\u8f93\u51fa\u4ee5\u4e0b\u9519\u8bef\uff0c\u6240\u4ee5\u8bf7\u52a1\u5fc5\u6ce8\u610f\uff01<br \/>\n\u203b\u4ee5\u4e0b\u793a\u4f8b\u4e2d\u7f3a\u5c11dynamodb:DescribeTable\u6743\u9650\u3002<\/p>\n<pre class=\"post-pre\"><code><span class=\"no\">Feb<\/span> <span class=\"mi\">13<\/span> <span class=\"mi\">16<\/span><span class=\"p\">:<\/span><span class=\"mo\">01<\/span><span class=\"p\">:<\/span><span class=\"mi\">39<\/span> <span class=\"n\">ip<\/span><span class=\"o\">-<\/span><span class=\"mi\">172<\/span><span class=\"o\">-<\/span><span class=\"mi\">31<\/span><span class=\"o\">-<\/span><span class=\"mi\">1<\/span><span class=\"o\">-<\/span><span class=\"mi\">190<\/span> <span class=\"ss\">logstash: <\/span><span class=\"no\">Caused<\/span> <span class=\"ss\">by: <\/span><span class=\"n\">com<\/span><span class=\"p\">.<\/span><span class=\"nf\">amazonaws<\/span><span class=\"p\">.<\/span><span class=\"nf\">services<\/span><span class=\"p\">.<\/span><span class=\"nf\">dynamodbv2<\/span><span class=\"p\">.<\/span><span class=\"nf\">model<\/span><span class=\"o\">.<\/span><span class=\"no\">AmazonDynamoDBException<\/span><span class=\"p\">:<\/span> <span class=\"no\">User<\/span><span class=\"p\">:<\/span> <span class=\"n\">arn<\/span><span class=\"ss\">:aws:sts<\/span><span class=\"o\">::&lt;<\/span><span class=\"no\">AWS<\/span><span class=\"err\">\u30a2\u30ab\u30a6\u30f3\u30c8<\/span><span class=\"o\">&gt;<\/span><span class=\"ss\">:assumed<\/span><span class=\"o\">-<\/span><span class=\"n\">role<\/span><span class=\"o\">\/<\/span><span class=\"n\">kinesis_get_role<\/span><span class=\"o\">\/<\/span><span class=\"n\">i<\/span><span class=\"o\">-<\/span><span class=\"mo\">03<\/span><span class=\"n\">c32b05117b3235a<\/span> <span class=\"n\">is<\/span> <span class=\"n\">not<\/span> <span class=\"n\">authorized<\/span> <span class=\"n\">to<\/span> <span class=\"ss\">perform: <\/span><span class=\"n\">dynamodb<\/span><span class=\"ss\">:DescribeTable<\/span> <span class=\"n\">on<\/span> <span class=\"ss\">resource: <\/span><span class=\"n\">arn<\/span><span class=\"ss\">:aws:dynamodb:us<\/span><span class=\"o\">-<\/span><span class=\"n\">east<\/span><span class=\"o\">-<\/span><span class=\"mi\">1<\/span><span class=\"ss\">:&lt;<\/span><span class=\"no\">AWS<\/span><span class=\"err\">\u30a2\u30ab\u30a6\u30f3\u30c8<\/span><span class=\"o\">&gt;<\/span><span class=\"ss\">:table<\/span><span class=\"o\">\/<\/span><span class=\"n\">logstash<\/span> <span class=\"p\">(<\/span><span class=\"no\">Service<\/span><span class=\"p\">:<\/span> <span class=\"no\">AmazonDynamoDBv2<\/span><span class=\"p\">;<\/span> <span class=\"no\">Status<\/span> <span class=\"no\">Code<\/span><span class=\"p\">:<\/span> <span class=\"mi\">400<\/span><span class=\"p\">;<\/span> <span class=\"no\">Error<\/span> <span class=\"no\">Code<\/span><span class=\"p\">:<\/span> <span class=\"no\">AccessDeniedException<\/span><span class=\"p\">;<\/span> <span class=\"no\">Request<\/span> <span class=\"no\">ID<\/span><span class=\"p\">:<\/span> <span class=\"no\">BQRBG0RSE187CLGTP802ONS5BFVV4KQNSO5AEMVJF66Q9ASUAAJG<\/span><span class=\"p\">)<\/span>\r\n<\/code><\/pre>\n<h4>3. \u5b89\u88c5\u63d2\u4ef6<\/h4>\n<p>\u60a8\u53ef\u4ee5\u901a\u8fc7\u67e5\u770b\/usr\/share\/logstash\/Gemfile\u6765\u4e86\u89e3\u9ed8\u8ba4\u5b89\u88c5\u7684\u63d2\u4ef6\u3002<br \/>\n\u987a\u4fbf\u63d0\u4e00\u4e0b\uff0c\u672c\u6b21\u6d89\u53ca\u7684\u4e24\u4e2a\u63d2\u4ef6\u90fd\u5c1a\u672a\u9ed8\u8ba4\u5b89\u88c5\uff0c\u8bf7\u8fdb\u884c\u989d\u5916\u5b89\u88c5\u3002<\/p>\n<pre class=\"post-pre\"><code>[root@ip-172-31-1-190 ~]# cat \/usr\/share\/logstash\/Gemfile\r\n# This is a Logstash generated Gemfile.\r\n# If you modify this file manually all comments and formatting will be lost.\r\n\r\nsource \"https:\/\/rubygems.org\"\r\ngem \"logstash-core\", :path =&gt; \".\/logstash-core\"\r\ngem \"logstash-core-plugin-api\", :path =&gt; \".\/logstash-core-plugin-api\"\r\ngem \"paquet\", \"~&gt; 0.2.0\"\r\ngem \"ruby-progressbar\", \"~&gt; 1.8.1\"\r\ngem \"builder\", \"~&gt; 3.2.2\"\r\ngem \"ci_reporter_rspec\", \"1.0.0\", :group =&gt; :development\r\ngem \"tins\", \"1.6\", :group =&gt; :development\r\ngem \"rspec\", \"~&gt; 3.5\", :group =&gt; :development\r\ngem \"logstash-devutils\", \"= 1.3.5\", :group =&gt; :development\r\ngem \"benchmark-ips\", :group =&gt; :development\r\ngem \"octokit\", \"3.8.0\", :group =&gt; :build\r\ngem \"stud\", \"~&gt; 0.0.22\", :group =&gt; :build\r\ngem \"fpm\", \"~&gt; 1.3.3\", :group =&gt; :build\r\ngem \"rubyzip\", \"~&gt; 1.2.1\", :group =&gt; :build\r\ngem \"gems\", \"~&gt; 0.8.3\", :group =&gt; :build\r\ngem \"rack-test\", :require =&gt; \"rack\/test\", :group =&gt; :development\r\ngem \"flores\", \"~&gt; 0.0.6\", :group =&gt; :development\r\ngem \"term-ansicolor\", \"~&gt; 1.3.2\", :group =&gt; :development\r\ngem \"json-schema\", \"~&gt; 2.6\", :group =&gt; :development\r\ngem \"belzebuth\", :group =&gt; :development\r\ngem \"pleaserun\", \"~&gt;0.0.28\"\r\ngem \"webrick\", \"~&gt; 1.3.1\"\r\ngem \"atomic\", \"&lt;= 1.1.99\"\r\ngem \"rake\", \"~&gt; 12.2.1\", :group =&gt; :build\r\ngem \"logstash-codec-cef\"\r\ngem \"logstash-codec-collectd\"\r\ngem \"logstash-codec-dots\"\r\ngem \"logstash-codec-edn\"\r\ngem \"logstash-codec-edn_lines\"\r\ngem \"logstash-codec-es_bulk\"\r\ngem \"logstash-codec-fluent\"\r\ngem \"logstash-codec-graphite\"\r\ngem \"logstash-codec-json\"\r\ngem \"logstash-codec-json_lines\"\r\ngem \"logstash-codec-line\"\r\ngem \"logstash-codec-msgpack\"\r\ngem \"logstash-codec-multiline\"\r\ngem \"logstash-codec-netflow\", \"&gt;=3.14.1\", \"&lt;4.0.0\"\r\ngem \"logstash-codec-plain\"\r\ngem \"logstash-codec-rubydebug\"\r\ngem \"logstash-filter-aggregate\"\r\ngem \"logstash-filter-anonymize\"\r\ngem \"logstash-filter-cidr\"\r\ngem \"logstash-filter-clone\"\r\ngem \"logstash-filter-csv\"\r\ngem \"logstash-filter-date\"\r\ngem \"logstash-filter-de_dot\"\r\ngem \"logstash-filter-dissect\"\r\ngem \"logstash-filter-dns\"\r\ngem \"logstash-filter-drop\"\r\ngem \"logstash-filter-elasticsearch\"\r\ngem \"logstash-filter-fingerprint\"\r\ngem \"logstash-filter-geoip\"\r\ngem \"logstash-filter-grok\"\r\ngem \"logstash-filter-http\"\r\ngem \"logstash-filter-jdbc_static\"\r\ngem \"logstash-filter-jdbc_streaming\"\r\ngem \"logstash-filter-json\"\r\ngem \"logstash-filter-kv\"\r\ngem \"logstash-filter-memcached\"\r\ngem \"logstash-filter-metrics\"\r\ngem \"logstash-filter-mutate\"\r\ngem \"logstash-filter-ruby\"\r\ngem \"logstash-filter-sleep\"\r\ngem \"logstash-filter-split\"\r\ngem \"logstash-filter-syslog_pri\"\r\ngem \"logstash-filter-throttle\"\r\ngem \"logstash-filter-translate\"\r\ngem \"logstash-filter-truncate\"\r\ngem \"logstash-filter-urldecode\"\r\ngem \"logstash-filter-useragent\"\r\ngem \"logstash-filter-xml\"\r\ngem \"logstash-input-beats\"\r\ngem \"logstash-input-azure_event_hubs\"\r\ngem \"logstash-input-dead_letter_queue\"\r\ngem \"logstash-input-elasticsearch\"\r\ngem \"logstash-input-exec\"\r\ngem \"logstash-input-file\"\r\ngem \"logstash-input-ganglia\"\r\ngem \"logstash-input-gelf\"\r\ngem \"logstash-input-generator\"\r\ngem \"logstash-input-graphite\"\r\ngem \"logstash-input-heartbeat\"\r\ngem \"logstash-input-http\"\r\ngem \"logstash-input-http_poller\"\r\ngem \"logstash-input-imap\"\r\ngem \"logstash-input-jdbc\"\r\ngem \"logstash-input-kafka\"\r\ngem \"logstash-input-pipe\"\r\ngem \"logstash-input-rabbitmq\"\r\ngem \"logstash-input-redis\"\r\ngem \"logstash-input-s3\"\r\ngem \"logstash-input-snmp\"\r\ngem \"logstash-input-snmptrap\"\r\ngem \"logstash-input-sqs\"\r\ngem \"logstash-input-stdin\"\r\ngem \"logstash-input-syslog\"\r\ngem \"logstash-input-tcp\"\r\ngem \"logstash-input-twitter\"\r\ngem \"logstash-input-udp\"\r\ngem \"logstash-input-unix\"\r\ngem \"logstash-output-elastic_app_search\"\r\ngem \"logstash-output-cloudwatch\"\r\ngem \"logstash-output-csv\"\r\ngem \"logstash-output-elasticsearch\"\r\ngem \"logstash-output-email\"\r\ngem \"logstash-output-file\"\r\ngem \"logstash-output-graphite\"\r\ngem \"logstash-output-http\"\r\ngem \"logstash-output-kafka\"\r\ngem \"logstash-output-lumberjack\"\r\ngem \"logstash-output-nagios\"\r\ngem \"logstash-output-null\"\r\ngem \"logstash-output-pagerduty\"\r\ngem \"logstash-output-pipe\"\r\ngem \"logstash-output-rabbitmq\"\r\ngem \"logstash-output-redis\"\r\ngem \"logstash-output-s3\", \"&gt;=4.0.9\", \"&lt;5.0.0\"\r\ngem \"logstash-output-sns\"\r\ngem \"logstash-output-sqs\"\r\ngem \"logstash-output-stdout\"\r\ngem \"logstash-output-tcp\"\r\ngem \"logstash-output-udp\"\r\ngem \"logstash-output-webhdfs\"\r\n<\/code><\/pre>\n<ul class=\"post-ul\">Logstash_put-kinesis\u306b\u306fkinesis output plugin\u3092\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3057\u307e\u3059\u3002<\/ul>\n<pre class=\"post-pre\"><code>[root@ip-172-31-1-235 ~]# \/usr\/share\/logstash\/bin\/logstash-plugin install logstash-output-kinesis\r\nValidating logstash-output-kinesis\r\nInstalling logstash-output-kinesis\r\nInstallation successful\r\n<\/code><\/pre>\n<ul class=\"post-ul\">Logstash_get-kinesis\u306b\u306fkinesis input plugin\u3092\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3057\u307e\u3059\u3002<\/ul>\n<pre class=\"post-pre\"><code>[root@ip-172-31-9-35 ~]# \/usr\/share\/logstash\/bin\/logstash-plugin install logstash-input-kinesis\r\nValidating logstash-input-kinesis\r\nInstalling logstash-input-kinesis\r\nInstallation successful\r\n<\/code><\/pre>\n<h4>4. logstash.conf\u7684\u914d\u7f6e\u6587\u4ef6<\/h4>\n<ul class=\"post-ul\">Logstash_put-kinesis\u306elogstash.conf\u3092\u4f5c\u6210\u3057\u307e\u3059\u3002<\/ul>\n<pre class=\"post-pre\"><code><span class=\"n\">input<\/span> <span class=\"p\">{<\/span>\r\n  <span class=\"n\">beats<\/span> <span class=\"p\">{<\/span>\r\n    <span class=\"n\">port<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"mi\">5044<\/span>\r\n  <span class=\"p\">}<\/span>\r\n<span class=\"p\">}<\/span>\r\n<span class=\"n\">output<\/span> <span class=\"p\">{<\/span>\r\n  <span class=\"n\">kinesis<\/span> <span class=\"p\">{<\/span>\r\n    <span class=\"n\">stream_name<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"s2\">\"kinesis_poc\"<\/span>\r\n    <span class=\"n\">max_pending_records<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"mi\">10000<\/span>\r\n    <span class=\"n\">randomized_partition_key<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"kp\">true<\/span>\r\n    <span class=\"n\">region<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"s2\">\"us-east-1\"<\/span>\r\n  <span class=\"p\">}<\/span>\r\n<span class=\"p\">}<\/span>\r\n<\/code><\/pre>\n<p>\u5c06\u4eceFilebeat\u63a5\u6536\u5230\u7684\u65e5\u5fd7\u6570\u636e\u6d41\u5411Kinesis\u3002<br \/>\n\u5728\u5206\u53d1\u5230\u591a\u4e2a\u5206\u7247\u65f6\uff0c\u6fc0\u6d3b\u968f\u673a\u5206\u533a\u952e\uff08randomized_partition_key\uff09\u529f\u80fd\u3002<\/p>\n<ul class=\"post-ul\">Logstash_get-kinesis\u306elogstash.conf\u3092\u4f5c\u6210\u3057\u307e\u3059\u3002<\/ul>\n<pre class=\"post-pre\"><code><span class=\"n\">input<\/span> <span class=\"p\">{<\/span>\r\n  <span class=\"n\">kinesis<\/span> <span class=\"p\">{<\/span>\r\n    <span class=\"n\">kinesis_stream_name<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"s2\">\"kinesis_poc\"<\/span>\r\n    <span class=\"n\">application_name<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"s2\">\"kinesis_poc\"<\/span>\r\n    <span class=\"n\">checkpoint_interval_seconds<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"mi\">10<\/span>\r\n    <span class=\"n\">metrics<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"s2\">\"cloudwatch\"<\/span>\r\n    <span class=\"n\">region<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"s2\">\"us-east-1\"<\/span>\r\n    <span class=\"n\">codec<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"n\">json<\/span> <span class=\"p\">{}<\/span>\r\n  <span class=\"p\">}<\/span>\r\n<span class=\"p\">}<\/span>\r\n<span class=\"n\">output<\/span> <span class=\"p\">{<\/span>\r\n  <span class=\"n\">elasticsearch<\/span> <span class=\"p\">{<\/span>\r\n    <span class=\"n\">hosts<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"p\">[<\/span> <span class=\"s2\">\"localhost:9200\"<\/span> <span class=\"p\">]<\/span>\r\n  <span class=\"p\">}<\/span>\r\n<span class=\"p\">}<\/span>\r\n<\/code><\/pre>\n<p>\u8fd9\u6b21\u662f\u5c06\u8f93\u51fa\u53d1\u9001\u5230\u5df2\u7ecf\u5728\u540c\u4e00\u64cd\u4f5c\u7cfb\u7edf\u4e2d\u5b89\u88c5\u7684Elasticsearch\uff0c\u5e76\u901a\u8fc7Kibana\u5b9e\u73b0\u53ef\u89c6\u5316\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d7d73913a08637a69fda0\/43-0.png\" alt=\"image.png\" \/><\/div>\n<h2>\u6267\u884c\u7ed3\u679c<\/h2>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d7d73913a08637a69fda0\/45-1.png\" alt=\"image.png\" \/><\/div>\n<p>\u5728\u516c\u5f0f\u7f51\u7ad9\u4e0a\u4e0d\u77e5\u9053\u4e3a\u4ec0\u4e48\u6ca1\u6709\u63d0\u5230\u4f5c\u4e3a\u9009\u9879\uff0c\u4f46\u5728GitHub\u4e0a\u5199\u7740\u53ea\u6709\u4e24\u4e2a\u8fed\u4ee3\u5668\u7684\u6307\u5b9a\u662f\u53ef\u4ee5\u7684\u3002<\/p>\n<h2>\u603b\u7ed3<\/h2>\n<p>\u60a8\u89c9\u5f97\u600e\u4e48\u6837\uff1f<\/p>\n<p>\u6211\u4ee5\u524d\u591a\u6b21\u4f7f\u7528AWS\u4e0a\u7684S3\u5b58\u50a8\u6876\u4f5c\u4e3aDataLake\uff0c\u4f46\u662f\u4e3a\u4e86\u4eceDataLake\u4e2d\u91c7\u96c6\u5927\u91cf\u65e5\u5fd7\u6570\u636e\u5e76\u5b9e\u65f6\u6295\u5165\u5230\u5206\u6790\u6240\u9700\u7684\u5b58\u50a8\u4e2d\uff0c\u6211\u9700\u8981\u4e00\u4e2a\u53ef\u6269\u5c55\u7684\u67b6\u6784\uff0c\u6240\u4ee5\u8fd9\u6b21\u6211\u5c1d\u8bd5\u4e86\u4e00\u4e0b\uff01<\/p>\n<p>\u7531\u4e8e Logstash \u548c Kinesis \u6570\u636e\u6d41\u4e4b\u95f4\u7684\u534f\u4f5c\u4fe1\u606f\u76f8\u5bf9\u8f83\u5c11\uff0c\u5e0c\u671b\u8fd9\u80fd\u5bf9\u67d0\u4e9b\u4eba\u6709\u5e2e\u52a9^^<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u9996\u5148 \u6211\u4f7f\u7528Elastic\u63d0\u4f9b\u7684ETL\u5de5\u5177Logstash\uff0c\u5c06\u65e5\u5fd7\u6570\u636e\u5bfc\u5165\u548c\u5bfc\u51fa\u5230Amazon Kinesis [&hellip;]<\/p>\n","protected":false},"author":10,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-47200","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.5 (Yoast SEO v21.5) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Logstash\u7684Kinesis\u8f93\u5165\u63d2\u4ef6\u548cKinesis\u8f93\u51fa\u63d2\u4ef6 - Blog - Silicon Cloud<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.silicloud.com\/zh\/blog\/logstash\u7684kinesis\u8f93\u5165\u63d2\u4ef6\u548ckinesis\u8f93\u51fa\u63d2\u4ef6\/\" \/>\n<meta property=\"og:locale\" content=\"zh_CN\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Logstash\u7684Kinesis\u8f93\u5165\u63d2\u4ef6\u548cKinesis\u8f93\u51fa\u63d2\u4ef6\" \/>\n<meta property=\"og:description\" content=\"\u9996\u5148 \u6211\u4f7f\u7528Elastic\u63d0\u4f9b\u7684ETL\u5de5\u5177Logstash\uff0c\u5c06\u65e5\u5fd7\u6570\u636e\u5bfc\u5165\u548c\u5bfc\u51fa\u5230Amazon Kinesis [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.silicloud.com\/zh\/blog\/logstash\u7684kinesis\u8f93\u5165\u63d2\u4ef6\u548ckinesis\u8f93\u51fa\u63d2\u4ef6\/\" \/>\n<meta property=\"og:site_name\" content=\"Blog - Silicon Cloud\" \/>\n<meta property=\"article:published_time\" content=\"2023-04-28T07:40:18+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-04-29T12:32:49+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d7d73913a08637a69fda0\/6-0.png\" \/>\n<meta name=\"author\" content=\"\u5b87, \u534e\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"\u4f5c\u8005\" \/>\n\t<meta name=\"twitter:data1\" content=\"\u5b87, \u534e\" \/>\n\t<meta name=\"twitter:label2\" content=\"\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 \u5206\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/logstash%e7%9a%84kinesis%e8%be%93%e5%85%a5%e6%8f%92%e4%bb%b6%e5%92%8ckinesis%e8%be%93%e5%87%ba%e6%8f%92%e4%bb%b6\/\",\"url\":\"https:\/\/www.silicloud.com\/zh\/blog\/logstash%e7%9a%84kinesis%e8%be%93%e5%85%a5%e6%8f%92%e4%bb%b6%e5%92%8ckinesis%e8%be%93%e5%87%ba%e6%8f%92%e4%bb%b6\/\",\"name\":\"Logstash\u7684Kinesis\u8f93\u5165\u63d2\u4ef6\u548cKinesis\u8f93\u51fa\u63d2\u4ef6 - Blog - Silicon Cloud\",\"isPartOf\":{\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#website\"},\"datePublished\":\"2023-04-28T07:40:18+00:00\",\"dateModified\":\"2024-04-29T12:32:49+00:00\",\"author\":{\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/513018e4e121d3add1b7c5de8be21458\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/logstash%e7%9a%84kinesis%e8%be%93%e5%85%a5%e6%8f%92%e4%bb%b6%e5%92%8ckinesis%e8%be%93%e5%87%ba%e6%8f%92%e4%bb%b6\/#breadcrumb\"},\"inLanguage\":\"zh-Hans\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.silicloud.com\/zh\/blog\/logstash%e7%9a%84kinesis%e8%be%93%e5%85%a5%e6%8f%92%e4%bb%b6%e5%92%8ckinesis%e8%be%93%e5%87%ba%e6%8f%92%e4%bb%b6\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/logstash%e7%9a%84kinesis%e8%be%93%e5%85%a5%e6%8f%92%e4%bb%b6%e5%92%8ckinesis%e8%be%93%e5%87%ba%e6%8f%92%e4%bb%b6\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"\u9996\u9875\",\"item\":\"https:\/\/www.silicloud.com\/zh\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Logstash\u7684Kinesis\u8f93\u5165\u63d2\u4ef6\u548cKinesis\u8f93\u51fa\u63d2\u4ef6\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#website\",\"url\":\"https:\/\/www.silicloud.com\/zh\/blog\/\",\"name\":\"Blog - Silicon Cloud\",\"description\":\"\",\"inLanguage\":\"zh-Hans\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/513018e4e121d3add1b7c5de8be21458\",\"name\":\"\u5b87, \u534e\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"zh-Hans\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/63cd45cbc05a35fc4ff7637a163c83c4962ef58d27472726c3a3e0c9c5194f0f?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/63cd45cbc05a35fc4ff7637a163c83c4962ef58d27472726c3a3e0c9c5194f0f?s=96&d=mm&r=g\",\"caption\":\"\u5b87, \u534e\"},\"url\":\"https:\/\/www.silicloud.com\/zh\/blog\/author\/yuhua\/\"},{\"@type\":\"ImageObject\",\"inLanguage\":\"zh-Hans\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/logstash%e7%9a%84kinesis%e8%be%93%e5%85%a5%e6%8f%92%e4%bb%b6%e5%92%8ckinesis%e8%be%93%e5%87%ba%e6%8f%92%e4%bb%b6\/#local-main-organization-logo\",\"url\":\"\",\"contentUrl\":\"\",\"caption\":\"Blog - Silicon Cloud\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Logstash\u7684Kinesis\u8f93\u5165\u63d2\u4ef6\u548cKinesis\u8f93\u51fa\u63d2\u4ef6 - Blog - Silicon Cloud","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.silicloud.com\/zh\/blog\/logstash\u7684kinesis\u8f93\u5165\u63d2\u4ef6\u548ckinesis\u8f93\u51fa\u63d2\u4ef6\/","og_locale":"zh_CN","og_type":"article","og_title":"Logstash\u7684Kinesis\u8f93\u5165\u63d2\u4ef6\u548cKinesis\u8f93\u51fa\u63d2\u4ef6","og_description":"\u9996\u5148 \u6211\u4f7f\u7528Elastic\u63d0\u4f9b\u7684ETL\u5de5\u5177Logstash\uff0c\u5c06\u65e5\u5fd7\u6570\u636e\u5bfc\u5165\u548c\u5bfc\u51fa\u5230Amazon Kinesis [&hellip;]","og_url":"https:\/\/www.silicloud.com\/zh\/blog\/logstash\u7684kinesis\u8f93\u5165\u63d2\u4ef6\u548ckinesis\u8f93\u51fa\u63d2\u4ef6\/","og_site_name":"Blog - Silicon Cloud","article_published_time":"2023-04-28T07:40:18+00:00","article_modified_time":"2024-04-29T12:32:49+00:00","og_image":[{"url":"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d7d73913a08637a69fda0\/6-0.png"}],"author":"\u5b87, \u534e","twitter_card":"summary_large_image","twitter_misc":{"\u4f5c\u8005":"\u5b87, \u534e","\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4":"4 \u5206"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.silicloud.com\/zh\/blog\/logstash%e7%9a%84kinesis%e8%be%93%e5%85%a5%e6%8f%92%e4%bb%b6%e5%92%8ckinesis%e8%be%93%e5%87%ba%e6%8f%92%e4%bb%b6\/","url":"https:\/\/www.silicloud.com\/zh\/blog\/logstash%e7%9a%84kinesis%e8%be%93%e5%85%a5%e6%8f%92%e4%bb%b6%e5%92%8ckinesis%e8%be%93%e5%87%ba%e6%8f%92%e4%bb%b6\/","name":"Logstash\u7684Kinesis\u8f93\u5165\u63d2\u4ef6\u548cKinesis\u8f93\u51fa\u63d2\u4ef6 - Blog - Silicon Cloud","isPartOf":{"@id":"https:\/\/www.silicloud.com\/zh\/blog\/#website"},"datePublished":"2023-04-28T07:40:18+00:00","dateModified":"2024-04-29T12:32:49+00:00","author":{"@id":"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/513018e4e121d3add1b7c5de8be21458"},"breadcrumb":{"@id":"https:\/\/www.silicloud.com\/zh\/blog\/logstash%e7%9a%84kinesis%e8%be%93%e5%85%a5%e6%8f%92%e4%bb%b6%e5%92%8ckinesis%e8%be%93%e5%87%ba%e6%8f%92%e4%bb%b6\/#breadcrumb"},"inLanguage":"zh-Hans","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.silicloud.com\/zh\/blog\/logstash%e7%9a%84kinesis%e8%be%93%e5%85%a5%e6%8f%92%e4%bb%b6%e5%92%8ckinesis%e8%be%93%e5%87%ba%e6%8f%92%e4%bb%b6\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.silicloud.com\/zh\/blog\/logstash%e7%9a%84kinesis%e8%be%93%e5%85%a5%e6%8f%92%e4%bb%b6%e5%92%8ckinesis%e8%be%93%e5%87%ba%e6%8f%92%e4%bb%b6\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"\u9996\u9875","item":"https:\/\/www.silicloud.com\/zh\/blog\/"},{"@type":"ListItem","position":2,"name":"Logstash\u7684Kinesis\u8f93\u5165\u63d2\u4ef6\u548cKinesis\u8f93\u51fa\u63d2\u4ef6"}]},{"@type":"WebSite","@id":"https:\/\/www.silicloud.com\/zh\/blog\/#website","url":"https:\/\/www.silicloud.com\/zh\/blog\/","name":"Blog - Silicon Cloud","description":"","inLanguage":"zh-Hans"},{"@type":"Person","@id":"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/513018e4e121d3add1b7c5de8be21458","name":"\u5b87, \u534e","image":{"@type":"ImageObject","inLanguage":"zh-Hans","@id":"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/63cd45cbc05a35fc4ff7637a163c83c4962ef58d27472726c3a3e0c9c5194f0f?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/63cd45cbc05a35fc4ff7637a163c83c4962ef58d27472726c3a3e0c9c5194f0f?s=96&d=mm&r=g","caption":"\u5b87, \u534e"},"url":"https:\/\/www.silicloud.com\/zh\/blog\/author\/yuhua\/"},{"@type":"ImageObject","inLanguage":"zh-Hans","@id":"https:\/\/www.silicloud.com\/zh\/blog\/logstash%e7%9a%84kinesis%e8%be%93%e5%85%a5%e6%8f%92%e4%bb%b6%e5%92%8ckinesis%e8%be%93%e5%87%ba%e6%8f%92%e4%bb%b6\/#local-main-organization-logo","url":"","contentUrl":"","caption":"Blog - Silicon Cloud"}]}},"_links":{"self":[{"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/posts\/47200","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/comments?post=47200"}],"version-history":[{"count":2,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/posts\/47200\/revisions"}],"predecessor-version":[{"id":87489,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/posts\/47200\/revisions\/87489"}],"wp:attachment":[{"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/media?parent=47200"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/categories?post=47200"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/tags?post=47200"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}