{"id":46745,"date":"2023-07-18T07:53:21","date_gmt":"2023-08-21T22:51:58","guid":{"rendered":"https:\/\/www.silicloud.com\/zh\/blog\/46745-2\/"},"modified":"2024-04-30T14:08:23","modified_gmt":"2024-04-30T06:08:23","slug":"46745-2","status":"publish","type":"post","link":"https:\/\/www.silicloud.com\/zh\/blog\/46745-2\/","title":{"rendered":""},"content":{"rendered":"<h1>\u306f\u3058\u3081\u306b<\/h1>\n<p>Oracle Cloud Infractructure(\u4ee5\u4e0bOCI)\u3067\u306f\u3001\u30b9\u30c8\u30ea\u30fc\u30df\u30f3\u30b0\u30c7\u30fc\u30bf\u3092\u30ea\u30a2\u30eb\u30bf\u30a4\u30e0\u306b\u53ce\u96c6\u30fb\u51e6\u7406\u304c\u51fa\u6765\u308b Streaming \u3068\u3044\u3046\u30b5\u30fc\u30d3\u30b9\u304c\u63d0\u4f9b\u3055\u308c\u3066\u3044\u307e\u3059\u3002Streaming \u3078\u30c7\u30fc\u30bf\u3092\u5165\u51fa\u529b\u3059\u308b\u65b9\u6cd5\u306f\u3001\u3056\u3063\u304f\u308a\u6b21\u306e\u3082\u306e\u304c\u3042\u308a\u307e\u3059<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">OCI SDK<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">Kafka \u4e92\u63dbAPI<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">Kafka Connect<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">OCI CLI<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">REST API<\/ul>\n<p>\u4eca\u56de\u306e\u8a18\u4e8b\u3067\u306f\u3001Kafka Connect \u3092\u4f7f\u3063\u3066 Streaming \u306b\u30c7\u30fc\u30bf\u3092\u5165\u51fa\u529b\u3057\u3066\u307f\u307e\u3059\u3002Kafka Connect \u306f\u3001Apache Kafka \u3067\u63d0\u4f9b\u3055\u308c\u3066\u3044\u308b API \u3068\u306a\u3063\u3066\u3044\u3066\u3001\u7c21\u5358\u306b\u30c7\u30fc\u30bf\u3092\u5165\u51fa\u529b\u51fa\u6765\u307e\u3059\u3002Connect \u306b\u63a5\u7d9a\u3059\u308b\u305f\u3081\u306e Connector \u304c\u8272\u3005\u516c\u958b\u3055\u308c\u3066\u3044\u308b\u305f\u3081\u3001\u81ea\u5206\u304c\u4f7f\u3044\u305f\u3044 Connector \u304c\u3042\u308c\u3070\u3001\u81ea\u5206\u3067\u5b9f\u88c5\u3059\u308b\u8ca0\u62c5\u3092\u6e1b\u3089\u305b\u307e\u3059\u3002\u4f7f\u3044\u305f\u3044 Connector \u304c\u7121\u3051\u308c\u3070\u3001\u81ea\u5206\u3067\u4f5c\u3063\u3066\u3044\u304f\u3053\u3068\u3082\u3067\u304d\u307e\u3059\u3002<\/p>\n<p>\u30a4\u30f3\u30bf\u30fc\u30cd\u30c3\u30c8\u4e0a\u306b\u516c\u958b\u3055\u308c\u3066\u3044\u308b\u30b3\u30cd\u30af\u30bf\u30fc\u306f\u3001\u30e9\u30a4\u30bb\u30f3\u30b9\u5f62\u614b\u306b\u3054\u6ce8\u610f\u304f\u3060\u3055\u3044\u3002<\/p>\n<h1>Auth Token\u751f\u6210<\/h1>\n<p>Streaming \u3067\u3001Kafka Connect \u3067\u63a5\u7d9a\u3059\u308b\u305f\u3081\u306b\u306f\u3001IAM \u30e6\u30fc\u30b6\u30fc\u3067 Auth Token \u304c\u5fc5\u8981\u3067\u3059\u3002\u81ea\u5206\u306e IAM\u30e6\u30fc\u30b6\u30fc\u306e\u8a73\u7d30\u753b\u9762\u3067\u3001Generate \u3057\u307e\u3059\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d666037434c4406d0a10c\/7-0.png\" alt=\"1588474181611.png\" \/><\/div>\n<p>\u9069\u5f53\u306b\u8aac\u660e\u3092\u3044\u308c\u307e\u3059\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d666037434c4406d0a10c\/9-0.png\" alt=\"1588474211704.png\" \/><\/div>\n<p>Token \u304c\u8868\u793a\u3055\u308c\u308b\u306e\u3067\u3001\u30e1\u30e2\u3063\u3066\u304a\u304d\u307e\u3059\u3002(\u753b\u50cf\u306e\u3082\u306e\u306f\u73fe\u5728\u4f7f\u3048\u307e\u305b\u3093)<\/p>\n<p>Token\u306e\u4e2d\u306b\u3001; \u306a\u3069\u306e\u8a18\u53f7\u304c\u542b\u307e\u308c\u3066\u3044\u308b\u3068\u3001\u6b63\u3057\u304f\u52d5\u304b\u306a\u3044\u6c17\u304c\u3057\u307e\u3059\u3002\u3082\u3057\u30cf\u30de\u3063\u3066\u6b63\u5e38\u306b\u52d5\u304b\u306a\u3044\u6642\u306f\u3001Token\u306e\u5165\u308c\u66ff\u3048\u3082\u691c\u8a0e\u3057\u3066\u307f\u3066\u304f\u3060\u3055\u3044\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d666037434c4406d0a10c\/12-0.png\" alt=\"1588474230416.png\" \/><\/div>\n<h1>Stream \u306e\u4f5c\u6210<\/h1>\n<p>OCI Console \u3067\u3001Analytics &gt; Streaming \u3092\u9078\u629e\u3057\u307e\u3059\u3002\u305d\u306e\u5f8c\u3001Create Stream \u3092\u9078\u3073\u307e\u3059\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d666037434c4406d0a10c\/15-0.png\" alt=\"1588519801919.png\" \/><\/div>\n<p>\u9069\u5f53\u306b\u30d1\u30e9\u30e1\u30fc\u30bf\u3092\u5165\u308c\u3066\u4f5c\u6210\u3057\u307e\u3059<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d666037434c4406d0a10c\/17-0.png\" alt=\"1588519908911.png\" \/><\/div>\n<p>Stream \u3092\u4f5c\u3063\u305f\u3068\u304d\u306b\u3001Stream Pool \u304c\u4f55\u3082\u306a\u3044\u5834\u5408\u306f\u3001Default \u306e Stream Pool \u304c\u81ea\u52d5\u7684\u306b\u4f5c\u6210\u3055\u308c\u307e\u3059\u3002teststream01\u306f\u3001DefaultPool \u306b\u81ea\u52d5\u914d\u7f6e\u3055\u308c\u307e\u3059\u3002Stream Pool \u306f\u3001\u8907\u6570\u306eStream\u3092\u7ba1\u7406\u3059\u308b\u6982\u5ff5\u3068\u306a\u3063\u3066\u3044\u3066\u3001Stream \u3078\u306e Endpoint \u3092 Public \u306b\u3059\u308b\u306e\u304b Private \u306b\u3059\u308b\u306e\u304b\u3001\u307e\u305f\u30c7\u30fc\u30bf\u306e\u6697\u53f7\u306b\u4f7f\u3046\u9375\u306f\u4f55\u3092\u4f7f\u7528\u3059\u308b\u306e\u304b\u3001\u3068\u3044\u3063\u305f\u4e8b\u3092\u307e\u3068\u3081\u3066\u7ba1\u7406\u3059\u308b\u305f\u3081\u306e\u6982\u5ff5\u3067\u3059\u3002<\/p>\n<p>\u4eca\u56de\u306f\u81ea\u52d5\u4f5c\u6210\u306b\u3057\u305f\u306e\u3067\u3001Endpoint \u306f Public \u306b\u306a\u3063\u3066\u3044\u3066\u3001\u6697\u53f7\u5316\u7528\u306e\u9375\u306f Oracle \u5074\u3067\u7ba1\u7406\u3055\u308c\u3066\u3044\u308b\u3082\u306e\u3092\u4f7f\u3046\u8a2d\u5b9a\u3067\u3059\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d666037434c4406d0a10c\/20-0.png\" alt=\"1588520017422.png\" \/><\/div>\n<p>\u4e0a\u8a18\u306e\u753b\u9762\u306b\u3042\u308b\u300cView Kafka Connection Settings\u300d\u3092\u62bc\u3057\u307e\u3059\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d666037434c4406d0a10c\/22-0.png\" alt=\"1588521024252.png\" \/><\/div>\n<p>Copy All \u3092\u62bc\u3057\u3066\u3001\u3059\u3079\u3066\u306e\u8a2d\u5b9a\u5024\u3092\u30e1\u30e2\u3063\u3066\u304a\u304d\u307e\u3059\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d666037434c4406d0a10c\/24-0.png\" alt=\"1588521056591.png\" \/><\/div>\n<h1>Kafka Connect Configration \u3092\u4f5c\u6210<\/h1>\n<p>Kafka Connector \u3092\u4f7f\u3063\u3066Streaming\u306b\u63a5\u7d9a\u3059\u308b\u305f\u3081\u306b\u3001Kafka Connect Configration \u3092\u4f5c\u6210\u3057\u307e\u3059\u3002\u3053\u308c\u3092\u4f5c\u6210\u3059\u308b\u3068\u3001Streaming \u5074\u3067 Kafka \u7528\u8a9e\u3067\u3044\u3046\u3068\u3053\u308d\u306e 3\u7a2e\u985e\u306eTopic \u304c\u4f5c\u6210\u3055\u308c\u307e\u3059\u3002\u8907\u6570\u306e Connector \u304c\u8ca0\u8377\u5206\u6563\u3057\u3066\u52d5\u4f5c\u3059\u308b\u305f\u3081\u306b\u306f\u3001\u4e92\u3044\u306b\u3069\u3046\u3044\u3046\u30b9\u30c6\u30fc\u30bf\u30b9\u3067\u3001\u3069\u3053\u307e\u3067\u306e\u30c7\u30fc\u30bf\u3092\u51e6\u7406\u3057\u3066\u3001\u3068\u3044\u3063\u305f\u60c5\u5831\u5171\u6709\u304c\u5fc5\u8981\u3067\u3059\u3002\u60c5\u5831\u5171\u6709\u3059\u308b\u305f\u3081\u306b\u30013\u7a2e\u985e\u306eTopic\u4e0a\u3067\u3084\u308a\u53d6\u308a\u3092\u884c\u3063\u3066\u3044\u307e\u3059\u3002<\/p>\n<p>\u305d\u308c\u3067\u306f\u3001Create \u30dc\u30bf\u30f3\u3092\u304a\u3057\u307e\u3059\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d666037434c4406d0a10c\/28-0.png\" alt=\"1588521439761.png\" \/><\/div>\n<p>\u540d\u524d\u3092\u9069\u5f53\u306b\u3044\u308c\u307e\u3059\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d666037434c4406d0a10c\/30-0.png\" alt=\"1588521472264.png\" \/><\/div>\n<p>\u8a73\u7d30\u753b\u9762\u304c\u81ea\u52d5\u7684\u306b\u8868\u793a\u3055\u308c\u307e\u3059\u3002Copy \u3092\u62bc\u3057\u3066\u30e1\u30e2\u3063\u3066\u304a\u304d\u307e\u3059\u3002<\/p>\n<div><img decoding=\"async\" class=\"post-images\" title=\"\" src=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d666037434c4406d0a10c\/32-0.png\" alt=\"1588521505413.png\" \/><\/div>\n<h1>Connector \u7528\u4eee\u60f3\u30de\u30b7\u30f3\u306e\u4f5c\u6210<\/h1>\n<p>\u4eca\u56de\u306e\u624b\u9806\u3067\u306f\u3001\u9069\u5f53\u306b CentOS7 \u306e\u4eee\u60f3\u30de\u30b7\u30f3\u3092\u4f5c\u6210\u3057\u307e\u3059\u3002<\/p>\n<h1>Open JDK 8 \u306e Install<\/h1>\n<p>\u4f5c\u6210\u3057\u305f CentOS7 \u3067 Kafka Connector \u3092\u52d5\u304b\u3059\u305f\u3081\u306b\u3001Open JDK 1.8 \u3092 Install \u3057\u307e\u3059\u3002\u597d\u304d\u306a JDK \u3067\u554f\u984c\u306a\u3044\u3068\u601d\u3044\u307e\u3059\u3002<\/p>\n<pre class=\"post-pre\"><code>sudo yum install -y java-1.8.0-openjdk java-1.8.0-openjdk-devel\r\n<\/code><\/pre>\n<h1>Kafka Download<\/h1>\n<p>Kafka Connector \u3092\u52d5\u304b\u3059\u305f\u3081\u306e\u30c4\u30fc\u30eb\u304c\u3001Kafka \u672c\u4f53\u306e tar gz ball \u306b\u542b\u307e\u308c\u3066\u3044\u308b\u306e\u3067\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u3057\u307e\u3059\u3002\u4ee5\u4e0b\u306e URL \u304b\u3089\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u3067\u304d\u307e\u3059\u3002<\/p>\n<pre class=\"post-pre\"><code>mkdir ~\/kafka\r\ncd ~\/kafka\r\nwget https:\/\/ftp.tsukuba.wide.ad.jp\/software\/apache\/kafka\/2.5.0\/kafka_2.12-2.5.0.tgz\r\n<\/code><\/pre>\n<p>\u89e3\u51cd<\/p>\n<pre class=\"post-pre\"><code>tar xfvz kafka_2.12-2.5.0.tgz\r\n<\/code><\/pre>\n<p>\u4f5c\u696d\u3057\u3084\u3059\u3044\u305f\u3081\u306b\u3001Kafka Home \u3092\u74b0\u5883\u5909\u6570\u306b\u8a2d\u5b9a<\/p>\n<pre class=\"post-pre\"><code>echo 'export KAFKA_HOME=$HOME\/kafka\/kafka_2.12-2.5.0\/' &gt;&gt; ~\/.bashrc\r\n<\/code><\/pre>\n<p>bachrc \u8aad\u307f\u8fbc\u307f<\/p>\n<pre class=\"post-pre\"><code>source ~\/.bashrc\r\n<\/code><\/pre>\n<p>Kafka \u306e targz ball \u306b\u542b\u307e\u308c\u3066\u3044\u308b\u30e9\u30a4\u30d6\u30e9\u30ea\u3092\u78ba\u8a8d\u3057\u307e\u3059\u3002<\/p>\n<pre class=\"post-pre\"><code>ls -la $KAFKA_HOME\/libs | grep -i connect\r\n<\/code><\/pre>\n<p>\u3044\u304f\u3064\u304b\u306e connector \u304c Kafka \u306b\u542b\u307e\u308c\u3066\u3044\u307e\u3059\u3002<\/p>\n<pre class=\"post-pre\"><code>[opc@kafka01 libs]$ ls -la $KAFKA_HOME\/libs | grep -i connect\r\n-rw-r--r--. 1 opc opc   101941 Apr  8 01:14 connect-api-2.5.0.jar\r\n-rw-r--r--. 1 opc opc    18607 Apr  8 01:16 connect-basic-auth-extension-2.5.0.jar\r\n-rw-r--r--. 1 opc opc    20506 Apr  8 01:16 connect-file-2.5.0.jar\r\n-rw-r--r--. 1 opc opc    45199 Apr  8 01:14 connect-json-2.5.0.jar\r\n-rw-r--r--. 1 opc opc    90298 Apr  8 01:16 connect-mirror-2.5.0.jar\r\n-rw-r--r--. 1 opc opc    29086 Apr  8 01:16 connect-mirror-client-2.5.0.jar\r\n-rw-r--r--. 1 opc opc   557008 Apr  8 01:16 connect-runtime-2.5.0.jar\r\n-rw-r--r--. 1 opc opc    91639 Apr  8 01:16 connect-transforms-2.5.0.jar\r\n[opc@kafka01 libs]$\r\n<\/code><\/pre>\n<h1>Kafka Connect \u7528\u8a2d\u5b9a<\/h1>\n<p>Kafka Connector \u7528\u306b\u5fc5\u8981\u306a\u8a2d\u5b9a\u30d5\u30a1\u30a4\u30eb\u3092\u4f5c\u6210\u3057\u307e\u3059\u3002\u305d\u308c\u305e\u308c\u306e\u30d1\u30e9\u30e1\u30fc\u30bf\u306f\u3001\u74b0\u5883\u306b\u5408\u308f\u305b\u3066\u9069\u5b9c\u5909\u66f4\u3057\u3066\u304f\u3060\u3055\u3044\u3002<\/p>\n<p>group.id : \u4efb\u610f\u306e\u540d\u524d\u3002\u8907\u6570\u306e Connect Worker \u3092\u52d5\u304b\u3059\u6642\u306f\u3001\u3069\u3046\u3044\u3046\u306e\u540d\u524d\u3092\u6307\u5b9a\u3059\u308b<\/p>\n<p>bootstrap.servers : OCI \u306e Stream Pool \u304b\u3089\u30b3\u30d4\u30fc\u3057\u305f\u5024\u3092\u6307\u5b9a<\/p>\n<p>sasl.jaas.config : OCI \u306e Stream Pool \u304b\u3089\u30b3\u30d4\u30fc\u3057\u305f\u5024\u3092\u6307\u5b9a\u3002password=&#8221;&#8221; \u306e\u90e8\u5206\u306f\u3001Auth Token \u306e\u6587\u5b57\u5217\u3092\u6307\u5b9a<\/p>\n<p>producer.sasl.jaas.config : sasl.jaas.config \u3068\u540c\u3058\u3082\u306e\u3092\u6307\u5b9a<\/p>\n<p>consumer.sasl.jaas.config : sasl.jaas.config \u3068\u540c\u3058\u3082\u306e\u3092\u6307\u5b9a<\/p>\n<p>config.storage.topic : Kafka Connect Configration \u304b\u3089\u30b3\u30d4\u30fc\u3057\u305f\u5024\u3092\u6307\u5b9a<\/p>\n<p>status.storage.topic : Kafka Connect Configration \u304b\u3089\u30b3\u30d4\u30fc\u3057\u305f\u5024\u3092\u6307\u5b9a<\/p>\n<p>offset.storage.topic : Kafka Connect Configration \u304b\u3089\u30b3\u30d4\u30fc\u3057\u305f\u5024\u3092\u6307\u5b9a<\/p>\n<pre class=\"post-pre\"><code>cat &lt;&lt;'EOF' &gt; $KAFKA_HOME\/config\/connect-distributed-demo.properties\r\ngroup.id=connect-demo-group\r\n\r\nbootstrap.servers=cell-1.streaming.ap-tokyo-1.oci.oraclecloud.com:9092\r\nsasl.mechanism=PLAIN\r\nsecurity.protocol=SASL_SSL\r\nsasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username=\"poc02\/oracleidentitycloudservice\/suguru.sugiyama@oracle.com\/ocid1.streampool.oc1.ap-tokyo-1.amaaaaaaycetm7yawtz56lnnerap4r45y4vheekgvhdaevxf3clfpuew6mla\" password=\"8t[shwUN}I-d+{}8Nx_a\";\r\n\r\nproducer.sasl.mechanism=PLAIN\r\nproducer.security.protocol=SASL_SSL\r\nproducer.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username=\"poc02\/oracleidentitycloudservice\/suguru.sugiyama@oracle.com\/ocid1.streampool.oc1.ap-tokyo-1.amaaaaaaycetm7yawtz56lnnerap4r45y4vheekgvhdaevxf3clfpuew6mla\" password=\"8t[shwUN}I-d+{}8Nx_a\";\r\n\r\nconsumer.sasl.mechanism=PLAIN\r\nconsumer.security.protocol=SASL_SSL\r\nconsumer.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username=\"poc02\/oracleidentitycloudservice\/suguru.sugiyama@oracle.com\/ocid1.streampool.oc1.ap-tokyo-1.amaaaaaaycetm7yawtz56lnnerap4r45y4vheekgvhdaevxf3clfpuew6mla\" password=\"8t[shwUN}I-d+{}8Nx_a\";\r\n\r\nconfig.storage.replication.factor=1\r\nconfig.storage.partitions=1\r\nconfig.storage.topic=ocid1.connectharness.oc1.ap-tokyo-1.amaaaaaaycetm7yapmslajk3scdwvngqw47c2iixkqltnb7zllk7ca3t33va-config\r\n\r\noffset.storage.replication.factor=1\r\noffset.storage.partitions=1\r\noffset.storage.topic=ocid1.connectharness.oc1.ap-tokyo-1.amaaaaaaycetm7yapmslajk3scdwvngqw47c2iixkqltnb7zllk7ca3t33va-offset\r\noffset.flush.interval.ms=10000\r\noffset.flush.timeout.ms=5000\r\n\r\nstatus.storage.replication.factor=1\r\nstatus.storage.partitions=1\r\nstatus.storage.topic=ocid1.connectharness.oc1.ap-tokyo-1.amaaaaaaycetm7yapmslajk3scdwvngqw47c2iixkqltnb7zllk7ca3t33va-status\r\n\r\nkey.converter=org.apache.kafka.connect.storage.StringConverter\r\nvalue.converter=org.apache.kafka.connect.json.JsonConverter\r\nkey.converter.schemas.enable=false\r\nvalue.converter.schemas.enable=true\r\n\r\ninternal.key.converter=org.apache.kafka.connect.json.JsonConverter\r\ninternal.value.converter=org.apache.kafka.connect.json.JsonConverter\r\n\r\ntask.shutdown.graceful.timeout.ms=10000\r\nEOF\r\n<\/code><\/pre>\n<p>Kafka Connect \u3092\u5b9f\u884c<\/p>\n<pre class=\"post-pre\"><code>$KAFKA_HOME\/bin\/connect-distributed.sh $KAFKA_HOME\/config\/connect-distributed-demo.properties\r\n<\/code><\/pre>\n<p>\u5b9f\u884c\u4f8b<br \/>\n\u305f\u304f\u3055\u3093\u306e\u6587\u5b57\u304c\u6d41\u308c\u3066\u884c\u3063\u3066\u3001\u672b\u5c3e\u306b\u3053\u3093\u306a\u6587\u5b57\u304c\u8868\u793a\u3055\u308c\u308c\u3070\u6b63\u5e38\u72b6\u614b\u3067\u3059\u3002<\/p>\n<pre class=\"post-pre\"><code>[2020-05-03 03:07:46,369] INFO [Worker clientId=connect-1, groupId=connect-demo-group] Finished reading to end of log and updated config snapshot, new config log offset: 886603188224 (org.apache.kafka.connect.runtime.distributed.DistributedHerder:1085)[2020-05-03 03:07:46,369] INFO [Worker clientId=connect-1, groupId=connect-demo-group] Starting connectors and tasks using config offset 886603188224 (org.apache.kafka.connect.runtime.distributed.DistributedHerder:1111)\r\n[2020-05-03 03:07:46,369] INFO [Worker clientId=connect-1, groupId=connect-demo-group] Finished starting connectors and tasks (org.apache.kafka.connect.runtime.distributed.DistributedHerder:1132)[2020-05-03 03:07:47,378] INFO [Worker clientId=connect-1, groupId=connect-demo-group] Session key updated (org.apache.kafka.connect.runtime.distributed.DistributedHerder:1447)\r\n<\/code><\/pre>\n<h1>Source Connector \u306e\u8a2d\u5b9a<\/h1>\n<p>\u4eca\u56de\u306e\u8a18\u4e8b\u3067\u306f\u3001File Connector \u3092\u4f7f\u7528\u3057\u3066\u52d5\u4f5c\u78ba\u8a8d\u3092\u3057\u307e\u3059\u3002\u52d5\u4f5c\u78ba\u8a8d\u306b\u4f7f\u3046 Input\u30d5\u30a1\u30a4\u30eb\u30fbOutput\u30d5\u30a1\u30a4\u30eb\u3092\u4f5c\u6210\u3057\u307e\u3059\u3002Input \u30d5\u30a1\u30a4\u30eb\u306f\u30011\u304b\u30891000\u307e\u3067\u306e\u6587\u5b57\u30921\u884c\u3054\u3068\u306b\u304b\u3044\u305f\u30d5\u30a1\u30a4\u30eb\u3067\u3059\u3002output.txt \u306f touch \u3067\u4f5c\u3063\u3066\u3044\u308b\u3060\u3051\u306a\u306e\u3067\u3001\u4e2d\u8eab\u306f\u7a7a\u3063\u307d\u3067\u3059\u3002<\/p>\n<pre class=\"post-pre\"><code>mkdir ~\/connect-quickstart\r\nseq 1000 &gt; ~\/connect-quickstart\/input.txt\r\ntouch ~\/connect-quickstart\/output.txt\r\n<\/code><\/pre>\n<p>\u30b3\u30cd\u30af\u30bf\u3092\u4f5c\u6210\u3059\u308b\u307e\u3048\u306b\u3001Connect Worker(\u4eee\u60f3\u30de\u30b7\u30f3) \u306e\u4e2d\u3067\u5b9a\u7fa9\u3055\u308c\u3066\u3044\u308b Connector \u4e00\u89a7\u3092\u8868\u793a\u3057\u307e\u3059\u3002\u4eca\u306f\u306a\u306b\u3082\u3057\u3066\u3044\u306a\u3044\u306e\u3067\u3001\u7a7a\u767d\u304c\u8fd4\u3063\u3066\u304d\u307e\u3059\u3002<\/p>\n<pre class=\"post-pre\"><code>[opc@kafka01 connect-quickstart]$ curl -s http:\/\/localhost:8083\/connectors | jq .\r\n[]\r\n[opc@kafka01 connect-quickstart]$\r\n<\/code><\/pre>\n<p>Connector \u3092\u8a2d\u5b9a\u3057\u307e\u3059\u3002<\/p>\n<p>topic : OCI Stream \u306e\u540d\u524d\u3092\u6307\u5b9a<\/p>\n<p>file : Input \u3068\u3057\u3066\u4f7f\u3046\u30d5\u30a1\u30a4\u30eb\u306ePath\u3092\u6307\u5b9a<\/p>\n<pre class=\"post-pre\"><code>curl -s -X POST -H \"Content-Type: application\/json\" --data '{\"name\": \"file-source\",\"config\": {\"connector.class\":\"org.apache.kafka.connect.file.FileStreamSourceConnector\",\"tasks.max\":\"1\",\"topic\":\"teststream01\",\"file\": \"\/home\/opc\/connect-quickstart\/input.txt\"}}' http:\/\/localhost:8083\/connectors\r\n<\/code><\/pre>\n<p>list \u3092\u78ba\u8a8d\u3057\u307e\u3059\u3002<\/p>\n<pre class=\"post-pre\"><code>[opc@kafka01 connect-quickstart]$ curl -s http:\/\/localhost:8083\/connectors | jq .\r\n[\r\n  \"file-source\"\r\n]\r\n[opc@kafka01 connect-quickstart]$\r\n<\/code><\/pre>\n<p>\u8a73\u7d30\u60c5\u5831\u3092\u78ba\u8a8d\u3057\u307e\u3059<\/p>\n<pre class=\"post-pre\"><code>[opc@kafka01 connect-quickstart]$ curl -s http:\/\/localhost:8083\/connectors\/file-source\/status | jq .\r\n{\r\n  \"name\": \"file-source\",\r\n  \"connector\": {\r\n    \"state\": \"RUNNING\",\r\n    \"worker_id\": \"10.0.0.14:8083\"\r\n  },\r\n  \"tasks\": [\r\n    {\r\n      \"id\": 0,\r\n      \"state\": \"RUNNING\",\r\n      \"worker_id\": \"10.0.0.14:8083\"\r\n    }\r\n  ],\r\n  \"type\": \"source\"\r\n}\r\n<\/code><\/pre>\n<p>memo : Delete\u30b3\u30de\u30f3\u30c9<\/p>\n<pre class=\"post-pre\"><code>curl -s -X DELETE http:\/\/localhost:8083\/connectors\/file-source | jq .\r\n<\/code><\/pre>\n<h1>Sink Connector \u306e\u8a2d\u5b9a<\/h1>\n<p>Connector \u306e Output(Sink) \u3068\u3057\u3066\u306e\u8a2d\u5b9a\u3092\u3057\u307e\u3059\u3002\u3053\u3061\u3089\u3082 Input \u3068\u540c\u69d8\u306b File Connector \u3092\u4f7f\u3044\u307e\u3059\u3002<\/p>\n<pre class=\"post-pre\"><code>curl -X POST -H \"Content-Type: application\/json\" --data '{\"name\": \"file-sink\", \"config\": {\"connector.class\":\"org.apache.kafka.connect.file.FileStreamSinkConnector\", \"tasks.max\":\"1\", \"topics\":\"teststream01\", \"file\": \"\/home\/opc\/connect-quickstart\/output.txt\"}}' http:\/\/localhost:8083\/connectors\r\n<\/code><\/pre>\n<p>file\u304c\u4f5c\u6210\u3055\u308c\u3066\u3044\u307e\u3059\u3002<\/p>\n<pre class=\"post-pre\"><code>[opc@kafka01 connect-quickstart]$ head -n 10 output.txt\r\n1\r\n2\r\n3\r\n4\r\n5\r\n6\r\n7\r\n8\r\n9\r\n10\r\n[opc@kafka01 connect-quickstart]$\r\n<\/code><\/pre>\n<h1>\u691c\u8a3c\u3057\u3066\u308f\u304b\u3063\u305f\u3053\u3068<\/h1>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">Streaming \u306e Stream \u306e\u540d\u524d\u304c\u3001Kafka\u306eTopicName\u306b\u5bfe\u5fdc\u3057\u3066\u3044\u308b<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\u4e00\u5ea6\u30b3\u30cd\u30af\u30bf\u30fc\u3092\u4f5c\u6210\u3057\u305f\u5f8c\u306b\u3001input file \u3092\u66f4\u65b0\u3057\u3066\u3082 output \u306b\u306f\u6d41\u308c\u306a\u3044<\/ul>\n<h1>\u53c2\u8003URL<\/h1>\n<p>Kafka Docs<br \/>\nhttps:\/\/kafka.apache.org\/documentation\/<\/p>\n<p>OCI Docs<br \/>\nhttps:\/\/docs.cloud.oracle.com\/en-us\/iaas\/Content\/Streaming\/Tasks\/kafkacompatibility.htm<\/p>\n<p>Oracle Blogs<br \/>\nhttps:\/\/blogs.oracle.com\/developers\/using-kafka-connect-with-oracle-streaming-service-and-autonomous-db<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u306f\u3058\u3081\u306b Oracle Cloud Infractructure(\u4ee5\u4e0bOCI)\u3067\u306f\u3001\u30b9\u30c8\u30ea\u30fc\u30df\u30f3\u30b0\u30c7\u30fc\u30bf\u3092\u30ea\u30a2 [&hellip;]<\/p>\n","protected":false},"author":12,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-46745","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.5 (Yoast SEO v21.5) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>- Blog - Silicon Cloud<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.silicloud.com\/zh\/blog\/46745-2\/\" \/>\n<meta property=\"og:locale\" content=\"zh_CN\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:description\" content=\"\u306f\u3058\u3081\u306b Oracle Cloud Infractructure(\u4ee5\u4e0bOCI)\u3067\u306f\u3001\u30b9\u30c8\u30ea\u30fc\u30df\u30f3\u30b0\u30c7\u30fc\u30bf\u3092\u30ea\u30a2 [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.silicloud.com\/zh\/blog\/46745-2\/\" \/>\n<meta property=\"og:site_name\" content=\"Blog - Silicon Cloud\" \/>\n<meta property=\"article:published_time\" content=\"2023-08-21T22:51:58+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-04-30T06:08:23+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d666037434c4406d0a10c\/7-0.png\" \/>\n<meta name=\"author\" content=\"\u9038, \u79d1\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"\u4f5c\u8005\" \/>\n\t<meta name=\"twitter:data1\" content=\"\u9038, \u79d1\" \/>\n\t<meta name=\"twitter:label2\" content=\"\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 \u5206\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/46745-2\/\",\"url\":\"https:\/\/www.silicloud.com\/zh\/blog\/46745-2\/\",\"name\":\"- Blog - Silicon Cloud\",\"isPartOf\":{\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#website\"},\"datePublished\":\"2023-08-21T22:51:58+00:00\",\"dateModified\":\"2024-04-30T06:08:23+00:00\",\"author\":{\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/85c1dae56e6ea1e695c73d33c684d487\"},\"inLanguage\":\"zh-Hans\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.silicloud.com\/zh\/blog\/46745-2\/\"]}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#website\",\"url\":\"https:\/\/www.silicloud.com\/zh\/blog\/\",\"name\":\"Blog - Silicon Cloud\",\"description\":\"\",\"inLanguage\":\"zh-Hans\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/85c1dae56e6ea1e695c73d33c684d487\",\"name\":\"\u9038, \u79d1\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"zh-Hans\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/c94f6d9cbbfbca863fab309840bd690c153c95f8490c290ad2ed54dd693dad16?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/c94f6d9cbbfbca863fab309840bd690c153c95f8490c290ad2ed54dd693dad16?s=96&d=mm&r=g\",\"caption\":\"\u9038, \u79d1\"},\"url\":\"https:\/\/www.silicloud.com\/zh\/blog\/author\/keyi\/\"},{\"@type\":\"ImageObject\",\"inLanguage\":\"zh-Hans\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/46745-2\/#local-main-organization-logo\",\"url\":\"\",\"contentUrl\":\"\",\"caption\":\"Blog - Silicon Cloud\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"- Blog - Silicon Cloud","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.silicloud.com\/zh\/blog\/46745-2\/","og_locale":"zh_CN","og_type":"article","og_description":"\u306f\u3058\u3081\u306b Oracle Cloud Infractructure(\u4ee5\u4e0bOCI)\u3067\u306f\u3001\u30b9\u30c8\u30ea\u30fc\u30df\u30f3\u30b0\u30c7\u30fc\u30bf\u3092\u30ea\u30a2 [&hellip;]","og_url":"https:\/\/www.silicloud.com\/zh\/blog\/46745-2\/","og_site_name":"Blog - Silicon Cloud","article_published_time":"2023-08-21T22:51:58+00:00","article_modified_time":"2024-04-30T06:08:23+00:00","og_image":[{"url":"https:\/\/cdn.silicloud.com\/blog-img\/blog\/img\/657d666037434c4406d0a10c\/7-0.png"}],"author":"\u9038, \u79d1","twitter_card":"summary_large_image","twitter_misc":{"\u4f5c\u8005":"\u9038, \u79d1","\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4":"4 \u5206"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.silicloud.com\/zh\/blog\/46745-2\/","url":"https:\/\/www.silicloud.com\/zh\/blog\/46745-2\/","name":"- Blog - Silicon Cloud","isPartOf":{"@id":"https:\/\/www.silicloud.com\/zh\/blog\/#website"},"datePublished":"2023-08-21T22:51:58+00:00","dateModified":"2024-04-30T06:08:23+00:00","author":{"@id":"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/85c1dae56e6ea1e695c73d33c684d487"},"inLanguage":"zh-Hans","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.silicloud.com\/zh\/blog\/46745-2\/"]}]},{"@type":"WebSite","@id":"https:\/\/www.silicloud.com\/zh\/blog\/#website","url":"https:\/\/www.silicloud.com\/zh\/blog\/","name":"Blog - Silicon Cloud","description":"","inLanguage":"zh-Hans"},{"@type":"Person","@id":"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/85c1dae56e6ea1e695c73d33c684d487","name":"\u9038, \u79d1","image":{"@type":"ImageObject","inLanguage":"zh-Hans","@id":"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/c94f6d9cbbfbca863fab309840bd690c153c95f8490c290ad2ed54dd693dad16?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/c94f6d9cbbfbca863fab309840bd690c153c95f8490c290ad2ed54dd693dad16?s=96&d=mm&r=g","caption":"\u9038, \u79d1"},"url":"https:\/\/www.silicloud.com\/zh\/blog\/author\/keyi\/"},{"@type":"ImageObject","inLanguage":"zh-Hans","@id":"https:\/\/www.silicloud.com\/zh\/blog\/46745-2\/#local-main-organization-logo","url":"","contentUrl":"","caption":"Blog - Silicon Cloud"}]}},"_links":{"self":[{"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/posts\/46745","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/comments?post=46745"}],"version-history":[{"count":2,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/posts\/46745\/revisions"}],"predecessor-version":[{"id":92367,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/posts\/46745\/revisions\/92367"}],"wp:attachment":[{"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/media?parent=46745"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/categories?post=46745"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/tags?post=46745"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}