{"id":46939,"date":"2023-12-06T03:32:05","date_gmt":"2023-10-01T07:00:29","guid":{"rendered":"https:\/\/www.silicloud.com\/zh\/blog\/%e5%8f%a6%e4%b8%80%e4%b8%aahadoop%e5%b3%b0%e4%bc%9a%e5%8f%82%e5%8a%a0%e6%8a%a5%e5%91%8a\/"},"modified":"2024-05-04T12:18:03","modified_gmt":"2024-05-04T04:18:03","slug":"%e5%8f%a6%e4%b8%80%e4%b8%aahadoop%e5%b3%b0%e4%bc%9a%e5%8f%82%e5%8a%a0%e6%8a%a5%e5%91%8a","status":"publish","type":"post","link":"https:\/\/www.silicloud.com\/zh\/blog\/%e5%8f%a6%e4%b8%80%e4%b8%aahadoop%e5%b3%b0%e4%bc%9a%e5%8f%82%e5%8a%a0%e6%8a%a5%e5%91%8a\/","title":{"rendered":"\u53e6\u4e00\u4e2aHadoop\u5cf0\u4f1a\u53c2\u52a0\u62a5\u544a"},"content":{"rendered":"<h2>\u8fd9\u7bc7\u6587\u7ae0\u662f\u5173\u4e8e\u4ec0\u4e48\u7684\uff1f<\/h2>\n<p>\u8fd9\u7bc7\u6587\u7ae0\u662f2016\u5e74Hortonworks Advent Calendar\u7684\u7b2c17\u7bc7\u6587\u7ae0\u3002<\/p>\n<h2>\u8fd9\u7bc7\u6587\u7ae0\u8bb2\u4e86\u4ec0\u4e48\uff1f<\/h2>\n<p>\u8fd9\u662f\u53e6\u4e00\u4e2aHadoop\u5cf0\u4f1a\u7684\u53c2\u52a0\u62a5\u544a\u3002<br \/>\n\u5305\u62ec\u6bcf\u4e2a\u6f14\u8bb2\u7684\u6458\u8981\uff08\u4f46\u5b9e\u9645\u4e0a\u6211\u4eec\u4e5f\u6709\u6f14\u8bb2\u8d44\u6599\uff0c\u6240\u4ee5\u8fd9\u53ea\u662f\u8865\u5145\u8bf4\u660e\uff09\u548c\u6211\u7684\u611f\u60f3\u3002<\/p>\n<h3>\u63a8\u52a8\u6570\u636e\u5e94\u7528\u7684\u201cPivotal HDB\uff08Apache HAWQ\uff09\u201d<\/h3>\n<p>\u9a6c\u4e4b\u5185\u6b63\u5e78\uff08Masayuki Matsushita\uff09\u63a8\u52a8\u6570\u636e\u5229\u7528\u7684\u201cPivotal HDB\uff08Apache HAWQ\uff08\u9e70\uff09\uff09\u201d\u6765\u8fdb\u884c\u6f14\u793a\u3002<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">SQL on Hadoop\u306e\u4e00\u7a2e\u3000HDFS\u4e0a\u306e\u9ad8\u901f\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u30a8\u30f3\u30b8\u30f3<\/ul>\n<\/li>\n<\/ul>\n<p>HDFS\u306b\u5bfe\u3059\u308b\u6a19\u6e96SQL\u306b\u3088\u308b\u9ad8\u901f\u30af\u30a8\u30ea\u51e6\u7406<br \/>\nHive\u3001HBase\u3001Avro\u306a\u3069Hadoop\u30c7\u30fc\u30bf\u3068\u306e\u9023\u643a<br \/>\n\u30c7\u30fc\u30bf\u30ed\u30fc\u30ab\u30ea\u30c6\u30a3\/\u30b7\u30e7\u30fc\u30c8\u30b5\u30fc\u30ad\u30c3\u30c8\u30ea\u30fc\u30c9\u5bfe\u5fdc<br \/>\n\u5404\u7a2e\u7d71\u8a08\u89e3\u6790\u95a2\u6570\u306b\u5bfe\u5fdc<\/p>\n<p>\u5143\u3005\u306fPivotal HDB\u3060\u3063\u305f\u3082\u306e\u3092OSS\u5316\u3057\u3001Apache HAWQ\u3068\u3044\u3046\u5f62\u3067\u516c\u958b\u3057\u305f\u3082\u306e<\/p>\n<p>MapReduce\u3084Hive\u3067\u51e6\u7406\u3092\u3057\u3066\u3044\u305f\u304c\u3001GPDB\u3067\u4e26\u5217\u5206\u6563DB\u304c\u3042\u308b<br \/>\n\u4e26\u5217\u5206\u6563DB\u306e\u30d5\u30a1\u30a4\u30eb\u3092HDFS\u4e0a\u306b\u914d\u7f6e\u53ef\u80fd\u3068\u3059\u308b\u3088\u3046\u306b\u3057\u3066\u958b\u767a\u3057\u305f\u306e\u304cHAWQ<\/p>\n<p>\u4ed6\u30d7\u30ed\u30c0\u30af\u30c8\u3068\u306e\u89aa\u548c\u6027<\/p>\n<p>Ambari\u304b\u3089\u65e2\u306b\u7d71\u5408\u7ba1\u7406\u304c\u53ef\u80fd<br \/>\nYARN\u4e0a\u3067\u30d7\u30ed\u30bb\u30b9\u3092\u5b9f\u884c\u53ef\u80fd<\/p>\n<p>TPC-DS Performance Review \u2013\u202f \u6bd4\u8f03: Pivotal HDB2.0 vs. Cloudera Impala 2.5<\/p>\n<p>\u3053\u3061\u3089\u306b\u3064\u3044\u3066\u306f\u516c\u958b\u306a\u3057\u3002<br \/>\n\u5b9f\u969b\u306e\u3068\u3053\u308d\u3053\u308c\u306f\u5fae\u5999\u306a\u5185\u5bb9\u3067\u3059\u306e\u3067\u3001\u8a73\u7d30\u306f\u611f\u60f3\u90e8\u5206\u306b\u3002<\/p>\n<p>\u3069\u306e\u3088\u3046\u306a\u6a5f\u80fd\u304c\u3042\u308b\u304b\uff1f<\/p>\n<p>\u30de\u30b9\u30bf\u30fc\uff0f\u30b9\u30ec\u30fc\u30d6\u65b9\u5f0f\u306e\u69cb\u9020<br \/>\n\u30de\u30b9\u30bf\u306f\u5197\u9577\u5316<\/p>\n<p>\u4f55\u6545\u65e9\u3044\u306e\u304b\uff1f<\/p>\n<p>\u30c7\u30fc\u30bf\u3092\u5404DataNode\u306b\u5b58\u5728\u3059\u308bHAWQ\u30bb\u30b0\u30e1\u30f3\u30c8\u30b5\u30fc\u30d0\u3067\u53d7\u3051\u53d6\u308a\u3001libhdfs3\u3067\u66f8\u304d\u8fbc\u307f\u3002<\/p>\n<p>\u826f\u304f\u4f7f\u7528\u3055\u308c\u3066\u3044\u308b\u7528\u9014\u306f\uff1f<\/p>\n<p>HDP\u3068\u4e00\u7dd2\u306b\u4f7f\u3063\u3066\u3044\u308b\u30b1\u30fc\u30b9\u304c\u591a\u3044\u3002SQL\u304c\u9ad8\u901f\u306a\u306e\u304c\u5927\u304d\u3044\u3002<\/p>\n<p>\u300cPivotal HDB2.0\u4e0eCloudera Impala 2.5\u300d\u7684\u6bd4\u8f83\u7ed3\u679c\u663e\u793a\uff0cHDB2.0\u7684\u901f\u5ea6\u662fImpala2.5\u7684\u6570\u500d\u3002\u7136\u800c\uff0cImpala2.5\u5df2\u7ecf\u662f\u8fc7\u53bb\u7684\u7248\u672c\uff0c\u5e76\u4e14\u6761\u4ef6\u4e5f\u4e0d\u660e\u786e\uff0c\u6240\u4ee5\u4e0d\u80fd\u592a\u4f9d\u8d56\u8fd9\u4e2a\u7ed3\u8bba\u3002<\/p>\n<p>\u8fd9\u662f\u4e00\u79cd\u4f7f\u7528\u6240\u8c13\u7684\u539f\u751fAPI\u6765\u9a71\u52a8\u7684SQL on Hadoop\uff0c\u901a\u8fc7HAWQ\u8fdb\u884c\u5199\u5165\u7684\u6570\u636e\u65e0\u6cd5\u4eceHadoop\u7684\u5176\u4ed6\u751f\u6001\u7cfb\u7edf\u4e2d\u8bfb\u53d6\uff08\u636e\u6211\u5728\u793e\u4ea4\u6d3b\u52a8\u4e2d\u542c\u8bf4\u7684\uff09\uff0c\u5b9e\u9645\u4e0a\u5b83\u66f4\u63a5\u8fd1\u4e8eImpala + Kudu\u7684\u5b58\u5728\u3002<\/p>\n<p>\u56e0\u6b64\uff0c\u6700\u7d42\u6211\u5e0c\u671b\u80fd\u5920\u770b\u5230Impala on Kudu\u548cApache HAWQ\u5728\u529f\u80fd\u548c\u5be6\u969b\u901f\u5ea6\u65b9\u9762\u9032\u884c\u771f\u6b63\u7684\u6bd4\u8f03&#8230;\u662f\u5426\u6709\u4efb\u4f55\u4eba\u53ef\u4ee5\u5e6b\u5fd9\u5462\uff1f\uff3e\uff3e\uff1b<\/p>\n<h3>\u5229\u7528Spark Streaming\u8fdb\u884c\u7cfb\u7edf\u9a8c\u8bc1\u7684\u7ed3\u679c\u548c\u8bbe\u8ba1\u65f6\u7684\u5b9d\u8d35\u7ecf\u9a8c\u3002<\/h3>\n<p>\u76ee\u524d\u6570\u636e\u5c1a\u672a\u516c\u5f00\u3002<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">Spark\u306f\uff1fSparkStreaming\u306f\uff1f<\/ul>\n<\/li>\n<\/ul>\n<p>\u3082\u3046\u304a\u306a\u3058\u307f\u306a\u306e\u3067\u3053\u3053\u3067\u306f\u7701\u304d\u307e\u3059\u3002<\/p>\n<p>SparkStreaming\u3067\u51fa\u6765\u308b\u3053\u3068<\/p>\n<p>Window\u30aa\u30da\u30ec\u30fc\u30b7\u30e7\u30f3<br \/>\n\u72b6\u614b\u66f4\u65b0\u30aa\u30da\u30ec\u30fc\u30b7\u30e7\u30f3\uff08\u904e\u53bb\u306e\u30c7\u30fc\u30bf\u306e\u5408\u8a08\u5024\u306b\u6700\u65b0\u306e\u30c7\u30fc\u30bf\u3092\u52a0\u7b97\uff09<\/p>\n<p>\u691c\u8a3c\u7d50\u679c\u3068\u8a2d\u8a08\u6642\u306e\u30ce\u30a6\u30cf\u30a6<\/p>\n<p>\u30e2\u30c7\u30eb\u30b7\u30b9\u30c6\u30e0<\/p>\n<p>\u904b\u52d5\u30ea\u30ba\u30e0\u306b\u5408\u308f\u305b\u305f\u97f3\u697d\u3092\u518d\u751f\u3059\u308b\u30b7\u30b9\u30c6\u30e0<\/p>\n<p>\u643a\u5e2f\u96fb\u8a71\u304b\u3089\u52a0\u901f\u5ea6\u306a\u3069\u3092\u53ce\u96c6<br \/>\n\u52d5\u4f5c\u72b6\u6cc1\u3092\u5224\u5b9a5\u79d2\u9593\u9694\u3067\u52d5\u4f5c\u72b6\u6cc1\u3092\u5224\u5b9a<\/p>\n<p>\u30b7\u30b9\u30c6\u30e0\u69cb\u6210<\/p>\n<p>\u30bb\u30f3\u30b5\u30c7\u30fc\u30bf\uff1eKafka\uff1eSparkStreamimg(MLLib\u4f7f\u7528)\uff1eElasticSearch\uff06HDFS<br \/>\nYARN\u4e0a\u3067\u52d5\u4f5c<br \/>\nSparkStreaming\u304c5\u79d2\u9593\u9694\u3067\u904b\u52d5\u72b6\u614b\u3092\u5224\u5b9a<br \/>\n\u5b66\u7fd2\u6e08\u307f\u306e\u5b66\u7fd2\u30e2\u30c7\u30eb\u3092\u4f7f\u7528\u3057\u3066\u5224\u5b9a<br \/>\nSparkStreaming\u306fDirectStreaming\u65b9\u5f0f\u3067\u5b9f\u884c<\/p>\n<p>Kafka\u306eParittion\u6570\u3068\u540c\u6570\u306eTask\u3092\u5b9f\u884c<\/p>\n<p>\u69cb\u7bc9\u5f8c\u306e\u30c1\u30e5\u30fc\u30cb\u30f3\u30b0<\/p>\n<p>\u4f7f\u7528\u3059\u308b\u30ce\u30fc\u30c9\u306e\u6570\u3092\u8abf\u6574\u3057\u3066\u5168\u4f53\u3068\u3057\u3066\u30b9\u30eb\u30fc\u30d7\u30c3\u30c8\u304c\u51fa\u308b\u3088\u3046\u30c1\u30e5\u30fc\u30cb\u30f3\u30b0<\/p>\n<p>Kafka Producer\u306e\u9001\u4fe1\u30e1\u30c3\u30bb\u30fc\u30b8\u30b5\u30a4\u30ba\u8abf\u6574<br \/>\nElasticSearch\u306e\u30ea\u30d5\u30ec\u30c3\u30b7\u30e5\u9593\u9694\u3092\u8abf\u6574<br \/>\nProducer\u3001Broker\u3092\u5897\u3084\u3059\u3053\u3068\u3067Kafka\u306e\u6027\u80fd\u3092\u7a4d\u307f\u5897\u3057<\/p>\n<p>\u767a\u751f\u3057\u305f\u554f\u984c<\/p>\n<p>\u30ce\u30fc\u30c9\u9593\u306e\u901a\u4fe1\u91cf\u3088\u308a\u3001Kafka\u30af\u30e9\u30b9\u30bf\u306e\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u5e2f\u57df\u304c\u30dc\u30c8\u30eb\u30cd\u30c3\u30af\u3068\u306a\u3063\u3066\u3044\u308b\u6a21\u69d8<br \/>\nKafkaBroker\u306e1\u30ce\u30fc\u30c9\u3042\u305f\u308a\u306e\u901a\u4fe1\u91cf\u304c\u5e2f\u57df\uff081Gbps\uff09\u3092\u8d85\u904e<br \/>\n\u30ec\u30d7\u30ea\u30b1\u30fc\u30b7\u30e7\u30f3\u304c\u9045\u5ef6\u5b9f\u884c\u3055\u308c\u308b\u305f\u3081\u3001\u901a\u4fe1\u91cf\u306f\u304e\u308a\u304e\u308a\u304a\u3055\u307e\u3063\u3066\u3044\u305f<\/p>\n<p>\u8a2d\u8a08\u6642\u306e\u30ce\u30a6\u30cf\u30a6<\/p>\n<p>Kafka\u306ePartition\u6570\u3092Spark\u306eExecutor\u306b\u5272\u308a\u5f53\u3066\u305fCPU\u30b3\u30a2\u6570\u4ee5\u4e0a\u306b\u8a2d\u5b9a\u3059\u308b\u5fc5\u8981\u304c\u3042\u308b<\/p>\n<p>\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u6570\u5c11\u306a\u3044\u3068\u30b3\u30a2\u3092\u4f7f\u3044\u304d\u308c\u306a\u3044<\/p>\n<p>Kafka\u306e\u30ec\u30d7\u30ea\u30b1\u30fc\u30b7\u30e7\u30f3\u3067\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u5e2f\u57df\u304c\u30dc\u30c8\u30eb\u30cd\u30c3\u30af\u3068\u306a\u308a\u3084\u3059\u3044<\/p>\n<p>10GB\u56de\u7dda\u306b\u3059\u308b<\/p>\n<p>Kafka\u306f\u30e1\u30c3\u30bb\u30fc\u30b8\u3092\u30c7\u30a3\u30b9\u30af\u306b\u4fdd\u5b58\u3059\u308b\u305f\u3081\u3001\u30c7\u30a3\u30b9\u30af\u6027\u80fd\u304c\u30dc\u30c8\u30eb\u30cd\u30c3\u30af\u306b\u306a\u308b\u5834\u5408\u3082\u3042\u308b<\/p>\n<p>\u30ec\u30d7\u30ea\u30ab\u6570\u3092\u8abf\u6574<br \/>\n\u30c7\u30a3\u30b9\u30af\u3092\u8ffd\u52a0<br \/>\nSSD\u5316<\/p>\n<p>\u901a\u8fc7\u589e\u52a0Kafka\u7684\u5206\u533a\u6570\uff0c\u53ef\u4ee5\u589e\u52a0Spark\u7684\u541e\u5410\u91cf\u3002\u4f46\u662f\uff0c\u5982\u679c\u589e\u52a0\u7684\u5206\u533a\u6570\u8fc7\u591a\uff0c\u53ef\u80fd\u4f1a\u5bfc\u81f4IO\u7ade\u4e89\uff0c\u6027\u80fd\u4f1a\u53d7\u5230\u4e25\u91cd\u5f71\u54cd\u3002\u56e0\u6b64\uff0c\u6211\u8ba4\u4e3a\u5728\u8c03\u6574\u8fd9\u4e9b\u65b9\u9762\u65f6\u9700\u8981\u5b9e\u9645\u5c1d\u8bd5\u6765\u786e\u5b9a\u6700\u4f73\u914d\u7f6e\u3002<\/p>\n<h3>Spark Streaming \u662f\u5426\u57fa\u4e8e\u54cd\u5e94\u5f0f\u6d41\uff1f<\/h3>\n<p>Spark Streaming\u662f\u5426\u57fa\u4e8e\u54cd\u5e94\u5f0f\u6d41\uff1f\u6765\u81eachibochibo\u7684\u53d1\u8868\u8d44\u6599\u3002<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">Back Pressure\u306e\u91cd\u8981\u6027<\/ul>\n<\/li>\n<\/ul>\n<p>\u305d\u3082\u305d\u3082Back Pressure\u3068\u306f<\/p>\n<p>\u30b9\u30c8\u30ea\u30fc\u30e0\u51e6\u7406\u306b\u3066\u30c7\u30fc\u30bf\u306e\u30d5\u30ed\u30fc\u5236\u5fa1\u7528<br \/>\n\u904e\u8ca0\u8377\u3067\u3042\u308b\u3053\u3068\u3092\u524d\u6bb5\u306e\u30b3\u30f3\u30dd\u30fc\u30cd\u30f3\u30c8\u306b\u30d5\u30a3\u30fc\u30c9\u30d0\u30c3\u30af\u3059\u308b\u4ed5\u7d44\u307f<\/p>\n<p>\u4f55\u6545\u91cd\u8981\u306a\u306e\u304b\uff1f<\/p>\n<p>\u9001\u4fe1\u5074\u3067\u5e38\u306b\u4e00\u5b9a\u306e\u30c7\u30fc\u30bf\u91cf\u3092\u4fdd\u3064\u306e\u306f\u96e3\u3057\u3044\u305f\u3081<br \/>\n\u4e00\u6642\u7684\u306a\u5897\u52a0\u306a\u3069\u306e\u6ce2\u306f\u3042\u3063\u3066\u3082\u30b7\u30b9\u30c6\u30e0\u5168\u4f53\u3068\u3057\u3066\u52d5\u304d\u7d9a\u3051\u308b\u3053\u3068\u306e\u65b9\u304c\u91cd\u8981<\/p>\n<p>Spark\u306f1.5\u304b\u3089Back Pressure\u306b\u5bfe\u5fdc<\/p>\n<p>Reactive Streams\u3068\u306f<\/p>\n<p>\u975e\u540c\u671f\u30b9\u30c8\u30ea\u30fc\u30e0\u51e6\u7406\u306e\u6a19\u6e96\u5316\u3092\u76ee\u6307\u3059\u898f\u683c<br \/>\nScala\u3060\u3068Akka Streams\u304c\u30b5\u30dd\u30fc\u30c8<br \/>\nJDK 9\u3067Flow API\u3068\u3057\u3066\u5c0e\u5165\u4e88\u5b9a<br \/>\nSpring 5\u306fReactive\u5bfe\u5fdc\u306b\u306a\u308a\u3001\u305d\u306e\u969b\u306bback pressure\u306b\u3082\u5bfe\u5fdc<br \/>\n\u52d5\u4f5c\u539f\u7406<\/p>\n<p>Subscriber\u5074\u3067\u30b5\u30a4\u30ba\u3092\u5236\u9650<br \/>\n\u904e\u8ca0\u8377\u306b\u76f4\u9762\u3059\u308b\u3068Subscriber\u306fback pressure\u306e\u30b7\u30b0\u30ca\u30eb\u3092\u9001\u4fe1<br \/>\nBack Pressure\u306e\u30b7\u30b0\u30ca\u30eb\u306f\u975e\u540c\u671f<br \/>\n\u8ca0\u8377\u72b6\u6cc1\u306b\u5fdc\u3058\u3066Pull\u578b\u304bPush\u578b\u304b\u306f\u5207\u308a\u66ff\u308f\u308b<\/p>\n<p>Spark Streaming\u306eBack Pressure\u5b9f\u88c5<\/p>\n<p>Reactive Streams\u306b\u306f\u6e96\u62e0\u3057\u3066\u3044\u306a\u3044\u3002<\/p>\n<p>Reactive Streams\u306e\u8a2d\u8a08\u65b9\u91dd\u304b\u3089\u30a4\u30f3\u30b9\u30d4\u30ec\u30fc\u30b7\u30e7\u30f3\u3092\u53d7\u3051\u3066\u3044\u308b\u3082\u306e\u306e\u3001Spark\u306e\u5185\u90e8\u304c\u3053\u306e\u4ed5\u69d8\u3092\u9075\u5b88\u3059\u308b\u3064\u3082\u308a\u306f\u306a\u3044\u3068\u306e\u3053\u3068\u3002<\/p>\n<p>\u3067\u306f\u3069\u3046\u3084\u3063\u3066Back Pressure\u3092\u5b9f\u73fe\u3057\u3066\u3044\u308b\u304b\uff1f<\/p>\n<p>StreamingListener#onBatchCompleted\u30e1\u30bd\u30c3\u30c9<\/p>\n<p>\u30df\u30cb\u30d0\u30c3\u30c1\u304c\u7d42\u4e86\u3057\u305f\u6bb5\u968e\u3067\u5b9f\u884c<br \/>\n\u4e0b\u8a18\u306e\u3088\u3046\u306a\u60c5\u5831\u304c\u53d6\u5f97\u53ef\u80fd<\/p>\n<p>\u30df\u30cb\u30d0\u30c3\u30c1\u5b9f\u884c\u6642\u9593<br \/>\n\u51e6\u7406\u958b\u59cb\u6642\u523b\u3001\u51e6\u7406\u7d42\u4e86\u6642\u523b<br \/>\n\u30b9\u30b1\u30b8\u30e5\u30fc\u30eb\u6e08\uff5e\u5b9f\u884c\u958b\u59cb\u307e\u3067\u306e\u5f85\u3061\u6642\u9593<br \/>\n\u51e6\u7406\u6642\u9593\uff08\u30df\u30cb\u30d0\u30c3\u30c1\u81ea\u4f53\u306e\u51e6\u7406\u6642\u9593\uff09<br \/>\n\u30b9\u30b1\u30b8\u30e5\u30fc\u30eb\u6e08\uff5e\u5b8c\u4e86\u307e\u3067\u306e\u7dcf\u6240\u8981\u6642\u9593<\/p>\n<p>\u4e0a\u8a18\u306e\u5024\u3092RateEstimator\u306b\u6e21\u3057\u3066\u65b0\u3057\u3044\u53d6\u5f97Rate\u3092\u8a08\u7b97\u3057\u3001\u6b21\u306e\u53d7\u4fe1\u91cf\u3092\u6c7a\u3081\u308b\u65b9\u5f0f<\/p>\n<p>\u611f\u60f3\uff1a<br \/>\n\u4f3c\u4e4e\u5e76\u4e0d\u7b26\u5408\u89c4\u8303\u3002<br \/>\n\u5b9e\u9645\u4e0a\uff0cSpark\u7684Subscriber\u672c\u8eab\u662f\u57fa\u4e8ePull\u6a21\u578b\u7684\uff0c<br \/>\n\u6240\u8c13\u7684\u5ef6\u8fdf\u662f\u6307\u201c\u5c0f\u6279\u5904\u7406\u6267\u884c\u95f4\u9694\u4e0d\u7b26\u5408\u5b9a\u4e49\u7684\u6267\u884c\u95f4\u9694\u201d\u3002<br \/>\n\u56e0\u6b64\uff0c\u9075\u5b88\u89c4\u8303\u672c\u8eab\u53ef\u80fd\u662f\u4e00\u4ef6\u56f0\u96be\u7684\u4e8b\u60c5\uff0c\u8fd9\u662f\u6211\u7684\u770b\u6cd5\u3002<\/p>\n<p>\u4e5f\u8bb8\uff0c\u57fa\u4e8e\u4e34\u65f6\u6d41\u91cf\u589e\u957f\u7684\u57fa\u672c\u6d41\u6570\u636e\u5904\u7406\u5e73\u53f0\u4ea7\u54c1\u53ef\u4ee5\u901a\u8fc7\u7c7b\u4f3cKafka\u3001Kinesis\u3001Cloud PubSub\u8fd9\u6837\u7684\u6d88\u606f\u961f\u5217\u6765\u5904\u7406\u3002\u8fd9\u610f\u5473\u7740\u53ef\u80fd\u4e0d\u9700\u8981\u9075\u5faaReactive Streams\u7684\u7acb\u573a\u3002<\/p>\n<h3>\u7528AWS\u521b\u5efa\u5c0f\u4e2d\u578bApache Kafka\u548c\u5404\u79cd\u70e6\u607c<\/h3>\n<p>\u6570\u636e\uff1a<br \/>\n\u6765\u81eaKeigo Suda\u7684\u5173\u4e8e\u4f7f\u7528AWS\u6784\u5efaApache Kafka\u548c\u5404\u79cd\u95ee\u9898\u7684\u8d44\u6599\u3002<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">AWS\u4e0a\u3067Kafka\u3092\u5229\u2f64\u3059\u308b\u305f\u3081\u306b\u8003\u3048\u305f\u3053\u3068<\/ul>\n<\/li>\n<\/ul>\n<p>\u3069\u306e\u3088\u3046\u306a\u30dd\u30a4\u30f3\u30c8\u304c\u3042\u3063\u305f\u304b<br \/>\n\u3069\u306e\u3088\u3046\u306b\u305d\u306e\u30dd\u30a4\u30f3\u30c8\u306b\u5bfe\u5fdc\u3057\u305f\u304b<\/p>\n<p>IoT\u306e\u305f\u3081\u306e\u30c7\u30fc\u30bf\u30d7\u30e9\u30c3\u30c8\u30d5\u30a9\u30fc\u30e0\u3092\u958b\u767a\u4e2d<\/p>\n<p>\u5de5\u5834\u5185\u306e\u5404\u30bb\u30f3\u30b5\u30fc\u30c7\u30fc\u30bf\u3092\u53ce\u96c6\u52a0\u5de5\u84c4\u7a4d\u5206\u6790\u3059\u308b\u305f\u3081\u306e\u57fa\u76e4<\/p>\n<p>IIoT(Industrial Internet of Things)<\/p>\n<p>\u4e16\u754c\u4e2d\u306b\u3042\u308b\u5de5\u5834\u306b\u5bfe\u3057\u3066\u5c55\u958b<\/p>\n<p>\u305d\u3082\u305d\u3082\u306a\u305cKafka on AWS\u306a\u306e\u304b\uff1f<\/p>\n<p>\u6a5f\u5bc6\u306a\u30c7\u30fc\u30bf\u3082\u6271\u3046\u306e\u3067VPC\u306b\u5bfe\u3057\u3066DirectConnect\u3067\u6295\u5165\u3057\u305f\u304b\u3063\u305f<\/p>\n<p>Amazon Kinesis\u3084Amazon IoT\u3067\u306f\u6697\u53f7\u5316\u3055\u308c\u3066\u3044\u308b\u3068\u306f\u3044\u3048\u3001\u30a4\u30f3\u30bf\u30fc\u30cd\u30c3\u30c8\u306b\u51fa\u3066\u3057\u307e\u3046\u3002\u305d\u308c\u306f\u907f\u3051\u305f\u3044\u3002<\/p>\n<p>Amazon Kinesis EndPoint\u306f\u306a\u3044\u305f\u3081\u3001\u30a4\u30f3\u30bf\u30fc\u30cd\u30c3\u30c8\u306b\u51fa\u306a\u3044\u7d4c\u8def\u3092\u4fdd\u8a3c\u3059\u308b\u3053\u3068\u304c\u3067\u304d\u306a\u3044\u3002<\/p>\n<p>Kafka\u3067\u30de\u30eb\u30c1AZ\u3092\u3069\u3046\u3059\u308b\u306e\u304b\uff1f<\/p>\n<p>AZ\u306e\u3069\u308c\u304b\u304c\u969c\u5bb3\u767a\u751f\u3057\u3066\u3082\u5168\u4f53\u3068\u3057\u3066\u306e\u505c\u6b62\u306f\u907f\u3051\u305f\u3044<\/p>\n<p>\u6771\u4eac\u30ea\u30fc\u30b8\u30e7\u30f3\u306f2AZ\u306e\u305f\u3081\u3001AZ\u6bce\u306b\u30af\u30e9\u30b9\u30bf\u3092\u914d\u7f6e\u3059\u308b\u3053\u3068\u306b<\/p>\n<p>\u3069\u3046\u3084\u3063\u3066Kafka\u306b\u30c7\u30fc\u30bf\u3092\u6295\u5165\u3059\u308b\u306e\u304b\uff1f<\/p>\n<p>API\/Producer\u51e6\u7406\u306fGo\u3067\u958b\u767a\u3001\u30af\u30e9\u30a4\u30a2\u30f3\u30c8\u30e9\u30a4\u30d6\u30e9\u30ea\u306fSarama\u3092\u5229\u7528<br \/>\nBroker\u306f\u5185\u90e8\u5411\u3051ELB\u306b\u30a2\u30bf\u30c3\u30c1\u3057\u3001\u63a5\u7d9a\u5148\u30af\u30e9\u30b9\u30bf\u3092\u5207\u308a\u66ff\u3048\u308b<\/p>\n<p>\u30c8\u30d4\u30c3\u30af\u8a2d\u8a08<\/p>\n<p>\u30c8\u30d4\u30c3\u30af\u3092\u7d30\u304b\u304f\u5206\u5272\u3059\u308b\u3068\u7206\u767a\u7684\u306b\u5897\u3048\u3066\u3044\u304f\u305f\u3081\u3001\u5de5\u5834\u5358\u4f4d\u306b\u96c6\u7d04\u3057\u3001\u5f8c\u6bb5\u306e\u30b9\u30c8\u30ea\u30fc\u30e0\u30c7\u30fc\u30bf\u51e6\u7406\u90e8\u3067\u5206\u5272<\/p>\n<p>\u30c7\u30fc\u30bf\u306e\u6b20\u640d\u304c\u7121\u3044\u3053\u3068\u3092\u3069\u3046\u3084\u3063\u3066\u4fdd\u8a3c\u3059\u308b\uff1f<\/p>\n<p>\u5404\u5de5\u5834\u306e\u30c7\u30fc\u30bf\u3068\u3001\u6700\u7d42\u7684\u306a\u51fa\u529b\u30c7\u30fc\u30bf\u306e\u6570\u3092\u7a81\u304d\u3042\u308f\u305b<br \/>\n\u5404\u5de5\u5834\u304b\u3089Kafka\u306b\u5bfe\u3057\u3066\u6295\u5165\u3057\u305f\u30c7\u30fc\u30bf\u3068\u3001Kafka\u306b\u5bfe\u3057\u3066\u6295\u5165\u3055\u308c\u305f\u30c7\u30fc\u30bf\u306e\u6570\u3092\u7a81\u304d\u5408\u308f\u305b<\/p>\n<p>\u5de5\u5834\u306e\u30c7\u30fc\u30bf\u3092\u96c6\u7d04\u3057\u3066Kafka\u306b\u6295\u5165\u3059\u308b\u305f\u3081\u3001\u7a81\u304d\u5408\u308f\u305b\u308b\u500b\u6240\u306f\uff12\u304b\u6240<\/p>\n<p>\u611f\u60f3\uff1a<br \/>\n\u300c\u7531\u4e8e\u7f3a\u4e4fAmazon Kinesis EndPoint\uff0c\u65e0\u6cd5\u4fdd\u8bc1\u4e0d\u7ecf\u8fc7\u4e92\u8054\u7f51\u7684\u8def\u5f84\u300d\u4ee5\u53ca<br \/>\n\u300c\u7531\u4e8e\u4e1c\u4eac\u5730\u533a\u53ea\u6709\u4e24\u4e2a\u53ef\u7528\u533a\uff0c\u96be\u4ee5\u914d\u7f6e\u9700\u8981\u51c6\u5907\u5947\u6570\u4e2a\u8282\u70b9\u7684\u96c6\u7fa4\u300d\u7b49\u7b49\uff0c<br \/>\nAWS\u7684\u82e6\u607c\u6545\u4e8b\u771f\u662f\u7433\u7405\u6ee1\u76ee\u7684&#8230;<\/p>\n<p>\u7136\u800c\uff0c\u4ed6\u5e76\u6ca1\u6709\u505a\u51fa\u7279\u522b\u7a81\u98de\u7684\u4e8b\u60c5\uff0c\u800c\u662f\u7a33\u5b9a\u5730\u79ef\u7d2f\u548c\u5efa\u8bbe\u4e86\u5e94\u8be5\u505a\u7684\u4e8b\u60c5\uff0c\u8fd9\u662f\u975e\u5e38\u6709\u53c2\u8003\u4ef7\u503c\u7684\u4e00\u6b21\u4f1a\u8bae\u3002<\/p>\n<h4>Hortonworks Data Cloud\u7684\u6982\u8981<\/h4>\n<p>\u8cc7\u6599\uff1a<br \/>\n\u7531\u59dc\u4e00\u5cf0\u63d0\u4f9b\u7684\u4ecb\u7d39Hortonworks Data Cloud for AWS<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">\u4f55\u6545Hortonworks Data Cloud\u304c\u3042\u308b\u306e\u304b\uff1f<\/ul>\n<\/li>\n<\/ul>\n<p>Ambari\u306b\u3088\u3063\u3066\u3001\u30af\u30e9\u30b9\u30bf\u306b\u5bfe\u3059\u308b\u30c7\u30d7\u30ed\u30a4\u306e\u624b\u9593\u306f\u5927\u5e45\u306b\u8efd\u6e1b\u3055\u308c\u305f\u3002<br \/>\n\u3060\u304c\u3001\u305d\u3082\u305d\u3082\u30de\u30b7\u30f3\u3092\u7528\u610f\u3059\u308b\u306e\u306b\u6642\u9593\u304c\u304b\u304b\u308b\u3053\u3068\u304c\u591a\u304f\u3001\u5c0e\u5165\u306b\u6642\u9593\u304c\u304b\u304b\u308b\u3002<br \/>\n\u306a\u3089\u30af\u30e9\u30a6\u30c9\u3060\uff01<\/p>\n<p>\u30af\u30e9\u30a6\u30c9\u4e0a\u3067Hortonworks\u306eData\u30af\u30e9\u30b9\u30bf\u3092\u69cb\u7bc9<\/p>\n<p>\u4f7f\u7528\u3057\u305f\u5206\u3060\u3051\u306e\u8ab2\u91d1<br \/>\nPrivate\u306a\u30d5\u30a9\u30fc\u30e9\u30e0\u4e0a\u3067\u306e\u30b3\u30df\u30e5\u30cb\u30c6\u30a3\u30b5\u30dd\u30fc\u30c8\u3092\u7121\u511f\u63d0\u4f9b<\/p>\n<p>\u30c7\u30e2<\/p>\n<p>\u30af\u30e9\u30a6\u30c9\u4e0a\u306b\u30af\u30e9\u30b9\u30bf\u3092\u5373\u5c55\u958b\u3057\u3001S3\u4e0a\u306e\u30c7\u30fc\u30bf\u3092\u5916\u90e8\u30c6\u30fc\u30d6\u30eb\u3068\u3057\u3066Hive\u306b\u30a4\u30f3\u30dd\u30fc\u30c8\u3057\u3001\u5373\u8aad\u3080\u3053\u3068\u304c\u51fa\u6765\u305f\uff01<\/p>\n<p>\u30aa\u30f3\u30d7\u30ec\u30df\u30b9\u306e\u30af\u30e9\u30b9\u30bf\u3068\u7570\u306a\u308b\u70b9<\/p>\n<p>\u30af\u30e9\u30b9\u30bf\u81ea\u4f53\u304c\u4e00\u6642\u7684\u306a\u3082\u306e\u3068\u306a\u308b\u305f\u3081\u3001\u6c38\u7d9a\u5316\u3059\u308b\u5fc5\u8981\u304c\u3042\u308b\u3082\u306e\u306f\u30af\u30e9\u30b9\u30bf\u3092\u8de8\u3044\u3067\u4fdd\u5b58<\/p>\n<p>HiveServer\u7528\u30c7\u30fc\u30bf\u306a\u3069\u306e\u30e1\u30bf\u30c7\u30fc\u30bf<br \/>\nS3\u4e0a\u306b\u914d\u7f6e\u3057\u305f\u30c7\u30fc\u30bf<\/p>\n<p>\u30b3\u30f3\u30d4\u30e5\u30fc\u30c8\u30ce\u30fc\u30c9\u3068\u30b9\u30c8\u30ec\u30fc\u30b8\u30ce\u30fc\u30c9\u3092\u5206\u96e2<\/p>\n<p>S3\u3078\u306e\u30a2\u30af\u30bb\u30b9\u304c\u30ea\u30e2\u30fc\u30c8\u3068\u306a\u308b\u305f\u3081\u3001\u6027\u80fd\u7279\u6027\u304c\u7570\u306a\u3063\u3066\u304f\u308b\u3002<br \/>\n\u30ec\u30b9\u30dd\u30f3\u30b9\u306f\u52a3\u308b\u304c\u3001\u30b9\u30eb\u30fc\u30d7\u30c3\u30c8\u306f\u51fa\u308b\u3002<\/p>\n<p>S3\u3078\u306e\u30db\u30c3\u30c8\u30b9\u30dd\u30c3\u30c8\u3092\u907f\u3051\u308b\u305f\u3081\u306b\u30e9\u30f3\u30c0\u30e0\u9806\u3067\u53d6\u5f97\u3057\u306b\u884c\u304f\u306a\u3069\u306e\u5bfe\u51e6\u304c\u5165\u3063\u3066\u3044\u308b<br \/>\n\u30af\u30e9\u30b9\u30bf\u5358\u4f4d\u306bIAM Role\u3092\u5272\u308a\u632f\u308a\u3001\u30a2\u30af\u30bb\u30b9\u53ef\u80fd\u306a\u30c7\u30fc\u30bf\u3092\u5236\u9650\u3059\u308b\u3053\u3068\u3082\u53ef\u80fd<br \/>\nApache Ranger\u3001Apache Atlas\u3068\u7d44\u307f\u5408\u308f\u305b\u308b\u3053\u3068\u3067\u30c7\u30fc\u30bf\u6a29\u9650\u7ba1\u7406\u304c\u67d4\u8edf\u30fb\u5bb9\u6613\u306b<\/p>\n<p>Apache Ranger\u3001Apache Atlas\u306f\u30de\u30eb\u30c1\u30af\u30e9\u30b9\u30bf\u306b\u5bfe\u3057\u3066\u5bfe\u5fdc\u53ef\u80fd\u306b\u306a\u3063\u3066\u3044\u308b\u3002<\/p>\n<p>\u4eca\u5f8c\u8ffd\u52a0\u3055\u308c\u308b\u6a5f\u80fd<\/p>\n<p>\u30b3\u30f3\u30d4\u30e5\u30fc\u30c8\u30ce\u30fc\u30c9\u306e\u81ea\u52d5\u5fa9\u65e7<\/p>\n<p>\u611f\u60f3\uff1a<br \/>\n\u7531\u65bcAWS\u5df2\u7d93\u5b58\u5728Elastic MapReduce\uff0c\u56e0\u6b64\u5b83\u4e0d\u50c5\u50c5\u662f\u5c07\u5206\u767c\u653e\u5728\u96f2\u7aef\u4e0a\uff0c\u9084\u63a1\u53d6\u4e86\u652f\u6301\u591a\u96c6\u7fa4\u548c\u5f9e\u5916\u90e8\u8b80\u53d6\u5143\u6578\u64da\u7b49\u63aa\u65bd\uff0c\u4ee5\u5be6\u73fe\u5dee\u7570\u5316\u3002<\/p>\n<p>\u6211\u5b9e\u9645\u4e0a\u89c9\u5f97\uff0c\u5728\u5c06\u6570\u636e\u653e\u5165\u4e91\u7aef\u65f6\u4f1a\u4ea7\u751f\u4e0e\u4e9a\u9a6c\u900a\u6216\u8c37\u6b4c\u539f\u6709\u7684\u5927\u6570\u636e\u57fa\u7840\u8bbe\u65bd\u7684\u7ade\u4e89\uff0c\u6240\u4ee5\u786e\u5b9a\u81ea\u5df1\u7684\u4f18\u52bf\u4f4d\u7f6e\u975e\u5e38\u91cd\u8981\u3002<\/p>\n<h3>\u603b\u7ed3<\/h3>\n<p>\u6574\u4f53\u800c\u8a00\uff0cHadoop\u5cf0\u4f1a\u7684\u5185\u5bb9\u548c\u8d8b\u52bf\u591a\u6837\uff0c\u771f\u53ef\u8c13\u662f\u4f5c\u4e3aHadoop\u5cf0\u4f1a\u7684\u756a\u5916\u7bc7\u5462\u3002<br \/>\n\u7279\u522b\u662f\u6700\u540e\u4e00\u6b21\u6f14\u8bb2\uff0c\u5c3d\u7ba1\u4e91\u8ba1\u7b97\u4e5f\u6210\u4e3a\u4e00\u4e2a\u9009\u62e9\uff0c\u4f46\u662f\u5e76\u975e\u7ade\u4e89\u76f8\u540c\u7684\u529f\u80fd\u548c\u6027\u80fd\uff0c\u800c\u662f\u671f\u5f85\u672a\u6765\u66f4\u52a0\u7a81\u51fa\u5dee\u5f02\u5316\u7684\u6f14\u8bb2\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u8fd9\u7bc7\u6587\u7ae0\u662f\u5173\u4e8e\u4ec0\u4e48\u7684\uff1f \u8fd9\u7bc7\u6587\u7ae0\u662f2016\u5e74Hortonworks Advent Calendar\u7684\u7b2c17\u7bc7\u6587 [&hellip;]<\/p>\n","protected":false},"author":5,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-46939","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.5 (Yoast SEO v21.5) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>\u53e6\u4e00\u4e2aHadoop\u5cf0\u4f1a\u53c2\u52a0\u62a5\u544a - Blog - Silicon Cloud<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.silicloud.com\/zh\/blog\/\u53e6\u4e00\u4e2ahadoop\u5cf0\u4f1a\u53c2\u52a0\u62a5\u544a\/\" \/>\n<meta property=\"og:locale\" content=\"zh_CN\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"\u53e6\u4e00\u4e2aHadoop\u5cf0\u4f1a\u53c2\u52a0\u62a5\u544a\" \/>\n<meta property=\"og:description\" content=\"\u8fd9\u7bc7\u6587\u7ae0\u662f\u5173\u4e8e\u4ec0\u4e48\u7684\uff1f \u8fd9\u7bc7\u6587\u7ae0\u662f2016\u5e74Hortonworks Advent Calendar\u7684\u7b2c17\u7bc7\u6587 [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.silicloud.com\/zh\/blog\/\u53e6\u4e00\u4e2ahadoop\u5cf0\u4f1a\u53c2\u52a0\u62a5\u544a\/\" \/>\n<meta property=\"og:site_name\" content=\"Blog - Silicon Cloud\" \/>\n<meta property=\"article:published_time\" content=\"2023-10-01T07:00:29+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-05-04T04:18:03+00:00\" \/>\n<meta name=\"author\" content=\"\u6e05, \u5b87\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"\u4f5c\u8005\" \/>\n\t<meta name=\"twitter:data1\" content=\"\u6e05, \u5b87\" \/>\n\t<meta name=\"twitter:label2\" content=\"\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 \u5206\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/%e5%8f%a6%e4%b8%80%e4%b8%aahadoop%e5%b3%b0%e4%bc%9a%e5%8f%82%e5%8a%a0%e6%8a%a5%e5%91%8a\/\",\"url\":\"https:\/\/www.silicloud.com\/zh\/blog\/%e5%8f%a6%e4%b8%80%e4%b8%aahadoop%e5%b3%b0%e4%bc%9a%e5%8f%82%e5%8a%a0%e6%8a%a5%e5%91%8a\/\",\"name\":\"\u53e6\u4e00\u4e2aHadoop\u5cf0\u4f1a\u53c2\u52a0\u62a5\u544a - Blog - Silicon Cloud\",\"isPartOf\":{\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#website\"},\"datePublished\":\"2023-10-01T07:00:29+00:00\",\"dateModified\":\"2024-05-04T04:18:03+00:00\",\"author\":{\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/1a6ecd3d914d22a5ac32791ffc1fbd8e\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/%e5%8f%a6%e4%b8%80%e4%b8%aahadoop%e5%b3%b0%e4%bc%9a%e5%8f%82%e5%8a%a0%e6%8a%a5%e5%91%8a\/#breadcrumb\"},\"inLanguage\":\"zh-Hans\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.silicloud.com\/zh\/blog\/%e5%8f%a6%e4%b8%80%e4%b8%aahadoop%e5%b3%b0%e4%bc%9a%e5%8f%82%e5%8a%a0%e6%8a%a5%e5%91%8a\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/%e5%8f%a6%e4%b8%80%e4%b8%aahadoop%e5%b3%b0%e4%bc%9a%e5%8f%82%e5%8a%a0%e6%8a%a5%e5%91%8a\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"\u9996\u9875\",\"item\":\"https:\/\/www.silicloud.com\/zh\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"\u53e6\u4e00\u4e2aHadoop\u5cf0\u4f1a\u53c2\u52a0\u62a5\u544a\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#website\",\"url\":\"https:\/\/www.silicloud.com\/zh\/blog\/\",\"name\":\"Blog - Silicon Cloud\",\"description\":\"\",\"inLanguage\":\"zh-Hans\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/1a6ecd3d914d22a5ac32791ffc1fbd8e\",\"name\":\"\u6e05, \u5b87\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"zh-Hans\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/4b2016c18459a605fc469c7566608f5686491baa112d0871ee613f61b7210565?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/4b2016c18459a605fc469c7566608f5686491baa112d0871ee613f61b7210565?s=96&d=mm&r=g\",\"caption\":\"\u6e05, \u5b87\"},\"url\":\"https:\/\/www.silicloud.com\/zh\/blog\/author\/qingyu\/\"},{\"@type\":\"ImageObject\",\"inLanguage\":\"zh-Hans\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/%e5%8f%a6%e4%b8%80%e4%b8%aahadoop%e5%b3%b0%e4%bc%9a%e5%8f%82%e5%8a%a0%e6%8a%a5%e5%91%8a\/#local-main-organization-logo\",\"url\":\"\",\"contentUrl\":\"\",\"caption\":\"Blog - Silicon Cloud\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"\u53e6\u4e00\u4e2aHadoop\u5cf0\u4f1a\u53c2\u52a0\u62a5\u544a - Blog - Silicon Cloud","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.silicloud.com\/zh\/blog\/\u53e6\u4e00\u4e2ahadoop\u5cf0\u4f1a\u53c2\u52a0\u62a5\u544a\/","og_locale":"zh_CN","og_type":"article","og_title":"\u53e6\u4e00\u4e2aHadoop\u5cf0\u4f1a\u53c2\u52a0\u62a5\u544a","og_description":"\u8fd9\u7bc7\u6587\u7ae0\u662f\u5173\u4e8e\u4ec0\u4e48\u7684\uff1f \u8fd9\u7bc7\u6587\u7ae0\u662f2016\u5e74Hortonworks Advent Calendar\u7684\u7b2c17\u7bc7\u6587 [&hellip;]","og_url":"https:\/\/www.silicloud.com\/zh\/blog\/\u53e6\u4e00\u4e2ahadoop\u5cf0\u4f1a\u53c2\u52a0\u62a5\u544a\/","og_site_name":"Blog - Silicon Cloud","article_published_time":"2023-10-01T07:00:29+00:00","article_modified_time":"2024-05-04T04:18:03+00:00","author":"\u6e05, \u5b87","twitter_card":"summary_large_image","twitter_misc":{"\u4f5c\u8005":"\u6e05, \u5b87","\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4":"1 \u5206"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.silicloud.com\/zh\/blog\/%e5%8f%a6%e4%b8%80%e4%b8%aahadoop%e5%b3%b0%e4%bc%9a%e5%8f%82%e5%8a%a0%e6%8a%a5%e5%91%8a\/","url":"https:\/\/www.silicloud.com\/zh\/blog\/%e5%8f%a6%e4%b8%80%e4%b8%aahadoop%e5%b3%b0%e4%bc%9a%e5%8f%82%e5%8a%a0%e6%8a%a5%e5%91%8a\/","name":"\u53e6\u4e00\u4e2aHadoop\u5cf0\u4f1a\u53c2\u52a0\u62a5\u544a - Blog - Silicon Cloud","isPartOf":{"@id":"https:\/\/www.silicloud.com\/zh\/blog\/#website"},"datePublished":"2023-10-01T07:00:29+00:00","dateModified":"2024-05-04T04:18:03+00:00","author":{"@id":"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/1a6ecd3d914d22a5ac32791ffc1fbd8e"},"breadcrumb":{"@id":"https:\/\/www.silicloud.com\/zh\/blog\/%e5%8f%a6%e4%b8%80%e4%b8%aahadoop%e5%b3%b0%e4%bc%9a%e5%8f%82%e5%8a%a0%e6%8a%a5%e5%91%8a\/#breadcrumb"},"inLanguage":"zh-Hans","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.silicloud.com\/zh\/blog\/%e5%8f%a6%e4%b8%80%e4%b8%aahadoop%e5%b3%b0%e4%bc%9a%e5%8f%82%e5%8a%a0%e6%8a%a5%e5%91%8a\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.silicloud.com\/zh\/blog\/%e5%8f%a6%e4%b8%80%e4%b8%aahadoop%e5%b3%b0%e4%bc%9a%e5%8f%82%e5%8a%a0%e6%8a%a5%e5%91%8a\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"\u9996\u9875","item":"https:\/\/www.silicloud.com\/zh\/blog\/"},{"@type":"ListItem","position":2,"name":"\u53e6\u4e00\u4e2aHadoop\u5cf0\u4f1a\u53c2\u52a0\u62a5\u544a"}]},{"@type":"WebSite","@id":"https:\/\/www.silicloud.com\/zh\/blog\/#website","url":"https:\/\/www.silicloud.com\/zh\/blog\/","name":"Blog - Silicon Cloud","description":"","inLanguage":"zh-Hans"},{"@type":"Person","@id":"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/1a6ecd3d914d22a5ac32791ffc1fbd8e","name":"\u6e05, \u5b87","image":{"@type":"ImageObject","inLanguage":"zh-Hans","@id":"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/4b2016c18459a605fc469c7566608f5686491baa112d0871ee613f61b7210565?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/4b2016c18459a605fc469c7566608f5686491baa112d0871ee613f61b7210565?s=96&d=mm&r=g","caption":"\u6e05, \u5b87"},"url":"https:\/\/www.silicloud.com\/zh\/blog\/author\/qingyu\/"},{"@type":"ImageObject","inLanguage":"zh-Hans","@id":"https:\/\/www.silicloud.com\/zh\/blog\/%e5%8f%a6%e4%b8%80%e4%b8%aahadoop%e5%b3%b0%e4%bc%9a%e5%8f%82%e5%8a%a0%e6%8a%a5%e5%91%8a\/#local-main-organization-logo","url":"","contentUrl":"","caption":"Blog - Silicon Cloud"}]}},"_links":{"self":[{"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/posts\/46939","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/comments?post=46939"}],"version-history":[{"count":2,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/posts\/46939\/revisions"}],"predecessor-version":[{"id":98272,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/posts\/46939\/revisions\/98272"}],"wp:attachment":[{"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/media?parent=46939"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/categories?post=46939"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/tags?post=46939"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}