{"id":47100,"date":"2023-03-03T12:09:58","date_gmt":"2023-05-25T17:17:46","guid":{"rendered":"https:\/\/www.silicloud.com\/zh\/blog\/%e6%80%9d%e8%80%83%e4%b8%b2%e6%b5%81%e5%a4%84%e7%90%86\/"},"modified":"2024-04-29T17:13:16","modified_gmt":"2024-04-29T09:13:16","slug":"%e6%80%9d%e8%80%83%e4%b8%b2%e6%b5%81%e5%a4%84%e7%90%86","status":"publish","type":"post","link":"https:\/\/www.silicloud.com\/zh\/blog\/%e6%80%9d%e8%80%83%e4%b8%b2%e6%b5%81%e5%a4%84%e7%90%86\/","title":{"rendered":"\u601d\u8003\u4e32\u6d41\u5904\u7406"},"content":{"rendered":"<p>\u6839\u636e\u300aDesigning Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems\u300b\u8fdb\u884c\u4e86\u6574\u7406\uff0c\u4ee5\u4fbf\u601d\u8003\u5173\u4e8e\u91cd\u65b0\u8fdb\u884c\u6d41\u5904\u7406\u7684\u95ee\u9898\u3002<\/p>\n<h2>\u6d3b\u52a8\u6d41<\/h2>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">stream processing\u3067\u306frecord\u306e\u3053\u3068\u3092event\u3068\u8a00\u3063\u305f\u308a\u3057\u307e\u3059<\/ul>\n<\/li>\n<\/ul>\n<p>event\u306f\u30bf\u30a4\u30e0\u30b9\u30bf\u30f3\u30d7\u3092\u542b\u3080\u5834\u5408\u304c\u591a\u3044<\/p>\n<p>event\u306e\u4f5c\u6210\u8005\u306fproducer, publisher, sender\u306a\u3069\u3068\u547c\u3070\u308c\u308b<br \/>\nevent\u306e\u4f7f\u7528\u8005\u306fconsumers, subscribers, recipents\u306a\u3069\u3068\u547c\u3070\u308c\u308b<br \/>\n\u3042\u308bevent\u306f1\u4eba\u306e\u4f5c\u6210\u8005\u306b\u3088\u3063\u3066\u4f5c\u3089\u308c\u3001\u8907\u6570\u306e\u4f7f\u7528\u8005\u306b\u3088\u3063\u3066\u4f7f\u7528\u3055\u308c\u308b<br \/>\n\u826f\u304f\u306a\u3044event stream\u306e\u4f8b<\/p>\n<p>producer\u306f\u4e00\u822c\u7684\u306a\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u306bevent\u3092\u633f\u5165<br \/>\nconsumers\u306f\u5b9a\u671f\u7684\u306b\u30dd\u30fc\u30ea\u30f3\u30b0\u3057\u3066\u633f\u5165\u3055\u308c\u305f\u30c7\u30fc\u30bf\u3092\u53d6\u5f97\u3059\u308b<br \/>\n\u4e00\u822c\u7684\u306a\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u306f\u5dee\u5206\u30c7\u30fc\u30bf\u3060\u3051\u53d6\u5f97\u3059\u308b\u306e\u306b\u6709\u52b9\u306a\u4ed5\u7d44\u307f\u3067\u306f\u306a\u3044\u3002<br \/>\n\u30a2\u30d7\u30ea\u30b1\u30fc\u30b7\u30e7\u30f3\u306f\u524d\u56de\u3069\u3053\u307e\u3067\u30c7\u30fc\u30bf\u3092\u53d6\u5f97\u3057\u305f\u306e\u304b\u899a\u3048\u3066\u306a\u3044\u3068\u3044\u3051\u306a\u3044\u3002<br \/>\n\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u304c\u306a\u3044\u3001\u52b9\u304b\u306a\u3044\u5834\u5408\u306f\u30dd\u30fc\u30ea\u30f3\u30b0\u306e\u305f\u3073\u306b\u30d5\u30eb\u30b9\u30ad\u30e3\u30f3\u304c\u8d70\u308b<\/p>\n<h2>\u5373\u65f6\u901a\u8baf\u7cfb\u7edf<\/h2>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">event\u304c\u4f5c\u3089\u308c\u305f\u3089consumers\u306b\u901a\u77e5\u304c\u3055\u308c\u308b\u4ed5\u7d44\u307f\u304c\u826f\u3044\u3002\u3053\u306e\u3088\u3046\u306a\u4ed5\u7d44\u307f\u3092messaging system\u3068\u547c\u3076\u3002<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">event\u3092\u542b\u3080\u30c7\u30fc\u30bf\u3092message\u3068\u547c\u3093\u3060\u308a\u3057\u307e\u3059<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">Unix pipe\u3084TCP\u30b3\u30cd\u30af\u30b7\u30e7\u30f3\u306f\u901a\u77e5\u306e\u4ed5\u7d44\u307f\u3092\u6709\u3059\u308b\u304c\u3001\u3053\u308c\u3089\u306f1\u5bfe1\u306e\u901a\u4fe1\u3067\u3042\u308a\u30011\u5bfe\u591a\u306e\u901a\u4fe1\u306f\u4e0d\u53ef\u80fd<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">messaging system\u3067\u306f\u4ee5\u4e0b\u306e\u3053\u3068\u3092\u6c17\u3092\u3064\u3051\u306a\u3044\u3068\u3044\u3051\u306a\u3044<\/ul>\n<\/li>\n<\/ul>\n<p>producer\u306e\u65b9\u304ccomsumers\u3088\u308a\u65e9\u3044\u30b9\u30d4\u30fc\u30c9\u3067event\u3092\u751f\u6210\u3059\u308b\u3068\u304d\u306fmessage\u3092\u7834\u68c4\u3059\u308b\u304b\u3001\u30d0\u30c3\u30d5\u30a1\u3059\u308b\u304b\u3001\u30d0\u30c3\u30af\u30d7\u30ec\u30c3\u30b7\u30e3\u30fc\u3059\u308b\u304b\u3057\u306a\u3044\u3068\u3044\u3051\u306a\u3044<br \/>\n\u4f55\u304b\u304c\u30af\u30e9\u30c3\u30b7\u30e5\u3057\u305f\u6642\u306b\u30e1\u30c3\u30bb\u30fc\u30b8\u30ed\u30b9\u30c8\u3092\u8a31\u5bb9\u3067\u304d\u308b\u306e\u304b\u3001\u305d\u308c\u3068\u3082\u30ed\u30b9\u30c8\u306f\u8a31\u3055\u308c\u306a\u3044\u306e\u304b\u3067\u30a2\u30fc\u30ad\u30c6\u30af\u30c1\u30e3\u306f\u5909\u308f\u308b<\/p>\n<h3>\u751f\u4ea7\u8005\u548c\u6d88\u8d39\u8005\u76f4\u63a5\u8fdb\u884c\u6d88\u606f\u4f20\u9012\u7684\u65b9\u5f0f<\/h3>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">\u4e2d\u9593\u30ce\u30fc\u30c9\u306f\u7528\u610f\u305b\u305aproducer\u304c\u76f4\u63a5\u305d\u308c\u305e\u308c\u306econsumers\u3068\u901a\u4fe1\u3092\u3059\u308b\u65b9\u5f0f<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">UDP\u306e\u30de\u30eb\u30c1\u30ad\u30e3\u30b9\u30c8\u304c\u826f\u304f\u4f7f\u308f\u308c\u3066\u3044\u308b<\/ul>\n<\/li>\n<\/ul>\n<p>UDP\u3060\u3051\u3060\u3068unreliable\u306a\u306e\u3067\u30a2\u30d7\u30ea\u30b1\u30fc\u30b7\u30e7\u30f3\u30d7\u30ed\u30c8\u30b3\u30eb\u3067\u30d1\u30b1\u30c3\u30c8\u30ed\u30b9\u3092\u6551\u3046\u4ed5\u7d44\u307f\u304c\u5fc5\u8981<\/p>\n<p>ZeroMQ\u306f\u3053\u306e\u65b9\u5f0f\u3092\u63a1\u7528\u3057\u3066\u3044\u308b<br \/>\n\u3053\u306e\u65b9\u5f0f\u306f\u4ee5\u4e0b\u306e\u70b9\u3067disadvantage\u304c\u3042\u308b<\/p>\n<p>\u30d1\u30b1\u30c3\u30c8\u30ed\u30b9\u3092\u6551\u3046\u4ed5\u7d44\u307f\u3092\u4f5c\u308a\u3053\u3080\u306e\u304c\u5927\u5909<br \/>\nconsumer\u304c\u6b7b\u3093\u3060\u5834\u5408\u306bproducer\u306b\u30ea\u30c8\u30e9\u30a4\u306e\u4ed5\u7d44\u307f\u3092\u5165\u308c\u306a\u3044\u3068\u3044\u3051\u306a\u3044<br \/>\nproducer\u304c\u6b7b\u3093\u3060\u5834\u5408\u306fconsumers\u3078\u672a\u9001\u4fe1\u306emessage\u3092\u30ed\u30b9\u30c8\u3055\u305b\u306a\u3044\u4ed5\u7d44\u307f\u3092\u4f5c\u308a\u3053\u3080\u306e\u304c\u5927\u5909<\/p>\n<h3>\u4f7f\u7528\u6d88\u606f\u961f\u5217\u7684\u65b9\u6cd5<\/h3>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">\u30e1\u30c3\u30bb\u30fc\u30b8\u30ad\u30e5\u30fc\uff08\u30e1\u30c3\u30bb\u30fc\u30b8\u30d6\u30ed\u30fc\u30ab\u30fc\u3068\u3082\u547c\u3076\uff09\u306fmessage stream\u3092\u6271\u3046\u306e\u306b\u6700\u9069\u5316\u3055\u308c\u305f\u30c7\u30fc\u30bf\u30b9\u30c8\u30a2<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">producers\u306f\u30e1\u30c3\u30bb\u30fc\u30b8\u30ad\u30e5\u30fc\u306bmessage\u3092\u66f8\u304d\u8fbc\u307f\u3001consumers\u306f\u30e1\u30c3\u30bb\u30fc\u30b8\u30ad\u30e5\u30fc\u304b\u3089message\u3092\u8aad\u3080<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">\u3053\u306e\u65b9\u5f0f\u306b\u3059\u308b\u3068\u30e1\u30c3\u30bb\u30fc\u30b8\u30ad\u30e5\u30fc\u3092durable\u306b\u3057\u3066\u304a\u3051\u3070\u8010\u969c\u5bb3\u6027\u3092\u62c5\u4fdd\u3067\u304d\u308b<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\u3053\u306e\u65b9\u5f0f\u3067\u306fproducers\u3068consumers\u306f\u975e\u540c\u671f\u3067\u51e6\u7406\u304c\u884c\u308f\u308c\u308b\u3053\u3068\u306b\u306a\u308b<\/ul>\n<h4>\u6d88\u606f\u961f\u5217\u4e0e\u6570\u636e\u5e93\u7684\u533a\u522b<\/h4>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">\u3044\u304f\u3064\u304b\u306e\u30e1\u30c3\u30bb\u30fc\u30b8\u30ad\u30e5\u30fc\u306f2phase commit\u3092\u4f7f\u3063\u305f\u308a\u306a\u3069\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u3068\u540c\u3058\u6280\u8853\u3092\u4f7f\u3063\u3066\u3044\u305f\u308a\u3082\u3059\u308b\u304c\u9055\u3044\u306f\u8272\u3005\u3042\u308b<\/ul>\n<\/li>\n<\/ul>\n<p>\u30e1\u30c3\u30bb\u30fc\u30b8\u30ad\u30e5\u30fc\u306fconsumers\u306bmessage\u304c\u4f1d\u9054\u3055\u308c\u305f\u3089\u305d\u306emessage\u306f\u524a\u9664\u3055\u308c\u308b\u3002\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u306fexplicit\u306b\u524a\u9664\u3057\u306a\u3044\u9650\u308a\u30c7\u30fc\u30bf\u306f\u4fdd\u5b58\u3055\u308c\u308b<br \/>\n\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u306f\u30bb\u30ab\u30f3\u30c0\u30ea\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u306e\u4ed8\u4e0e\u3084\u591a\u69d8\u306a\u691c\u7d22\u304c\u53ef\u80fd\u3060\u304c\u30e1\u30c3\u30bb\u30fc\u30b8\u30ad\u30e5\u30fc\u306f\u30d1\u30bf\u30fc\u30f3\u30de\u30c3\u30c1\u30f3\u30b0\u3067message\u3092filter\u3067\u304d\u308b\u304f\u3089\u3044<br \/>\n\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u306e\u691c\u7d22\u306fpoint-in-time\u306esnapshot\u3092\u898b\u305b\u308b\u3002\u30e1\u30c3\u30bb\u30fc\u30b8\u30ad\u30e5\u30fc\u306f\u8ffd\u52a0\u3055\u308c\u305fmessage\u3092\u898b\u305b\u308b<\/p>\n<h4>\u591a\u4e2a\u6d88\u8d39\u8005<\/h4>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">\u8907\u6570\u306econsumer\u304c\u540c\u3058topic\u3092\u8aad\u3080\u3068\u304d\u306b\u4ee3\u8868\u7684\u306a\u30d1\u30bf\u30fc\u30f3<\/ul>\n<\/li>\n<\/ul>\n<p>Load balancing<br \/>\n\u8907\u6570\u306econsumer\u306bmessage\u3092\u632f\u308a\u5206\u3051\u308b\u65b9\u5f0f\u3002message\u306e\u91cf\u304c\u591a\u304f\u3066consumer\u306e\u51e6\u7406\u304c\u8ffd\u3044\u3064\u304b\u306a\u3044\u3068\u304d\u306b\u8ca0\u8377\u5206\u6563\u76ee\u7684\u3067\u4f7f\u3046<\/p>\n<p>Fan-out<br \/>\n\u540c\u3058message\u3092\u305d\u308c\u305e\u308c\u306econsumer\u306b\u5272\u308a\u5f53\u3066\u308b\u65b9\u5f0f\u3002\u540c\u3058message\u3092\u4f7f\u3046\u304c\u51e6\u7406\u306f\u5225\u3005\u306e\u6642\u306b\u4f7f\u3063\u305f\u308a\u3059\u308b<\/p>\n<h4>\u5f53\u6d88\u8d39\u8005\u5d29\u6e83\u65f6<\/h4>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">\u30e1\u30c3\u30bb\u30fc\u30b8\u30ed\u30b9\u30c8\u3092\u9632\u3050\u305f\u3081\u306bconsumers\u306f\u660e\u793a\u7684\u306bmessage\u3092\u53d7\u3051\u53d6\u3063\u305f\u3053\u3068\u3092\u30e1\u30c3\u30bb\u30fc\u30b8\u30ad\u30e5\u30fc\u306b\u4f1d\u3048\u308b\u3002\u30e1\u30c3\u30bb\u30fc\u30b8\u30ad\u30e5\u30fc\u306b\u4f1d\u3048\u308b\u3053\u3068\u3092acknowledgments\u3068\u3044\u3046<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">\u30e1\u30c3\u30bb\u30fc\u30b8\u30ad\u30e5\u30fc\u306f\u5168\u3066\u306econsumers\u304b\u3089acknowledgments\u3092\u53d7\u3051\u53d6\u3063\u3066\u3044\u306a\u3044\u3068\u304d\u306fmessage\u3092\u524a\u9664\u3057\u306a\u3044<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">Load balancing\u65b9\u5f0f\u3067\u3042\u308c\u3070\u5225\u306econsumer\u306b\u518d\u9001\u3059\u308b\u3002\u305f\u3060\u3057\u518d\u9001\u304c\u884c\u308f\u308c\u308b\u5834\u5408\u306b\u306fproducer\u304cmessage\u3092\u9001\u3063\u305f\u9806\u5e8f\u3068\u305a\u308c\u308b\u5834\u5408\u304c\u3042\u308b\u3002\uff08\u518d\u9001\u306e\u9593\u306b\u5f8c\u7d9a\u306emessage\u304c\u3059\u3067\u306b\u9001\u3089\u308c\u3066\u3044\u308b\u53ef\u80fd\u6027\u3082\u3042\u308b\u305f\u3081\uff09<\/ul>\n<\/li>\n<\/ul>\n<p>\u9806\u5e8f\u304c\u305a\u308c\u308b\u3068\u307e\u305a\u3044\u5834\u5408\u306fLoad balancing\u65b9\u5f0f\u306f\u4f7f\u3063\u3066\u306f\u30c0\u30e1<\/p>\n<h3>\u6d88\u606f\u961f\u5217\u7684\u5b9e\u73b0<\/h3>\n<p>AMQP\/JMS-style<\/p>\n<p>\u30e1\u30c3\u30bb\u30fc\u30b8\u30ad\u30e5\u30fc\u306fconsumer\u304b\u3089acknowlegments\u3092\u53d7\u3051\u53d6\u3063\u305f\u5f8c\u3001message\u3092\u524a\u9664\u3059\u308b\u65b9\u5f0f<br \/>\n\u53e4\u3044\u30e1\u30c3\u30bb\u30fc\u30b8\u3092\u8aad\u3080\u5fc5\u8981\u306e\u306a\u3044\u5834\u5408\u306a\u3069\u306b\u5229\u7528\u3055\u308c\u308b<\/p>\n<p>Log-based<\/p>\n<p>append-only\u3067disk\u306bmessage\u3092\u66f8\u3044\u3066\u3044\u304f\u65b9\u5f0f<\/p>\n<p>consumer\u306flog\u306e\u6700\u5f8c\u306e\u4f4d\u7f6e\u307e\u3067\u304d\u305f\u3089notification\u304c\u6765\u308b\u307e\u3067\u5f85\u3064<br \/>\n\u3088\u308a\u9ad8\u3044\u30b9\u30eb\u30fc\u30d7\u30c3\u30c8\u3092\u5b9f\u73fe\u3059\u308b\u305f\u3081\u306b\u306flog\u3092\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30cb\u30f3\u30b0\u3059\u308b<br \/>\n\u7570\u306a\u308b\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u306f\u7570\u306a\u308b\u30ce\u30fc\u30c9\u306b\u7f6e\u304f\u3053\u3068\u3082\u53ef\u80fd<br \/>\n\u305d\u308c\u305e\u308c\u306e\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u3067\u306f\u9806\u5e8f\u4fdd\u8a3c\u306e\u305f\u3081\u306b\u305d\u308c\u305e\u308c\u306emessage\u306b\u30b7\u30fc\u30b1\u30f3\u30b9\u30ca\u30f3\u30d0\u30fc\u3092\u5272\u308a\u632f\u308b<br \/>\n\u7570\u306a\u308b\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u9593\u3067\u306f\u30b7\u30fc\u30b1\u30f3\u30b9\u30ca\u30f3\u30d0\u30fc\u306f\u5171\u6709\u3057\u306a\u3044\u306e\u3067\u3001\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u9593\u306e\u9806\u5e8f\u4fdd\u8a3c\u306f\u306a\u3044<br \/>\nApache Kafka\u3084Amazon Kinesis Streams\u306f\u3053\u306e\u3088\u3046\u306a\u5b9f\u88c5\u3092\u3057\u3066\u3044\u308b<br \/>\n\u30ed\u30fc\u30c9\u30d0\u30e9\u30f3\u30b7\u30f3\u30b0\u3059\u308b\u3068\u304d\u306f\u5404consumer\u306b\u62c5\u5f53\u3059\u308b\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u3092\u5272\u308a\u5f53\u3066\u308b\u306e\u304c\u901a\u5e38\u3060\u304c\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u6570\u4ee5\u4e0a\u306bconsumer\u3092\u5897\u3084\u305b\u306a\u3044\u30c7\u30e1\u30ea\u30c3\u30c8\u306f\u3042\u308b<br \/>\n\u30e1\u30c3\u30bb\u30fc\u30b8\u30ad\u30e5\u30fc\u3067\u306f\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u30ec\u30d7\u30ea\u30b1\u30fc\u30b7\u30e7\u30f3\u3068\u540c\u3058\u3088\u3046\u306bmessage\u306blog sequence number\u3092\u5272\u308a\u632f\u3063\u3066\u3001consumer\u3068\u306e\u63a5\u7d9a\u304c\u5207\u308c\u3001\u518d\u63a5\u7d9a\u3059\u308b\u6642\u306b\u30c7\u30fc\u30bf\u30ed\u30b9\u30c8\u306a\u304f\u8aad\u3081\u308b\u4ed5\u7d44\u307f\u304c\u3042\u308b<\/p>\n<h4>\u78c1\u76d8\u7ba1\u7406<\/h4>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">log\u306f\u3042\u308b\u4e00\u5b9a\u306e\u5358\u4f4d\uff08\u30bb\u30b0\u30e1\u30f3\u30c8\u3068\u547c\u3076\uff09\u3067\u5206\u5272\u3055\u308c\u30c7\u30a3\u30b9\u30af\u306b\u4fdd\u5b58\u3055\u308c\u308b\u3002\u53e4\u3044\u30bb\u30b0\u30e1\u30f3\u30c8\u306f\u5b9a\u671f\u7684\u306b\u524a\u9664\u307e\u305f\u306f\u30a2\u30fc\u30ab\u30a4\u30d6\u30b9\u30c8\u30ec\u30fc\u30b8\u3078\u79fb\u52d5\u3055\u305b\u308b<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">\u975e\u5e38\u306b\u9045\u3044consumer\u304c\u3044\u308b\u5834\u5408\u3001\u30bb\u30b0\u30e1\u30f3\u30c8\u3092\u57fa\u672c\u7684\u306b\u524a\u9664\u3057\u306a\u3044\u304c\u30c7\u30a3\u30b9\u30af\u304c\u4e00\u676f\u306b\u306a\u308a\u305d\u3046\u3067\u3042\u308c\u3070\u524a\u9664\u3092\u884c\u3046\u3002\u3053\u308c\u306fcircular buffer, ring buffer\u3068\u547c\u3070\u308c\u308b\u6280\u8853\u3067<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\u5b9f\u88c5\u3092\u884c\u3046<\/ul>\n<h4>\u6162\u7684\u6d88\u8d39\u8005\u7b56\u7565<\/h4>\n<ul class=\"post-ul\">consumer\u304c\u3069\u308c\u3060\u3051\u9045\u308c\u3066\u3044\u308b\u306e\u304b\u3092\u76e3\u8996\u3057\u3066\u304a\u304d\u3001\u9045\u308c\u304c\u9855\u8457\u306a\u5834\u5408\u306f\u30a2\u30e9\u30fc\u30c8\u3092\u3042\u3052\u308b\u3002\u305d\u308c\u3092\u53d7\u3051\u3066\u30aa\u30da\u30ec\u30fc\u30bf\u304c\u624b\u52d5\u3067consumer\u306e\u9045\u308c\u5bfe\u7b56\u3092\u884c\u3046\u3002<\/ul>\n<h2>\u6570\u636e\u5e93\u4e2d\u7684\u6570\u636e\u6d41<\/h2>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">\u73fe\u5728\u306f\u69d8\u3005\u306a\u7a2e\u985e\u306e\u30c7\u30fc\u30bf\u30b9\u30c8\u30ec\u30fc\u30b8\u304c\u4f7f\u308f\u308c\u3066\u304a\u308a(\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u3001DWH\u3001\u30e1\u30c3\u30bb\u30fc\u30b8\u30ad\u30e5\u30fc\u306a\u3069\uff09\u305d\u308c\u3089\u3092\u9023\u643a\u3057\u3066\u30b7\u30b9\u30c6\u30e0\u3092\u4f5c\u308b\u3053\u3068\u304c\u591a\u3044\u3002\u305d\u306e\u6642\u306b\u5fc5\u8981\u306b\u306a\u308b\u306e\u304c\u30c7\u30fc\u30bf\u9023\u643a\u3002\u4e00\u822c\u7684\u306b\u30c7\u30fc\u30bf\u9023\u643a\u3067\u306fETL\u304c\u4f7f\u308f\u308c\u308b<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">ETL\u3067\u306f\u6642\u3068\u3057\u3066\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u304b\u3089\u30c7\u30fc\u30bf\u3092\u53d6\u5f97\u3059\u308b\u6642\u306b\u30c7\u30fc\u30bf\u306e\u30d5\u30eb\u30b3\u30d4\u30fc\u3092\u53d6\u308b\u3053\u3068\u304c\u3042\u308b\u3002\u3057\u304b\u3057\u3053\u308c\u306f\u66f4\u65b0\u30c7\u30fc\u30bf\u4ee5\u5916\u3082\u542b\u3080\u305f\u3081\u660e\u3089\u304b\u306b\u975e\u52b9\u7387\u3002<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\u56de\u907f\u3059\u308b\u624b\u6bb5\u3068\u3057\u3066dual writes\u3068\u3044\u3046\u65b9\u6cd5\u304c\u4f7f\u308f\u308c\u308b\u3053\u3068\u304c\u3042\u308b\u3002\u3053\u308c\u306f\u30a2\u30d7\u30ea\u30b1\u30fc\u30b7\u30e7\u30f3\u304c\uff12\u3064\u306e\u30b7\u30b9\u30c6\u30e0\u306b\u66f8\u304d\u8fbc\u3080\u65b9\u6cd5\u3002\u305f\u3060\u3001\u3053\u306e\u65b9\u6cd5\u306frace condition\u304c\u767a\u751f\u3059\u308b\u53ef\u80fd\u6027\u304c\u3042\u308b\u3053\u3068\u306b\u6ce8\u610f\u3057\u306a\u3044\u3068\u3044\u3051\u306a\u3044\u3002\u307e\u305f\u7247\u5074\u306e\u30b7\u30b9\u30c6\u30e0\u306b\u66f8\u304d\u8fbc\u307f\u306f\u6210\u529f\u3057\u305f\u304c\u3001\u3082\u3046\u4e00\u3064\u306b\u306f\u5931\u6557\u3057\u305f\u5834\u5408\u306e\u5236\u5fa1\u306a\u3069\u3092\u30a2\u30d7\u30ea\u30b1\u30fc\u30b7\u30e7\u30f3\u5074\u3067\u884c\u3046\u306e\u306f\u975e\u5e38\u306b\u53b3\u3057\u3044\u3002<\/ul>\n<h3>\u53d8\u66f4\u6570\u636e\u6355\u83b7\uff08CDC\uff09<\/h3>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">\u3053\u308c\u3089\u306e\u554f\u984c\u3092\u89e3\u6c7a\u3059\u308b\u65b9\u6cd5\u3068\u3057\u3066Change Data Capture\u304c\u51fa\u3066\u304d\u305f\u3002\u3053\u308c\u306f\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u306e\u30c8\u30e9\u30f3\u30b6\u30af\u30b7\u30e7\u30f3\u30ed\u30b0\u3092\u30c7\u30fc\u30bf\u9023\u643a\u3059\u308b\u6280\u8853\u3002<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">\u30c8\u30e9\u30f3\u30b6\u30af\u30b7\u30e7\u30f3\u30ed\u30b0\u306f\u4eca\u307e\u3067\u4ed5\u69d8\u304c\u516c\u958b\u3055\u308c\u3066\u3044\u306a\u304b\u3063\u305f\u304c\u3001API\u3092\u4f7f\u3063\u3066\u30c8\u30e9\u30f3\u30b6\u30af\u30b7\u30e7\u30f3\u30ed\u30b0\u3092\u53d6\u5f97\u3067\u304d\u308b\u3088\u3046\u306b\u3057\u305f\u306e\u304cChange Data Capture\u3002<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">\u66f4\u65b0\u30c7\u30fc\u30bf\u3092\u53d6\u5f97\u3059\u308b\u65b9\u6cd5\u3068\u3057\u3066\u306f\u30c8\u30ea\u30ac\u30fc\u3092\u5b9a\u7fa9\u3057\u3066\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u3078\u306e\u5168\u3066\u306e\u66f4\u65b0\u64cd\u4f5c\u3092\u7279\u5b9a\u306e\u30c6\u30fc\u30d6\u30eb\uff08changelog\u30c6\u30fc\u30d6\u30eb\u3068\u8a00\u3063\u305f\u308a\u3059\u308b\uff09\u306b\u53cd\u6620\u3059\u308b\u3084\u308a\u65b9\u3082\u3042\u308b\u304c\u3001\u3053\u308c\u306f\u30af\u30a8\u30ea\u3092\uff12\u500d\u5b9f\u884c\u3059\u308b\u3053\u3068\u306b\u306a\u308b\u306e\u3067\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9\u30aa\u30fc\u30d0\u30fc\u30d8\u30c3\u30c9\u304c\u5927\u304d\u3044<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">\u521d\u671f\u306e\u69cb\u7bc9\u3068\u3057\u3066\u306f\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u306e\u30b9\u30ca\u30c3\u30d7\u30b7\u30e7\u30c3\u30c8\u3092\u3068\u308a\u3001\u30b3\u30d4\u30fc\u5148\u306e\u30b7\u30b9\u30c6\u30e0\u306b\u53cd\u6620\u3055\u305b\u308b\u3002\u30b9\u30ca\u30c3\u30d7\u30b7\u30e7\u30c3\u30c8\u306f\u30ed\u30b0\u306e\u30aa\u30d5\u30bb\u30c3\u30c8\u304c\u8a18\u9332\u3055\u308c\u3066\u3044\u308b\u306e\u3067\u3001CDC\u3067\u30c7\u30fc\u30bf\u3092\u53cd\u6620\u3055\u305b\u308b\u6642\u306b\u3069\u306e\u30ed\u30b0\u304b\u3089\u53cd\u6620\u3055\u305b\u308c\u3070\u826f\u3044\u304b\u308f\u304b\u308b\u3002<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\u3042\u308b\u30d7\u30e9\u30a4\u30de\u30ea\u30ad\u30fc\u3092\u6301\u3064\u30ec\u30b3\u30fc\u30c9\u304c\u8907\u6570\u56de\u66f4\u65b0\u3055\u308c\u305f\u6642\u306b\u6700\u65b0\u306e\u30ed\u30b0\u3060\u3051\u3092\u53cd\u6620\u3059\u308c\u3070\u30c7\u30fc\u30bf\u306e\u6574\u5408\u6027\u306f\u4fdd\u305f\u308c\u308b\u305f\u3081\u3001\u6700\u65b0\u306e\u30ed\u30b0\u3060\u3051\u3092\u6301\u3064\u3088\u3046\u306bcompaction\u304c\u884c\u308f\u308c\u308b<\/ul>\n<h3>\u4e8b\u4ef6\u6eaf\u6e90<\/h3>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">Event Sourcing\u306fevent\u3068\u3044\u3046\u5f62\u3067\u66f4\u65b0\u64cd\u4f5c\u3092\u8ffd\u52a0\u3057\u3066\u3044\u304f\u30b7\u30b9\u30c6\u30e0\u30c7\u30b6\u30a4\u30f3\u30a2\u30fc\u30ad\u30c6\u30af\u30c1\u30e3<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\u30a2\u30d7\u30ea\u30b1\u30fc\u30b7\u30e7\u30f3\u5074\u3067\u884c\u3046\u306e\u3067\u3001\u66f4\u65b0\u64cd\u4f5c\u3092\u8ffd\u52a0\u3057\u3066\u3044\u304f\u3068\u3044\u3046\u70b9\u3067\u306fCDC\u3068\u540c\u3058\u3060\u304c\u5b9f\u73fe\u3059\u308b\u30ec\u30a4\u30e4\u304c\u7570\u306a\u308b<\/ul>\n<h2>\u5904\u7406\u6570\u636e\u6d41<\/h2>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">\u30b9\u30c8\u30ea\u30fc\u30e0\u51e6\u7406\u3067\u306f\u4f55\u304c\u3067\u304d\u308b\u304b\u3002\u30d0\u30c3\u30c1\u3068\u306e\u9055\u3044\u306f\u30b9\u30c8\u30ea\u30fc\u30e0\u51e6\u7406\u306f\u7d42\u308f\u308a\u304c\u306a\u3044\u3053\u3068\u3002\u305d\u306e\u305f\u3081\u3001\u30bd\u30fc\u30c8\u306a\u3069\u7d42\u308f\u308a\u304c\u306a\u3044\u30c7\u30fc\u30bf\u306b\u5bfe\u3057\u3066\u306f\u3067\u304d\u306a\u3044<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">CEP(Complex Event Processing)\u3067\u306f\u8907\u96d1\u306a\u30d1\u30bf\u30fc\u30f3\u30de\u30c3\u30c1\u30f3\u30b0\u3092\u3059\u308b\u3053\u3068\u304c\u3067\u304d\u308b<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">\u7279\u5b9a\u306e\u30bf\u30a4\u30e0\u30b9\u30d1\u30f3(window\u3068\u547c\u3076)\u3067\u96c6\u8a08\u3092\u884c\u3046<\/ul>\n<\/li>\n<\/ul>\n<p>\u6642\u523b\u306f\u30ce\u30fc\u30c9\u306e\u30ed\u30fc\u30ab\u30eb\u6642\u523b\u3092\u4f7f\u3046\u3053\u3068\u304c\u591a\u3044<br \/>\n\u3057\u304b\u3057\u969c\u5bb3\u306a\u3069\u3067\u30a4\u30d9\u30f3\u30c8\u306e\u767a\u751f\u6642\u523b\u3068\u305d\u306e\u30a4\u30d9\u30f3\u30c8\u304c\u51e6\u7406\u3055\u308c\u308b\u30ce\u30fc\u30c9\u306e\u30ed\u30fc\u30ab\u30eb\u6642\u523b\u304c\u5927\u304d\u304f\u7570\u306a\u308b\u5834\u9762\u304c\u51fa\u308b\u3053\u3068\u304c\u3042\u308b<br \/>\n\u30a4\u30d9\u30f3\u30c8\u306e\u767a\u751f\u6642\u523b\u3067window\u3059\u308b\u3053\u3068\u3082\u3042\u308b\u304c\u3001\u305d\u308c\u3060\u3068\u5168\u3066\u306e\u30a4\u30d9\u30f3\u30c8\u304c\u3044\u3064\u5230\u7740\u3059\u308b\u304b\u306f\u4fdd\u8a3c\u3059\u308b\u3053\u3068\u304c\u3067\u304d\u306a\u3044<br \/>\n\u305d\u306e\u305f\u3081\u7279\u5b9a\u306e\u6642\u9593\u3060\u3051\u5f85\u3063\u305f\u5f8c\u306bwindow\u3092\u5b8c\u4e86\u3055\u305b\u308b\u3002\u5b8c\u4e86\u3057\u305f\u5f8c\u306b\u30a4\u30d9\u30f3\u30c8\u304c\u304d\u305f\u5834\u5408\u306f\u4ee5\u4e0b\u306e\uff12\u3064\u306e\u3069\u3061\u3089\u304b\u3092\u9078\u629e\u3059\u308b<br \/>\n\u30a4\u30d9\u30f3\u30c8\u3092\u6368\u3066\u308b<br \/>\n\u66f4\u65b0\u3055\u308c\u305f\u30c7\u30fc\u30bf\u306b\u30a4\u30d9\u30f3\u30c8\u3092\u4ed8\u52a0\u3059\u308b<\/p>\n<p>window\u306e\u30bf\u30a4\u30d7<\/p>\n<p>Tumbling Window<br \/>\n\u56fa\u5b9a\u306e\u6642\u9593\u306ewindow\u3067\u7279\u5b9a\u306e\u30a4\u30d9\u30f3\u30c8\u306f\u5fc5\u305a\uff11\u3064\u306ewindow\u306b\u5165\u308b\u3002\u4f8b\u3048\u30705\u5206\u9593\u9694\u306709:00 &#8211; 09:04, 09:05 &#8211; 09:09, 09:10 &#8211; 09:14\u306e\u3088\u3046\u306bwindow\u51e6\u7406\u304c\u884c\u308f\u308c\u308b<br \/>\nHopping Window<br \/>\n\u56fa\u5b9a\u306e\u6642\u9593\u306ewindow\u3060\u304cTumbling Window\u3068\u306f\u9055\u3044window\u304c\u30aa\u30fc\u30d0\u30fc\u30e9\u30c3\u30d7\u3059\u308b\u3002\u4f8b\u3048\u3070\uff15\u5206\u306ewindow\u30671\u5206\u9593\u9694\u306e\u5834\u5408\u300109:00 &#8211; 09:04, 09:01 &#8211; 09:05, 09:02 &#8211; 09:06\u306e\u3088\u3046\u306bwindow\u51e6\u7406\u304c\u884c\u308f\u308c\u308b<br \/>\nSliding Window<br \/>\n\u30a4\u30d9\u30f3\u30c8\u306e\u767a\u751f\u3092\u5951\u6a5f\u306b\u30a4\u30d9\u30f3\u30c8\u306e\u767a\u751f\u6642\u523b\u304b\u3089X\u5206\u524d\u306e\u9593\u3092window\u3068\u3057\u3066\u51e6\u7406\u3059\u308b\u3002Apache beam\u3067\u3044\u3046Sliding Window\u306f\u3053\u3053\u3067\u306eHopping Window\u306e\u3053\u3068\u3067\u3059<br \/>\nSession Window<br \/>\n\u30e6\u30fc\u30b6\u30bb\u30c3\u30b7\u30e7\u30f3\u3054\u3068\u306bwindow\u3092\u5b9a\u7fa9\u3059\u308b\u3002\u305d\u306e\u305f\u3081\u3001window\u306e\u6642\u9593\u9593\u9694\u306f\u4e8b\u524d\u306b\u306f\u6c7a\u307e\u3063\u3066\u304a\u3089\u305a\u3001\u30e6\u30fc\u30b6\u30bb\u30c3\u30b7\u30e7\u30f3\u306e\u9577\u3055\u306b\u4f9d\u5b58\u3059\u308b<\/p>\n<h3>\u6d41\u52a0\u5165<\/h3>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">stream-stream joins(window join)<\/ul>\n<\/li>\n<\/ul>\n<p>window\u540c\u58eb\u3092join\u3059\u308b\u65b9\u5f0f<br \/>\n\u4f8b\u3048\u3070CTR\u3092\u898b\u305f\u3044\u6642\u306b\u4f7f\u3046\u3002\u691c\u7d22\u306e\u30ed\u30b0\u3068\u691c\u7d22\u7d50\u679c\u304b\u3089\u4f55\u3092\u30af\u30ea\u30c3\u30af\u3057\u305f\u304b\u306e\u30ed\u30b0\u3092join\u3059\u308b\u3002<\/p>\n<p>stream-table join<\/p>\n<p>\u30a4\u30d9\u30f3\u30c8\u3068\u56fa\u5b9a\u306e\u30c6\u30fc\u30d6\u30eb\u3092join\u3059\u308b\u65b9\u5f0f<br \/>\n\u4f8b\u3048\u3070\u30a4\u30d9\u30f3\u30c8\u306b\u542b\u307e\u308c\u3066\u3044\u308b\u30e6\u30fc\u30b6ID\u3092\u4f7f\u3063\u3066\u30e6\u30fc\u30b6\u306e\u30de\u30b9\u30bf\u8868\u304b\u3089\u30e6\u30fc\u30b6\u540d\u306a\u3069\u3092\u53d6\u5f97\u3059\u308b<br \/>\n\u30b9\u30c8\u30ea\u30fc\u30e0\u30a8\u30f3\u30b8\u30f3\u304b\u3089\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u306b\u63a5\u7d9a\u3059\u308b\u3084\u308a\u65b9\u3082\u3042\u308b\u304c\u3001\u3053\u308c\u3060\u3068\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u901a\u4fe1\u306b\u3088\u308b\u30aa\u30fc\u30d0\u30fc\u30d8\u30c3\u30c9\u3001\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u306e\u8ca0\u8377\u4e0a\u6607\u304c\u8d77\u3053\u308b\u53ef\u80fd\u6027\u304c\u3042\u308b\u3002\u305d\u306e\u305f\u3081\u3001\u5834\u5408\u306b\u3088\u3063\u3066\u306f\u3053\u306e\u30c6\u30fc\u30d6\u30eb\u3092\u30b9\u30c8\u30ea\u30fc\u30e0\u30a8\u30f3\u30b8\u30f3\u5185\u306e\u30e1\u30e2\u30ea\u306b\u30cf\u30c3\u30b7\u30e5\u8868\u3068\u3057\u3066\u6301\u305f\u305b\u308b\u306a\u3069\u3057\u305f\u65b9\u304c\u826f\u3044<\/p>\n<p>table-table join<\/p>\n<p>\u56fa\u5b9a\u306e\u30c6\u30fc\u30d6\u30eb\u540c\u58eb\u3092join\u3059\u308b\u65b9\u5f0f<br \/>\n\u4f55\u304b\u306e\u30a4\u30d9\u30f3\u30c8\u304c\u767a\u751f\u3057\u305f\u6642\u306b\u30ad\u30e3\u30c3\u30b7\u30e5\u306e\u66f4\u65b0\u306a\u3069\u30e1\u30f3\u30c6\u30ca\u30f3\u30b9\u76ee\u7684\u3067\u4f7f\u7528\u3057\u305f\u308a\u3059\u308b<\/p>\n<p>time-dependence join<\/p>\n<p>\u4f8b\u3048\u3070tax rate\u30c6\u30fc\u30d6\u30eb\uff08tax rate\u304c\u5909\u308f\u308b\u305f\u3073\u306b\u66f4\u65b0\u3055\u308c\u308b\uff09\u3068\u30a4\u30d9\u30f3\u30c8\u3092join\u3059\u308b\u6642\u306b\u3069\u306e\u6642\u70b9\u306etax rate\u30c6\u30fc\u30d6\u30eb\u306a\u306e\u304b\u3067\u7d50\u679c\u304c\u5909\u308f\u308b\u3002\u3053\u306e\u3088\u3046\u306a\u6642\u523b\u306b\u4f9d\u5b58\u3059\u308bjoin\u304c\u3042\u308b<br \/>\njoin\u3092\u3059\u308b\u6642\u523b\u306b\u3088\u3063\u3066\u7d50\u679c\u304c\u5909\u308f\u308b\u305f\u3081\u975e\u6c7a\u5b9a\u7684\u3067\u3042\u308a\u597d\u307e\u3057\u304f\u306a\u3044\u3002\u89e3\u6c7a\u3059\u308b\u624b\u9806\u3068\u3057\u3066tax rate\u30c6\u30fc\u30d6\u30eb\u3092historical\u306b\u3057\u3001\u5404\u30ec\u30b3\u30fc\u30c9\u306bidentifier\u3092\u5272\u308a\u5f53\u3066\u308b\u3002\u3053\u306e\u3088\u3046\u306b\u3059\u308c\u3070\u3069\u306e\u6642\u70b9\u306e\u30c7\u30fc\u30bf\u304cjoin\u3055\u308c\u305f\u304b\u304c\u308f\u304b\u308b\u3002<\/p>\n<h2>\u5bb9\u9519\u6027<\/h2>\n<ul class=\"post-ul\">\n<li style=\"list-style-type: none;\">\n<ul class=\"post-ul\">\u51e6\u7406\u304c\u5931\u6557\u3057\u3066\u3082exactly once\u3092\u4fdd\u8a3c\u3059\u308b\u306b\u306f\u3069\u3046\u3057\u305f\u3089\u826f\u3044\u304b\u3002\u30d0\u30c3\u30c1\u51e6\u7406\u306b\u95a2\u3057\u3066\u306f\u5931\u6557\u3057\u305f\u5834\u5408\u306f\u30ea\u30c8\u30e9\u30a4\u3059\u308c\u3070\u826f\u3044\u3060\u3051\u306a\u306e\u3067\u7c21\u5358\u3002\u30b9\u30c8\u30ea\u30fc\u30e0\u51e6\u7406\u306b\u95a2\u3057\u3066\u306f\u3044\u304f\u3064\u304b\u306e\u65b9\u6cd5\u304c\u3042\u308b\u3002<\/ul>\n<\/li>\n<\/ul>\n<p>microbatching<br \/>\n\u30d0\u30c3\u30c1\u51e6\u7406\u3092\u7d30\u304b\u3044\u30bf\u30a4\u30df\u30f3\u30b0\uff08\uff11\u79d2\u9593\u9694\u306a\u3069\uff09\u3067\u5b9f\u884c\u3057\u30b9\u30c8\u30ea\u30fc\u30e0\u51e6\u7406\u306b\u898b\u305b\u308b\u65b9\u5f0f\u3002\u3053\u306e\u65b9\u5f0f\u3067\u3042\u308c\u3070\u30d0\u30c3\u30c1\u51e6\u7406\u3068\u540c\u3058\u3088\u3046\u306b\u30ea\u30c8\u30e9\u30a4\u306e\u6982\u5ff5\u3092\u9069\u7528\u3067\u304d\u308b\u3002<br \/>\nmicrobatching\u306f\u5fc5\u7136\u7684\u306btumbling window\u306b\u306a\u308b<br \/>\nspark streaming\u306fmicrobatching\u3067\u3042\u308b<br \/>\ncheckpointing<br \/>\n\u3042\u308b\u30bf\u30a4\u30df\u30f3\u30b0\u3067\u72b6\u614b\u3092\u4fdd\u5b58\u3059\u308b\u30c1\u30a7\u30c3\u30af\u30dd\u30a4\u30f3\u30c8\u3092\u30b9\u30c8\u30ec\u30fc\u30b8\u306b\u4fdd\u5b58\u3059\u308b\u65b9\u5f0f\u3002\u30d0\u30ea\u30a2\u547d\u4ee4\u3092\u30e1\u30c3\u30bb\u30fc\u30b8\u30b9\u30c8\u30ea\u30fc\u30e0\u306b\u6d41\u3057\u3001\u305d\u306e\u30d0\u30ea\u30a2\u547d\u4ee4\u3092\u30c8\u30ea\u30ac\u306b\u3057\u3066\u30c1\u30a7\u30c3\u30af\u30dd\u30a4\u30f3\u30c8\u3092\u884c\u3046\u3002<br \/>\nApache Flink\u306fcheckpointing\u3092\u884c\u3046<br \/>\natomic commit<br \/>\ndownstream\u306b\u72b6\u614b\u3092\u6e21\u3059\u6642\u306batomic commit(two-phase commit\u3068\u304b)\u3092\u884c\u3044\u72b6\u614b\u3092\u78ba\u5b9f\u306b\u78ba\u5b9a\u3055\u305b\u308b\u65b9\u5f0f<br \/>\nGoogle Cloud Dataflow, VoltDB\u306fatomic commit\u3092\u884c\u3046<\/p>\n<p>\u51aa\u7b49\u6027<\/p>\n<p>\u30a4\u30f3\u30af\u30ea\u30e1\u30f3\u30c8\u51e6\u7406\u306f\u51aa\u7b49\u3067\u306f\u306a\u3044\u3002\u51aa\u7b49\u306b\u3059\u308b\u305f\u3081\u306bApache Kafka\u3067\u306fmessage\u306e\u30aa\u30d5\u30bb\u30c3\u30c8\u3092\u3082\u3064\u3002\u30b9\u30c8\u30ea\u30fc\u30e0\u51e6\u7406\u306e\u4e2d\u3067\u30aa\u30d5\u30bb\u30c3\u30c8\u3092\u78ba\u8a8d\u3057\u3059\u3067\u306b\u51e6\u7406\u3057\u305fmessage\u306a\u306e\u304b\u3069\u3046\u304b\u3092\u5224\u65ad\u3057\u3066\u51aa\u7b49\u6027\u3092\u7dad\u6301\u3059\u308b\u3002<\/p>\n<p>\u5fa9\u65e7<\/p>\n<p>Apache Flink\u3067\u306f\u30b9\u30ca\u30c3\u30d7\u30b7\u30e7\u30c3\u30c8\u3092HDFS\u306a\u3069\u306e\u5171\u6709\u30b9\u30c8\u30ec\u30fc\u30b8\u306b\u4fdd\u5b58\u3059\u308b\u3002node failure\u304c\u8d77\u304d\u305f\u6642\u306b\u5225\u306enode\u304c\u30b9\u30ca\u30c3\u30d7\u30b7\u30e7\u30c3\u30c8\u3092\u8aad\u307f\u8fbc\u307f\u51e6\u7406\u3092\u518d\u958b\u3059\u308b\u3002<br \/>\nSamza, Kafka streams\u306f\u72b6\u614b\u3092Kafka\u306etopic\u306b\u4fdd\u5b58\u3059\u308b<br \/>\nVoltDB\u306f\u72b6\u614b\u3092\u5225\u306e\u30ce\u30fc\u30c9\u306b\u30ec\u30d7\u30ea\u30b1\u30fc\u30c8\u3059\u308b<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u6839\u636e\u300aDesigning Data-Intensive Applications: The Big Ideas [&hellip;]<\/p>\n","protected":false},"author":8,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-47100","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.5 (Yoast SEO v21.5) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>\u601d\u8003\u4e32\u6d41\u5904\u7406 - Blog - Silicon Cloud<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.silicloud.com\/zh\/blog\/\u601d\u8003\u4e32\u6d41\u5904\u7406\/\" \/>\n<meta property=\"og:locale\" content=\"zh_CN\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"\u601d\u8003\u4e32\u6d41\u5904\u7406\" \/>\n<meta property=\"og:description\" content=\"\u6839\u636e\u300aDesigning Data-Intensive Applications: The Big Ideas [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.silicloud.com\/zh\/blog\/\u601d\u8003\u4e32\u6d41\u5904\u7406\/\" \/>\n<meta property=\"og:site_name\" content=\"Blog - Silicon Cloud\" \/>\n<meta property=\"article:published_time\" content=\"2023-05-25T17:17:46+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-04-29T09:13:16+00:00\" \/>\n<meta name=\"author\" content=\"\u96c5, \u609f\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"\u4f5c\u8005\" \/>\n\t<meta name=\"twitter:data1\" content=\"\u96c5, \u609f\" \/>\n\t<meta name=\"twitter:label2\" content=\"\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 \u5206\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/%e6%80%9d%e8%80%83%e4%b8%b2%e6%b5%81%e5%a4%84%e7%90%86\/\",\"url\":\"https:\/\/www.silicloud.com\/zh\/blog\/%e6%80%9d%e8%80%83%e4%b8%b2%e6%b5%81%e5%a4%84%e7%90%86\/\",\"name\":\"\u601d\u8003\u4e32\u6d41\u5904\u7406 - Blog - Silicon Cloud\",\"isPartOf\":{\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#website\"},\"datePublished\":\"2023-05-25T17:17:46+00:00\",\"dateModified\":\"2024-04-29T09:13:16+00:00\",\"author\":{\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/f044a4b7fa4ee2701702942002419ca6\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/%e6%80%9d%e8%80%83%e4%b8%b2%e6%b5%81%e5%a4%84%e7%90%86\/#breadcrumb\"},\"inLanguage\":\"zh-Hans\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.silicloud.com\/zh\/blog\/%e6%80%9d%e8%80%83%e4%b8%b2%e6%b5%81%e5%a4%84%e7%90%86\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/%e6%80%9d%e8%80%83%e4%b8%b2%e6%b5%81%e5%a4%84%e7%90%86\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"\u9996\u9875\",\"item\":\"https:\/\/www.silicloud.com\/zh\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"\u601d\u8003\u4e32\u6d41\u5904\u7406\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#website\",\"url\":\"https:\/\/www.silicloud.com\/zh\/blog\/\",\"name\":\"Blog - Silicon Cloud\",\"description\":\"\",\"inLanguage\":\"zh-Hans\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/f044a4b7fa4ee2701702942002419ca6\",\"name\":\"\u96c5, \u609f\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"zh-Hans\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/e71a913e914f1aad1efc391f92084294bac54bc782acd289638580134cf667a6?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/e71a913e914f1aad1efc391f92084294bac54bc782acd289638580134cf667a6?s=96&d=mm&r=g\",\"caption\":\"\u96c5, \u609f\"},\"url\":\"https:\/\/www.silicloud.com\/zh\/blog\/author\/yawu\/\"},{\"@type\":\"ImageObject\",\"inLanguage\":\"zh-Hans\",\"@id\":\"https:\/\/www.silicloud.com\/zh\/blog\/%e6%80%9d%e8%80%83%e4%b8%b2%e6%b5%81%e5%a4%84%e7%90%86\/#local-main-organization-logo\",\"url\":\"\",\"contentUrl\":\"\",\"caption\":\"Blog - Silicon Cloud\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"\u601d\u8003\u4e32\u6d41\u5904\u7406 - Blog - Silicon Cloud","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.silicloud.com\/zh\/blog\/\u601d\u8003\u4e32\u6d41\u5904\u7406\/","og_locale":"zh_CN","og_type":"article","og_title":"\u601d\u8003\u4e32\u6d41\u5904\u7406","og_description":"\u6839\u636e\u300aDesigning Data-Intensive Applications: The Big Ideas [&hellip;]","og_url":"https:\/\/www.silicloud.com\/zh\/blog\/\u601d\u8003\u4e32\u6d41\u5904\u7406\/","og_site_name":"Blog - Silicon Cloud","article_published_time":"2023-05-25T17:17:46+00:00","article_modified_time":"2024-04-29T09:13:16+00:00","author":"\u96c5, \u609f","twitter_card":"summary_large_image","twitter_misc":{"\u4f5c\u8005":"\u96c5, \u609f","\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4":"1 \u5206"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.silicloud.com\/zh\/blog\/%e6%80%9d%e8%80%83%e4%b8%b2%e6%b5%81%e5%a4%84%e7%90%86\/","url":"https:\/\/www.silicloud.com\/zh\/blog\/%e6%80%9d%e8%80%83%e4%b8%b2%e6%b5%81%e5%a4%84%e7%90%86\/","name":"\u601d\u8003\u4e32\u6d41\u5904\u7406 - Blog - Silicon Cloud","isPartOf":{"@id":"https:\/\/www.silicloud.com\/zh\/blog\/#website"},"datePublished":"2023-05-25T17:17:46+00:00","dateModified":"2024-04-29T09:13:16+00:00","author":{"@id":"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/f044a4b7fa4ee2701702942002419ca6"},"breadcrumb":{"@id":"https:\/\/www.silicloud.com\/zh\/blog\/%e6%80%9d%e8%80%83%e4%b8%b2%e6%b5%81%e5%a4%84%e7%90%86\/#breadcrumb"},"inLanguage":"zh-Hans","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.silicloud.com\/zh\/blog\/%e6%80%9d%e8%80%83%e4%b8%b2%e6%b5%81%e5%a4%84%e7%90%86\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.silicloud.com\/zh\/blog\/%e6%80%9d%e8%80%83%e4%b8%b2%e6%b5%81%e5%a4%84%e7%90%86\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"\u9996\u9875","item":"https:\/\/www.silicloud.com\/zh\/blog\/"},{"@type":"ListItem","position":2,"name":"\u601d\u8003\u4e32\u6d41\u5904\u7406"}]},{"@type":"WebSite","@id":"https:\/\/www.silicloud.com\/zh\/blog\/#website","url":"https:\/\/www.silicloud.com\/zh\/blog\/","name":"Blog - Silicon Cloud","description":"","inLanguage":"zh-Hans"},{"@type":"Person","@id":"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/f044a4b7fa4ee2701702942002419ca6","name":"\u96c5, \u609f","image":{"@type":"ImageObject","inLanguage":"zh-Hans","@id":"https:\/\/www.silicloud.com\/zh\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/e71a913e914f1aad1efc391f92084294bac54bc782acd289638580134cf667a6?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e71a913e914f1aad1efc391f92084294bac54bc782acd289638580134cf667a6?s=96&d=mm&r=g","caption":"\u96c5, \u609f"},"url":"https:\/\/www.silicloud.com\/zh\/blog\/author\/yawu\/"},{"@type":"ImageObject","inLanguage":"zh-Hans","@id":"https:\/\/www.silicloud.com\/zh\/blog\/%e6%80%9d%e8%80%83%e4%b8%b2%e6%b5%81%e5%a4%84%e7%90%86\/#local-main-organization-logo","url":"","contentUrl":"","caption":"Blog - Silicon Cloud"}]}},"_links":{"self":[{"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/posts\/47100","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/comments?post=47100"}],"version-history":[{"count":2,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/posts\/47100\/revisions"}],"predecessor-version":[{"id":86566,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/posts\/47100\/revisions\/86566"}],"wp:attachment":[{"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/media?parent=47100"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/categories?post=47100"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.silicloud.com\/zh\/blog\/wp-json\/wp\/v2\/tags?post=47100"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}