{"id":22607,"date":"2024-03-15T23:52:09","date_gmt":"2024-03-15T23:52:09","guid":{"rendered":"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/"},"modified":"2024-03-21T23:17:16","modified_gmt":"2024-03-21T23:17:16","slug":"how-to-read-incremental-data-from-hbase","status":"publish","type":"post","link":"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/","title":{"rendered":"How to read incremental data from HBase?"},"content":{"rendered":"<p>There are two methods you can use to read incremental data from HBase.<\/p>\n<ol>\n<li>Create a connection object for HBase and specify the table name and column family to be read.<\/li>\n<li>Use the Scan object to set the scanning range and filter conditions in order to only retrieve incremental data.<\/li>\n<li>Get a ResultScanner object by using the getScanner method of the Table object.<\/li>\n<li>Iterate through the ResultScanner object, using the Result object to retrieve the data from each row.<\/li>\n<\/ol>\n<p>Here is an example code snippet demonstrating how to perform incremental reading using the Java API of HBase.<\/p>\n<pre class=\"post-pre\"><code><span class=\"hljs-type\">Configuration<\/span> <span class=\"hljs-variable\">config<\/span> <span class=\"hljs-operator\">=<\/span> HBaseConfiguration.create();\r\n<span class=\"hljs-type\">Connection<\/span> <span class=\"hljs-variable\">connection<\/span> <span class=\"hljs-operator\">=<\/span> ConnectionFactory.createConnection(config);\r\n<span class=\"hljs-type\">TableName<\/span> <span class=\"hljs-variable\">tableName<\/span> <span class=\"hljs-operator\">=<\/span> TableName.valueOf(<span class=\"hljs-string\">\"your_table_name\"<\/span>);\r\n<span class=\"hljs-type\">Table<\/span> <span class=\"hljs-variable\">table<\/span> <span class=\"hljs-operator\">=<\/span> connection.getTable(tableName);\r\n\r\n<span class=\"hljs-type\">Scan<\/span> <span class=\"hljs-variable\">scan<\/span> <span class=\"hljs-operator\">=<\/span> <span class=\"hljs-keyword\">new<\/span> <span class=\"hljs-title class_\">Scan<\/span>();\r\nscan.setStartRow(Bytes.toBytes(<span class=\"hljs-string\">\"start_row_key\"<\/span>));\r\nscan.setStopRow(Bytes.toBytes(<span class=\"hljs-string\">\"stop_row_key\"<\/span>));\r\n\r\n<span class=\"hljs-type\">ResultScanner<\/span> <span class=\"hljs-variable\">scanner<\/span> <span class=\"hljs-operator\">=<\/span> table.getScanner(scan);\r\n<span class=\"hljs-keyword\">for<\/span> (Result result : scanner) {\r\n    <span class=\"hljs-comment\">\/\/ \u5904\u7406\u6bcf\u4e00\u884c\u7684\u6570\u636e<\/span>\r\n    <span class=\"hljs-keyword\">for<\/span> (Cell cell : result.rawCells()) {\r\n        <span class=\"hljs-comment\">\/\/ \u5904\u7406\u6bcf\u4e00\u4e2a\u5355\u5143\u683c\u7684\u6570\u636e<\/span>\r\n        <span class=\"hljs-type\">byte<\/span>[] row = CellUtil.cloneRow(cell);\r\n        <span class=\"hljs-type\">byte<\/span>[] family = CellUtil.cloneFamily(cell);\r\n        <span class=\"hljs-type\">byte<\/span>[] qualifier = CellUtil.cloneQualifier(cell);\r\n        <span class=\"hljs-type\">byte<\/span>[] value = CellUtil.cloneValue(cell);\r\n        <span class=\"hljs-comment\">\/\/ \u5904\u7406\u6570\u636e<\/span>\r\n    }\r\n}\r\n\r\nscanner.close();\r\ntable.close();\r\nconnection.close();\r\n<\/code><\/pre>\n<ol>\n<li>Create an HBase connection object and specify the table name and column family to read from.<\/li>\n<li>Use the TableMapReduceUtil class to create a Job object and configure its input and output formats.<\/li>\n<li>Set the scanning range and filter conditions using the Scan object to only retrieve incremental data.<\/li>\n<li>Use the initTableMapperJob method of the TableMapReduceUtil class to configure the Mapper class, input table name, and Scan object.<\/li>\n<li>Use the `initTableReducerJob` method of the `TableMapReduceUtil` class to specify the Reducer class, output table name, and connection object.<\/li>\n<li>Execute the Job object.<\/li>\n<\/ol>\n<p>Here is a code snippet demonstrating how to use HBase&#8217;s MapReduce for incremental reading.<\/p>\n<pre class=\"post-pre\"><code><span class=\"hljs-type\">Configuration<\/span> <span class=\"hljs-variable\">config<\/span> <span class=\"hljs-operator\">=<\/span> HBaseConfiguration.create();\r\n<span class=\"hljs-type\">Connection<\/span> <span class=\"hljs-variable\">connection<\/span> <span class=\"hljs-operator\">=<\/span> ConnectionFactory.createConnection(config);\r\n<span class=\"hljs-type\">TableName<\/span> <span class=\"hljs-variable\">inputTableName<\/span> <span class=\"hljs-operator\">=<\/span> TableName.valueOf(<span class=\"hljs-string\">\"your_input_table_name\"<\/span>);\r\n<span class=\"hljs-type\">TableName<\/span> <span class=\"hljs-variable\">outputTableName<\/span> <span class=\"hljs-operator\">=<\/span> TableName.valueOf(<span class=\"hljs-string\">\"your_output_table_name\"<\/span>);\r\n\r\n<span class=\"hljs-type\">Scan<\/span> <span class=\"hljs-variable\">scan<\/span> <span class=\"hljs-operator\">=<\/span> <span class=\"hljs-keyword\">new<\/span> <span class=\"hljs-title class_\">Scan<\/span>();\r\nscan.setStartRow(Bytes.toBytes(<span class=\"hljs-string\">\"start_row_key\"<\/span>));\r\nscan.setStopRow(Bytes.toBytes(<span class=\"hljs-string\">\"stop_row_key\"<\/span>));\r\n\r\n<span class=\"hljs-type\">Job<\/span> <span class=\"hljs-variable\">job<\/span> <span class=\"hljs-operator\">=<\/span> Job.getInstance(config);\r\njob.setJarByClass(IncrementalRead.class);\r\njob.setMapperClass(IncrementalReadMapper.class);\r\njob.setReducerClass(IncrementalReadReducer.class);\r\njob.setInputFormatClass(TableInputFormat.class);\r\njob.setOutputFormatClass(TableOutputFormat.class);\r\nTableMapReduceUtil.initTableMapperJob(inputTableName, scan, IncrementalReadMapper.class, ImmutableBytesWritable.class, Put.class, job);\r\nTableMapReduceUtil.initTableReducerJob(outputTableName.getNameAsString(), IncrementalReadReducer.class, job);\r\n\r\njob.waitForCompletion(<span class=\"hljs-literal\">true<\/span>);\r\n\r\nconnection.close();\r\n<\/code><\/pre>\n<p>Please note that the sample code above is just a simple example, you will need to adjust and expand it according to your specific needs.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>There are two methods you can use to read incremental data from HBase. Create a connection object for HBase and specify the table name and column family to be read. Use the Scan object to set the scanning range and filter conditions in order to only retrieve incremental data. Get a ResultScanner object by using [&hellip;]<\/p>\n","protected":false},"author":12,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_import_markdown_pro_load_document_selector":0,"_import_markdown_pro_submit_text_textarea":"","footnotes":""},"categories":[1],"tags":[],"class_list":["post-22607","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.5 (Yoast SEO v21.5) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>How to read incremental data from HBase? - Blog - Silicon Cloud<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How to read incremental data from HBase?\" \/>\n<meta property=\"og:description\" content=\"There are two methods you can use to read incremental data from HBase. Create a connection object for HBase and specify the table name and column family to be read. Use the Scan object to set the scanning range and filter conditions in order to only retrieve incremental data. Get a ResultScanner object by using [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/\" \/>\n<meta property=\"og:site_name\" content=\"Blog - Silicon Cloud\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/SiliCloudGlobal\/\" \/>\n<meta property=\"article:published_time\" content=\"2024-03-15T23:52:09+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-03-21T23:17:16+00:00\" \/>\n<meta name=\"author\" content=\"Liam\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@SiliCloudGlobal\" \/>\n<meta name=\"twitter:site\" content=\"@SiliCloudGlobal\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Liam\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/\"},\"author\":{\"name\":\"Liam\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/23786905eb7b377f45ddb01c17da7671\"},\"headline\":\"How to read incremental data from HBase?\",\"datePublished\":\"2024-03-15T23:52:09+00:00\",\"dateModified\":\"2024-03-21T23:17:16+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/\"},\"wordCount\":233,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#organization\"},\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/\",\"url\":\"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/\",\"name\":\"How to read incremental data from HBase? - Blog - Silicon Cloud\",\"isPartOf\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#website\"},\"datePublished\":\"2024-03-15T23:52:09+00:00\",\"dateModified\":\"2024-03-21T23:17:16+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.silicloud.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How to read incremental data from HBase?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#website\",\"url\":\"https:\/\/www.silicloud.com\/blog\/\",\"name\":\"Silicon Cloud Blog\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#organization\"},\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#organization\",\"name\":\"Silicon Cloud Blog\",\"url\":\"https:\/\/www.silicloud.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png\",\"contentUrl\":\"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png\",\"width\":1024,\"height\":1024,\"caption\":\"Silicon Cloud Blog\"},\"image\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/SiliCloudGlobal\/\",\"https:\/\/twitter.com\/SiliCloudGlobal\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/23786905eb7b377f45ddb01c17da7671\",\"name\":\"Liam\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/8d37ed3e7f770dde8bf069ba0b4298688028c3abaacf1131742fc1352d174ebd?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/8d37ed3e7f770dde8bf069ba0b4298688028c3abaacf1131742fc1352d174ebd?s=96&d=mm&r=g\",\"caption\":\"Liam\"},\"sameAs\":[\"http:\/\/Wilson\"],\"url\":\"https:\/\/www.silicloud.com\/blog\/author\/liamwilson\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"How to read incremental data from HBase? - Blog - Silicon Cloud","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/","og_locale":"en_US","og_type":"article","og_title":"How to read incremental data from HBase?","og_description":"There are two methods you can use to read incremental data from HBase. Create a connection object for HBase and specify the table name and column family to be read. Use the Scan object to set the scanning range and filter conditions in order to only retrieve incremental data. Get a ResultScanner object by using [&hellip;]","og_url":"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/","og_site_name":"Blog - Silicon Cloud","article_publisher":"https:\/\/www.facebook.com\/SiliCloudGlobal\/","article_published_time":"2024-03-15T23:52:09+00:00","article_modified_time":"2024-03-21T23:17:16+00:00","author":"Liam","twitter_card":"summary_large_image","twitter_creator":"@SiliCloudGlobal","twitter_site":"@SiliCloudGlobal","twitter_misc":{"Written by":"Liam","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/#article","isPartOf":{"@id":"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/"},"author":{"name":"Liam","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/23786905eb7b377f45ddb01c17da7671"},"headline":"How to read incremental data from HBase?","datePublished":"2024-03-15T23:52:09+00:00","dateModified":"2024-03-21T23:17:16+00:00","mainEntityOfPage":{"@id":"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/"},"wordCount":233,"commentCount":0,"publisher":{"@id":"https:\/\/www.silicloud.com\/blog\/#organization"},"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/","url":"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/","name":"How to read incremental data from HBase? - Blog - Silicon Cloud","isPartOf":{"@id":"https:\/\/www.silicloud.com\/blog\/#website"},"datePublished":"2024-03-15T23:52:09+00:00","dateModified":"2024-03-21T23:17:16+00:00","breadcrumb":{"@id":"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.silicloud.com\/blog\/how-to-read-incremental-data-from-hbase\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.silicloud.com\/blog\/"},{"@type":"ListItem","position":2,"name":"How to read incremental data from HBase?"}]},{"@type":"WebSite","@id":"https:\/\/www.silicloud.com\/blog\/#website","url":"https:\/\/www.silicloud.com\/blog\/","name":"Silicon Cloud Blog","description":"","publisher":{"@id":"https:\/\/www.silicloud.com\/blog\/#organization"},"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.silicloud.com\/blog\/#organization","name":"Silicon Cloud Blog","url":"https:\/\/www.silicloud.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png","contentUrl":"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png","width":1024,"height":1024,"caption":"Silicon Cloud Blog"},"image":{"@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/SiliCloudGlobal\/","https:\/\/twitter.com\/SiliCloudGlobal"]},{"@type":"Person","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/23786905eb7b377f45ddb01c17da7671","name":"Liam","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/8d37ed3e7f770dde8bf069ba0b4298688028c3abaacf1131742fc1352d174ebd?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/8d37ed3e7f770dde8bf069ba0b4298688028c3abaacf1131742fc1352d174ebd?s=96&d=mm&r=g","caption":"Liam"},"sameAs":["http:\/\/Wilson"],"url":"https:\/\/www.silicloud.com\/blog\/author\/liamwilson\/"}]}},"_links":{"self":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts\/22607","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/comments?post=22607"}],"version-history":[{"count":1,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts\/22607\/revisions"}],"predecessor-version":[{"id":56537,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts\/22607\/revisions\/56537"}],"wp:attachment":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/media?parent=22607"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/categories?post=22607"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/tags?post=22607"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}