{"id":3361,"date":"2024-03-13T06:52:22","date_gmt":"2024-03-13T06:52:22","guid":{"rendered":"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/"},"modified":"2025-07-30T15:07:52","modified_gmt":"2025-07-30T15:07:52","slug":"how-do-kafka-consumers-manage-message-offsets","status":"publish","type":"post","link":"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/","title":{"rendered":"Kafka Consumer Offsets: Manual vs Auto"},"content":{"rendered":"

Consumers in Kafka can manage message offsets in two ways: manually or automatically.<\/p>\n

    \n
  1. Manual management: Consumers can manually submit the offset of messages by invoking the commitSync or commitAsync methods. In manual management mode, consumers have the flexibility to decide when to submit the offset and which offset to submit.<\/li>\n<\/ol>\n

    The sample code is as follows:<\/p>\n

    while<\/span> (true<\/span>) {\r\n    ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100<\/span>));\r\n    for<\/span> (ConsumerRecord<String, String> record : records) {\r\n        \/\/ \u5904\u7406\u6d88\u606f<\/span>\r\n    }\r\n    consumer.commitSync();\r\n}\r\n<\/code><\/pre>\n
      \n
    1. Automatic management: Consumers can set the enable.auto.commit parameter to true, allowing Kafka to automatically manage message offsets. In automatic management mode, Kafka will periodically automatically submit message offsets.<\/li>\n<\/ol>\n
      The sample code is as follows:<\/p>\n
      Properties<\/span> props<\/span> =<\/span> new<\/span> Properties<\/span>();\r\nprops.put(\"bootstrap.servers\"<\/span>, \"localhost:9092\"<\/span>);\r\nprops.put(\"group.id\"<\/span>, \"test-group\"<\/span>);\r\nprops.put(\"enable.auto.commit\"<\/span>, \"true\"<\/span>);\r\n\r\nKafkaConsumer<String, String> consumer = new<\/span> KafkaConsumer<\/span><>(props);\r\nconsumer.subscribe(Collections.singletonList(\"test-topic\"<\/span>));\r\n\r\nwhile<\/span> (true<\/span>) {\r\n    ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100<\/span>));\r\n    for<\/span> (ConsumerRecord<String, String> record : records) {\r\n        \/\/ \u5904\u7406\u6d88\u606f<\/span>\r\n    }\r\n}\r\n<\/code><\/pre>\n
      Consumers have the option to choose between manually managing or automatically managing message offsets based on their actual needs. Manual management offers more precise control, but requires consumers to write more code to handle offset submission. Automatic management is more convenient, but may result in message duplication due to periodic offset submission.<\/p>\n","protected":false},"excerpt":{"rendered":"
      Consumers in Kafka can manage message offsets in two ways: manually or automatically. Manual management: Consumers can manually submit the offset of messages by invoking the commitSync or commitAsync methods. In manual management mode, consumers have the flexibility to decide when to submit the offset and which offset to submit. The sample code is as […]<\/p>\n","protected":false},"author":14,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_import_markdown_pro_load_document_selector":0,"_import_markdown_pro_submit_text_textarea":"","footnotes":""},"categories":[1],"tags":[1439,879,1436,1438,1437],"class_list":["post-3361","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-commitsync","tag-kafka","tag-kafka-consumer","tag-message-offsets","tag-offset-management"],"yoast_head":"\nKafka Consumer Offsets: Manual vs Auto - Blog - Silicon Cloud<\/title>\n<meta name=\"description\" content=\"Learn manual (commitSync\/commitAsync) vs automatic methods for Kafka consumer offset control.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Kafka Consumer Offsets: Manual vs Auto\" \/>\n<meta property=\"og:description\" content=\"Learn manual (commitSync\/commitAsync) vs automatic methods for Kafka consumer offset control.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/\" \/>\n<meta property=\"og:site_name\" content=\"Blog - Silicon Cloud\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/SiliCloudGlobal\/\" \/>\n<meta property=\"article:published_time\" content=\"2024-03-13T06:52:22+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-07-30T15:07:52+00:00\" \/>\n<meta name=\"author\" content=\"Noah Thompson\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@SiliCloudGlobal\" \/>\n<meta name=\"twitter:site\" content=\"@SiliCloudGlobal\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Noah Thompson\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/\"},\"author\":{\"name\":\"Noah Thompson\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/2e83cc6ab9f60d36921c2d0f9f280f4a\"},\"headline\":\"Kafka Consumer Offsets: Manual vs Auto\",\"datePublished\":\"2024-03-13T06:52:22+00:00\",\"dateModified\":\"2025-07-30T15:07:52+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/\"},\"wordCount\":150,\"publisher\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#organization\"},\"keywords\":[\"CommitSync\",\"Kafka\",\"Kafka Consumer\",\"Message Offsets\",\"Offset Management\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/\",\"url\":\"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/\",\"name\":\"Kafka Consumer Offsets: Manual vs Auto - Blog - Silicon Cloud\",\"isPartOf\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#website\"},\"datePublished\":\"2024-03-13T06:52:22+00:00\",\"dateModified\":\"2025-07-30T15:07:52+00:00\",\"description\":\"Learn manual (commitSync\/commitAsync) vs automatic methods for Kafka consumer offset control.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.silicloud.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Kafka Consumer Offsets: Manual vs Auto\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#website\",\"url\":\"https:\/\/www.silicloud.com\/blog\/\",\"name\":\"Silicon Cloud Blog\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#organization\"},\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#organization\",\"name\":\"Silicon Cloud Blog\",\"url\":\"https:\/\/www.silicloud.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png\",\"contentUrl\":\"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png\",\"width\":1024,\"height\":1024,\"caption\":\"Silicon Cloud Blog\"},\"image\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/SiliCloudGlobal\/\",\"https:\/\/twitter.com\/SiliCloudGlobal\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/2e83cc6ab9f60d36921c2d0f9f280f4a\",\"name\":\"Noah Thompson\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/350e537e1530ede2762ee0237e877d6693f4f7163ab4f303202cc9a6b27b6cb4?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/350e537e1530ede2762ee0237e877d6693f4f7163ab4f303202cc9a6b27b6cb4?s=96&d=mm&r=g\",\"caption\":\"Noah Thompson\"},\"url\":\"https:\/\/www.silicloud.com\/blog\/author\/noahthompson\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Kafka Consumer Offsets: Manual vs Auto - Blog - Silicon Cloud","description":"Learn manual (commitSync\/commitAsync) vs automatic methods for Kafka consumer offset control.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/","og_locale":"en_US","og_type":"article","og_title":"Kafka Consumer Offsets: Manual vs Auto","og_description":"Learn manual (commitSync\/commitAsync) vs automatic methods for Kafka consumer offset control.","og_url":"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/","og_site_name":"Blog - Silicon Cloud","article_publisher":"https:\/\/www.facebook.com\/SiliCloudGlobal\/","article_published_time":"2024-03-13T06:52:22+00:00","article_modified_time":"2025-07-30T15:07:52+00:00","author":"Noah Thompson","twitter_card":"summary_large_image","twitter_creator":"@SiliCloudGlobal","twitter_site":"@SiliCloudGlobal","twitter_misc":{"Written by":"Noah Thompson","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/#article","isPartOf":{"@id":"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/"},"author":{"name":"Noah Thompson","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/2e83cc6ab9f60d36921c2d0f9f280f4a"},"headline":"Kafka Consumer Offsets: Manual vs Auto","datePublished":"2024-03-13T06:52:22+00:00","dateModified":"2025-07-30T15:07:52+00:00","mainEntityOfPage":{"@id":"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/"},"wordCount":150,"publisher":{"@id":"https:\/\/www.silicloud.com\/blog\/#organization"},"keywords":["CommitSync","Kafka","Kafka Consumer","Message Offsets","Offset Management"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/","url":"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/","name":"Kafka Consumer Offsets: Manual vs Auto - Blog - Silicon Cloud","isPartOf":{"@id":"https:\/\/www.silicloud.com\/blog\/#website"},"datePublished":"2024-03-13T06:52:22+00:00","dateModified":"2025-07-30T15:07:52+00:00","description":"Learn manual (commitSync\/commitAsync) vs automatic methods for Kafka consumer offset control.","breadcrumb":{"@id":"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.silicloud.com\/blog\/how-do-kafka-consumers-manage-message-offsets\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.silicloud.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Kafka Consumer Offsets: Manual vs Auto"}]},{"@type":"WebSite","@id":"https:\/\/www.silicloud.com\/blog\/#website","url":"https:\/\/www.silicloud.com\/blog\/","name":"Silicon Cloud Blog","description":"","publisher":{"@id":"https:\/\/www.silicloud.com\/blog\/#organization"},"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.silicloud.com\/blog\/#organization","name":"Silicon Cloud Blog","url":"https:\/\/www.silicloud.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png","contentUrl":"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png","width":1024,"height":1024,"caption":"Silicon Cloud Blog"},"image":{"@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/SiliCloudGlobal\/","https:\/\/twitter.com\/SiliCloudGlobal"]},{"@type":"Person","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/2e83cc6ab9f60d36921c2d0f9f280f4a","name":"Noah Thompson","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/350e537e1530ede2762ee0237e877d6693f4f7163ab4f303202cc9a6b27b6cb4?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/350e537e1530ede2762ee0237e877d6693f4f7163ab4f303202cc9a6b27b6cb4?s=96&d=mm&r=g","caption":"Noah Thompson"},"url":"https:\/\/www.silicloud.com\/blog\/author\/noahthompson\/"}]}},"_links":{"self":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts\/3361","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/users\/14"}],"replies":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/comments?post=3361"}],"version-history":[{"count":2,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts\/3361\/revisions"}],"predecessor-version":[{"id":147996,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts\/3361\/revisions\/147996"}],"wp:attachment":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/media?parent=3361"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/categories?post=3361"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/tags?post=3361"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}