{"id":4010,"date":"2024-03-13T07:49:05","date_gmt":"2024-03-13T07:49:05","guid":{"rendered":"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/"},"modified":"2025-07-31T00:25:39","modified_gmt":"2025-07-31T00:25:39","slug":"what-is-the-structure-of-the-big-data-atlas","status":"publish","type":"post","link":"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/","title":{"rendered":"Big Data Atlas Structure Guide"},"content":{"rendered":"

Big Data Atlas is an open-source tool for data classification and metadata management, used to create and maintain an inventory of data assets. Its architecture mainly consists of the following components:<\/p>\n

    \n
  1. Data Collectors: These are used to gather metadata information from various data sources such as databases, file systems, data warehouses, etc. The collectors regularly scan the data sources and send the metadata information to the Atlas central component for processing.<\/li>\n
  2. The Atlas Core component in Atlas: is responsible for receiving and processing metadata information from data collectors, as well as providing storage, retrieval, and management functions for metadata. The Atlas Core component also includes metadata type definition, relationship modeling, and query functions to support user queries and browsing of metadata information.<\/li>\n
  3. Metadata Store: Used to persistently store the collected metadata information. Atlas offers various implementations for metadata storage backends, such as HBase, MySQL, etc., allowing users to choose the storage method that best suits their needs.<\/li>\n
  4. Metadata Search Service: This service allows users to quickly search and query metadata information using keywords, tags, relationships, and other methods. Atlas provides a metadata search service based on Solr to enhance search performance and efficiency.<\/li>\n
  5. Metadata Update Service: a service used to manage changes and updates to metadata information. When metadata information in the data source changes, the data collector informs the Metadata Update Service to ensure the accuracy and consistency of the metadata information.<\/li>\n<\/ol>\n

    Overall, the architecture of the big data Atlas adopts a distributed design approach, enabling comprehensive management and monitoring of data assets through collaboration and coordination among various components. Users can easily manage and utilize data assets through various functions and interfaces provided by Atlas, thus improving the efficiency and quality of data governance.<\/p>\n","protected":false},"excerpt":{"rendered":"

    Big Data Atlas is an open-source tool for data classification and metadata management, used to create and maintain an inventory of data assets. Its architecture mainly consists of the following components: Data Collectors: These are used to gather metadata information from various data sources such as databases, file systems, data warehouses, etc. The collectors regularly […]<\/p>\n","protected":false},"author":13,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_import_markdown_pro_load_document_selector":0,"_import_markdown_pro_submit_text_textarea":"","footnotes":""},"categories":[1],"tags":[924,2342,3107,2330,3108],"class_list":["post-4010","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-big-data-atlas","tag-data-architecture","tag-data-classification","tag-metadata-management","tag-open-source-tools"],"yoast_head":"\nBig Data Atlas Structure Guide - Blog - Silicon Cloud<\/title>\n<meta name=\"description\" content=\"Explore the architecture of Big Data Atlas: its core components like Data Collectors and Atlas Core for metadata management.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Big Data Atlas Structure Guide\" \/>\n<meta property=\"og:description\" content=\"Explore the architecture of Big Data Atlas: its core components like Data Collectors and Atlas Core for metadata management.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/\" \/>\n<meta property=\"og:site_name\" content=\"Blog - Silicon Cloud\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/SiliCloudGlobal\/\" \/>\n<meta property=\"article:published_time\" content=\"2024-03-13T07:49:05+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-07-31T00:25:39+00:00\" \/>\n<meta name=\"author\" content=\"Isabella Edwards\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@SiliCloudGlobal\" \/>\n<meta name=\"twitter:site\" content=\"@SiliCloudGlobal\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Isabella Edwards\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/\"},\"author\":{\"name\":\"Isabella Edwards\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/5579144e23c225c8188167f3e3f888dd\"},\"headline\":\"Big Data Atlas Structure Guide\",\"datePublished\":\"2024-03-13T07:49:05+00:00\",\"dateModified\":\"2025-07-31T00:25:39+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/\"},\"wordCount\":291,\"publisher\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#organization\"},\"keywords\":[\"Big Data Atlas\",\"Data Architecture\",\"data classification\",\"metadata management\",\"open-source tools\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/\",\"url\":\"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/\",\"name\":\"Big Data Atlas Structure Guide - Blog - Silicon Cloud\",\"isPartOf\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#website\"},\"datePublished\":\"2024-03-13T07:49:05+00:00\",\"dateModified\":\"2025-07-31T00:25:39+00:00\",\"description\":\"Explore the architecture of Big Data Atlas: its core components like Data Collectors and Atlas Core for metadata management.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.silicloud.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Big Data Atlas Structure Guide\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#website\",\"url\":\"https:\/\/www.silicloud.com\/blog\/\",\"name\":\"Silicon Cloud Blog\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#organization\"},\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#organization\",\"name\":\"Silicon Cloud Blog\",\"url\":\"https:\/\/www.silicloud.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png\",\"contentUrl\":\"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png\",\"width\":1024,\"height\":1024,\"caption\":\"Silicon Cloud Blog\"},\"image\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/SiliCloudGlobal\/\",\"https:\/\/twitter.com\/SiliCloudGlobal\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/5579144e23c225c8188167f3e3f888dd\",\"name\":\"Isabella Edwards\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/d4d4dec47f553ac7961d9fa4cc9bdcdcf5b7ce5106594330b6d25c5694fdbaec?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/d4d4dec47f553ac7961d9fa4cc9bdcdcf5b7ce5106594330b6d25c5694fdbaec?s=96&d=mm&r=g\",\"caption\":\"Isabella Edwards\"},\"url\":\"https:\/\/www.silicloud.com\/blog\/author\/isabellaedwards\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Big Data Atlas Structure Guide - Blog - Silicon Cloud","description":"Explore the architecture of Big Data Atlas: its core components like Data Collectors and Atlas Core for metadata management.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/","og_locale":"en_US","og_type":"article","og_title":"Big Data Atlas Structure Guide","og_description":"Explore the architecture of Big Data Atlas: its core components like Data Collectors and Atlas Core for metadata management.","og_url":"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/","og_site_name":"Blog - Silicon Cloud","article_publisher":"https:\/\/www.facebook.com\/SiliCloudGlobal\/","article_published_time":"2024-03-13T07:49:05+00:00","article_modified_time":"2025-07-31T00:25:39+00:00","author":"Isabella Edwards","twitter_card":"summary_large_image","twitter_creator":"@SiliCloudGlobal","twitter_site":"@SiliCloudGlobal","twitter_misc":{"Written by":"Isabella Edwards","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/#article","isPartOf":{"@id":"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/"},"author":{"name":"Isabella Edwards","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/5579144e23c225c8188167f3e3f888dd"},"headline":"Big Data Atlas Structure Guide","datePublished":"2024-03-13T07:49:05+00:00","dateModified":"2025-07-31T00:25:39+00:00","mainEntityOfPage":{"@id":"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/"},"wordCount":291,"publisher":{"@id":"https:\/\/www.silicloud.com\/blog\/#organization"},"keywords":["Big Data Atlas","Data Architecture","data classification","metadata management","open-source tools"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/","url":"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/","name":"Big Data Atlas Structure Guide - Blog - Silicon Cloud","isPartOf":{"@id":"https:\/\/www.silicloud.com\/blog\/#website"},"datePublished":"2024-03-13T07:49:05+00:00","dateModified":"2025-07-31T00:25:39+00:00","description":"Explore the architecture of Big Data Atlas: its core components like Data Collectors and Atlas Core for metadata management.","breadcrumb":{"@id":"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.silicloud.com\/blog\/what-is-the-structure-of-the-big-data-atlas\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.silicloud.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Big Data Atlas Structure Guide"}]},{"@type":"WebSite","@id":"https:\/\/www.silicloud.com\/blog\/#website","url":"https:\/\/www.silicloud.com\/blog\/","name":"Silicon Cloud Blog","description":"","publisher":{"@id":"https:\/\/www.silicloud.com\/blog\/#organization"},"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.silicloud.com\/blog\/#organization","name":"Silicon Cloud Blog","url":"https:\/\/www.silicloud.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png","contentUrl":"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png","width":1024,"height":1024,"caption":"Silicon Cloud Blog"},"image":{"@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/SiliCloudGlobal\/","https:\/\/twitter.com\/SiliCloudGlobal"]},{"@type":"Person","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/5579144e23c225c8188167f3e3f888dd","name":"Isabella Edwards","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/d4d4dec47f553ac7961d9fa4cc9bdcdcf5b7ce5106594330b6d25c5694fdbaec?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/d4d4dec47f553ac7961d9fa4cc9bdcdcf5b7ce5106594330b6d25c5694fdbaec?s=96&d=mm&r=g","caption":"Isabella Edwards"},"url":"https:\/\/www.silicloud.com\/blog\/author\/isabellaedwards\/"}]}},"_links":{"self":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts\/4010","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/users\/13"}],"replies":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/comments?post=4010"}],"version-history":[{"count":2,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts\/4010\/revisions"}],"predecessor-version":[{"id":148670,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts\/4010\/revisions\/148670"}],"wp:attachment":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/media?parent=4010"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/categories?post=4010"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/tags?post=4010"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}