{"id":2159,"date":"2024-03-12T09:20:54","date_gmt":"2024-03-12T09:20:54","guid":{"rendered":"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/"},"modified":"2024-04-08T15:24:06","modified_gmt":"2024-04-08T15:24:06","slug":"how-to-regularize-models-in-keras","status":"publish","type":"post","link":"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/","title":{"rendered":"How to regularize models in Keras?"},"content":{"rendered":"<p>In <a href=\"https:\/\/keras.io\/\">Keras<\/a>, regularization can be applied to a model by adding regularization terms within the layers of the model. Regularization terms can be specified in the parameters of each layer, for example:<\/p>\n<pre class=\"post-pre\"><code><span class=\"hljs-keyword\">from<\/span> keras <span class=\"hljs-keyword\">import<\/span> regularizers\r\n\r\nmodel = Sequential()\r\nmodel.add(Dense(<span class=\"hljs-number\">64<\/span>, input_dim=<span class=\"hljs-number\">64<\/span>, kernel_regularizer=regularizers.l2(<span class=\"hljs-number\">0.01<\/span>)))\r\nmodel.add(Activation(<span class=\"hljs-string\">'relu'<\/span>))\r\nmodel.add(Dense(<span class=\"hljs-number\">64<\/span>, kernel_regularizer=regularizers.l2(<span class=\"hljs-number\">0.01<\/span>)))\r\nmodel.add(Activation(<span class=\"hljs-string\">'relu'<\/span>))\r\nmodel.add(Dense(<span class=\"hljs-number\">10<\/span>, kernel_regularizer=regularizers.l2(<span class=\"hljs-number\">0.01<\/span>)))\r\nmodel.add(Activation(<span class=\"hljs-string\">'softmax'<\/span>))\r\n<\/code><\/pre>\n<p>In the above example, we added L2 regularization terms with a parameter value of 0.01 to each fully connected layer. Different regularization methods can be chosen as needed, such as L1 regularization, L1L2 regularization, etc. By adding regularization terms, the model will constrain the weights during training to prevent overfitting.<\/p>\n<p>&nbsp;<\/p>\n<p>More tutorials<\/p>\n<p><a class=\"LinkSuggestion__Link-sc-1gewdgc-4 cLBplk\" href=\"https:\/\/www.silicloud.com\/blog\/how-to-evaluate-and-test-models-in-keras\/\" target=\"_blank\" rel=\"noopener\">How to evaluate and test models in Keras?<span class=\"sc-gswNZR eASTkv\">(Opens in a new browser tab)<\/span><\/a><\/p>\n<p><a class=\"LinkSuggestion__Link-sc-1gewdgc-4 cLBplk\" href=\"https:\/\/www.silicloud.com\/blog\/how-to-implement-sequence-to-sequence-learning-in-keras\/\" target=\"_blank\" rel=\"noopener\">How to implement sequence-to-sequence learning in Keras?<span class=\"sc-gswNZR eASTkv\">(Opens in a new browser tab)<\/span><\/a><\/p>\n<p><a class=\"LinkSuggestion__Link-sc-1gewdgc-4 cLBplk\" href=\"https:\/\/www.silicloud.com\/blog\/python-keywords-and-identifiers-have-been-revised\/\" target=\"_blank\" rel=\"noopener\">Python Keywords and Identifiers have been revised.<span class=\"sc-gswNZR eASTkv\">(Opens in a new browser tab)<\/span><\/a><\/p>\n<p><a class=\"LinkSuggestion__Link-sc-1gewdgc-4 cLBplk\" href=\"https:\/\/www.silicloud.com\/blog\/how-to-use-custom-loss-functions-in-keras\/\" target=\"_blank\" rel=\"noopener\">How to use custom loss functions in Keras.<span class=\"sc-gswNZR eASTkv\">(Opens in a new browser tab)<\/span><\/a><\/p>\n<p><a class=\"LinkSuggestion__Link-sc-1gewdgc-4 cLBplk\" href=\"https:\/\/www.silicloud.com\/blog\/how-to-retrieve-data-from-a-qtableview-in-qt\/\" target=\"_blank\" rel=\"noopener\">How to retrieve data from a QTableView in Qt?<span class=\"sc-gswNZR eASTkv\">(Opens in a new browser tab)<\/span><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In Keras, regularization can be applied to a model by adding regularization terms within the layers of the model. Regularization terms can be specified in the parameters of each layer, for example: from keras import regularizers model = Sequential() model.add(Dense(64, input_dim=64, kernel_regularizer=regularizers.l2(0.01))) model.add(Activation(&#8216;relu&#8217;)) model.add(Dense(64, kernel_regularizer=regularizers.l2(0.01))) model.add(Activation(&#8216;relu&#8217;)) model.add(Dense(10, kernel_regularizer=regularizers.l2(0.01))) model.add(Activation(&#8216;softmax&#8217;)) In the above example, we added [&hellip;]<\/p>\n","protected":false},"author":12,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_import_markdown_pro_load_document_selector":0,"_import_markdown_pro_submit_text_textarea":"","footnotes":""},"categories":[1],"tags":[],"class_list":["post-2159","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.5 (Yoast SEO v21.5) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>How to regularize models in Keras? - Blog - Silicon Cloud<\/title>\n<meta name=\"description\" content=\"In Keras, regularization can be applied to a model by adding regularization terms within the layers of the model. Regularization terms can be\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How to regularize models in Keras?\" \/>\n<meta property=\"og:description\" content=\"In Keras, regularization can be applied to a model by adding regularization terms within the layers of the model. Regularization terms can be\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/\" \/>\n<meta property=\"og:site_name\" content=\"Blog - Silicon Cloud\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/SiliCloudGlobal\/\" \/>\n<meta property=\"article:published_time\" content=\"2024-03-12T09:20:54+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-04-08T15:24:06+00:00\" \/>\n<meta name=\"author\" content=\"Liam\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@SiliCloudGlobal\" \/>\n<meta name=\"twitter:site\" content=\"@SiliCloudGlobal\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Liam\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/\"},\"author\":{\"name\":\"Liam\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/23786905eb7b377f45ddb01c17da7671\"},\"headline\":\"How to regularize models in Keras?\",\"datePublished\":\"2024-03-12T09:20:54+00:00\",\"dateModified\":\"2024-04-08T15:24:06+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/\"},\"wordCount\":160,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#organization\"},\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/\",\"url\":\"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/\",\"name\":\"How to regularize models in Keras? - Blog - Silicon Cloud\",\"isPartOf\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#website\"},\"datePublished\":\"2024-03-12T09:20:54+00:00\",\"dateModified\":\"2024-04-08T15:24:06+00:00\",\"description\":\"In Keras, regularization can be applied to a model by adding regularization terms within the layers of the model. Regularization terms can be\",\"breadcrumb\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.silicloud.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How to regularize models in Keras?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#website\",\"url\":\"https:\/\/www.silicloud.com\/blog\/\",\"name\":\"Silicon Cloud Blog\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#organization\"},\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#organization\",\"name\":\"Silicon Cloud Blog\",\"url\":\"https:\/\/www.silicloud.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png\",\"contentUrl\":\"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png\",\"width\":1024,\"height\":1024,\"caption\":\"Silicon Cloud Blog\"},\"image\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/SiliCloudGlobal\/\",\"https:\/\/twitter.com\/SiliCloudGlobal\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/23786905eb7b377f45ddb01c17da7671\",\"name\":\"Liam\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/8d37ed3e7f770dde8bf069ba0b4298688028c3abaacf1131742fc1352d174ebd?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/8d37ed3e7f770dde8bf069ba0b4298688028c3abaacf1131742fc1352d174ebd?s=96&d=mm&r=g\",\"caption\":\"Liam\"},\"sameAs\":[\"http:\/\/Wilson\"],\"url\":\"https:\/\/www.silicloud.com\/blog\/author\/liamwilson\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"How to regularize models in Keras? - Blog - Silicon Cloud","description":"In Keras, regularization can be applied to a model by adding regularization terms within the layers of the model. Regularization terms can be","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/","og_locale":"en_US","og_type":"article","og_title":"How to regularize models in Keras?","og_description":"In Keras, regularization can be applied to a model by adding regularization terms within the layers of the model. Regularization terms can be","og_url":"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/","og_site_name":"Blog - Silicon Cloud","article_publisher":"https:\/\/www.facebook.com\/SiliCloudGlobal\/","article_published_time":"2024-03-12T09:20:54+00:00","article_modified_time":"2024-04-08T15:24:06+00:00","author":"Liam","twitter_card":"summary_large_image","twitter_creator":"@SiliCloudGlobal","twitter_site":"@SiliCloudGlobal","twitter_misc":{"Written by":"Liam","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/#article","isPartOf":{"@id":"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/"},"author":{"name":"Liam","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/23786905eb7b377f45ddb01c17da7671"},"headline":"How to regularize models in Keras?","datePublished":"2024-03-12T09:20:54+00:00","dateModified":"2024-04-08T15:24:06+00:00","mainEntityOfPage":{"@id":"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/"},"wordCount":160,"commentCount":0,"publisher":{"@id":"https:\/\/www.silicloud.com\/blog\/#organization"},"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/","url":"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/","name":"How to regularize models in Keras? - Blog - Silicon Cloud","isPartOf":{"@id":"https:\/\/www.silicloud.com\/blog\/#website"},"datePublished":"2024-03-12T09:20:54+00:00","dateModified":"2024-04-08T15:24:06+00:00","description":"In Keras, regularization can be applied to a model by adding regularization terms within the layers of the model. Regularization terms can be","breadcrumb":{"@id":"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.silicloud.com\/blog\/how-to-regularize-models-in-keras\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.silicloud.com\/blog\/"},{"@type":"ListItem","position":2,"name":"How to regularize models in Keras?"}]},{"@type":"WebSite","@id":"https:\/\/www.silicloud.com\/blog\/#website","url":"https:\/\/www.silicloud.com\/blog\/","name":"Silicon Cloud Blog","description":"","publisher":{"@id":"https:\/\/www.silicloud.com\/blog\/#organization"},"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.silicloud.com\/blog\/#organization","name":"Silicon Cloud Blog","url":"https:\/\/www.silicloud.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png","contentUrl":"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png","width":1024,"height":1024,"caption":"Silicon Cloud Blog"},"image":{"@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/SiliCloudGlobal\/","https:\/\/twitter.com\/SiliCloudGlobal"]},{"@type":"Person","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/23786905eb7b377f45ddb01c17da7671","name":"Liam","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/8d37ed3e7f770dde8bf069ba0b4298688028c3abaacf1131742fc1352d174ebd?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/8d37ed3e7f770dde8bf069ba0b4298688028c3abaacf1131742fc1352d174ebd?s=96&d=mm&r=g","caption":"Liam"},"sameAs":["http:\/\/Wilson"],"url":"https:\/\/www.silicloud.com\/blog\/author\/liamwilson\/"}]}},"_links":{"self":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts\/2159","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/comments?post=2159"}],"version-history":[{"count":3,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts\/2159\/revisions"}],"predecessor-version":[{"id":77622,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts\/2159\/revisions\/77622"}],"wp:attachment":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/media?parent=2159"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/categories?post=2159"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/tags?post=2159"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}