{"id":5370,"date":"2024-03-14T02:45:10","date_gmt":"2024-03-14T02:45:10","guid":{"rendered":"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/"},"modified":"2025-08-01T14:13:48","modified_gmt":"2025-08-01T14:13:48","slug":"how-to-adjust-the-learning-rate-in-pytorch","status":"publish","type":"post","link":"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/","title":{"rendered":"PyTorch Learning Rate Adjustment Guide"},"content":{"rendered":"

In PyTorch, there are several ways to adjust the learning rate.<\/p>\n

    \n
  1. Automatically adjust the learning rate using the learning rate scheduler from the torch.optim.lr_scheduler module. Various learning rate scheduling strategies such as StepLR, ReduceLROnPlateau, CosineAnnealingLR can be chosen. Simply call the step method of the learning rate scheduler at the end of each epoch or batch to update the learning rate.<\/li>\n<\/ol>\n
    import<\/span> torch.optim as<\/span> optim\r\nfrom<\/span> torch.optim.lr_scheduler import<\/span> StepLR\r\n\r\noptimizer = optim.SGD(model.parameters(), lr=0.1<\/span>)\r\nscheduler = StepLR(optimizer, step_size=30<\/span>, gamma=0.1<\/span>)\r\n\r\nfor<\/span> epoch in<\/span> range<\/span>(num_epochs):\r\n    # Train the model<\/span>\r\n    ...\r\n    \r\n    # Update learning rate<\/span>\r\n    scheduler.step()\r\n<\/code><\/pre>\n
      \n
    1. Manually adjust the learning rate. You can manually change the learning rate during training as needed, for example, by adjusting it at specific epochs or under certain conditions.<\/li>\n<\/ol>\n
      optimizer = optim.SGD(model.parameters(), lr=0.1<\/span>)\r\n\r\nfor<\/span> epoch in<\/span> range<\/span>(num_epochs):\r\n    # Train the model<\/span>\r\n    ...\r\n    \r\n    if<\/span> epoch == 30<\/span>:\r\n        for<\/span> param_group in<\/span> optimizer.param_groups:\r\n            param_group['lr'<\/span>] = 0.01<\/span>\r\n<\/code><\/pre>\n
        \n
      1. Adjust the learning rate by using optimizer.param_groups in the torch.optim module. Update the learning rate by modifying the lr parameter in optimizer.param_groups.<\/li>\n<\/ol>\n
        optimizer = optim.SGD(model.parameters(), lr=0.1<\/span>)\r\n\r\nfor<\/span> epoch in<\/span> range<\/span>(num_epochs):\r\n    # Train the model<\/span>\r\n    ...\r\n    \r\n    if<\/span> epoch % 10<\/span> == 0<\/span>:\r\n        for<\/span> param_group in<\/span> optimizer.param_groups:\r\n            param_group['lr'<\/span>] *= 0.1<\/span>\r\n<\/code><\/pre>\n
        The above are several common methods for adjusting the learning rate, which can be chosen based on the actual situation when training a neural network.<\/p>\n","protected":false},"excerpt":{"rendered":"
        In PyTorch, there are several ways to adjust the learning rate. Automatically adjust the learning rate using the learning rate scheduler from the torch.optim.lr_scheduler module. Various learning rate scheduling strategies such as StepLR, ReduceLROnPlateau, CosineAnnealingLR can be chosen. Simply call the step method of the learning rate scheduler at the end of each epoch or […]<\/p>\n","protected":false},"author":10,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_import_markdown_pro_load_document_selector":0,"_import_markdown_pro_submit_text_textarea":"","footnotes":""},"categories":[1],"tags":[960,2428,944,601,1239],"class_list":["post-5370","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-deep-learning","tag-learning-rate","tag-neural-networks","tag-optimization","tag-pytorch"],"yoast_head":"\nPyTorch Learning Rate Adjustment Guide - Blog - Silicon Cloud<\/title>\n<meta name=\"description\" content=\"Master PyTorch learning rate scheduling with StepLR, ReduceLROnPlateau & CosineAnnealingLR. Code examples included.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"PyTorch Learning Rate Adjustment Guide\" \/>\n<meta property=\"og:description\" content=\"Master PyTorch learning rate scheduling with StepLR, ReduceLROnPlateau & CosineAnnealingLR. Code examples included.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/\" \/>\n<meta property=\"og:site_name\" content=\"Blog - Silicon Cloud\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/SiliCloudGlobal\/\" \/>\n<meta property=\"article:published_time\" content=\"2024-03-14T02:45:10+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-08-01T14:13:48+00:00\" \/>\n<meta name=\"author\" content=\"Jackson Davis\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@SiliCloudGlobal\" \/>\n<meta name=\"twitter:site\" content=\"@SiliCloudGlobal\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Jackson Davis\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/\"},\"author\":{\"name\":\"Jackson Davis\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/55a10b8b0457c35884c25677889ad350\"},\"headline\":\"PyTorch Learning Rate Adjustment Guide\",\"datePublished\":\"2024-03-14T02:45:10+00:00\",\"dateModified\":\"2025-08-01T14:13:48+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/\"},\"wordCount\":149,\"publisher\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#organization\"},\"keywords\":[\"Deep Learning\",\"Learning Rate\",\"Neural Networks\",\"Optimization\",\"PyTorch\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/\",\"url\":\"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/\",\"name\":\"PyTorch Learning Rate Adjustment Guide - Blog - Silicon Cloud\",\"isPartOf\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#website\"},\"datePublished\":\"2024-03-14T02:45:10+00:00\",\"dateModified\":\"2025-08-01T14:13:48+00:00\",\"description\":\"Master PyTorch learning rate scheduling with StepLR, ReduceLROnPlateau & CosineAnnealingLR. Code examples included.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.silicloud.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"PyTorch Learning Rate Adjustment Guide\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#website\",\"url\":\"https:\/\/www.silicloud.com\/blog\/\",\"name\":\"Silicon Cloud Blog\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#organization\"},\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#organization\",\"name\":\"Silicon Cloud Blog\",\"url\":\"https:\/\/www.silicloud.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png\",\"contentUrl\":\"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png\",\"width\":1024,\"height\":1024,\"caption\":\"Silicon Cloud Blog\"},\"image\":{\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/SiliCloudGlobal\/\",\"https:\/\/twitter.com\/SiliCloudGlobal\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/55a10b8b0457c35884c25677889ad350\",\"name\":\"Jackson Davis\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/2fdb47d6df1226e92380d96973782572a97b0675d098bb914410dec348eb5d29?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/2fdb47d6df1226e92380d96973782572a97b0675d098bb914410dec348eb5d29?s=96&d=mm&r=g\",\"caption\":\"Jackson Davis\"},\"url\":\"https:\/\/www.silicloud.com\/blog\/author\/jacksondavis\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"PyTorch Learning Rate Adjustment Guide - Blog - Silicon Cloud","description":"Master PyTorch learning rate scheduling with StepLR, ReduceLROnPlateau & CosineAnnealingLR. Code examples included.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/","og_locale":"en_US","og_type":"article","og_title":"PyTorch Learning Rate Adjustment Guide","og_description":"Master PyTorch learning rate scheduling with StepLR, ReduceLROnPlateau & CosineAnnealingLR. Code examples included.","og_url":"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/","og_site_name":"Blog - Silicon Cloud","article_publisher":"https:\/\/www.facebook.com\/SiliCloudGlobal\/","article_published_time":"2024-03-14T02:45:10+00:00","article_modified_time":"2025-08-01T14:13:48+00:00","author":"Jackson Davis","twitter_card":"summary_large_image","twitter_creator":"@SiliCloudGlobal","twitter_site":"@SiliCloudGlobal","twitter_misc":{"Written by":"Jackson Davis","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/#article","isPartOf":{"@id":"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/"},"author":{"name":"Jackson Davis","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/55a10b8b0457c35884c25677889ad350"},"headline":"PyTorch Learning Rate Adjustment Guide","datePublished":"2024-03-14T02:45:10+00:00","dateModified":"2025-08-01T14:13:48+00:00","mainEntityOfPage":{"@id":"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/"},"wordCount":149,"publisher":{"@id":"https:\/\/www.silicloud.com\/blog\/#organization"},"keywords":["Deep Learning","Learning Rate","Neural Networks","Optimization","PyTorch"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/","url":"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/","name":"PyTorch Learning Rate Adjustment Guide - Blog - Silicon Cloud","isPartOf":{"@id":"https:\/\/www.silicloud.com\/blog\/#website"},"datePublished":"2024-03-14T02:45:10+00:00","dateModified":"2025-08-01T14:13:48+00:00","description":"Master PyTorch learning rate scheduling with StepLR, ReduceLROnPlateau & CosineAnnealingLR. Code examples included.","breadcrumb":{"@id":"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.silicloud.com\/blog\/how-to-adjust-the-learning-rate-in-pytorch\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.silicloud.com\/blog\/"},{"@type":"ListItem","position":2,"name":"PyTorch Learning Rate Adjustment Guide"}]},{"@type":"WebSite","@id":"https:\/\/www.silicloud.com\/blog\/#website","url":"https:\/\/www.silicloud.com\/blog\/","name":"Silicon Cloud Blog","description":"","publisher":{"@id":"https:\/\/www.silicloud.com\/blog\/#organization"},"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.silicloud.com\/blog\/#organization","name":"Silicon Cloud Blog","url":"https:\/\/www.silicloud.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png","contentUrl":"https:\/\/www.silicloud.com\/blog\/wp-content\/uploads\/2023\/11\/EN-SILICON-Full.png","width":1024,"height":1024,"caption":"Silicon Cloud Blog"},"image":{"@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/SiliCloudGlobal\/","https:\/\/twitter.com\/SiliCloudGlobal"]},{"@type":"Person","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/55a10b8b0457c35884c25677889ad350","name":"Jackson Davis","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.silicloud.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/2fdb47d6df1226e92380d96973782572a97b0675d098bb914410dec348eb5d29?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/2fdb47d6df1226e92380d96973782572a97b0675d098bb914410dec348eb5d29?s=96&d=mm&r=g","caption":"Jackson Davis"},"url":"https:\/\/www.silicloud.com\/blog\/author\/jacksondavis\/"}]}},"_links":{"self":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts\/5370","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/comments?post=5370"}],"version-history":[{"count":2,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts\/5370\/revisions"}],"predecessor-version":[{"id":150117,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/posts\/5370\/revisions\/150117"}],"wp:attachment":[{"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/media?parent=5370"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/categories?post=5370"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.silicloud.com\/blog\/wp-json\/wp\/v2\/tags?post=5370"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}