How can transfer learning be implemented in PyTorch?
Implementing transfer learning in PyTorch usually involves the following steps:
- Load pretrained model: Begin by loading a pretrained model, such as one trained on the ImageNet dataset.
import torch
import torchvision.models as models
model = models.resnet18(pretrained=True)
- To modify the final layer of the model: Typically, the goal of transfer learning is to apply a pre-trained model to a new task, so it is necessary to replace the final layer of the model with the output layer for the new task. This can be achieved by modifying the model’s fully connected layer.
n_features = model.fc.in_features
model.fc = torch.nn.Linear(n_features, num_classes) # num_classes为新任务的类别数
- Freeze the parameters of the model: In transfer learning, it is common practice to freeze the parameters of the pre-trained model and only train the parameters of newly added layers. This can be achieved by setting the requires_grad attribute of the parameters.
for param in model.parameters():
param.requires_grad = False
- Define a loss function and optimizer: Specify a loss function and optimizer appropriate for the new task.
criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.001)
- Train the model: Train the model with a new dataset.
for epoch in range(num_epochs):
for inputs, labels in dataloader:
optimizer.zero_grad()
outputs = model(inputs)
loss = criterion(outputs, labels)
loss.backward()
optimizer.step()
This completes the implementation process of transfer learning. By following the steps above, you can quickly train your model on a new task using a pre-trained model.