How to use dropout in PyTorch?

In PyTorch, Dropout can be implemented using torch.nn.Dropout. Dropout is a commonly used regularization method that randomly sets some neurons’ outputs to zero during training to prevent overfitting.

Here is an example code using Dropout:

import torch
import torch.nn as nn

# 定义一个简单的神经网络
class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.fc1 = nn.Linear(10, 20)
        self.dropout = nn.Dropout(p=0.5)  # 定义一个Dropout层
        self.fc2 = nn.Linear(20, 2)

    def forward(self, x):
        x = self.fc1(x)
        x = self.dropout(x)  # 在中间层应用Dropout
        x = torch.relu(x)
        x = self.fc2(x)
        return x

# 创建一个输入数据的示例
x = torch.randn(1, 10)

# 创建网络实例
net = Net()

# 将网络设为训练模式
net.train()

# 前向传播
output = net(x)

# 输出结果
print(output)

In the above example, we first defined a simple neural network class called Net, which includes an input layer, a Dropout layer, and an output layer. In the forward method, we pass the input data through the layers of the network, applying the Dropout operation in the middle layer. Next, we created an example input data x and created an instance of the network called net. When performing forward propagation calculations, we need to set the network to training mode by calling net.train(), so that the Dropout operation is applied in this mode. Finally, we output the network’s output result.

It is important to note that Dropout should only be applied during the training phase and not during testing. This means calling net.eval() in order to obtain more stable output results during testing.