How do I write a PyTorch sequential model?

Question

How do I write a sequential model in PyTorch, just like what we can do with Keras? I tried:

import torch
import torch.nn as nn

net = nn.Sequential()
net.add(nn.Linear(3, 4))
net.add(nn.Sigmoid())
net.add(nn.Linear(4, 1))
net.add(nn.Sigmoid())
net.float()

But I get the error:

AttributeError: 'Sequential' object has no attribute 'add'

score 48 · Answer 1 · edited Jun 21 '23 at 16:24

Sequential does not have an add method at the moment, though there is some debate about adding this functionality.

As you can read in the documentation nn.Sequential takes as argument the layers separated as a sequence of arguments or an OrderedDict.

If you have a model with lots of layers, you can create a list first and then use the * operator to expand the list into positional arguments, like this:

layers = []
layers.append(nn.Linear(3, 4))
layers.append(nn.Sigmoid())
layers.append(nn.Linear(4, 1))
layers.append(nn.Sigmoid())

net = nn.Sequential(*layers)

This will result in a similar structure of your code, as adding directly.

score 23 · Accepted Answer · edited Jun 09 '22 at 20:40

As described by the correct answer, this is what it would look as a sequence of arguments:

device = torch.device('cpu')
if torch.cuda.is_available():
    device = torch.device('cuda')
 
net = nn.Sequential(
      nn.Linear(3, 4),
      nn.Sigmoid(),
      nn.Linear(4, 1),
      nn.Sigmoid()
      ).to(device)

print(net)

Sequential(
  (0): Linear(in_features=3, out_features=4, bias=True)
  (1): Sigmoid()
  (2): Linear(in_features=4, out_features=1, bias=True)
  (3): Sigmoid()
  )

score 6 · Answer 3 · edited Jun 22 '21 at 14:25

layerlist = []
for i in layers:
    layerlist.append(nn.Linear(n_in, i))  # n_in input neurons connected to i number of output neurons
    layerlist.append(nn.ReLU(inplace=True))  # Apply activation function - ReLU
    layerlist.append(nn.BatchNorm1d(i))  # Apply batch normalization
    layerlist.append(nn.Dropout(p))  # Apply dropout to prevent overfitting
    n_in = i  # Reassign number of input neurons as the number of neurons from previous last layer

    # Establish the FCC between the last hidden layer and output layer
    layerlist.append(nn.Linear(layers[-1], out_sz))

    self.layers = nn.Sequential(*layerlist)

score 2 · Answer 4 · answered Apr 08 '19 at 04:35

As McLawrence said nn.Sequential doesn't have the add method. I think maybe the codes in which you found the using of add could have lines that modified the torch.nn.Module.add to a function like this:

def add_module(self,module):
    self.add_module(str(len(self) + 1 ), module)

torch.nn.Module.add = add_module

after doing this, you can add a torch.nn.Module to a Sequential like you posted in the question.

How do I write a PyTorch sequential model?

4 Answers4