Regression Case for Graph Neural Network

Question

I am trying to train a simple graph neural network (and tried both torch_geometric and dgl libraries) in a regression problem with 1 node feature and 1 node level target. My issue is that the optimizer trains the model such that it gives the same values for all nodes in the graph.

The problem is simple. In a 5 node graph, each node has one feature (x), and one target value for each node (y). The problem is a regression problem meaning that I want to predict the y values (that is a continuous number). The problem is that after the training, the values of the predicted y for all of the nodes are the same (that is an average value of all of the expected target values of y). I do not know what is the problem as I tried both torch_geometric and dgl libraries. Thank you for your help :).

The code can be like the below:

class GAT(torch.nn.Module):
    def __init__(self,num_features):
        super(GAT, self).__init__()
        self.hid = 1
        self.in_head = 8
        self.out_head = 1
        # self.conv1 = GATConv(num_features, self.hid, heads=self.in_head, dropout=0.6)
        # self.conv2 = GATConv(self.hid*self.in_head, 1, concat=False,
        #                      heads=self.out_head, dropout=0.3)
        self.mlp1 = MLP(in_channels=num_features, hidden_channels=32,
                                out_channels=self.out_head, num_layers=1)

    def forward(self, data):
        x, edge_index = data.x, data.edge_index
                
        # x = F.dropout(x, p=0.1, training=self.training)
        # x = self.conv1(x, edge_index)
        # x = F.elu(x)
        x = self.mlp1(x)       
        # x = F.dropout(x, p=0.1, training=self.training)
        # x = self.conv2(x, edge_index)
        return x

Here the model has an MLP layer, but different combinations such as GraphConv networks (as commented in the model) give the same results.

and for the training block:

model = GAT(1).to(device)
data1_ =train_dataset[2] # dataset[0].to(device)
data=data0
optimizer = torch.optim.Adam(model.parameters(), lr=0.005, weight_decay=5e-4)

model.train()
for epoch in range(3000):
    model.train()
    optimizer.zero_grad()
    out = model(data)
    loss = torch.mean((out-data.y)**2) 
    
    if epoch%200 == 0:
        print(loss)
    
    loss.backward()
    optimizer.step()

And the results are like below for a simple graph:

a simple result of predictions - blue line is the prediction where it is constant for all of the nodes; x is the feature of nodes, where changes in each node

Answer 1

I think that you have to pass the graph and the node features values in model(), like this:

out = model(graph, nodes_features)