0

Question

I have code that is based on Part 2, Chapter 11 of Deep Learning with PyTorch, by Luca Pietro Giovanni Antiga, Thomas Viehmann, and Eli Stevens. It's working just fine. It predicts the value of a Boolean variable. I want to convert this so that it predicts the value of a real number variable that happens to be always between 0 and 34.

There are two parts that I don't know how to convert. First, this part:

    pos_t = torch.tensor([
            not candidateInfo_tup.isNodule_bool,
            candidateInfo_tup.isNodule_bool
        ],
        dtype=torch.long,
    )

(Why are two values passed in here when one is completely determined by the other?)

and then this part:

    self.head_linear = nn.Linear(1152, 2)
    self.head_softmax = nn.Softmax(dim=1)

How do I do this?

Guess

I don't want people to think I haven't thought about this at all, so here is my guess:

First part:

    age_t = torch.tensor(candidateInfo_tup.age_int, dtype=torch.double)

Second part:

    self.head_linear = nn.Linear(299520, 1)
    self.head_relu = nn.ReLU()

I'm also guessing that I need to change this:

    loss_func = nn.CrossEntropyLoss(reduction='none')

to something like this:

    loss_func = nn.L1Loss()

My guesses are based on this article by Christian Versloot.

Poe Dator
  • 4,535
  • 2
  • 14
  • 35
Paul Reiners
  • 8,576
  • 33
  • 117
  • 202
  • 1
    Quick question, are you sure what you are looking for is regression? If the variable you want to predict is an integer with 35 patterns ranged from 0 to 34, it is called "multiclass classification", not regression. I was curious because your code had `age_int` in it. – ken Apr 29 '22 at 03:08
  • I did think about that. I decided that regression made more sense. The number is proportional to the progression of a disease which is gradual, rather than, say, episodic. Radiologists look at 34 parts of a brain and report 0 or 1 depending on the absence or presence of a lesion. So that's how you get the int from 0 to 34. But an individual lesion may vary in severity. To simplify things, the doctor only needs to report a 0 or 1. I may end up rounding it off to an int, but I think it really is a regression. – Paul Reiners Apr 29 '22 at 14:37
  • I see. This is not an answer either, but the example you just described is a "multi-label regression" that predicts 34 variables that range from 0 to 1, instead of 1 variable that ranges from 0 to 34. A multi-label is a bit complicated, so I don't think you need to get into it right now. However, with a multi-label, you might achieve higher accuracy, and you will be able to measure the accuracy per lesion, which will greatly help you to further improvements. – ken Apr 30 '22 at 11:21

1 Answers1

1

The example from the book is working but it has some redundant elements which confuse you. Normally output size of 1 is enough for a binary classification problem. To bring it to 0 or 1, one may use sigmoid and then rounding, like in the example here: PyTorch Binary Classification - same network structure, 'simpler' data, but worse performance? Or just put after single output neuron this:

y_pred_binary = torch.round(torch.sigmoid(y_pred))

The book example uses output size of 2 and then applies softmax to get to 0 and 1. This works, but such technique is typically used in multi-class classification.

For prediction of 0-34 variable:

  • if these are discrete variables - it is called "multiclass classification" as Ken indicated. Use size 35 output and softmax in this case. Search for "pytorch multiclass classification" for examples.
  • if this is a regression - than your changes seem in the right direction, except 'Second part'. instead of RELU - clip output at both ends at [0, 34]. Also 299520 - is too much for the previous layer. Use whatever input size there was before. Search for "pytorch regression" for examples.
Poe Dator
  • 4,535
  • 2
  • 14
  • 35