3

I am trying to implement simple NEAT. I read from various sources that there are 4 types of "nodes": input neurons, hidden neurons, output neurons and so-called bias neurons. I don't see which process may create bias neurons, that are depicted in this paper at page 16.

I understand that new neurons may be created while mutating, but it requires an existing connection between two neurons that will be split by this new neuron (basing on paper already mentioned, page 10). However, bias neuron has no "input" connection, so it clearly can't be created in the mentioned way. Then how, in details, does NEAT create bias neurons?

FindingTruth
  • 69
  • 8
  • 1
    I am aware that bias is part of neuron when we are talking about neural networks in general. However, it's kind of different way when we are talking about NEAT though. In this particular paper (that I mentioned) it is indeed called "bias unit", not "bias neuron", but still it's a separeted node, not a part of neuron. Here is much better example: https://www.heatonresearch.com/encog/neat/neat_structure.html In the very first picture, you can clearly see there is a "bias neuron" (it is so-called in this article) that is only connected to some other hidden neurons. – FindingTruth Feb 23 '19 at 09:09
  • 1
    I quite understand what you mean by "it would probably generate new chromosomes", but as I said, according to mentioned paper and other sources, neurons are added only by mutation - new neuron splits existing connection. Author doesn't say anything about mutating neurons genome that would add a new neuron arbitrarily. Also, crossover cannot lead to create new neurons, which, I guess, is clear. Then, I don't really see how author intends to create bias neurons during NEAT processing, which is basically my issue. – FindingTruth Feb 25 '19 at 15:06

1 Answers1

5

A bias neuron (node) in the NEAT context is simply a special input neuron that is always active. It is always included by construction since it seems to help evolution in many cases.

So, in short, you do not create bias neurons just like you won't create new input or output nodes; these are defined by your problem.

You are correct in that the standard NEAT implementation introduces new hidden nodes by splitting existing connections. Hidden nodes are the only neurons you will create or destroy in NEAT (and, to my knowledge, in neuroevolution in general).

Pablo
  • 1,373
  • 16
  • 36