140

Is there a rule of thumb (or set of examples) to determine when to use genetic algorithms as opposed to neural networks (and vice-versa) to solve a problem?

I know there are cases in which you can have both methods mixed, but I am looking for a high-level comparison between the two methods.

nbro
  • 15,395
  • 32
  • 113
  • 196
Neil N
  • 24,862
  • 16
  • 85
  • 145
  • 3
    It's worth pointing out there are two types of neural network - supervised and unsupervised. Supervised get training data from a human, unsupervised feedback into themselves and are more like GAs in that respect. – Chris S Oct 25 '12 at 22:53
  • 3
    I don't think it's "list-y" at all. The answers compare two methods, and clarify when to use one vs the other. – Neil N Dec 23 '12 at 19:36
  • Short answer: use GA when the function to model is non-continuous/discrete, or when the dataset is astronomically high-dimensional. Use neural networks or GA (or another model) for the rest depending on what gives you the best results. For more infos, see my answer here: https://stackoverflow.com/a/49684833/1121352 – gaborous Apr 06 '18 at 03:32

8 Answers8

134

From wikipedia:

A genetic algorithm (GA) is a search technique used in computing to find exact or approximate solutions to optimization and search problems.

and:

Neural networks are non-linear statistical data modeling tools. They can be used to model complex relationships between inputs and outputs or to find patterns in data.

If you have a problem where you can quantify the worth of a solution, a genetic algorithm can perform a directed search of the solution space. (E.g. find the shortest route between two points)

When you have a number of items in different classes, a neural network can "learn" to classify items it has not "seen" before. (E.g. face recognition, voice recognition)

Execution times must also be considered. A genetic algorithm takes a long time to find an acceptable solution. A neural network takes a long time to "learn", but then it can almost instantly classify new inputs.

Dawie Strauss
  • 3,706
  • 3
  • 23
  • 26
  • 14
    I just want to add a bit to the GA definition. Sometimes people think of the solution space of a GA problem as a set of states or values. Such as "Find all of the ways a set of four chess pieces could be arranged on a chessboard to create a checkmate." However the solution space can also be a set of algorithms. This is where the real power of genetic algorithms come into play. They help you to answer a question like "Find a sequence of moves with a given set of chess pieces that will result in a checkmate." – lfalin Mar 13 '14 at 10:46
46

A genetic algorithm (despite its sexy name) is, for most purposes, an optimization technique. It primarily boils down to you having a number of variables and wanting to find the best combination of values for these variables. It just borrows techniques from natural evolution to get there.

Neural networks are useful for recognizing patterns. They follow a simplistic model of the brain, and by changing a number of weights between them, attempt to predict outputs based on inputs.

They are two fundamentally different entities, but sometimes the problems they are capable of solving overlap.

nbro
  • 15,395
  • 32
  • 113
  • 196
zenna
  • 9,006
  • 12
  • 73
  • 101
  • 25
    Neural networks are just an interpolation technique, actually. :) – Don Reba May 05 '11 at 09:52
  • 3
    +1 for genetic algorithms (optimization) and neural networks (supervised learning) have almost nothing in common. – alfa Mar 04 '12 at 17:58
  • 2
    The only common element is that they dynamically rearrange themselves as they approach a goal. – lfalin Mar 13 '14 at 10:49
  • 2
    I'm not sure there is any overlap, really. Neural network are a model with a cost function, GA are an optimization technique that can be used to minimize this cost function, as @zenna said. – Matthieu Brucher Nov 18 '18 at 09:07
40

GAs generate new patterns in a structure that you define.

NNs classify (or recognize) existing patterns based on training data that you provide.

GAs perform well at efficiently searching a large state-space of solutions, and converging on one or more good solutions, but not necessarily the 'best' solution.

NNs can learn to recognize patterns (via training), but it is notoriously difficult to figure out what they have learned, i.e. to extract the knowledge from them once trained, and reuse the knowledge in some other (non-NN).

nbro
  • 15,395
  • 32
  • 113
  • 196
Steven A. Lowe
  • 60,273
  • 18
  • 132
  • 202
  • Which genetic algorithm to use, if the number of parameters you are trying to optimize is large. In my case - between 30 and 100 – Alexander Cska Jul 29 '22 at 10:02
28

You are comparing two totally different things here.

Neural Networks are used for regression/classification - given a set of (x, y) examples, you want regress the unknown y for some given x.

Genetic algorithms are an optimization technique. Given a function f(x), you want to determine the x which minimizes/maximizes f(x).

bayer
  • 6,854
  • 24
  • 35
  • 10
    Indeed. They are really 'orthogonal' techniques. You can use a GA to find neural net weights and/or architecture. – redcalx Oct 12 '09 at 21:10
  • Calling them "totally different" or "orthogonal" seems a bit excessive given the other answers provided on this and other threads on the topic. – Johan Jan 13 '21 at 10:50
23

There are many similarities between them, so I will only try to outline their differences.

Neural networks

Are able to analyze online patterns (those that change over time). Generally, this is a time-varying sample that needs to be matched and predicted.

Examples:

  • Graph extrapolation
  • Facial recognition

Genetic algorithms

Used when you can code attributes that you think may contribute to a specific, non-changing problem. The emphasis is on being able to code these attributes (sometimes you know what they are) and that the problem is to a large degree unchanging (otherwise evolutions don't converge).

Examples:

  • Scheduling airplanes/shipping
  • Timetables.
  • Finding the best characteristics for a simple agent in an artificial environment
  • Rendering an approximation of a picture with random polygons
nbro
  • 15,395
  • 32
  • 113
  • 196
Unknown
  • 45,913
  • 27
  • 138
  • 182
14

You can use genetic algorithms as an alternative to the backpropagation algorithm to update weights in neural networks. For an example of this refer to: http://www.ai-junkie.com/ann/evolved/nnt1.html.

nbro
  • 15,395
  • 32
  • 113
  • 196
Amro
  • 123,847
  • 25
  • 243
  • 454
8

Genetic algorithms (usually) work on discrete data (enums, integer ranges, etc.). A typical application for GAs is searching a discrete space for a "good enough" solution when the only available alternative is a brute-force search (evaluating all combinations).

Neural networks, on the other hand, (usually) work on continuous data (floats, etc.). A typical application for NNs is function approximation, where you've got a set X of inputs and a set Y of related outputs, but the analytical function f: X → Y.

Of course, there are thousands of variants of both, so the line between them is somewhat blurred.

nbro
  • 15,395
  • 32
  • 113
  • 196
5

There is no rule of thumb. In many cases you can formulate your problem to make use of either of them. Machine learning is still an active area of research and which learning model to use can be debatable.

GA's take sexy languages from evolution but you're waiting for your computer to stumble upon a solution through a random process. Study your data, make good assumptions, try to know what you want and pick an approach that can make good use of these. If your first choice gives poor results, know why it was so, and improve the algorithm itself or pick a better one.

tilish
  • 1,093
  • 1
  • 9
  • 20