3

Do you know if anyone has tried to compile high level programming languages (java, c#, etc') into a recurrent neural network and then evolve them?

I mean that the whole process including memory usage is stored in a graph of a neural net, and I'm talking about complex programs (thinking about natural language processing problems).

When I say neural net I mean a directed weighted graphs that spreads activation, and the nodes are functions of their inputs (linear, sigmoid and multiplicative to keep it simple).

Furthermore, is that what people mean in genetic programming or is there a difference?

Uri
  • 25,622
  • 10
  • 45
  • 72
  • 2
    Java is not a functional programming language. – SLaks Feb 23 '11 at 21:10
  • @SLaks It can be if that's how you use it. It's not a particularly convenient functional programming language (nor a particularly convenient object-oriented language, some would argue) but it certainly doesn't ban you from using functional programming. – Zooba Feb 23 '11 at 21:12
  • you may have your terms mixed up. When you say, "Functional" do you mean a language that is organized into methods (often referred to as "functions"? or do you mean a language that is organized into side-effect free series of transformations? – Chris Pfohl Feb 23 '11 at 21:14
  • I just meant hight level languages like java. I'll change it. – Uri Feb 23 '11 at 21:24

5 Answers5

2

Neural networks are not particularly well suited for evolving programs; their strength tends to be in classification. If anyone has tried, I haven't heard about it (which considering I barely touch neural networks is not a surprise, but I am active in the general AI field at the moment).

The main reason why neural networks aren't useful for generating programs is that they basically represent a mathematical equation (numeric, rather than functional). Given some numeric input, you get a numeric output. It is difficult to interpret these in the context of a program any more complicated than simple arithmetic.

Genetic Programming traditionally uses Lisp, which is a pure functional language, and often programs are often shown as tree diagrams (which occasionally look similar to some neural network diagrams - is this the source of your confusion?). The programs are evolved by exchanging entire branches of a tree (a function and all its parameters) between programs or regenerating an entire branch randomly.

There are certainly a lot of good (and a lot of bad) references on both of these topics out there - I refrain from listing them because it isn't clear what you are actually interested in. Wikipedia covers each of these techniques, and is a good starting point.

Zooba
  • 11,221
  • 3
  • 37
  • 40
  • I disagree/didn't understand: you could have many output neurons and interpret them however you like... you could sort an array if you have enough output units. Also, of course I checked wikipedia and google first and didn't get enough info. – Uri Feb 23 '11 at 21:33
  • 2
    @Uri Sure, with enough neurons you can do anything, that's the theory behind it. However, practical limitations (RAM, time) prevent these from being realised. If you are interested in evolving imperative languages rather than functional, you may want to look into Grammatical Evolution. If you are interested in evolving programs within a NN, you are quite possibly going outside the existing range of research, which means you'll have to try it and see what happens (and write it up so the next person who's interested can find out whether it works). – Zooba Feb 23 '11 at 21:53
0

Genetic programming is very different from Neural networks. What you are suggesting is more along the lines of genetic programming - making small random changes to a program, possibly "breeding" successful programs. It is not easy, and I have my doubts that it can be done successfully across a large program.

You may have more luck extracting a small but critical part of your program, one which has a few particular "aspects" (such as parameter values) that you can try to evolve.

Google is your friend.

BeeOnRope
  • 60,350
  • 16
  • 207
  • 386
  • I know they are different, but is there a difference between what I'm suggesting and genetic programming? If I can model the basic operations of programming using a neural net, why not all of the program? – Uri Feb 23 '11 at 21:37
  • As I understand it, you are suggesting genetic algorithms layered on top of a neural net. That is, the latter evolving the former. So in that sense it's an application of genetic programming, but having a neural net underneath isn't "typical". – BeeOnRope Feb 23 '11 at 22:13
0

Some sophisticated anti-virus programs as well as sophisticated malware use formal grammar and genetic operators to evolve against each other using neural networks.

Here is an example paper on the topic: http://nexginrc.org/nexginrcAdmin/PublicationsFiles/raid09-sadia.pdf

Sources: A class on Artificial Intelligence I took a couple years ago.

Alain
  • 26,663
  • 20
  • 114
  • 184
0

With regards to your main question, no one has ever tried that on programming languages to the best of my knowledge, but there is some research in the field of evolutionary computation that could be compared to something like that (but it's obviously a far-fetched comparison). As a matter of possible interest, I asked a similar question about sel-improving compilers a while ago.

For a difference between genetic algorithms and genetic programming, have a look at this question.

Neural networks have nothing to do with genetic algorithms or genetic programming, but you can obviously use either to evolve neural nets (as any other thing for that matters).

Community
  • 1
  • 1
JohnIdol
  • 48,899
  • 61
  • 158
  • 242
0

You could have look at genetic-programming.org where they claim that they have found some near human competitive results produced by genetic programming.

I have not heard of self-evolving and self-imrpvoing programs before. They may exist as special research tools like genetic-programming.org have but nothing solid for generic use. And even if they exist they are very limited to special purpose operations like malware detection as Alain mentioned.

Unmanned Player
  • 1,109
  • 9
  • 30