1

I have heard that neural networks are very good when implementing solutions to sequential-decision tasks. However, I assume that the qualifier "sequential" exists because there must likewise be a "non-sequential" realm. Just wondering if there were some standard, canonical examples some people knew of the top of their head. Thanks :)

Chuck Testa
  • 5,937
  • 3
  • 16
  • 11

2 Answers2

1

Driving is a good example of a sequential task: You are always deciding whether to steer straight, left, or right. It is sequential because there is some uncertainty in the vehicle; you might steer straight but the car goes a little bit to the right. When you notice this, you correct by steering a little to the left. There would be no way to make all the decisions ahead of time, close your eyes, and drive without crashing.

Planning a path from A to B is an example of a non-sequential task (e.g., Google maps). You know the map, you know the start & goal, and you know the road network. You can make all decisions at once (not sequential) and complete the task (finding a path).

More generally, you'll be making sequential decisions anytime you have some uncertainty and observe something, then react appropriately. From a machine learning perspective, this is the difference between Supervised learning and Online learning.

Felix
  • 2,064
  • 3
  • 21
  • 31
  • "From a machine learning perspective, this is the difference between Supervised learning and Online learning." Thanks man! Can you explain how this relates? :) – Chuck Testa Dec 24 '11 at 03:36
  • Supervised learning: You have all the data now, and use it to train some regressor/classifier (your "decision" maker). Online learning: You have some small amount of data. You use it to train a regressor/classifier, and make a prediction/decision. You then receive more data, and __update__ your decision maker. This process repeats. – Felix Dec 24 '11 at 10:25
0

Recognizing a picture of your mother. Deciding what you're going to have for breakfast. Coming up with examples of non-sequential decision tasks.

David Schwartz
  • 179,497
  • 17
  • 214
  • 278
  • Deciding what you're going to have for breakfast? That seems pretty sequential? I'm not sure how that becomes a non-sequential task? – Chuck Testa Dec 22 '11 at 05:03
  • Think about how you decide what to have for breakfast. Then see if you can write them out as a sequence of steps. Surely it *can* be done in a sequential way (rate each food you have, determine for all recipes which you can make, rate each recipe on difficulty and quality, consider all combinations of recipe items to see which fit into sensible breakfasts) but nobody actually decides that way. – David Schwartz Dec 22 '11 at 05:49
  • But when a neural network or computer does it, don't they have to *compute* everything sequentially anyway, regardless of whether it's considered a sequential or non-sequential decision? Is there a standard example of a sequential decision to compare our non-sequential examples with? – Chuck Testa Dec 22 '11 at 06:26
  • In theory, it should be possible in principle to map any non-sequential method to a sequential method and vice-versa. – David Schwartz Dec 22 '11 at 06:28
  • Would you then say that a neural network (that corresponds to an agent in a game) that has sensor data feeding into its input neurons, and certain actions corresponding to each output neuron, a non-sequential model? I'm confused because that's what it would seem like, but I always hear that example used when people are referring to "sequential" decision tasks.... – Chuck Testa Dec 22 '11 at 16:54
  • If you implement that network in hardware, then it's non-sequential. If you implement it in software, it's sequential. For any computable, non-sequential model, there's a corresponding sequential model. (Whether there are sequential models for non-compatible, non-sequential models is mostly a philosophical question). – David Schwartz Dec 22 '11 at 20:42