1

I am looking for a way to generate Kolmogorov-Chapman equations for MathCad to solve Markov Chain problem. Problem is to find probability of the system being in one of the states. System has N components. I have a graph with 2^N nodes (states), and 2*N parameter: N a's, which is probability that Nth component will break down and N b's, which is the probability that broken component will become healthy again.

N can be close to 10, which means, that there will be 1024 equations at least, so I'm looking for a way to generate these equations.

Mathcad is not a requirement, any other math engine is OK (even Python if then can be solved with smth like scipy)

Rahul
  • 1,727
  • 3
  • 18
  • 33
  • Start by writing one equation, then generalize that. What does "looking for a way to generate" mean? Are you having trouble writing the equations or programming them? The former is not a good SO question; the latter might be if you'd post the code you've written. – duffymo Sep 10 '17 at 13:50
  • Are you using the probability matrix form of the Makov Chain? If so you just need matrix multiplication. [Matrix Form](https://www.youtube.com/watch?v=W5P4kCpdhho) – Felix Castor Sep 10 '17 at 15:32

1 Answers1

2

Problem is to find probability of the system being in one of the states.

As your stochastic process is a combination of N independent Markov chains, the problem is much easier to approach by solving the equations for each chain separately than solving a system of 22*N equations for the whole process.

The transition matrix for an individual chain is:

enter image description here

So, the Kolmogorov-Chapman equations are:

enter image description here

enter image description here

And if you still need the equations for the entire system:

enter image description here

Anton
  • 3,113
  • 14
  • 12
  • Do you mean that the power-iteration might be a good heuristic? (only for well-behaved systems, I guess) – wildplasser Sep 10 '17 at 22:12
  • 1
    @wildplasser I don't think you need power-iteration to calculate a 2x2 matrix raised to a power (or the limit of such matrices, or their eigenvectors). All of it can be done analytically. – Anton Sep 10 '17 at 22:30