To plot the development of the transition probabilities over time, you (by definition) need multiple observations for the transition from one action a
to another action b
.
Once you have multiple transaction probability matrices, it is actually very straightforward to plot their development over time. In my example below, most of the code is actually devoted to (re-)create your data and the different transition probability matrices.
library(ggplot2)
num_samples <- 1000
num_actions <- 20
# 1. generate activities dataframe (dropped ID column):
activ <- data.frame(v1 = sample(1:num_actions, num_samples, replace = TRUE),
v2 = sample(1:num_actions, num_samples, replace = TRUE),
v3 = sample(1:num_actions, num_samples, replace = TRUE),
v4 = sample(1:num_actions, num_samples, replace = TRUE),
v5 = sample(1:num_actions, num_samples, replace = TRUE),
v6 = sample(1:num_actions, num_samples, replace = TRUE),
v7 = sample(1:num_actions, num_samples, replace = TRUE))
num_transp <- ncol(activ) - 1
# 2. calculate transition probabilities for each time step:
l_transp <- vector("list", num_transp)
for (t in 1:num_transp){
transp <- matrix(0, nrow = num_actions, ncol = num_actions)
data = activ[, t:(t+1)]
for (action in 1:num_actions){
rows_to_keep <- data[,1] == action
counts <- table(data[rows_to_keep,])
probs <- as.data.frame(counts/sum(rows_to_keep))
follow_actions <- as.integer(as.character(probs[,2]))
transp[action, follow_actions] <- probs$Freq
}
l_transp[[t]] <- transp
}
# 3. get development of transition probability from action 2 to action 3 over time:
to_plot <- vector("numeric", num_transp)
for (i in 1:num_transp){
to_plot[i] <- l_transp[[i]][2, 3]
}
# plot development of transition probability from action 2 to action 3 over time:
ggplot(data.frame(x=1:num_transp, y=to_plot), aes(x=x, y=y)) +
geom_line() +
xlab("time") +
ylab("transition probability")
Running this will give you something like 
It looks very much like what one would expect from a scenario where 20 different actions were chosen randomly and independently over time.