官术网_书友最值得收藏!

How it works...

In Step 2, we calculated the transition probability after k steps, which is the kth power of the transition matrix. You will see the following output:

>>> print("Transition probability after 2 steps:\n{}".format(T_2))
Transition probability after 2 steps:
tensor([[0.6400, 0.3600],
[0.4800, 0.5200]])
>>> print("Transition probability after 5 steps:\n{}".format(T_5))
Transition probability after 5 steps:
tensor([[0.5670, 0.4330],
[0.5773, 0.4227]])
>>> print(
"Transition probability after 10 steps:\n{}".format(T_10))
Transition probability after 10 steps:
tensor([[0.5715, 0.4285],
[0.5714, 0.4286]])
>>> print(
"Transition probability after 15 steps:\n{}".format(T_15))
Transition probability after 15 steps:
tensor([[0.5714, 0.4286],
[0.5714, 0.4286]])
>>> print(
"Transition probability after 20 steps:\n{}".format(T_20))
Transition probability after 20 steps:
tensor([[0.5714, 0.4286],
[0.5714, 0.4286]])

We can see that, after 10 to 15 steps, the transition probability converges. This means that, no matter what state the process is in, it has the same probability of transitioning to s0 (57.14%) and s1 (42.86%).

In Step 4, we calculated the state distribution after k = 1, 2, 5, 10, 15, and 20 steps, which is the multiplication of the initial state distribution and the transition probability. You can see the results here:

>>> print("Distribution of states after 1 step:\n{}".format(v_1))
Distribution of states after 1 step:
tensor([[0.5200, 0.4800]])
>>> print("Distribution of states after 2 steps:\n{}".format(v_2))
Distribution of states after 2 steps:
tensor([[0.5920, 0.4080]])
>>> print("Distribution of states after 5 steps:\n{}".format(v_5))
Distribution of states after 5 steps:
tensor([[0.5701, 0.4299]])
>>> print(
"Distribution of states after 10 steps:\n{}".format(v_10))
Distribution of states after 10 steps:
tensor([[0.5714, 0.4286]])
>>> print(
"Distribution of states after 15 steps:\n{}".format(v_15))
Distribution of states after 15 steps:
tensor([[0.5714, 0.4286]])
>>> print(
"Distribution of states after 20 steps:\n{}".format(v_20))
Distribution of states after 20 steps:
tensor([[0.5714, 0.4286]])

We can see that, after 10 steps, the state distribution converges. The probability of being in s0 (57.14%) and the probability of being in s1 (42.86%) remain unchanged in the long run.

Starting with [0.7, 0.3], the state distribution after one iteration becomes [0.52, 0.48]. Details of its calculation are illustrated in the following diagram:

After another iteration, the state distribution becomes [0.592, 0.408] as calculated in the following diagram:

As time progresses, the state distribution reaches equilibrium.

主站蜘蛛池模板: 临安市| 敦化市| 泰安市| 喀喇| 巴青县| 康马县| 剑河县| 临颍县| 广饶县| 南涧| 玛沁县| 象州县| 镇赉县| 岳阳市| 扎鲁特旗| 昂仁县| 托克托县| 凤庆县| 昌乐县| 崇州市| 怀安县| 林芝县| 敖汉旗| 光山县| 济阳县| 莒南县| 古丈县| 保亭| 宝清县| 河南省| 色达县| 观塘区| 前郭尔| 子长县| 饶平县| 阿勒泰市| 宣化县| 屏东市| 安西县| 邢台市| 静乐县|