Question

Exercise 2 (Discounted Gain) Consider the discrete-time Markov chain with the graph below. Let $\pi_i(t)$ denote the probability of being in state $i$ at time $t$, and let $\pi_i$ denote the proportion of time spent in state $i$ over an infinite horizon. The rewards are $r_{12} = +2$ and $r_{32} = -1$ (all other rewards are zero). Let $\gamma \le 1$ be the discount factor, $v(i)$ the expected total discounted reward starting from state $i$, and $g(i)$ the average reward (per period) starting from state $i$. 1. Determine $\pi_1, \pi_2, \pi_3$. 2. Deduce $g(1), g(2), and g(3)$. 3. Write a linear system to compute $v(1), v(2), and v(3)$. 4. When $\gamma = 0$, what are $v(1), v(2), and v(3)$? 5. When $\gamma = 0.5$, what are $v(1), v(2), and v(3)$? 6. When $\gamma = 1$, what are $v(1), v(2), and v(3)$?

          Exercise 2 (Discounted Gain) Consider the discrete-time Markov chain with the graph below. Let $\pi_i(t)$ denote the probability of being in state $i$ at time $t$, and let $\pi_i$ denote the proportion of time spent in state $i$ over an infinite horizon. The rewards are $r_{12} = +2$ and $r_{32} = -1$ (all other rewards are zero). Let $\gamma \le 1$ be the discount factor, $v(i)$ the expected total discounted reward starting from state $i$, and $g(i)$ the average reward (per period) starting from state $i$.

1. Determine $\pi_1, \pi_2, \pi_3$.
2. Deduce $g(1), g(2), and g(3)$.
3. Write a linear system to compute $v(1), v(2), and v(3)$.
4. When $\gamma = 0$, what are $v(1), v(2), and v(3)$?
5. When $\gamma = 0.5$, what are $v(1), v(2), and v(3)$?
6. When $\gamma = 1$, what are $v(1), v(2), and v(3)$?

exercise 2 discounted gain consider the discrete time markov chain with the graph below let piit denote the probability of being in state i at time t and let pii denote the proportion of t 54113

Added by Vicenta S.

Question

Please give Ace some feedback