Question 3:
In a Markov Decision Process (MDP), the state $s \in \{1,2,3,...,99\}$ is defined as the total capital
and action $a \in \{0,1,..., \text{min}(s, 100-s)\}$ is the current investment. In each step, an investment
will be successful with $p$ probability that lead to double the invested money. If it fails, the
investment will be lost. The termination condition is defined as reaching the total capital to 100 or
0. The reward is zero in each transition except the game ends with full capital. In this case, the
reward is +1.
a) For $p = 0.25$ and $p = 0.4$, implement value iteration via MATLAB to obtain optimum
amount of investment with respect to the capital, and illustrate value estimates-capital (i.e,
y-axis is value estimates, x-axis is capital) and optimum policy-capital graphs.
b) For $p = 0.25$ and $p = 0.4$, implement policy iteration via MATLAB to obtain optimum
amount of investment with respect to the capital, and illustrate value estimates-capital (i.e,
y-axis is value estimates, x-axis is capital) and optimum policy-capital graphs.
