Chapter 15, Probabilistic Reasoning over Time Video Solutions, Artificial Intelligence. A Modern Approach [Global Edition]

Problem 1

Show that any second-order Markov process can be rewritten as a first-order Markov process with an augmented set of state variables. Can this always be done parsimoniously, i.e., without increasing the number of parameters needed to specify the transition model?

Check back soon!

01:12

Problem 2

In this exercise, we examine what happens to the probabilities in the umbrella world in the limit of long time sequences.
a. Suppose we observe an unending sequence of days on which the umbrella appears. Show that, as the days go by, the probability of rain on the current day increases monotonically toward a fixed point. Calculate this fixed point.
b. Now consider forecasting further and further into the future, given just the first two umbrella observations. First, compute the probability $(2+k \mid 1,2)$ for $=120$ and plot the results. You should see that the probability converges towards a fixed point. Prove that the exact value of this fixed point is 0.5 .

Dominador Tan

Numerade Educator

Problem 3

This exercise develops a space-efficient variant of the forward-backward algorithm described in Figure 15.4 (page 576). We wish to compute $\mathbf{P}\left(\mathbf{X}_k \mid \mathbf{e}_{1: t}\right)$ for $=1$. .t. This will be done with a divide-and-conquer approach.
a. Suppose, for simplicity, that $t$ is odd, and let the halfway point be $h=(t+1) 2$. Show that $\mathbf{P}\left(\mathbf{X}_k \mid \mathbf{e}_{1: t}\right)$ can be computed for $=1$, $h$ given just the initial forward message $\mathbf{f}_{1: 0}$, the backward message $\mathbf{b}_{h+1: t}$, and the evidence $\mathbf{e}_{1: h}$.
b. Show a similar result for the second half of the sequence.
c. Given the results of (a) and (b), a recursive divide-and-conquer algorithm can be constructed by first running forward along the sequence and then backward from the end, storing just the required messages at the middle and the ends. Then the algorithm is called on each half. Write out the algorithm in detail.
d. Compute the time and space complexity of the algorithm as a function of $t$, the length of the sequence. How does this change if we divide the input into more than two pieces?

Check back soon!

Problem 4

On page 577, we outlined a flawed procedure for finding the most likely state sequence, given an observation sequence. The procedure involves finding the most likely state at each time step, using smoothing, and returning the sequence composed of these states. Show that, for some temporal probability models and observation sequences, this procedure returns an impossible state sequence (i.e., the posterior probability of the sequence is zero).

Check back soon!

Problem 5

Equation (15.12) describes the filtering process for the matrix formulation of HMMs. Give a similar equation for the calculation of likelihoods, which was described generically in Equation (15.7).

Check back soon!

02:28

Problem 6

In Section 15.3.2, the prior distribution over locations is uniform and the transition model assumes an equal probability of moving to any neighboring square. What if those assumptions are wrong? Suppose that the initial location is actually chosen uniformly from the northwest quadrant of the room and the Move action actually tends to move southeast. Keeping the HMM model fixed, explore the effect on localization and path accuracy as the southeasterly tendency increases, for different values of .

Victor Salazar

Numerade Educator

01:17

Problem 7

Consider a version of the vacuum robot (page 582) that has the policy of going straight for as long as it can; only when it encounters an obstacle does it change to a new (randomly selected) heading. To model this robot, each state in the model consists of a (location, heading) pair. Implement this model and see how well the Viterbi algorithm can track a robot with this model. The robot's policy is more constrained than the random-walk robot; does that mean that predictions of the most likely path are more accurate?

Liam Haas-Neill

Numerade Educator

Problem 8

We have described three policies for the vacuum robot: (1) a uniform random walk, (2) a bias for wandering southeast, as described in Exercise 15.6, and (3) the policy described in Exercise 15.7. Suppose an observer is given the observation sequence from a vacuum robot, but is not sure which of the three policies the robot is following. What approach should the observer use to find the most likely path, given the observations? Implement the approach and test it. How much does the localization accuracy suffer, compared to the case in which the observer knows which policy the robot is following?

Check back soon!

10:19

Problem 9

This exercise is concemed with filtering in an environment with no landmarks. Consider a vacuum robot in an empty room, represented by an $\times$ rectangular grid. The robot's location is hidden; the only evidence available to the observer is a noisy location sensor that gives an approximation to the robot's location. If the robot is at location ( . ) then with probability .1 the sensor gives the correct location, with probability .05 each it reports one of the 8 locations immediately surrounding ( . ), with probability . 025 each it reports one of the 16 locations that surround those 8 , and with the remaining probability of .1 it reports "no reading." The robot's policy is to pick a direction and follow it with probability .7 on each step; the robot switches to a randomly selected new heading with probability .3 (or with probability 1 if it encounters a wall). Implement this as an HMM and do filtering to track the robot. How accurately can we track the robot's path?

Will Erickson

Numerade Educator

Problem 10

Often, we wish to monitor a continuous-state system whose behavior switches unpredictably among a set of distinct "modes." For example, an aircraft trying to evade a missile can execute a series of distinct maneuvers that the missile may attempt to track. A Bayesian network representation of such a switching Kalman filter model is shown in Figure 15.21.
a. Suppose that the discrete state $t$ has possible values and that the prior continuous state estimate $\mathbf{P}\left(\mathbf{X}_0\right)$ is a multivariate Gaussian distribution. Show that the prediction $\mathbf{P}\left(\mathbf{X}_1\right)$ is a mixture of Gaussians - that is, a weighted sum of Gaussians such that the weights sum to 1 .
b. Show that if the current continuous state estimate $\mathbf{P}\left(\mathbf{X}_t \mid \mathbf{e}_{1: t}\right)$ is a mixture of Gaussians, then in the general case the updated state estimate $\mathbf{P}\left(\mathbf{X}_{t+1} \mid \mathbf{e}_{1: t+1}\right)$ will be a mixture of Gaussians.
c. What aspect of the temporal process do the weights in the Gaussian mixture represent?

The results in (a) and (b) show that the representation of the posterior grows without limit even for switching Kalman filters, which are among the simplest hybrid dynamic models.

Check back soon!

Problem 11

Complete the missing step in the derivation of Equation (15.19) on page 586, the first update step for the one-dimensional Kalman filter.

Check back soon!

02:18

Problem 12

Let us examine the behavior of the variance update in Equation (15.20) (page 587).
a. Plot the value of $\frac{2}{t}$ as a function of $t$, given various values for $\frac{2}{x}$ and $\frac{2}{z}$.
b. Show that the update has a fixed point ${ }^2$ such that ${ }_t^2 \rightarrow{ }^2$ as $t \rightarrow \infty$, and calculate the value of 2 .
c. Give a qualitative explanation for what happens as $\frac{2}{x} \rightarrow 0$ and as $\underset{z}{2} \rightarrow 0$.

Jerelyn Nevil

Numerade Educator

03:06

Problem 13

A professor wants to know if students are getting enough sleep. Each day, the professor observes whether the students sleep in class, and whether they have red eyes. The professor has the following domain theory:
$\bullet$ The prior probability of getting enough sleep, with no observations, is 0.6 .
$\bullet$ The probability of getting enough sleep on night $t$ is 0.8 given that the student got enough sleep the previous night, and 0.2 if not.
$\bullet$ The probability of having red eyes is 0.2 if the student got enough sleep, and 0.7 if not.
$\bullet$The probability of sleeping in class is 0.1 if the student got enough sleep, and 0.3 if not. Formulate this information as a dynamic Bayesian network that the professor could use to filter or predict from a sequence of observations. Then reformulate it as a hidden Markov model that has only a single observation variable. Give the complete probability tables for the model.

Sheryl Ezze

Numerade Educator

03:11

Problem 14

For the DBN specified in Exercise 15.13 and for the evidence values
$\mathbf{e}_1=$ not red eyes, not sleeping in class
$\mathbf{e}_2=$ red eyes, not sleeping in class
$\mathbf{e}_3=$ red eyes, sleeping in class
perform the following computations:
a. State estimation: Compute (EnoughSleep ${ }_t \mid \mathbf{e}_{1: t}$ ) for each of $t=1.2,3$.
b. Smoothing: Compute (EnoughSleep $\mathbf{e}_t \mid \mathbf{e}_{1: 3}$ ) for each of $t=1,2,3$.
c. Compare the filtered and smoothed probabilities for $t=1$ and $t=2$.

Maxime Rossetti

Numerade Educator

00:52

Problem 15

Suppose that a particular student shows up with red eyes and sleeps in class every day. Given the model described in Exercise 15.13, explain why the probability that the student had enough sleep the previous night converges to a fixed point rather than continuing to go down as we gather more days of evidence. What is the fixed point? Answer this both numerically (by computation) and analytically.

Tara Palmer

Numerade Educator

01:36

Problem 16

This exercise analyzes in more detail the persistent-failure model for the battery sensor in Figure 15.15(a) (page 594).
a. Figure 15.15 (b) stops at $t=32$. Describe qualitatively what should happen as $t \rightarrow \infty$ if the sensor continues to read 0 .
b. Suppose that the external temperature affects the battery sensor in such a way that transient failures become more likely as temperature increases. Show how to augment the DBN structure in Figure 15.15(a), and explain any required changes to the CPTs.
c. Given the new network structure, can battery readings be used by the robot to infer the current temperature?

Zachary Warner

Numerade Educator

04:30

Problem 17

Consider applying the variable elimination algorithm to the umbrella DBN unrolled algorithm - the size of the largest factor-is the same, regardless of whether the rain variables are eliminated in forward or backward order.

Bryan Lynn

Numerade Educator