Refer back to Exercise 1 in the previous section. Consider...

Refer back to Exercise 1 in the previous section. Consider two negotiators, $A$ and $B,$ who employ strategies according to the Markov chain model described. $$ \begin{array}{l}{\text { (a) Construct the one-step transition matrix for the Markov chain } X_{n}=\text { strategy employed at }} \\ {\text { the } n \text { th stage of a negotiation, assuming the states are }(1) \text { cooperative and }(2) \text { competitive. }}\end{array} $$ $$ \begin{array}{l}{\text { (b) If negotiator A employs a cooperative strategy at some stage, what is the probability she }} \\ {\text { uses a competitive strategy the next time? [Don't forget that A's turns are two time steps }} \\ {\text { apart, since } B \text { counter-negotiates in between.] }}\end{array} $$ $$ \begin{array}{l}{\text { (c) Now introduce a third state, }(3) \text { end of the negotiation. Assume that a Markov chain model }} \\ {\text { with the following one-step transition matrix applies: }}\end{array} $$ $$ \mathbf{P}=\left[ \begin{array}{ccc}{.6} & {.2} & {.2} \\ {3} & {.4} & {.3} \\ {0} & {0} & {1}\end{array}\right] $$ Given that the initial strategy presented is cooperative, what is the probability the negotiations end within three time steps? $$ \begin{array}{l}{\text { (d) Refer back to (c). Given that the initial strategy presented is competitive, what is the }} \\ {\text { probability the negotiations end within three time steps? }}\end{array} $$

Refer back to Exercise 1 in the previous section. Consider two negotiators, $A$ and $B,$ who
employ strategies according to the Markov chain model described.
$$
\begin{array}{l}{\text { (a) Construct the one-step transition matrix for the Markov chain } X_{n}=\text { strategy employed at }} \\ {\text { the } n \text { th stage of a negotiation, assuming the states are }(1) \text { cooperative and }(2) \text { competitive. }}\end{array}
$$
$$
\begin{array}{l}{\text { (b) If negotiator A employs a cooperative strategy at some stage, what is the probability she }} \\ {\text { uses a competitive strategy the next time? [Don't forget that A's turns are two time steps }} \\ {\text { apart, since } B \text { counter-negotiates in between.] }}\end{array}
$$
$$
\begin{array}{l}{\text { (c) Now introduce a third state, }(3) \text { end of the negotiation. Assume that a Markov chain model }} \\ {\text { with the following one-step transition matrix applies: }}\end{array}
$$
$$
\mathbf{P}=\left[ \begin{array}{ccc}{.6} & {.2} & {.2} \\ {3} & {.4} & {.3} \\ {0} & {0} & {1}\end{array}\right]
$$
Given that the initial strategy presented is cooperative, what is the probability the
negotiations end within three time steps?
$$
\begin{array}{l}{\text { (d) Refer back to (c). Given that the initial strategy presented is competitive, what is the }} \\ {\text { probability the negotiations end within three time steps? }}\end{array}
$$

Key Concepts

Multi-Step Transition Probabilities

Multi-step transition probabilities refer to the likelihood of transitioning from one state to another over several time steps, as opposed to only one. These probabilities are calculated by raising the transition matrix to a power corresponding to the number of steps, which provides insight into the potential evolution of the system over time, including reaching certain states through various pathways.

Absorbing State

An absorbing state is a state in a Markov chain that, once entered, cannot be left. When a chain reaches an absorbing state, the process terminates or remains in that state indefinitely. This concept is particularly important in problems where a process or negotiation ends, and it is used to determine the likelihood and timing of the transition to this terminating condition.

Markov Chain

A Markov chain is a stochastic model that describes a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. This property, known as the Markov property, allows the process to be modeled as a memoryless process, making it particularly useful for predicting future states based solely on the current state without requiring knowledge of the full history.

Transition Matrix

The transition matrix is a fundamental tool in the study of Markov chains, representing the probabilities of moving from one state to another in a single time step. Each entry in the matrix indicates the probability of transitioning from one specified state to another, and the rows of the matrix typically sum to one. This matrix is essential for calculating multi-step probabilities and for understanding the overall behavior of the system over time.

Transcript

00:01 So in this question, we refer back to exercise one, which was in the previous chapter, which was the situation where we had two negotiators who could either be cooperating or be competitive.

00:13 And we determined in exercise one that the single step transition probabilities were these four probabilities here.

00:24 And so for part a of this exercise, we're asked to construct the one -step transition matrix for this markov chain, where x sub n is the strategy at the nth stage of negotiation.

00:38 So it is straightforward to construct that, given these transition probabilities, so p sub 1 .1 is simply entry 1 -1 in this matrix.

00:49 So we have 0 .6.

00:53 P sub 1 -2 is entry 1 -2, so 0 .4 goes here, and then we have 0 .3 here, and 0 .7.

01:03 So this is the one -step transition matrix.

01:07 Next, in part b we are asked, we are essentially asked what is the two -step probability that we will go from state 1 to state 1.

01:20 So remember we had two states.

01:22 We had cooperative and competitive.

01:28 So if we label this 1 and this 2, we're asking what is the probability we go from cooperative to competitive and two steps.

01:41 So i should have said competitive.

01:42 It's the probability, the two -step probability, i've going from 1 to 2.

01:53 And so the way to calculate this is to find the two -step transition matrix, and then look at entry 1 -2 in that matrix.

02:03 So using software, i've calculated this, and this matrix is as follows, 0 .48, 0 .52, 0 .39, and 0 .61...

Please give Ace some feedback