At the start of each week, the condition of a machine is
determined by measuring the amount of electrical current it uses.
According to its amperage reading, the machine is categorized as
being in one of the following four states: low, medium, high,
failed. A machine in the low state has a probability of 0.05, 0.03,
and 0.02 of being in the medium, high, or failed state,
respectively, at the start of the next week. A machine in the
medium state has a probability of 0.09 and 0.06 of being in the
high or failed state, respectively, at the start of the next week
(it cannot, by itself, go to the low state). And, a machine in the
high state has a probability of 0.1 of being in the failed state at
the start of the next week (it cannot, by itself, go to the low or
medium state). If a machine is in the failed state at the start of
a week, repair is immediately begun on the machine so that it will
(with probability 1) be in the low state at the start of the
following week. Let X be a Markov chain
where Xn is the state of the machine at
the start of week n.
(a) Give the Markov matrix for X.
(b) A new machine always starts in the low state. What is the
probability that the machine is in the failed state three weeks
after it is new?
(c) What is the probability that a machine has at least one failure
three weeks after it is new?
(d) On the average, how many weeks per year is the machine
working?
(e) Each week that the machine is in the low state, a profit of
$1,000 is realized; each week that the machine is in the medium
state, a profit of $500 is realized; each week that the machine is
in the high state, a profit of $400 is realized; and the week in
which a failure is fixed, a cost of $700 is incurred. What is the
long-run average profit per week realized by the machine?
(f) A suggestion has been made to change the maintenance policy for
the machine. If at the start of a week the machine is in the high
state, the machine will be taken out of service and repaired so
that at the start of the next week it will again be in the low
state. When a repair is made due to the machine being in the high
state instead of a failed state, a cost of $600 is incurred. Is
this new policy worthwhile?