Solution to Markov Analysis Problems using Formula