Competitive Markov Decision Processes

This book is devoted to a unified treatment of Competitive Markov Decision Processes. It examines these processes from the standpoints of modeling and of optimization, providing newcomers to the field with an accessible account of algorithms, theory, and applications, while also supplying specialists with a comprehensive survey of recent developments. The treatment is self-contained, requiring only some knowledge of linear algebra and real analysis. Topics covered include: Mathematical programming: Markov decision processes (the non-competitive case), and shastic games via mathematical programming.- Existence, structure and applications: Summable shastic games, average-reward shastic games and applications and special classes of shastic games.- Appendices on: matrix games, bimatrix games and nonlinear programming; a theorem of Hardy and Littlewood; Markov chains; and complex varieties and the limit discount equation.