Stella X. Yu : Papers / Google Scholar

Optimization Models for the First Arrival Target Distribution Function in Discrete Time Stella X. Yu and Yuanlie Lin and Pingfan Yan Journal of Mathematical Analysis and Applications, 225(1):193-223, 1998 Paper Abstract This paper deals with countable state, countable action MDP endowed with a distribution function optimality criterion for the positive first arrival target total return. Based on the basic properties of the objective functions, convex combination and cut-and-paste properties of the optimal policies, the optimality equations for the value functions and optimality conditions are obtained. If the complete or the local stochastic order optimal policies exist, there must be deterministic stationary optimal policies. If the single point stochastic order optimal policies exist, there must be deterministic nonstationary policies. These results are applied to systems with finite state space and action space. It is shown that the single point stochastic order optimal policies must exist. An algorithm is developed to compute the value functions and the optimal action sets, from which all optimal policies can be constructed. Numerical results are given and they indicate possible directions of further research on the optimality constraints on system parameters. Keywords Markov Decision Process, distribution function, first arrival target, stochastic order, optimal policy