
Optimization Models for the First Arrival Target Distribution Function in Discrete Time

Stella X. Yu and Yuanlie Lin and Pingfan Yan

Journal of Mathematical Analysis and Applications, 225(1):193223, 1998

Paper

Abstract

This paper deals with countable state, countable action MDP endowed with a distribution function optimality criterion for the positive first arrival target total return. Based on the basic properties of the objective functions, convex combination and cutandpaste properties of the optimal policies, the optimality equations for the value functions and optimality conditions are obtained. If the complete or the local stochastic order optimal policies exist, there must be deterministic stationary optimal policies. If the single point stochastic order optimal policies exist, there must be deterministic nonstationary policies. These results are applied to systems with finite state space and action space. It is shown that the single point stochastic order optimal policies must exist. An algorithm is developed to compute the value functions and the optimal action sets, from which all optimal policies can be constructed. Numerical results are given and they indicate possible directions of further research on the optimality constraints on system parameters.

Keywords

Markov Decision Process, distribution function, first arrival target, stochastic order, optimal policy
