Numerical Methods for Controlled Stochastic Delay Systems

This is the first comprehensive book on the subject
Useful forms of numerical algorithms and system approximations are given
Various examples are presented with applications to control and modern communications systems
For a broad audience of graduate students, researchers, and practitioners in control theory, stochastic systems, numerical analysis, operations research, probability theory, and dynamical systems.
May be used as a graduate-level textbook in a special topics course or seminar on numerical methods in stochastic control
The Markov chain approximation methods are widely used for the numerical solution of nonlinear stochastic control problems in continuous time. This book extends the methods to stochastic systems with delays. Because such problems are infinite-dimensional, many new issues arise in getting good numerical approximations and in the convergence proofs. Useful forms of numerical algorithms and system approximations are developed in this work, and the convergence proofs are given. All of the usual cost functions are treated as well as singular and impulsive controls. A major concern is on representations and approximations that use minimal memory.