Research Catalog

Title

Numerical methods for stochastic control problems in continuous time / Harold J. Kushner, Paul G. Dupuis.

Author

Kushner, Harold J. (Harold Joseph), 1933-

Publication

New York : Springer-Verlag, c1992.

Items in the Library & Off-site

Holdings

Details

Additional Authors

Dupuis, Paul

Description

ix, 439 p. : ill.; 25 cm.

Series Statement

Applications of mathematics ; 24

Uniform Title

Applications of mathematics 24.

Subject

• Markov Chains
• Markov processes
• Numerical analysis
• Stochastic control theory

Bibliography (note)

• Includes bibliographical references (p. [423]-431) and indexes.

Processing Action (note)

• committed to retain

Contents

• 1. Review of Continuous Time Models -- 1.1. Martingales and Martingale Inequalities -- 1.2. Stochastic Integration -- 1.3. Stochastic Differential Equations Diffusions -- 1.4. Reflected Diffusions -- 1.5. Processes with Jumps -- 2. Controlled Markov Chains -- 2.1. Recursive Equations for the Cost -- 2.2. Optimal Stopping Problems -- 2.3. Discounted Cost -- 2.4. Control to a Target Set and Contraction Mappings -- 2.5. Finite Time Control Problems -- 3. Dynamic Programming Equations -- 3.1. Functionals of Uncontrolled Processes -- 3.2. Optimal Stopping Problen -- 3.3. Control Until a Target Set Is Reached -- 3.4. Discounted Problem with a Target Set and Reflection -- 3.5. Average Cost Per Unit Time -- 4. Markov Chain Approximation Method: Introduction -- 4.1. Markov Chain Approximation Method -- 4.2. Continuous Time Interpolation and Approximating Cost Function -- 4.3. Continuous Time Markov Chain Interpolation -- 4.4. Random Walk Approximation to the Wiener Process -- 4.5. Deterministic Discounted Problem -- 4.6. Deterministic Relaxed Controls -- 5. Construction of the Approximating Markov Chain -- 5.1. Finite Difference Type Approximations: One Dimensional Examples -- 5.2. Numerical Simplifications and Alternatives for Example 4 -- 5.3. General Finite Difference Method -- 5.4. Direct Construction of the Approximating Markov Chain -- 5.5. Variable Grids -- 5.6. Jump Diffusion Processes -- 5.7. Approximations for Reflecting Boundaries -- 5.8. Dynamic Programming Equations -- 6. Computational Methods for Controlled Markov Chains -- 6.1. Problem Formulation -- 6.2. Classical Iterative Methods: Approximation in Policy and Value Space -- 6.3. Error Bounds for Discounted Problems -- 6.4. Accelerated Jacobi and Gauss-Seidel Methods -- 6.5. Domain Decomposition and Implementation on Parallel Processors -- 6.6. State Aggregation Method
• 6.7. Coarse Grid-Fine Grid Solutions -- 6.8. Multigrid Method -- 6.9. Linear Programming Formulations and Constraints -- 7. Ergodic Cost Problem: Formulations and Algorithms -- 7.1. Control Problem for the Markov Chain: Formulation -- 7.2. Jacobi Type Iteration -- 7.3. Approximation in Policy Space -- 7.4. Numerical Methods for the Solution of (3.4) -- 7.5. Control Problem for the Approximating Markov Chain -- 7.6. Continuous Parameter Markov Chain Interpolation -- 7.7. Computations for the Approximating Markov Chain -- 7.8. Boundary Costs and Controls -- 8. Heavy Traffic and Singular Control Problems: Examples and Markov Chain Approximations -- 8.1. Motivating Examples -- 8.2. Heavy Traffic Problem: A Markov Chain Approximation -- 8.3. Singular Control: A Markov Chain Approximation -- 9. Weak Convergence and the Characterization of Processes -- 9.1. Weak Convergence -- 9.2. Criteria for Tightness in D[superscript k] [actual symbol not reproducible] -- 9.3. Characterization of Processes -- 9.4. Example -- 9.5. Relaxed Controls -- 10. Convergence Proofs -- 10.1. Limit Theorems and Approximations of Relaxed Controls -- 10.2. Existence of an Optimal Control: Absorbing Boundary -- 10.3. Approximating the Optimal Control -- 10.4. Approximating Markov Chain: Weak Convergence -- 10.5. Convergence of the Costs: Discounted Cost and Absorbing Boundary -- 10.6. Optimal Stopping Problem -- 11. Convergence for Reflecting Boundaries, Singular Coutrol and Ergodic Cost Problems -- 11.1. Reflecting Boundary Problem -- 11:2. Singular Control Problem -- 11.3. Ergodic Cost Problem -- 12. Finite Time Problems and Nonlinear Filtering -- 12.1. Explicit Approximation Method: An Example -- 12.2. General Explicit Approximation Method -- 12.3. Implicit Approximation Method: An Example -- ^ 12.4. General Implicit Approximation Method
• 12.5. Optimal Control Problem: Approximations and Dynamic Programing Equations -- 12.6. Methods of Solution, Decomposition and Convergence -- 12.7. Nonlinear Filtering -- 13. Problems from the Calculus of Variations -- 13.1. Problems of Interest -- 13.2. Numerical Schemes and Convergence for the Finite Time Problem -- 13.3. Problems with a Controlled Stopping Time -- 13.4. Problems with a Discontinuous Running Cost -- 14. Viscosity Solution Approach to Proving Convergence of Numerical Schemes -- 14.1. Definitions and Some Properties of Viscosity Solutions -- 14.2. Numerical Schemes -- 14.3. Proof of Convergence.

ISBN

• 0387978348 (New York : acid-free paper)
• 3540978348 (Berlin : acid-free paper)

LCCN

^^^92010350^

OCLC

25552666

Owning Institutions

Harvard Library