Some studies on the approximations to the solution of Stochastic Differential games

(1)

SOME STUDIES ON THE APPROXIMATIONS TO THE SaUTION OF STOCHASTIC DIFFERENTIAL GAYE'S

RISHI SHUMSHERE RANA

Th

e

sis submitted to the Indian Institute of Technology for the award of

DOCTOR OF PHILOSOPHY

Department of Electrical Engineering Indian Institute of Tecthology, Delhi

February, 1977

(2)

ACKNOWLMLIDENT

I hereby express my deep sense of gratitude to Prof. A.K. Mahalanabis for his persistent help and

encouragement throughout the course of this work. It was entirely due to his efforts that I have been able

to complete this work.

I would also like to express my thanks to my friends and colleagues Dr. S. Prasad, Dr. K.K. Biswas, Dr. B.B. Hadan, Dr. V.F. Bhatkarl Mr. J. Kurien and

Hr. B.S. Bede for their, helpful suggestions and valuable discussions.

Finally, I would like to thank Mr. Ranjit Kumar • for his Tatient and competent typing of the manuscript.

(3)

CONTENTS

SECTION ^page

CHAPTER I INTRODUCTION AND SCOPE OF THE THESIS

1.1 Introduction 1

1.2 Linear Quadratic Differential Games 2 1.-5 Stochastic LQDG

1.4 Scope of the Thesis 12

CHAPTER II SOLUTION OF INFINITE DURATION DISCRETE TIME STOCHASTIC CONTROL AND STOCHASTIC DIFFEREN- TIAL GAME PROBTENO

2.1 Introduction 20

2.2 A Stochastic Control Problem 22 2.2.1 Problem Statement 22 2.2.2 Solution of the Finite Duration

Problem 24

2.2.3 Infinite Duration Case 27 2.3 New Solutions of the Control Problem 28

2.3.1 Solution Through the Separation

Principle 29

2.3.2 An Alternative Solution 36

2.4 A

Stochastic Multi-stage Game 39 2.4.1 Problem S catement 39 2.4 .2 Solution of the LQDG 42 2.5, A New Solution of the Infinite Duration

Discrete rime Game 46

2.5.1 Problem Formulation 47 2.5.2 Development of the Solution 49 2.5.3 Detailed Relations for an nth Order

System 52

(4)

CHAPTER II (Continued) Page 2.6 Solution of an Infinite Duration Caitinuous

Game 55

CHAPTER 11I EFFECT OF RANDOM PARAMETER VARIATIONS

3.1 Introduction Go

3.2 A Stochastic LQ Control Problem 63 3.2.1 System Description 63 3.2.2 Rev iew the Solution of the LQ

Optimal Control Probl.em 67 3.2.3 Approximation of the State Depen-

dent Noise 70

3.2.4 An Example 74

3.3 Direct Computation of the Feed-back Gain

for the Control Problem 76 3.3.1 Problem Formulation 76 3.3.2 Development of the Solution 78

3.3,3 An Example 80

Li A Stochastic LQDG Problem 82 3.4.1 Problem Description 82 3./1.2 Review of the Solution 84 3.k.3 Approximation of the State

Dependent Noise 87

3.4,4 An Example 88

3.5 Direct Computation of the Game Solution 90 3.5.1 Problem Formulation 90

3.5.2

Development of the Solution 92

3.5.3 An Example 94

(5)

page CHAPTER IV DEVELOPIvENT OF OUTPUT FEED-BACK

SOLUTIONS

4.1 In tr oduc ti on 97

4.2 A Discrete Time Control Problem 98

4.2.1 Problem Formulation 98 4.2.2 Development of the Solution 99

4.2.3 An Example , 104

4.3 A Stochastic Discrete Time Game Problem 106

4,3.1 Problem Formulation 106

4.3.2 Development of the Solution 109

4.3.3 An Example 111

A Continuous Time Stochastic Control

Problem 113

4 .4 .1 Re able ^III Formulation 113 The Development of the Solution 115 CHAPiER V SUMMARY AND • CONCLUDING REMARKS

`...) • 1 Summary 123

5.2 Concluding Remarks 127

REFERENCES