Pontryagin's minimum principle (case : Linear sytem, Quadratic performance index, finite time horizon)

ALAZARD Daniel — 2016-11-09T10:17:54Z

Optimal control

Simplified case : Linear system, quadratic performance index, fixed horizon and final state

Problem :
Solution using Pontryagin's minimum principle :
Exercises

Problem :

Let us consider the linear system :

From a given initial state , the objective is to bring back the state to 0 within a given time horizon () while minimizing the quadratic performance index :

where and are given weighting matrices with and 0" title="\mathbf{R}>0" />.

Solution using Pontryagin's minimum principle :

The Hamiltonian reads :

where is the costate vector.

the optimal control minimizes :

Costate dynamics :

State-costate dynamics : (1), (2) and (3) leads to :

with

is the Hamiltonian matrix associated to such a control problem. (4) can be intregrated taken into account boundary conditions on the state-costate augmented vector :

initial conditions on : (5),
terminal conditions on : (6).

The set of equations (4), (5) and (6) is also called a two point boundary-value problem.

Integration of the two point boundary-value problem :

where , are the 4 submatrices partionning (WARNING !! : ).

Then one can easily derive the initial value of the costate :

where depends only on the problem data : , , , , and not on .

Optimal control initial value : from equation (2) :

Closed-loop optimal control at any time : at time , assuming that the current state is known (using a measurement system), the objective is still to bring back the final state to () but the time horizon is now . The calculus of the current optimal control is the same problem than the previous one, just changing by and by . Thus :

with :

the time-varying state feedback to be implemented in closed-loop according to the following Figure :

Remark : is not defined since and is not invertible.

Optimal state trajectories : The integration of equation (4) between 0 and () leads to (first row) :

where :

is called the transition matrix.

Optimal performance index :

For any and a current state one can define the cost-to-go function (or value-function) as :

and the optimal cost-to-go function as :

From equation (4) : one can derive that :

Thus (after simplification) :

Thus :

From this last equation, on can find again the definition of the costate used to solve the Hamilton–Jacobi–Bellman equation ; i.e. : the gradient of the optimal cost-to-go function w.r.t. :

The optimal performance index is : .

Exercises

Exo #1 : show that is the solution of the matrix Riccati differential equation :

also written as :

Exo #2 : considering now that , compute the time-variant state feedback gain and the time-variant feedforward gain of the optimal closed-loop control law to be implemented according to the following Figure.

Reverse Engineering in Control Design

ALAZARD Daniel — 2012-08-02T09:08:58Z

Some tools and demo files in relation with the book : Reverse engineering in control design

If you publish work that uses this material, please refer to the following book :

Daniel Alazard ,
“Reserve Engineering in control design”.
Wiley, ISTE, 2013.
http://www.iste.co.uk/index.php?f=a&ACTION=View&id=556

Download

Matlab toolbox bib_obr.zip :

Some basic Matlab tools bib1.zip :

Demo files (including toolboxes) Matlab_files.zip :

Materials for JDMACS 2011 - GT MOSAR

ALAZARD Daniel — 2011-06-01T15:02:28Z

Slides :
Matlab toolbox for Observer-based Realization :

bib_obr.zip
some basic Matlab tools :

bib1.zip
MATLAB training lab-work :

See also : Commande Robuste des Systèmes Linéaires - Techniques LMI - Boîte à outil RoMulOC for the course on Robustness Analysis using LMI and RoMUlOC toolbox.

Pontryagin's minimum principle (case : Linear sytem, Quadratic performance index, finite time horizon)

Pontryagin's minimum principle (case : Linear sytem, Quadratic performance index, finite time horizon)

Optimal control

Contents

Problem :

Solution using Pontryagin's minimum principle :

Exercises

Reverse Engineering in Control Design

Materials for JDMACS 2011 - GT MOSAR