L’excellence passionnément
Optimal control
Simplified case : Linear system, quadratic performance index, fixed horizon and final state
Contents
- Problem :
- Solution using Pontryagin’s minimum principle :
- Exercises
Problem :
Let us consider the linear system :
From a given initial state , the objective is to bring back the state to 0 within a given time horizon () while minimizing the quadratic performance index :
where and are given weighting matrices with and .
Solution using Pontryagin’s minimum principle :
- The Hamiltonian reads :
where is the costate vector.
- the optimal control minimizes :
- Costate dynamics :
- State-costate dynamics : (1), (2) and (3) leads to :
with
is the Hamiltonian matrix associated to such a control problem. (4) can be intregrated taken into account boundary conditions on the state-costate augmented vector :
- initial conditions on : (5),
- terminal conditions on : (6).
The set of equations (4), (5) and (6) is also called a two point boundary-value problem.
- Integration of the two point boundary-value problem :
where , are the 4 submatrices partionning (WARNING !! : ).
Then one can easily derive the initial value of the costate :
where depends only on the problem data : , , , , and not on .
- Optimal control initial value : from equation (2) :
- Closed-loop optimal control at any time : at time , assuming that the current state is known (using a measurement system), the objective is still to bring back the final state to () but the time horizon is now . The calculus of the current optimal control is the same problem than the previous one, just changing by and by . Thus :
with :
the time-varying state feedback to be implemented in closed-loop according to the following Figure :
Remark : is not defined since and is not invertible.
- Optimal state trajectories : The integration of equation (4) between 0 and () leads to (first row) :
where :
is called the transition matrix.
- Optimal performance index :
For any and a current state one can define the cost-to-go function (or value-function) as :
and the optimal cost-to-go function as :
From equation (4) : one can derive that :
Thus (after simplification) :
Thus :
From this last equation, on can find again the definition of the costate used to solve the Hamilton–Jacobi–Bellman equation ; i.e. : the gradient of the optimal cost-to-go function w.r.t. :
The optimal performance index is : .
Exercises
- Exo #1 : show that is the solution of the matrix Riccati differential equation :
also written as :
- Exo #2 : considering now that , compute the time-variant state feedback gain and the time-variant feedforward gain of the optimal closed-loop control law to be implemented according to the following Figure.
