Class 03 - Q&A

[andrewrosemberg]

Please use this space to ask clarification questions, propose changes/corrections to the presented topics that should be accounted for in the final chapter, or point out interesting facts!

[andrewrosemberg]

Why do we define the Hamiltonian for an optimal control problem?

[jwestrenen3]

The Hamiltonian transforms a dynamic optimization problem into a set of differential equations. It uses the co-state
𝜆, acting like a Lagrange multiplier in time. The co-state can kind of be seen as a shadow price so: how much would the cost increase if the state were increased by a little? This balances the immediate cost and the future cost impact.

[andrewrosemberg]

What is the relationship between the Pontryagin’s Maximum (Minimum) Principle(s) and the KKT conditions presented in lecture 2?

[PedroGatech]

My understanding, as someone who understands physics more than optimization, is that the KKT is analogous to Lagrangian Mechanics, while the PMP is analogous to Hamiltonian mechanics, with the Lagrangian in KKT playing a similar role to the Hamiltonian in PMP. This is further solidified because we define the Hamiltonian based on the Lagrangian $H = L + \lambda^T f$.
And the fact that, for the optimal $u^$, we have $\cfrac{\partial H}{\partial u^} = 0$ is analogous to the stationarity conditions of the KKT $\nabla f(x^)+J_C (x^)^T \lambda^*=0$.
But my understanding is that we discretized the problem we defined KKT, while we first defined the PMP using continuous time. So I'd guess that PMP is essentially KKT but for continuous time.

[andrewrosemberg]

What is the multiple shooting method trying to solve from the single shooting approach?

[ArnaudDeza]

When we introduce the Hamiltonian in class, are we simply folding the dynamics into a Lagrangian, or is the real point that it yields a forward–backward pair (state/costate) we can actually solve by shooting on real problems? I am trying to make a connection to content in the lectures slides of class 02 where constrained optimization methods push constraints into objectives and then use linear system solvers. This is something I tried asking in lecture but I am still slightly confused about the differences between the two.

[ArnaudDeza]

Could we restate Pontryagin’s Maximum/Minimum Principle in the KKT language from Lecture 2? In particular I am wondering about what plays the role of multipliers and how to find/view the constraints for system of equations relating to dynamics and stationarity?

[ArnaudDeza]

For path constraints (e.g., torque or state bounds), would you prefer handling them via penalization in an indirect (PMP) setup or moving to a direct multiple-shooting NLP. Is there examples where you can show one is preferable or more commonly used than the other (maybe in the examples you have in the slides).

[PedroGatech]

In physics, Hamiltonian systems are known to be more "general" than Lagrangian systems, in the sense that we can describe more systems with Hamiltonian Mechanics than Lagrangian Mechanics (one example: we need to have a 1-1 correlation between velocity and momentum for this equivalence to hold, so photons that don't have mass, always have the same velocity but have different momenta, are not well described by Lagrangian Mechanics).
In this paper, Martin state the following:

The Hamiltonian approach is here regarded as more fundamental than the Lagrangian approach, for two reasons. It seems more natural in the first instance to treat the behaviour of a system in time in terms of a continuously unfolding transformation, rather than in terms of a variational principle applying simultaneously to the whole of a time range. More practically, it will be found that the Hamiltonian approach is the wider of the two, in that some of the Hamiltonian systems to be considered do not have Lagrangian forms.

and

The Lagrangian formulation can therefore exist only when the number of [dynamical variables]'s is even; this leads to the characteristic pairing of dynamical conjugates in a Lagrangian theory.

This leads to a simple 'rotation' system, with rotations around the three axes $(r_1,r_2,r_3)$, to have no (unique) Lagrangian, exactly because it has an odd number of dynamical variables.

Is this translatable to KKT vs PMP? In the sense that PMP is somehow more "fundamental" than KKT, and we can use PMP to describe a wider set of systems?

[andrewrosemberg]

In class, we talked about the ill-conditioning of some systems that makes it hard to solve with single shooting. Can anyone give me a concrete example where single shooting has "exploding" gradients? i.e., in the backward pass, these gradients tend to be exponentially larger with increasing horizon. If a good example is given, it should also be easy to change it a bit to make the case for vanishing gradients.