Merge branch 'main' into at/class22

Author: andrewrosemberg

README.md

@@ -11,6 +11,9 @@
 ## Overview
 This student-led course explores modern techniques for controlling — and learning to control — dynamical systems. Topics range from classical optimal control and numerical optimization to reinforcement learning, PDE-constrained optimization (finite-element methods, Neural DiffEq, PINNs, neural operators), and GPU-accelerated workflows.
 
+## Objective
+Create an online book at the end using the materials from all lectures.
+
 ## Prerequisites
 * Solid linear-algebra background  
 * Programming experience in Julia, Python, *or* MATLAB  
@@ -19,9 +22,22 @@ This student-led course explores modern techniques for controlling — and learn
 ## Grading
 | Component | Weight |
 |-----------|--------|
-| Participation & paper critiques | **25 %** |
-| In-class presentations | **50 %** |
-| Projects | **25 %** |
+| Participation | **25 %** |
+| In-class Presentations and Chapter | **50 %** |
+| Projects (Liaison work & Scribe & Admin & ...) | **25 %** |
+
+**Class material is due one week before the lecture!** No exceptions apart from the first 2 lectures.
+
+**Issues outlining references that will be used for lecture preparation are due at the end of the 3rd week (10/05/2025)!** 
+20 minutes of research should give you an initial idea of what you need to read.
+
+🎯🚲 **Guessing Game**
+
+Here’s how the presentation grading works: we already know the lecture content we expect from you. Any deviations will be penalized **exponentially**. Your mission is twofold:  
+1. **Check your understanding** — use [discussions](https://github.com/LearningToOptimize/LearningToControlClass/discussions) from previous lectures to ensure you’ve mastered earlier topics. We expect lectures to be extremely linked to each other.
+2. **Test your hypotheses** — validate your lecture content by raising and resolving issues, focusing primarily on your *main task issue* (see this example from [class 03](https://github.com/LearningToOptimize/LearningToControlClass/issues/18)).  
+
+All interactions will happen **only through GitHub** — no in-person hints will be given. 
 
 ## Weekly Schedule (Fall 2025 – Fridays 2 p.m. ET)
 
@@ -30,13 +46,13 @@ This student-led course explores modern techniques for controlling — and learn
 | #  | Date (MM/DD) | Format / Presenter | Topic & Learning Goals | Prep / Key Resources |
 |----|--------------|--------------------|------------------------|----------------------|
 | 1  | 08/22/2025   | Lecture — Andrew Rosemberg | Course map; why PDE-constrained **optimization**; tooling overview; stability & state-space dynamics; Lyapunov; discretization issues | [📚](https://learningtooptimize.github.io/LearningToControlClass/dev/class01/class01/) |
-| 2  | 08/29/2025   | Lecture - Arnaud Deza | Numerical **optimization** for control (grad/SQP/QP); ALM vs. interior-point vs. penalty methods | |
+| 2  | 08/29/2025   | Lecture - Arnaud Deza | Numerical **optimization** for control (grad/SQP/QP); ALM vs. interior-point vs. penalty methods | [📚](https://learningtooptimize.github.io/LearningToControlClass/dev/class02/overview/) |
 | 3  | 09/05/2025   | Lecture - Zaowei Dai | Pontryagin’s Maximum Principle; shooting & multiple shooting; LQR, Riccati, QP viewpoint (finite / infinite horizon) | |
 | 4  | 09/12/2025   | **External seminar 1** - Joaquim Dias Garcia| Dynamic Programming & Model-Predictive Control | |
 | 5  | 09/19/2025   | Lecture - Guancheng "Ivan" Qiu | **Nonlinear** trajectory **optimization**; collocation; implicit integration | |
 | 6  | 09/26/2025   | **External seminar 2** - Henrique Ferrolho | Trajectory **optimization** on robots in Julia Robotics | |
 | 7  | 10/03/2025   | Lecture - Jouke van Westrenen | Stochastic optimal control, Linear Quadratic Gaussian (LQG), Kalman filtering, robust control under uncertainty, unscented optimal control; | |
-| 8  | 10/10/2025   | **External seminar 3** TBD (speaker to be confirmed) | Topology **optimization** | |
+| 8  | 10/10/2025   | Lecture - Kevin Wu | Distributed optimal control & multi-agent coordination; Consensus, distributed MPC, and optimization over graphs (ADMM) ||
 | 9  | 10/17/2025   | **External seminar 4** — François Pacaud | GPU-accelerated optimal control | |
 |10  | 10/24/2025   | Lecture - Michael Klamkin | Physics-Informed Neural Networks (PINNs): formulation & pitfalls | |
 |11  | 10/31/2025   | **External seminar 5** - Chris Rackauckas | Neural Differential Equations: PINNs + classical solvers | |
@@ -57,7 +73,7 @@ Students must provide materials equivalent to those used in an in-person session
 | 18 | Lecture - Joe Ye | Robust control & min-max DDP (incl. PDE cases); chance constraints; Data-driven control & Model-Based RL-in-the-loop | |
 | 19 | Lecture - TBD | Contact Explict and Contact Implicit; Trajectory Optimization for Hybrid and Composed Systems;  | |
 | 20 | Lecture - TBD | Probabilistic Programming; Bayesian numerical methods; Variational Inference; probabilistic solvers for ODEs/PDEs; Bayesian optimization in control;  | |
-| 21 | Lecture - TBD | Distributed optimal control & multi-agent coordination; Consensus, distributed MPC, and optimization over graphs (ADMM). | |
+| 21 | Lecture - Kevin Wu | Distributed optimal control & multi-agent coordination; Consensus, distributed MPC, and optimization over graphs (ADMM). | |
 | 22 | Lecture - Shuaicheng (Allen) Tong | Dynamic Optimal Control of Power Systems; Generators swing equations, Transmission lines electromagnetic transients, dynamic load models, and inverters. | |
 
 ## Reference Material

class01/background_materials/math_basics.html

@@ -39,6 +39,8 @@ md"
 |  Lecturer   | : | Rosemberg, Andrew |
 |  Date   | : | 28 of July, 2025 |
 
+Special thanks to **Guancheng Qiu** for helping fix some of the code!
+
 # Background Math (_Welcome to Pluto!_)
 
 This background material will use Pluto!

class01/background_materials/math_basics.jl

@@ -37,6 +37,8 @@ md"""
 |  Lecturer   | : | Rosemberg, Andrew |
 |  Date   | : | 28 of July, 2025 |
 
+Special thanks to **Guancheng Qiu** for helping fix some of the code!
+
 """
 
 # ╔═╡ eeceb82e-abfb-4502-bcfb-6c9f76a0879d
@@ -436,21 +438,32 @@ begin
  [7  1  3  9  2  4  8  5  6];
  [9  6  1  5  3  7  2  8  4];
  [2  8  7  4  1  9  6  3  5];
- [3  4  5  2  8  6  1  7  9]])
-
-	anss = missing
-    try
-        anss = (
-            x_ss   = haskey(sudoku, :x_s) ? JuMP.value.(sudoku[:x_s]) : missing,
-        )
-    catch
-        anss = missing
+ [3  4  5  2  8  6  1  7  9]],)
+
+    anss = (;
+        x_ss = haskey(sudoku, :x_s) && JuMP.is_solved_and_feasible(sudoku) ? JuMP.value.(sudoku[:x_s]) : missing
+    )
+
+    # Convert 3D binary matrix to 2D solution matrix
+    function convert_3d_to_solution(x_3d)
+        if ismissing(x_3d)
+            return missing
+        end
+        solution = zeros(Int, 9, 9)
+        for i in 1:9, j in 1:9, k in 1:9
+            if x_3d[i, j, k] ≈ 1.0
+                solution[i, j] = k
+            end
+        end
+        return solution
     end
 
-    goods = !ismissing(anss) &&
-           all(isapprox.(anss.x_ss, ground_truth_s.x_ss; atol=1e-3))
+    solution_matrix = ismissing(anss) ? missing : convert_3d_to_solution(anss.x_ss)
+
+    goods = !ismissing(anss) && !ismissing(solution_matrix) &&
+           all(isapprox.(solution_matrix, ground_truth_s.x_ss; atol=1e-3))
 
-    if ismissing(anss)
+    if ismissing(anss.x_ss)
         still_missing()
     elseif goods
         correct()
@@ -578,8 +591,8 @@ begin
 model_nlp = Model(Ipopt.Optimizer)
 
 # Required named variables
-@variable(model_nlp, x)
-@variable(model_nlp, y)
+@variable(model_nlp, x_nlp)
+@variable(model_nlp, y_nlp)
 
 # --- YOUR CODE HERE ---
 
@@ -707,7 +720,7 @@ begin
     # Decide which badge to show
     if ismissing(ansd)               # nothing yet
         still_missing()
-    elseif x == 25.0
+    elseif ansd == 25.0
         correct()
     else
         keep_working()
@@ -721,8 +734,8 @@ begin
 	ans3=missing
     try
         ans3 = (
-            x   = safeval(model_nlp, :x),
-            y   = safeval(model_nlp, :y),
+            x   = safeval(model_nlp, :x_nlp),
+            y   = safeval(model_nlp, :y_nlp),
             obj = objective_value(model_nlp),
         )
     catch
@@ -799,7 +812,7 @@ end
 # ╟─bca712e4-3f1c-467e-9209-e535aed5ab0a
 # ╟─3997d993-0a31-435e-86cd-50242746c305
 # ╠═3f56ec63-1fa6-403c-8d2a-1990382b97ae
-# ╟─0e8ed625-df85-4bd2-8b16-b475a72df566
+# ╠═0e8ed625-df85-4bd2-8b16-b475a72df566
 # ╟─fa5785a1-7274-4524-9e54-895d46e83861
 # ╟─5e3444d0-8333-4f51-9146-d3d9625fe2e9
 # ╠═0e190de3-da60-41e9-9da5-5a0c7fefd1d7

class01/background_materials/optimization_basics.html

@@ -0,0 +1,7 @@
+# This file is machine-generated - editing it directly is not advised
+
+julia_version = "1.11.6"
+manifest_format = "2.0"
+project_hash = "da39a3ee5e6b4b0d3255bfef95601890afd80709"
+
+[deps]

class01/background_materials/optimization_basics.jl

@@ -0,0 +1 @@
+[deps]

class02/ISYE_8803___Lecture_2___Slides.pdf

@@ -0,0 +1,123 @@
+\section{Sequential Quadratic Programming (SQP)}
+
+% ------------------------------------------------
+\begin{frame}{What is SQP?}
+\textbf{Idea:} Solve a nonlinear, constrained problem by repeatedly solving a \emph{quadratic program (QP)} built from local models.\\[4pt]
+\begin{itemize}
+  \item Linearize constraints; quadratic model of the Lagrangian/objective.
+  \item Each iteration: solve a QP to get a step \(d\), update \(x \leftarrow x + \alpha d\).
+  \item Strength: strong local convergence (often superlinear) with good Hessian info.
+\end{itemize}
+\end{frame}
+
+% ------------------------------------------------
+\begin{frame}{Target Problem (NLP)}
+\[
+\min_{x \in \R^n} \ f(x)
+\quad
+\text{s.t.}\quad
+g(x)=0,\quad h(x)\le 0
+\]
+\begin{itemize}
+  \item \(f:\R^n\!\to\!\R\), \(g:\R^n\!\to\!\R^{m}\) (equalities), \(h:\R^n\!\to\!\R^{p}\) (inequalities).
+  \item KKT recap (at candidate optimum \(x^\star\)): 
+\[
+\exists \ \lambda \in \R^{m},\ \mu \in \R^{p}_{\ge 0}:
+\ \grad f(x^\star) + \nabla g(x^\star)^T\lambda + \nabla h(x^\star)^T \mu = 0,
+\]
+\[
+g(x^\star)=0,\quad h(x^\star)\le 0,\quad \mu \ge 0,\quad \mu \odot h(x^\star) = 0.
+\]
+\end{itemize}
+\end{frame}
+
+% ------------------------------------------------
+\begin{frame}{From NLP to a QP (Local Model)}
+At iterate \(x_k\) with multipliers \((\lambda_k,\mu_k)\):\\[4pt]
+\textbf{Quadratic model of the Lagrangian}
+\[
+m_k(d) = \ip{\grad f(x_k)}{d} + \tfrac{1}{2} d^T B_k d
+\]
+with \(B_k \approx \nabla^2_{xx}\Lag(x_k,\lambda_k,\mu_k)\).\\[6pt]
+\textbf{Linearized constraints}
+\[
+g(x_k) + \nabla g(x_k)\, d = 0,\qquad
+h(x_k) + \nabla h(x_k)\, d \le 0.
+\]
+\end{frame}
+
+% ------------------------------------------------
+\begin{frame}{The SQP Subproblem (QP)}
+\[
+\begin{aligned}
+\min_{d \in \R^n}\quad & \grad f(x_k)^T d + \tfrac{1}{2} d^T B_k d \\
+\text{s.t.}\quad & \nabla g(x_k)\, d + g(x_k) = 0, \\
+& \nabla h(x_k)\, d + h(x_k) \le 0.
+\end{aligned}
+\]
+\begin{itemize}
+  \item Solve QP \(\Rightarrow\) step \(d_k\) and updated multipliers \((\lambda_{k+1},\mu_{k+1})\).
+  \item Update \(x_{k+1} = x_k + \alpha_k d_k\) (line search or trust-region).
+\end{itemize}
+\end{frame}
+
+% ------------------------------------------------
+\begin{frame}{Algorithm Sketch (SQP)}
+\begin{enumerate}
+  \item Start with \(x_0\), multipliers \((\lambda_0,\mu_0)\), and \(B_0 \succ 0\).
+  \item Build QP at \(x_k\) with \(B_k\), linearized constraints.
+  \item Solve QP \(\Rightarrow\) get \(d_k\), \((\lambda_{k+1},\mu_{k+1})\).
+  \item Globalize: line search on merit or use filter/TR to choose \(\alpha_k\).
+  \item Update \(x_{k+1} = x_k + \alpha_k d_k\), update \(B_{k+1}\) (e.g., BFGS).
+\end{enumerate}
+\end{frame}
+
+% ------------------------------------------------
+\begin{frame}{Toy Example (Local Models)}
+\textbf{Problem:}
+\[
+\min_{x\in\R^2} \ \tfrac{1}{2}\norm{x}^2
+\quad \text{s.t.} \quad g(x)=x_1^2 + x_2 - 1 = 0,\ \ h(x)=x_2 - 0.2 \le 0.
+\]
+At \(x_k\), build QP with
+\[
+\grad f(x_k)=x_k,\quad B_k=I,\quad
+\nabla g(x_k) = \begin{bmatrix} 2x_{k,1} & 1 \end{bmatrix},\ 
+\nabla h(x_k) = \begin{bmatrix} 0 & 1 \end{bmatrix}.
+\]
+Solve for \(d_k\), then \(x_{k+1}=x_k+\alpha_k d_k\).
+\end{frame}
+ 
+
+% ------------------------------------------------
+\begin{frame}{Globalization: Making SQP Robust}
+SQP is an important method, and there are many issues to be considered to obtain an \textbf{efficient} and \textbf{reliable} implementation:
+\begin{itemize}
+  \item Efficient solution of the linear systems at each Newton Iteration (Matrix block structure can be exploited.
+  \item Quasi-Newton approximations to the Hessian.
+  \item Trust region, line search, etc. to improve robustnes (i.e TR: restrict \(\norm{d}\) to maintain model validity.)
+  \item Treatment of constraints (equality and inequality) during the iterative process.
+  \item Selection of good starting guess for $\lambda$.
+\end{itemize}
+\end{frame}
+
+ 
+   
+ 
+ 
+
+% ------------------------------------------------
+\begin{frame}{Final Takeaways on SQP}
+\textbf{When SQP vs.\ Interior-Point?}
+\begin{itemize}
+  \item \textbf{SQP}: strong local convergence; warm-start friendly; natural for NMPC.
+  \item \textbf{IPM}: very robust for large, strictly feasible problems; good for dense inequality sets.
+  \item In practice: both are valuable—choose to match problem structure and runtime needs.
+\end{itemize} 
+\textbf{Takeaways of SQP} 
+\begin{itemize}
+  \item SQP = Newton-like method using a sequence of structured QPs.
+  \item Globalization (merit/filter/TR) makes it reliable from poor starts.
+  \item Excellent fit for control (NMPC/trajectory optimization) due to sparsity and warm starts.
+\end{itemize}
+\end{frame}

class02/Manifest.toml

@@ -6,5 +6,50 @@
 
 ---
 
-Add notes, links, and resources below.
+## Overview
+
+This class covers the fundamental numerical optimization techniques essential for optimal control problems. We explore gradient-based methods, Sequential Quadratic Programming (SQP), and various approaches to handling constraints including Augmented Lagrangian Methods (ALM), interior-point methods, and penalty methods.
+
+## Interactive Materials
+
+The class is structured around 1 slide deck and four interactive Jupyter notebooks:
+
+1. **[Part 1a: Root Finding & Backward Euler](https://learningtooptimize.github.io/LearningToControlClass/dev/class02/part1_root_finding.html)**
+   - Root-finding algorithms for implicit integration
+   - Fixed-point iteration vs. Newton's method
+   - Application to pendulum dynamics
+
+
+2. **[Part 1b: Minimization via Newton's Method](https://learningtooptimize.github.io/LearningToControlClass/dev/class02/part1_minimization.html)**
+   - Unconstrained optimization fundamentals
+   - Newton's method implementation
+   - Globalization strategies: Hessian matrix and regularization
+
+3. **[Part 2: Equality Constraints](https://learningtooptimize.github.io/LearningToControlClass/dev/class02/part2_eq_constraints.html)**
+   - Lagrange multiplier theory
+   - KKT conditions for equality constraints
+   - Quadratic programming implementation
+
+4. **[Part 3: Interior-Point Methods](https://learningtooptimize.github.io/LearningToControlClass/dev/class02/part3_ipm.html)**
+   - Inequality constraint handling
+   - Barrier methods and log-barrier functions
+   - Comparison with penalty methods
+
+## Additional Resources
+
+- **[Lecture Slides (PDF)](https://learningtooptimize.github.io/LearningToControlClass/dev/class02/ISYE_8803___Lecture_2___Slides.pdf)** - Complete slide deck
+- **[LaTeX Source](https://learningtooptimize.github.io/LearningToControlClass/dev/class02/main.tex)** - Source code for lecture slides
+
+## Key Learning Outcomes
+
+- Understand gradient-based optimization methods
+- Implement Newton's method for minimization
+- Apply root-finding techniques for implicit integration
+- Solve equality-constrained optimization problems
+- Compare different constraint handling methods
+- Implement Sequential Quadratic Programming (SQP)
+
+## Next Steps
+
+This class provides the foundation for advanced topics in subsequent classes, including Pontryagin's Maximum Principle, nonlinear trajectory optimization, and stochastic optimal control.