Lagrange 乘子法

我们来推导一下 Lagrange 乘子法与含约束问题的等价性. 以关于双变量的函数为例,: \[ Q(y_1, y_2): \mathbb{R^2} \mapsto \mathbb{R} \]

现在希望获取该函数的最小值, 同时满足约束 \[ A(y_1, y_2) = 0 \] 最直接的思路是, 通过约束方程 \(A\) , 用 \(y_1\) 表示 \(y_2\), 这样, 代入到目标函数 \(Q\) 中, 就得到关于 \(y_1\) 的单变量方程 \[ R(y_1) \triangleq Q(y_1, y_2(y_1)) \] 这样之后, 极值问题就等价于对函数 \(R\) 的导数为 0: \[ \frac{\mathrm{d} R}{\,\mathrm{d} y_1} = 0 \iff \frac{\partial Q}{\,\partial y_1} + \frac{\partial Q}{\,\partial y_2} \frac{\mathrm{d}y_1}{\mathrm{d} y_2} = 0 \] 同样的, 对约束方程关于 \(y_1\) 求导数得到 \[ \frac{\partial A}{\,\partial y_1} + \frac{\partial A}{\,\partial y_2} \frac{\mathrm{d}y_1}{\mathrm{d} y_2} = 0 \] 两个方程中都出现了 \(\mathrm{d} y_1 / \mathrm{d} y_2\) 项, 因此 \[ \left. \frac{\partial Q}{\,\partial y_1} \right/ \frac{\partial Q}{\,\partial y_2} = \left. \frac{\partial A}{\,\partial y_1} \right/ \frac{\partial A}{\,\partial y_2} \] 或者 \[ \left. \frac{\partial Q}{\,\partial y_1} \right/ \frac{\partial A}{\,\partial y_1} = \left. \frac{\partial Q}{\,\partial y_2} \right/ \frac{\partial A}{\,\partial y_2} \] 如果引入新的变量 \(x\), 表示偏导数之间的比值 \[ \left. \frac{\partial Q}{\,\partial y_1} \right/ \frac{\partial A}{\,\partial y_1} = \left. \frac{\partial Q}{\,\partial y_2} \right/ \frac{\partial A}{\,\partial y_2} = -x \] 就得到 \[ \begin{aligned} \frac{\partial Q}{\,\partial y_1} + x \frac{\partial A}{\,\partial y_1} = 0 \\ \frac{\partial Q}{\,\partial y_2} + x \frac{\partial A}{\,\partial y_2} = 0 \end{aligned} \] 使用 Lagrange 乘子组成的目标函数为 \[ L(y_1,y_2,x) = Q(y_1, y_2) + x A(y_1, y_2) \] 对 \((y_1, y_2, x)\) 分别求偏导数, 并令其等于 0, 得到 \[ \begin{aligned} \frac{\partial L}{\,\partial y_1} = 0 &\iff \frac{\partial Q}{\,\partial y_1} + x \frac{\partial A}{\,\partial y_1} = 0 \\ \frac{\partial L}{\,\partial y_2} = 0 &\iff \frac{\partial Q}{\,\partial y_2} + x \frac{\partial A}{\,\partial y_2} = 0 \\ \frac{\partial L}{\partial x} = 0 &\iff A = 0 \\ \end{aligned} \]