【矩阵论】2. 矩阵分解——SVD

矩阵分解可以得到简化的乘积矩阵，可以简化后续的计算与处理度

2.1.2 奇异值分解SVD

a. 正SVD

$$\begin{array}{rl}& 设A=A_{m\times n}，r(A)>0,正奇值\sqrt{{\lambda }_1},\cdots ,\sqrt{{\lambda }_r},则有分解A=P\Delta Q^H\\ & 其中\Delta =\left(\begin{array}{cccc}& \sqrt{{\lambda }_1}& & \\ & & \ddots & \\ & & & \sqrt{{\lambda }_r}\end{array}\right)，P_{m\times r}，Q_{n\times r}为半U阵,P^{H}P=I_r=Q^{H}Q\\ & 可写正SVD公式A=P\left(\begin{array}{cccc}& \sqrt{{\lambda }_1}& & \\ & & \ddots & \\ & & & \sqrt{{\lambda }_r}\end{array}\right)Q^H\end{array}$$

证明

1. $P_{m\times r}，Q_{n\times r}$ 的构造

$$\begin{array}{rl}& A^{H}A为Hermite阵，由Hermite分解定理，存在U阵\\ & 使U^H(A^{H}A)U={\left(\begin{array}{cccc}& {\lambda }_1& & \\ & & \ddots & \\ & & & {\lambda }_n\end{array}\right)}_{n\times n}，\\ & ,且A^{H}A为半正定阵，有{\lambda }_1,\cdots ,{\lambda }_r>0,{\lambda }_{r+1}=\cdots ={\lambda }_n=0,r(A)=r\\ & 而U的列向量(q_1,q_2,\cdots ,q_n)为A^{H}A的特征向量\\ & 且A^{H}A{q}_1={\lambda }_1{q}_1,\cdots ,A^{H}A{q}_r={\lambda }_r{q}_r,A^{H}A{q}_{r+1}=0,A^{H}A{q}_n=0\\ & \\ & 令Q=(q_1,q_2,\cdots ,q_r{)}_{n\times r}，P=(\frac{A{q}_1}{|A{q}_1|},\cdots ,\frac{A{q}_r}{|A{q}_r|}{)}_{m\times r}\end{array}$$

2. P与Q为半U阵
   $$\begin{array}{rl}& 已知Q中列向量为U阵的r个非零列向量，则Q为半U阵\\ & 而(A{q}_1,A{q}_2)=(A{q}_2{)}^H(A{q}_1)=q_2{A}^{H}A{q}_1={\lambda }_1{q}_2^H{q}_1=({\lambda }_1{q}_1,q_2)=0\\ & 同理，A{q}_i与A{q}_j都正交\\ & |A{q}_1{|}^2=(A{q}_1{)}^H(A{q}_1)=q_1^H{A}^{H}A{q}_1={\lambda }_1{q}_1^H{q}_1={\lambda }_1\ge 0\\ & \Rightarrow P=\left(\frac{A{q}_1}{|A{q}_1|},\frac{A{q}_2}{|A{q}_2|},\cdots ,\frac{A{q}_r}{|A{q}_r|}\right)=\left(\frac{A{q}_1}{\sqrt{{\lambda }_1}},\frac{A{q}_2}{\sqrt{{\lambda }_2}},\cdots ,\frac{A{q}_r}{\sqrt{{\lambda }_r}}\right)\\ & 即P阵为半U阵\end{array}$$

3. 代入P,Q
   $$\begin{array}{rl}& P\Delta Q^H=\left(\frac{A{q}_1}{\sqrt{{\lambda }_1}},\frac{A{q}_2}{\sqrt{{\lambda }_2}},\cdots ,\frac{A{q}_r}{\sqrt{{\lambda }_r}}\right)\left(\begin{array}{cccc}& \sqrt{{\lambda }_1}& & \\ & & \ddots & \\ & & & \sqrt{{\lambda }_r}\end{array}\right)\left(\begin{array}{c}{q}_1^H\\ q_2^H\\ ⋮\\ q_r^H\end{array}\right)\\ & =(A{q}_1,A{q}_2,\cdots ,A{q}_r)\left(\begin{array}{c}{q}_1^H\\ q_2^H\\ ⋮\\ q_r^H\end{array}\right)=\left(A{q}_1{q}_1^H,A{q}_2{q}_2^H,，\cdots ,A{q}_r{q}_r^H,\right)\end{array}$$

4. 验证 $\left(A{q}_1{q}_1^H,A{q}_2{q}_2^H,，\cdots ,A{q}_r{q}_r^H,\right)=A$
   $$\begin{array}{rl}& 由A^{H}A为半正定Hermite阵，r(A^{H}A)=r\\ & 由同解定理，A^{H}Ax=0⟺Ax=0\\ & A^{H}A{q}_{r+1}=\cdots =A^{H}A{q}_n=0\Rightarrow A{q}_{r+1}=\cdots =A{q}_n=0\\ & \Rightarrow A(q_{r+1}{q}_{r+1}^H+\cdots +q_n{q}_n^H)=0\\ & 而A(q_1{q}_1^H+\cdots +q_r{q}_r^H+q_{r+1}{q}_{r+1}^H+\cdots +q_n{q}_n^H)=AI=A\\ & 即A(q_1{q}_1^H+\cdots +q_r{q}_r^H)=A\end{array}$$
   最后得证 正SVD公式，$A=P\Delta Q^H=P\left(\begin{array}{cccc}& \sqrt{{\lambda }_1}& & \\ & & \ddots & \\ & & & \sqrt{{\lambda }_r}\end{array}\right)Q^H$

分解方法

1. 求 $A^{H}A$ 的特征值，${\lambda }_1\ge \cdots ,\ge {\lambda }_r>0$

   正奇值为 $\sqrt{{\lambda }_1},\cdots \sqrt{{\lambda }_r}$

2. 求 ${\lambda }_1,\cdots ,{\lambda }_r$ 的正交特征向量（不必单位化）

3. 令列半U阵 $Q=\left(\frac{X_1}{|X_1|},\cdots ,\frac{X_r}{|X_r|}\right),P=\left(\frac{A{X}_1}{|A{X}_1|},\cdots ,\frac{A{X}_r}{|A{X}_r|}\right)$

eg

$$\begin{array}{r}A=\left(\begin{array}{cc}1& 1\\ 0& 0\\ 1& 1\end{array}\right)\end{array}$$

A H A = ( 2 2 2 2 ) , 由 A H A 为 秩 1 矩 阵 ， 得 λ = { 4 , 0 } ， 正 奇 值 为 ( λ ) = 2 而 λ 1 = 4 的 特 征 向 量 为 X 1 = ( 1 1 ) , 则 Δ = ( 2 ) 令 Q = ( X 1 ∣ X 1 ∣ ) = ( 1 2 1 2 ) , P = ( A X 1 ∣ A X 1 ∣ ) = ( 1 2 0 1 2 ) 则 A 的 正 S V D 为 A = P Δ Q H = ( 1 2 0 1 2 ) ( 2 ) ( 1 2 , 1 2 ) \begin{aligned} &A^HA=\left( \begin{matrix} 2&2\\ 2&2 \end{matrix} \right),由A^HA为秩1矩阵，得\lambda=\{4,0\}，正奇值为\sqrt(\lambda)=\sqrt{2}\\ &而\lambda_1=4的特征向量为X_1=\left( \begin{matrix} 1\\1 \end{matrix} \right),则\Delta=\left( 2 \right)\\ &令Q=\left(\frac{X_1}{\vert X_1\vert}\right)=\left( \begin{matrix} \frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}} \end{matrix} \right),P=\left( \begin{matrix} \frac{AX_1}{\vert AX_1\vert} \end{matrix} \right)=\left( \begin{matrix} \frac{1}{\sqrt{2}}\\0\\\frac{1}{\sqrt{2}} \end{matrix} \right)\\ &则A的正SVD为A=P\Delta Q^H=\left( \begin{matrix} \frac{1}{\sqrt{2}}\\0\\\frac{1}{\sqrt{2}} \end{matrix} \right)\left(2\right)\left( \frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}} \right) \end{aligned} ​AHA=(22​22​),由AHA为秩1矩阵，得λ={4,0}，正奇值为( ​λ)=2 ​而λ1​=4的特征向量为X1​=(11​),则Δ=(2)令Q=(∣X1​∣X1​​)=(2 ​1​2 ​1​​),P=(∣AX1​∣AX1​​​)=⎝⎛​2 ​1​02 ​1​​⎠⎞​则A的正SVD为A=PΔQH=⎝⎛​2 ​1​02 ​1​​⎠⎞​(2)(2 ​1​,2 ​1​)​

考虑 $B=\left(\begin{array}{ccc}1& 0& 1\\ 1& 0& 1\end{array}\right)$ 的正SVD

$$\begin{array}{rl}& 已知B=A^H,且有A的正SVD，A=P\Delta Q^H=\left(\begin{array}{c}\frac{1}{\sqrt{2}}\\ 0\\ \frac{1}{\sqrt{2}}\end{array}\right)\left(2\right)\left(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}}\right)\\ & B=(P\Delta Q^H{)}^H=Q\Delta P^H=\left(\begin{array}{c}\frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}}\end{array}\right)\left(2\right)\left(\frac{1}{\sqrt{2}},0,\frac{1}{\sqrt{2}}\right)\end{array}$$

b. SVD(将P，Q，D都扩充为方阵)

$$\begin{array}{rl}& 设A=A_{m\times n}，r(A)>0,正奇值\sqrt{{\lambda }_1},\cdots ,\sqrt{{\lambda }_r}>0,则有分解A=W\Delta V^H\\ & 其中D={\left(\begin{array}{cc}\Delta & 0\\ 0& 0\end{array}\right)}_{m\times n}，W_{m\times m}=(P,P_1),V_{n\times n}=(Q,Q_1)\\ & 可写SVD公式A=W\left(\begin{array}{cc}\Delta & 0\\ 0& 0\end{array}\right)V^H=P\Delta Q^H,其中\left(\begin{array}{ccc}\sqrt{{\lambda }_1}& & \\ & \ddots & \\ & & \sqrt{{\lambda }_r}\end{array}\right)\end{array}$$

证明:

$$\begin{array}{rl}& 由正SVD：A=P\Delta Q^H,分别把P，Q扩充为U阵，即W=(P,P_1),V=(Q,Q_1)，\\ & V^H=\left(\begin{array}{c}{Q}^H\\ A_1^H\end{array}\right),则WD{V}^H=(P,P_1)\left(\begin{array}{cc}\Delta & 0\\ 0& 0\end{array}\right)\left(\begin{array}{c}{Q}^H\\ A_1^H\end{array}\right)=\left(\begin{array}{c}P\Delta 0\end{array}\right)\left(\begin{array}{c}{Q}^H\\ A_1^H\end{array}\right)=P\Delta Q^H\end{array}$$

分解方法

1. 求 $A^{H}A$ 的特征值，${\lambda }_1\ge \cdots ,\ge {\lambda }_r>0$

   正奇值为 $\sqrt{{\lambda }_1},\cdots \sqrt{{\lambda }_r}$

2. 求 ${\lambda }_1,\cdots ,{\lambda }_r$ 的正交特征向量（不必单位化）

3. 令列U半阵 $Q=\left(\frac{X_1}{|X_1|},\cdots ,\frac{X_r}{|X_r|}\right),P=\left(\frac{A{X}_1}{|A{X}_1|},\cdots ,\frac{A{X}_r}{|A{X}_r|}\right)$ ，则有正SVD：$A=P\Delta Q^H$

4. 将P，Q扩充为W，V 扩充方法不唯一 由证明可知，不管 $P_1,Q_1$ 为何，都会被消去

   得SVD公式 $A=WD{V}^H$
   

eg

$$\begin{array}{r}A=\left(\begin{array}{cc}1& 1\\ 0& 0\\ 1& 1\end{array}\right)\end{array}$$

$$\begin{array}{rl}& 扩充为两个U阵V=\left(Q,X\right)=\left(\begin{array}{cc}\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}}& -\frac{1}{\sqrt{2}}\end{array}\right),W=\left(P,Y\right)=\left(\begin{array}{ccc}\frac{1}{\sqrt{2}}& 0& \frac{1}{\sqrt{2}}\\ 0& 0& 0\\ \frac{1}{\sqrt{2}}& 0& -\frac{1}{\sqrt{2}}\end{array}\right)\\ & 则有奇异分解SVD，A=\left(\begin{array}{ccc}\frac{1}{\sqrt{2}}& 0& \frac{1}{\sqrt{2}}\\ 0& 0& 0\\ \frac{1}{\sqrt{2}}& 0& -\frac{1}{\sqrt{2}}\end{array}\right)\left(\begin{array}{cc}2& 0\\ 0& 0\\ 0& 0\end{array}\right)\left(\begin{array}{cc}\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}}& -\frac{1}{\sqrt{2}}\end{array}\right)\end{array}$$

$$\begin{array}{r}A=\left(\begin{array}{cc}2i& 1\\ 1& i\\ 1& i\end{array}\right)\end{array}$$
A H A = ( − 2 i 1 1 1 − i − i ) ( 2 i 1 1 i 1 i ) = ( 6 0 0 3 ) ， 可 知 正 奇 值 S ( A ) + = { 6 , 3 } λ 1 = 6 , X 1 = ( 1 0 ) , A X 1 = ( 2 i 1 1 ) ∣ A X 1 ∣ = 6 ; λ 2 = 3 , X 2 = ( 0 1 ) , A X 2 = ( 1 i i ) 令 Q = ( X 1 , X 2 ) = ( 1 0 0 1 ) , P = ( A X 1 ∣ A X 1 ∣ , A X 2 ∣ A X 2 ∣ ) = ( 2 i 6 1 3 1 6 i 3 1 6 i 3 ) 则 有 正 S V D ， A = P Δ Q H = ( 2 i 6 1 3 1 6 i 3 1 6 i 3 ) ( 6 3 ) ( 1 0 0 1 ) H 可 知 S V D ， A = W D V = ( 2 i 6 1 3 0 1 6 i 3 1 2 1 6 i 3 − 1 2 ) ( 6 3 ) ( 1 0 0 1 ) H \begin{aligned} &A^HA=\left( \begin{matrix} -2i&1&1\\ 1&-i&-i \end{matrix} \right)\left( \begin{matrix} 2i&1\\ 1&i\\ 1&i \end{matrix} \right)=\left( \begin{matrix} 6&0\\ 0&3 \end{matrix} \right)，可知正奇值S(A)_+=\{\sqrt{6},\sqrt{3}\}\\ &\lambda_1=6,X_1=\left( \begin{matrix} 1\\0 \end{matrix} \right),AX_1=\left( \begin{matrix} 2i\\1\\1 \end{matrix} \right)\vert AX_1\vert=\sqrt{6};\lambda_2=3,X_2=\left( \begin{matrix} 0\\1 \end{matrix} \right),AX_2=\left( \begin{matrix} 1\\i\\i \end{matrix} \right)\\ &令Q=\left( \begin{matrix} X_1,X_2 \end{matrix} \right)=\left( \begin{matrix} 1&0\\0&1 \end{matrix} \right),P=\left( \begin{matrix} \frac{AX_1}{\vert AX_1\vert},\frac{AX_2}{\vert AX_2\vert} \end{matrix} \right)=\left( \begin{matrix} \frac{2i}{\sqrt{6}}&\frac{1}{\sqrt{3}}\\ \frac{1}{\sqrt{6}}&\frac{i}{\sqrt{3}}\\ \frac{1}{\sqrt{6}}&\frac{i}{\sqrt{3}} \end{matrix} \right)\\ &则有正SVD，A=P\Delta Q^H=\left( \begin{matrix} \frac{2i}{\sqrt{6}}&\frac{1}{\sqrt{3}}\\ \frac{1}{\sqrt{6}}&\frac{i}{\sqrt{3}}\\ \frac{1}{\sqrt{6}}&\frac{i}{\sqrt{3}} \end{matrix} \right)\left( \begin{matrix} \sqrt{6}&\\ &\sqrt{3} \end{matrix} \right)\left( \begin{matrix} 1&0\\0&1 \end{matrix} \right)^H\\ &可知SVD，A=WDV=\left( \begin{matrix} \frac{2i}{\sqrt{6}}&\frac{1}{\sqrt{3}}&0\\ \frac{1}{\sqrt{6}}&\frac{i}{\sqrt{3}}&\frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{6}}&\frac{i}{\sqrt{3}}&-\frac{1}{\sqrt{2}} \end{matrix} \right)\left( \begin{matrix} \sqrt{6}&\\ &\sqrt{3} \end{matrix} \right)\left( \begin{matrix} 1&0\\0&1 \end{matrix} \right)^H \end{aligned} ​AHA=(−2i1​1−i​1−i​)⎝⎛​2i11​1ii​⎠⎞​=(60​03​)，可知正奇值S(A)+​={6 ​,3 ​}λ1​=6,X1​=(10​),AX1​=⎝⎛​2i11​⎠⎞​∣AX1​∣=6 ​;λ2​=3,X2​=(01​),AX2​=⎝⎛​1ii​⎠⎞​令Q=(X1​,X2​​)=(10​01​),P=(∣AX1​∣AX1​​,∣AX2​∣AX2​​​)=⎝⎜⎛​6 ​2i​6 ​1​6 ​1​​3 ​1​3 ​i​3 ​i​​⎠⎟⎞​则有正SVD，A=PΔQH=⎝⎜⎛​6 ​2i​6 ​1​6 ​1​​3 ​1​3 ​i​3 ​i​​⎠⎟⎞​(6 ​​3 ​​)(10​01​)H可知SVD，A=WDV=⎝⎜⎛​6 ​2i​6 ​1​6 ​1​​3 ​1​3 ​i​3 ​i​​02 ​1​−2 ​1​​⎠⎟⎞​(6 ​​3 ​​)(10​01​)H​

一个SVD解答
$$\begin{array}{r}A=\left(\begin{array}{cc}i& 2\\ 1& i\\ 1& i\end{array}\right)，求正SVD与SVD\end{array}$$

$$\begin{array}{rl}& \because A^{H}A=\left(\begin{array}{ccc}-i& 1& 1\\ 2& -i& -i\end{array}\right)\left(\begin{array}{cc}i& 2\\ 1& i\\ 1& i\end{array}\right)=\left(\begin{array}{cc}3& 0\\ 0& 6\end{array}\right)为对角阵，令{\lambda }_1=3,{\lambda }_2=6\\ & A^H有两个特向X_1=\left(\begin{array}{c}1\\ 0\end{array}\right),X_2=\left(\begin{array}{c}0\\ 1\end{array}\right)(互正交)，正奇值为\sqrt{{\lambda }_1}=\sqrt{3},\sqrt{{\lambda }_2}=\sqrt{6}\\ & A{X}_1=\left(\begin{array}{c}i\\ 1\\ 1\end{array}\right),A{X}_2=\left(\begin{array}{c}2\\ i\\ i\end{array}\right),|A{X}_1|=\sqrt{3},|A{X}_2|=\sqrt{6}\\ & 令P=\left(\frac{A{X}_1}{|A{X}_1|},\frac{A{X}_2}{|A{X}_2|}\right)=\left(\begin{array}{cc}\frac{i}{\sqrt{3}}& \frac{2}{\sqrt{6}}\\ \frac{1}{\sqrt{3}}& \frac{i}{\sqrt{6}}\\ \frac{i}{\sqrt{3}}& \frac{i}{\sqrt{6}}\end{array}\right),则正SVD\\ & A=P\Delta Q^H=\left(\begin{array}{cc}\frac{i}{\sqrt{3}}& \frac{2}{\sqrt{6}}\\ \frac{1}{\sqrt{3}}& \frac{i}{\sqrt{6}}\\ \frac{i}{\sqrt{3}}& \frac{i}{\sqrt{6}}\end{array}\right)\left(\begin{array}{cc}\sqrt{3}& \\ & \sqrt{6}\end{array}\right)\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)\end{array}$$

$$\begin{array}{rl}& 令W=(P,P_1)=\left(\begin{array}{ccc}\frac{i}{\sqrt{3}}& \frac{2}{\sqrt{6}}& 0\\ \frac{1}{\sqrt{3}}& \frac{i}{\sqrt{6}}& \frac{1}{\sqrt{2}}\\ \frac{i}{\sqrt{3}}& \frac{i}{\sqrt{6}}& -\frac{1}{\sqrt{2}}\end{array}\right),或\left(\begin{array}{ccc}\frac{i}{\sqrt{3}}& \frac{2}{\sqrt{6}}& 0\\ \frac{1}{\sqrt{3}}& \frac{i}{\sqrt{6}}& \frac{i}{\sqrt{2}}\\ \frac{i}{\sqrt{3}}& \frac{i}{\sqrt{6}}& -\frac{i}{\sqrt{2}}\end{array}\right),\\ & 有SVD，A=\left(\begin{array}{ccc}\frac{i}{\sqrt{3}}& \frac{2}{\sqrt{6}}& 0\\ \frac{1}{\sqrt{3}}& \frac{i}{\sqrt{6}}& \frac{1}{\sqrt{2}}\\ \frac{i}{\sqrt{3}}& \frac{i}{\sqrt{6}}& -\frac{1}{\sqrt{2}}\end{array}\right)\left(\begin{array}{cc}\sqrt{3}& 0\\ 0& \sqrt{6}\\ 0& 0\end{array}\right)\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)\end{array}$$

c. 当A是向量时

$$\begin{array}{rl}& A=\left(\begin{array}{c}{a}_1\\ ⋮\\ a_n\end{array}\right)\ne 0,则其正SVD\end{array}$$

$$\begin{array}{rl}& A^{H}A=\left(\overline{a_1},\cdots ,\overline{a_n}\right)\left(\begin{array}{c}{a}_1\\ ⋮\\ a_n\end{array}\right)=|a_1{|}^2+\cdots +|a_n{|}^2>0\\ & {\lambda }_1=|a_1{|}^2+\cdots +|a_n{|}^2，令\Delta =\left(\sqrt{{\lambda }_1}\right),Q=(1)\\ & X_1=(1)，P=\frac{1}{\sqrt{|a_1{|}^2+|a_2{|}^2+\cdots +|a_n{|}^n}}\left(\begin{array}{c}{a}_1\\ ⋮\\ a_n\end{array}\right),\\ & 正SVD：A=P\Delta Q^H=\\ & \frac{1}{\sqrt{|a_1{|}^2+|a_2{|}^2+\cdots +|a_n{|}^n}}\left(\begin{array}{c}{a}_1\\ ⋮\\ a_n\end{array}\right)\left(\sqrt{|a_1{|}^2+|a_2{|}^2+\cdots +|a_n}\right)\left(1\right)\end{array}$$

d. $A^H$与A的SVD只需求一个

$$若已知正SVD，A=P\Delta Q^H,可得A^H的正SVD=(P\Delta Q^H{)}^H$$

$$\begin{array}{r}A=\left(\begin{array}{ccc}2& 0& 0\\ 3& 0& 0\end{array}\right)与B=A^H的正SVD与SVD\end{array}$$

2.1.3 正SVD的等价写法

$$\begin{array}{rl}& A=P\Delta Q^H,\Delta =\left(\begin{array}{ccc}{s}_1& & \\ & \ddots & \\ & & s_r\end{array}\right),令P=\left(\frac{A{X}_1}{|A{X}_1|},\cdots ,\frac{A{X}_r}{|A{X}_r|}\right)=\left(p_1,\cdots ,p_r\right)\\ & Q=\left(X_1,\cdots ,X_r\right)=\left(q_1,\cdots ,q_r\right),则有正SVD\\ & A=s_1{p}_1{q}_1^H+\cdots +s_r{p}_r{q}_r^H\end{array}$$