content_views" class="htmledit_views">

1. 问题

      在线性回归中࿰c;我们使用直线来拟合样本点࿰c;寻找n维特征向量X和输出结果（或者叫做label）Y之间的线性关系。其中clip_image002" border="0" alt="clip_image002" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202015572875.png" width="41" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;clip_image004" border="0" alt="clip_image004" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202015589353.png" width="33" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />。然而当Y也是多维时࿰c;或者说Y也有多个特征时࿰c;我们希望分析出X和Y的关系。

      当然我们仍然可以使用回归的方法来分析࿰c;做法如下：

      假设clip_image002[1]" border="0" alt="clip_image002[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202015581055.png" width="41" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;clip_image006" border="0" alt="clip_image006" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202015599.png" width="44" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;那么可以建立等式Y=AX如下

      clip_image008" border="0" alt="clip_image008" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202015593630.png" width="222" height="62" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      其中clip_image010" border="0" alt="clip_image010" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202015593106.png" width="55" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;形式和线性回归一样࿰c;需要训练m次得到m个clip_image012" border="0" alt="clip_image012" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016001220.png" width="15" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />。

      这样做的一个缺点是࿰c;Y中的每个特征都与X的所有特征关联࿰c;Y中的特征之间没有什么联系。

      我们想换一种思路来看这个问题࿰c;如果将X和Y都看成整体࿰c;考察这两个整体之间的关系。我们将整体表示成X和Y各自特征间的线性组合࿰c;也就是考察clip_image014" border="0" alt="clip_image014" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016006586.png" width="23" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和clip_image016" border="0" alt="clip_image016" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016018811.png" width="23" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />之间的关系。

      这样的应用其实很多࿰c;举个简单的例子。我们想考察一个人解题能力X（解题速度clip_image018" border="0" alt="clip_image018" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016015289.png" width="13" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;解题正确率clip_image020" border="0" alt="clip_image020" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016021767.png" width="14" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />）与他/她的阅读能力Y（阅读速度clip_image022" border="0" alt="clip_image022" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016031833.png" width="13" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;理解程度clip_image024" border="0" alt="clip_image024" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016031899.png" width="14" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />）之间的关系࿰c;那么形式化为：

      clip_image026" border="0" alt="clip_image026" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016034964.png" width="119" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" /> 和 clip_image028" border="0" alt="clip_image028" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016041617.png" width="116" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      然后使用Pearson相关系数

      clip_image030" border="0" alt="clip_image030" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016048062.png" width="428" height="40" style="border-bottom:0px; border-left:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      来度量u和v的关系࿰c;我们期望寻求一组最优的解a和b࿰c;使得Corr(u, v)最大࿰c;这样得到的a和b就是使得u和v就有最大关联的权重。

      到这里࿰c;基本上介绍了典型相关分析的目的。

2. CCA表示与求解

      给定两组向量clip_image032" border="0" alt="clip_image032" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016054540.png" width="13" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和clip_image034" border="0" alt="clip_image034" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016056765.png" width="13" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />（替换之前的x为clip_image032[1]" border="0" alt="clip_image032[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016069655.png" width="13" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;y为clip_image034[1]" border="0" alt="clip_image034[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016063833.png" width="13" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />）࿰c;clip_image032[2]" border="0" alt="clip_image032[2]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016073899.png" width="13" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />维度为clip_image036" border="0" alt="clip_image036" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016072329.png" width="14" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;clip_image034[2]" border="0" alt="clip_image034[2]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016085393.png" width="13" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />维度为clip_image038" border="0" alt="clip_image038" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016083507.png" width="14" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;默认clip_image040" border="0" alt="clip_image040" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016091622.png" width="46" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />。形式化表示如下：

      clip_image042" border="0" alt="clip_image042" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016097510.png" width="326" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      clip_image044" border="0" alt="clip_image044" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016091688.png" width="8" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />是x的协方差矩阵；左上角是clip_image032[3]" border="0" alt="clip_image032[3]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201610641.png" width="13" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />自己的协方差矩阵；右上角是clip_image046" border="0" alt="clip_image046" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201610118.png" width="65" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />；左下角是clip_image048" border="0" alt="clip_image048" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016106770.png" width="65" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;也是clip_image050" border="0" alt="clip_image050" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016117393.png" width="20" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />的转置；右下角是clip_image034[3]" border="0" alt="clip_image034[3]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016127459.png" width="13" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />的协方差矩阵。

      与之前一样࿰c;我们从clip_image032[4]" border="0" alt="clip_image032[4]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016125889.png" width="13" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和clip_image034[4]" border="0" alt="clip_image034[4]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016135955.png" width="13" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />的整体入手࿰c;定义

      clip_image052" border="0" alt="clip_image052" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016137416.png" width="62" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" /> clip_image054" border="0" alt="clip_image054" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201613481.png" width="61" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      我们可以算出u和v的方差和协方差：

      clip_image056" border="0" alt="clip_image056" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016149957.png" width="115" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" /> clip_image058" border="0" alt="clip_image058" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016146610.png" width="114" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" /> clip_image060" border="0" alt="clip_image060" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016146087.png" width="131" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      上面的结果其实很好算࿰c;推导一下第一个吧：

      clip_image062" border="0" alt="clip_image062" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016143611.png" width="460" height="62" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      最后࿰c;我们需要算Corr(u,v)了

      clip_image064" border="0" alt="clip_image064" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016154724.png" width="209" height="62" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      我们期望Corr(u,v)越大越好࿰c;关于Pearson相关系数࿰c;《class="tags" href="/tags/ShuJuWaJue.html" title=数据挖掘>数据挖掘导论》给出了一个很好的图来说明：

      clip_image066" border="0" alt="clip_image066" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016163777.jpg" width="554" height="370" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      横轴是u࿰c;纵轴是v࿰c;这里我们期望通过调整a和b使得u和v的关系越像最后一个图越好。其实第一个图和最后一个图有联系的࿰c;我们可以调整a和b的符号࿰c;使得从第一个图变为最后一个。

      接下来我们求解a和b。

      回想在LDA中࿰c;也得到了类似Corr(u,v)的公式࿰c;我们在求解时固定了分母࿰c;来求分子（避免a和b同时扩大n倍仍然符号解条件的情况出现）。这里我们同样这么做。

      这个class="tags" href="/tags/YouHua.html" title=优化>优化问题的条件是：

cellspacing="0" cellpadding="0">

Maximize clip_image068" border="0" alt="clip_image068" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016163253.png" width="48" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />       Subject to: clip_image070" border="0" alt="clip_image070" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016162730.png" width="166" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      求解方法是构造Lagrangian等式࿰c;这里我简单推导如下：

      clip_image072" border="0" alt="clip_image072" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016178303.png" width="319" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      求导࿰c;得

      clip_image074" border="0" alt="clip_image074" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016174955.png" width="130" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      clip_image076" border="0" alt="clip_image076" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016176384.png" width="135" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      令导数为0后࿰c;得到方程组：

      clip_image078" border="0" alt="clip_image078" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016177812.png" width="120" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      clip_image080" border="0" alt="clip_image080" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201617877.png" width="125" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      第一个等式左乘clip_image082" border="0" alt="clip_image082" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201618420.png" width="15" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;第二个左乘clip_image084" border="0" alt="clip_image084" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016192678.png" width="15" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;再根据clip_image086" border="0" alt="clip_image086" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201619519.png" width="143" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;得到

      clip_image088" border="0" alt="clip_image088" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016195535.png" width="108" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      也就是说求出的clip_image090" border="0" alt="clip_image090" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016201174.png" width="7" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />即是Corr(u,v)࿰c;只需找最大clip_image090[1]" border="0" alt="clip_image090[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016214305.png" width="7" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />即可。

      让我们把上面的方程组进一步简化࿰c;并写成矩阵形式࿰c;得到

      clip_image092" border="0" alt="clip_image092" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016213781.png" width="94" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      clip_image094" border="0" alt="clip_image094" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016216846.png" width="95" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      写成矩阵形式

      clip_image096" border="0" alt="clip_image096" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201621783.png" width="230" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      令

      clip_image098" border="0" alt="clip_image098" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016229944.png" width="280" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      那么上式可以写作：

      clip_image100" border="0" alt="clip_image100" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016227469.png" width="90" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      显然࿰c;又回到了求特征值的老路上了࿰c;只要求得clip_image102" border="0" alt="clip_image102" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016232551.png" width="32" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />的最大特征值clip_image104" border="0" alt="clip_image104" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016243522.png" width="28" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;那么Corr(u,v)和a和b都可以求出。

      在上面的推导过程中࿰c;我们假设了clip_image106" border="0" alt="clip_image106" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016242160.png" width="20" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和clip_image108" border="0" alt="clip_image108" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016255291.png" width="20" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />均可逆。一般情况下都是可逆的࿰c;只有存在特征间线性相关时会出现不可逆的情况࿰c;在本文最后会提到不可逆的处理办法。

      再次审视一下࿰c;如果直接去计算clip_image102[1]" border="0" alt="clip_image102[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201626373.png" width="32" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />的特征值࿰c;复杂度有点高。我们将第二个式子代入第一个࿰c;得

      clip_image110" border="0" alt="clip_image110" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016263438.png" width="149" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      这样先对clip_image112" border="0" alt="clip_image112" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016261278.png" width="83" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />求特征值clip_image114" border="0" alt="clip_image114" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016271901.png" width="13" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和特征向量clip_image116" border="0" alt="clip_image116" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016283603.png" width="8" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;然后根据第二个式子求得b。

待会举个例子说明求解过程。

      假设按照上述过程࿰c;得到了clip_image090[2]" border="0" alt="clip_image090[2]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016286493.png" width="7" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />最大时的clip_image118" border="0" alt="clip_image118" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016298195.png" width="14" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和clip_image120" border="0" alt="clip_image120" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016296625.png" width="13" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />。那么clip_image118[1]" border="0" alt="clip_image118[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016306691.png" width="14" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和clip_image120[1]" border="0" alt="clip_image120[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201631901.png" width="13" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />称为典型变量（canonical variates）࿰c;clip_image090[3]" border="0" alt="clip_image090[3]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016317903.png" width="7" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />即是u和v的相关系数。

      最后࿰c;我们得到u和v的等式为：

      clip_image122" border="0" alt="clip_image122" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201632444.png" width="61" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" /> clip_image124" border="0" alt="clip_image124" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016323509.png" width="61" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      我们也可以接着去寻找第二组典型变量对࿰c;其最class="tags" href="/tags/YouHua.html" title=优化>优化条件是

cellspacing="0" cellpadding="0">

Maximize       clip_image126" border="0" alt="clip_image126" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016329397.png" width="54" height="21" style="border-bottom:0px; border-left:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />       Subject to:   clip_image128" border="0" alt="clip_image128" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201632510.png" width="179" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />                         clip_image130" border="0" alt="clip_image130" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016334131.png" width="175" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      其实第二组约束条件就是clip_image132" border="0" alt="clip_image132" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016333608.png" width="191" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />。

      计算步骤同第一组计算方法࿰c;只不过是clip_image090[4]" border="0" alt="clip_image090[4]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016335069.png" width="7" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />取clip_image112[1]" border="0" alt="clip_image112[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016338134.png" width="83" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />的第二大特征值。

      得到的clip_image134" border="0" alt="clip_image134" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016342660.png" width="14" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和clip_image136" border="0" alt="clip_image136" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016357186.png" width="13" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />其实也满足

      clip_image138" border="0" alt="clip_image138" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016358299.png" width="175" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" /> 即 clip_image140" border="0" alt="clip_image140" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016357776.png" width="218" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      总结一下࿰c;i和j分别表示clip_image142" border="0" alt="clip_image142" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016364777.png" width="12" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和clip_image144" border="0" alt="clip_image144" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016364843.png" width="12" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />得到结果

      clip_image146" border="0" alt="clip_image146" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016367908.png" width="228" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      clip_image148" border="0" alt="clip_image148" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016375432.png" width="269" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

3. CCA计算例子

      我们回到之前的评价一个人解题和其阅读能力的关系的例子。假设我们通过对样本计算协方差矩阵得到如下结果：

      clip_image150" border="0" alt="clip_image150" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016376545.png" width="150" height="83" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      clip_image152" border="0" alt="clip_image152" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201637482.png" width="442" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      然后求clip_image112[2]" border="0" alt="clip_image112[2]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016377135.png" width="83" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;得

      clip_image154" border="0" alt="clip_image154" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016381071.png" width="239" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      这里的A和前面的clip_image156" border="0" alt="clip_image156" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016384136.png" width="79" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />中的A不是一回事（这里符号有点乱࿰c;不好意思）。

      然后对A求特征值和特征向量࿰c;得到

      clip_image158" border="0" alt="clip_image158" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016388073.png" width="332" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      然后求b࿰c;之前我们说的方法是根据clip_image160" border="0" alt="clip_image160" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016384725.png" width="86" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />求b࿰c;这里࿰c;我们也可以采用类似求a的方法来求b。

      回想之前的等式

      clip_image092[1]" border="0" alt="clip_image092[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016382566.png" width="94" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      clip_image094[1]" border="0" alt="clip_image094[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016393679.png" width="95" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      我们将上面的式子代入下面的࿰c;得

      clip_image162" border="0" alt="clip_image162" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016396743.png" width="148" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      然后直接对clip_image164" border="0" alt="clip_image164" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016398172.png" width="83" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />求特征向量即可࿰c;注意clip_image164[1]" border="0" alt="clip_image164[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016391237.png" width="83" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和clip_image112[3]" border="0" alt="clip_image112[3]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201639713.png" width="83" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />的特征值相同࿰c;这个可以自己证明下。

      不管使用哪种方法࿰c;

      clip_image166" border="0" alt="clip_image166" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016406286.png" width="238" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      clip_image168" border="0" alt="clip_image168" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016405763.png" width="157" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      这里我们得到a和b的两组向量࿰c;到这还没完࿰c;我们需要让它们满足之前的约束条件

      clip_image170" border="0" alt="clip_image170" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016408827.png" width="171" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      这里的clip_image172" border="0" alt="clip_image172" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016411402.png" width="12" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />应该是我们之前得到的VecA中的列向量的m倍࿰c;我们只需要求得m࿰c;然后将VecA中的列向量乘以m即可。

      clip_image174" border="0" alt="clip_image174" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016412830.png" width="99" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      这里的clip_image176" border="0" alt="clip_image176" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016414532.png" width="16" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />是VecA的列向量。

      clip_image178" border="0" alt="clip_image178" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016422929.jpg" width="463" height="59" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      因此最后的a和b为：

      clip_image180" border="0" alt="clip_image180" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201642138.jpg" width="365" height="55" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      第一组典型变量为

      clip_image182" border="0" alt="clip_image182" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016424075.png" width="313" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      相关系数

      clip_image184" border="0" alt="clip_image184" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016437696.png" width="214" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      第二组典型变量为

      clip_image186" border="0" alt="clip_image186" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016431633.png" width="303" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      相关系数

      clip_image188" border="0" alt="clip_image188" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016435569.png" width="215" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      这里的clip_image190" border="0" alt="clip_image190" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/2011062020164496.png" width="12" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />（解题速度）࿰c;clip_image192" border="0" alt="clip_image192" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016446258.png" width="13" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />（解题正确率）࿰c;clip_image194" border="0" alt="clip_image194" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016454372.png" width="13" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />（阅读速度）࿰c;clip_image196" border="0" alt="clip_image196" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016464438.png" width="13" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />（阅读理解程度）。他们前面的系数意思不是特征对单个u或v的贡献比重࿰c;而是从u和v整体关系看࿰c;当两者关系最密切时࿰c;特征计算时的权重。

4. Kernel Canonical Correlation Analysis（KCCA）

      通常当我们发现特征的线性组合效果不够好或者两组集合关系是非线性的时候࿰c;我们会尝试核函数方法࿰c;这里我们继续介绍Kernel CCA。

      在《支持向量机-核函数》那一篇中࿰c;大致介绍了一下核函数࿰c;这里再简单提一下：

      当我们对两个向量作内积的时候

      clip_image198" border="0" alt="clip_image198" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201646327.png" width="110" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      我们可以使用clip_image200" border="0" alt="clip_image200" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016464504.png" width="30" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;clip_image202" border="0" alt="clip_image202" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016472618.png" width="31" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />来替代clip_image204" border="0" alt="clip_image204" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016482684.png" width="8" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和clip_image206" border="0" alt="clip_image206" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016487211.png" width="7" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;比如原来的clip_image204[1]" border="0" alt="clip_image204[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016499785.png" width="8" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />特征向量为clip_image208" border="0" alt="clip_image208" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016499261.png" width="70" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;那么

      我们可以定义

      clip_image210" border="0" alt="clip_image210" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016502883.jpg" width="135" height="182" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      如果clip_image202[1]" border="0" alt="clip_image202[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016507409.png" width="31" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />与clip_image200[1]" border="0" alt="clip_image200[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016517475.png" width="30" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />的构造一样࿰c;那么

      clip_image212" border="0" alt="clip_image212" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016513048.png" width="510" height="62" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

                        clip_image214" border="0" alt="clip_image214" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201651888.png" width="132" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      这样࿰c;仅通过计算x和y的内积的平方就可以达到在高维空间（这里为clip_image216" border="0" alt="clip_image216" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016512001.png" width="15" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />）中计算clip_image218" border="0" alt="clip_image218" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016528479.png" width="30" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和clip_image220" border="0" alt="clip_image220" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201653181.png" width="31" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />内积的效果。

      由核函数࿰c;我们可以得到核矩阵K࿰c;其中

      clip_image222" border="0" alt="clip_image222" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016531610.png" width="123" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      即第clip_image224" border="0" alt="clip_image224" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016546136.png" width="5" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />行第clip_image226" border="0" alt="clip_image226" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016547838.png" width="5" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />列的元素是第clip_image224[1]" border="0" alt="clip_image224[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016556791.png" width="5" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />个和第clip_image226[1]" border="0" alt="clip_image226[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016556857.png" width="5" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />个样例在核函数下的内积。

      一个很好的核函数定义：

      clip_image228" border="0" alt="clip_image228" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016569398.jpg" width="508" height="29" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      其中样例x有n个特征࿰c;经过clip_image218[1]" border="0" alt="clip_image218[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016569465.png" width="30" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />变换后࿰c;从n维特征上升到了N维特征࿰c;其中每一个特征是clip_image230" border="0" alt="clip_image230" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016562529.png" width="142" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />。

      回到CCA࿰c;我们在使用核函数之前

      clip_image232" border="0" alt="clip_image232" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016572595.png" width="56" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" /> clip_image234" border="0" alt="clip_image234" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016571548.png" width="55" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      这里假设x和y都是n维的࿰c;引入核函数后࿰c;clip_image236" border="0" alt="clip_image236" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016582138.png" width="35" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和clip_image238" border="0" alt="clip_image238" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016585203.png" width="36" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />变为了N维。

      使用核函数后࿰c;u和v的公式为：

      clip_image240" border="0" alt="clip_image240" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016586631.png" width="86" height="21" style="border-bottom:0px; border-left:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      clip_image242" border="0" alt="clip_image242" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016583284.png" width="88" height="42" style="border-bottom:0px; border-left:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      这里的c和d都是N维向量。

      现在我们有样本clip_image244" border="0" alt="clip_image244" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016592760.png" width="69" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;这里的clip_image246" border="0" alt="clip_image246" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202016593383.png" width="12" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />表示样本x的第i个样例࿰c;是n维向量。

根据前面说过的相关系数࿰c;构造拉格朗日公式如下：

      clip_image248" border="0" alt="clip_image248" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017009828.jpg" width="305" height="109" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      其中

      clip_image250" border="0" alt="clip_image250" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201700941.png" width="151" height="62" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      clip_image252" border="0" alt="clip_image252" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201700417.png" width="225" height="62" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      然后让L对a求导࿰c;令导数等于0࿰c;得到（这一步我没有验证࿰c;待会从宏观上解释一下）

      clip_image254" border="0" alt="clip_image254" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017003482.png" width="112" height="62" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      同样对b求导࿰c;令导数等于0࿰c;得到

      clip_image256" border="0" alt="clip_image256" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017004910.png" width="113" height="62" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      求出c和d干嘛呢？c和d只是clip_image258" border="0" alt="clip_image258" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017015500.png" width="10" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />的系数而已࿰c;按照原始的CCA做法去做就行了呗࿰c;为了再引入clip_image260" border="0" alt="clip_image260" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017016089.png" width="8" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和clip_image262" border="0" alt="clip_image262" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017026679.png" width="8" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />？

      回答这个问题要从核函数的意义上来说明。核函数初衷是希望在式子中有clip_image264" border="0" alt="clip_image264" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017029536.png" width="66" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;然后用K替换之࿰c;根本没有打算去计算出实际的clip_image258[1]" border="0" alt="clip_image258[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017034269.png" width="10" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />。因此即是按照原始CCA的方式计算出了c和d࿰c;也是没用的࿰c;因为根本有没有实际的clip_image258[2]" border="0" alt="clip_image258[2]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201703747.png" width="10" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />让我们去做clip_image266" border="0" alt="clip_image266" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017033812.png" width="44" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />。另一个原因是核函数比如高斯径向基核函数可以上升到无限维࿰c;N是无穷的࿰c;因此c和d也是无穷维的࿰c;根本没办法直接计算出来。我们的思路是在原始的空间中构造出权重clip_image260[1]" border="0" alt="clip_image260[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201704290.png" width="8" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和clip_image262[1]" border="0" alt="clip_image262[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017051992.png" width="8" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;然后利用clip_image258[3]" border="0" alt="clip_image258[3]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017052058.png" width="10" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />将clip_image260[2]" border="0" alt="clip_image260[2]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017064076.png" width="8" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和clip_image262[2]" border="0" alt="clip_image262[2]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017072506.png" width="8" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />上升到高维࿰c;他们在高维对应的权重就是c和d。

      虽然clip_image260[3]" border="0" alt="clip_image260[3]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017074208.png" width="8" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和clip_image262[3]" border="0" alt="clip_image262[3]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017082322.png" width="8" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />是在原始空间中（维度为样例个数M）࿰c;但其作用点不是在原始特征上࿰c;而是原始样例上。看上面得出的c和d的公式就知道。clip_image260[4]" border="0" alt="clip_image260[4]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017088800.png" width="8" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />通过控制每个高维样例的权重࿰c;来控制c。

      好了࿰c;接下来我们看看使用clip_image260[5]" border="0" alt="clip_image260[5]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017096915.png" width="8" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和clip_image262[4]" border="0" alt="clip_image262[4]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017105344.png" width="8" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />后࿰c;u和v的变化

      clip_image268" border="0" alt="clip_image268" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017106457.png" width="249" height="62" style="border-bottom:0px; border-left:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      clip_image270" border="0" alt="clip_image270" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017103982.png" width="251" height="62" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      clip_image272" border="0" alt="clip_image272" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017105411.png" width="39" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />表示可以将第i个样例上升到的N维向量࿰c;clip_image274" border="0" alt="clip_image274" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017117952.png" width="35" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />意义可以类比原始CCA的x。

      鉴于这样表示接下来会越来越复杂࿰c;改用矩阵形式表示。

      clip_image276" border="0" alt="clip_image276" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017119065.png" width="307" height="83" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      简写为

      clip_image278" border="0" alt="clip_image278" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017111606.png" width="56" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      其中X（M×N）为

            clip_image280" border="0" alt="clip_image280" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017125543.png" width="120" height="83" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      我们发现

      clip_image282" border="0" alt="clip_image282" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017122195.png" width="68" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      我们可以算出u和v的方差和协方差（这里实际上事先对样本clip_image204[2]" border="0" alt="clip_image204[2]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017121149.png" width="8" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和clip_image284" border="0" alt="clip_image284" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017127801.png" width="8" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />做了均值归0处理）：

      clip_image286" border="0" alt="clip_image286" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017135326.png" width="439" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      clip_image288" border="0" alt="clip_image288" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017134803.png" width="131" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      clip_image290" border="0" alt="clip_image290" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017134279.png" width="512" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      这里clip_image274[1]" border="0" alt="clip_image274[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017134869.png" width="35" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和clip_image292" border="0" alt="clip_image292" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017146297.png" width="36" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />维度可以不一样。

      最后࿰c;我们得到Corr(u,v)

      clip_image294" border="0" alt="clip_image294" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017141870.png" width="235" height="83" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      可以看到࿰c;在将clip_image018[1]" border="0" alt="clip_image018[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017144935.png" width="13" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />和clip_image020[1]" border="0" alt="clip_image020[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017141936.png" width="14" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />处理成clip_image296" border="0" alt="clip_image296" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017153365.png" width="57" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />࿰c;clip_image298" border="0" alt="clip_image298" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/2011062020171517.png" width="58" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />后࿰c;得到的结果和之前形式基本一样࿰c;只是将clip_image044[1]" border="0" alt="clip_image044[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017156495.png" width="8" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />替换成了两个K乘积。

      因此࿰c;得到的结果也是一样的࿰c;之前是

      clip_image100[1]" border="0" alt="clip_image100[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017169560.png" width="90" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      其中

      clip_image098[1]" border="0" alt="clip_image098[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017163497.png" width="280" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      引入核函数后࿰c;得到

      clip_image100[2]" border="0" alt="clip_image100[2]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017162973.png" width="90" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      其中

      clip_image300" border="0" alt="clip_image300" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017168546.png" width="333" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      注意这里的两个w有点区别࿰c;前面的clip_image116[1]" border="0" alt="clip_image116[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017173072.png" width="8" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />维度和x的特征数相同࿰c;clip_image302" border="0" alt="clip_image302" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017189235.png" width="8" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />维度和y的特征数相同。后面的clip_image304" border="0" alt="clip_image304" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017181809.png" width="9" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />维度和x的样例数相同࿰c;clip_image306" border="0" alt="clip_image306" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/20110620201719762.png" width="9" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />维度和y的样例数相同࿰c;严格来说“clip_image304[1]" border="0" alt="clip_image304[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017193303.png" width="9" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />维度=clip_image306[1]" border="0" alt="clip_image306[1]" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017205006.png" width="9" height="21" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />维度”。

5. 其他话题

      1、当协方差矩阵不可逆时࿰c;怎么办？

      要进行regularization。

      一种方法是将前面的KCCA中的拉格朗日等式加上二次正则化项࿰c;即：

      clip_image308" border="0" alt="clip_image308" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017208070.png" width="153" height="42" style="border-bottom:0px; border-left:0px; margin:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      这样求导后得到的等式中࿰c;等式右边的矩阵一定是正定矩阵。

      第二种方法是在Pearson系数的分母上加入正则化项࿰c;同样结果也一定可逆。

      clip_image310" border="0" alt="clip_image310" src="http://images.cnblogs.com/cnblogs_com/jerrylead/201106/201106202017208975.jpg" width="405" height="112" style="border-bottom:0px; border-left:0px; padding-left:0px; padding-right:0px; display:inline; border-top:0px; border-right:0px; padding-top:0px" />

      2、求Kernel矩阵效率不高怎么办？

      使用Cholesky decomposition压缩法或者部分Gram-Schmidt正交化法࿰c;。

      3、怎么使用CCA用来做预测？

c="http://pic002.cnblogs.com/images/2011/279228/2011082622053268.png" />

　　

     c="http://pic002.cnblogs.com/images/2011/279228/2011082622010459.png" />

      4、如果有多个集合怎么办？X、Y、Z…？怎么衡量多个样本集的关系？

      这个称为Generalization of the Canonical Correlation。方法是使得两两集合的距离差之和最小。可以参考文献2。

6. 参考文献

      1、 http://www.stat.tamu.edu/~rrhocking/stat636/LEC-9.636.pdf

      2、 Canonical correlation analysis: An overview with class="tags" href="/tags/APPLICATION.html" title=application>application to learning methods. David R. Hardoon , Sandor Szedmak and John Shawe-Taylor

      3、 A kernel method for canonical correlation analysis. Shotaro Akaho

      4、 Canonical Correlation a Tutorial. Magnus Borga

      5、 Kernel Canonical Correlation Analysis. Max Welling