Loading [MathJax]/jax/output/HTML-CSS/jax.js

常用无监督降维方法简述

简介: Unsupervised Dimension ReductionData with high dimension is always difficult to tackle. One hand is that it requires tremendous computation resource. On the other hand, it is not so objec

Unsupervised Dimension Reduction

Data with high dimension is always difficult to tackle. One hand is that it requires tremendous computation resource. On the other hand, it is not so objective as the one with low dimension. Therefore, dimension reduction is one of the key tricks to tackle it.

Linear Dimension Reduction

In order to reduce the dimension of samples, such as transform {xi}ni=1 {xi}ni=1 into {zi}ni=1 {zi}ni=1 with little lose of information, we can use linear transformation :

zi=Txi
zi=Txi
Before doing that, it is necessary to make sure the average of training set {xi}ni=1 {xi}ni=1 to be zero, i.e. centralization. So if it were not true, we should move the frame :
xixi1ni=1nxi
xixi1nni=1xi

Principal Component Analysis, PCA

PCA, as you can see in the following contents, is the simplest linear dimension reduction method. Suppose that zi zi is the orthogonal projection of xi xi. Then we require that TTT=Im TTT=Im. By the same way in LS methods we try to reduce the lose of information as little as possible, i.e. we try to minimize:

i=1nTTTxixi2=tr(TCTT)+tr(C)
ni=1TTTxixi2=tr(TCTT)+tr(C)
where C C is the covariance of training set:
C=i=1nxixTi
C=ni=1xixTi
In summary, PCA is defined as
maxTRm×dtr(TCTT)s.t.TTT=Im
maxTRm×dtr(TCTT)s.t.TTT=Im
Consider the eigenvalues of C C:
Cξ=λξ
Cξ=λξ
Define the eigenvalues and corresponded eigen vectors as λ1λ2λd0 λ1λ2λd0 and ξ1,,ξd ξ1,,ξd respectively. Then we get :
T=(ξ1,,ξm)T
T=(ξ1,,ξm)T

Here is a simple example:

n=100;
%x=[2*randn(n,1) randn(n,1)];
x=[2*randn(n,1) 2*round(rand(n,1))-1+randn(n,1)/3];
x=x-repmat(mean(x),[n,1]);
[t,v]=eigs(x'*x,1);

figure(1); clf; hold on; axis([-6 6 -6 6]);
plot(x(:,1),x(:,2),'rx');
plot(9*[-t(1) t(1)], 9*[-t(2) t(2)]);
AI 代码解读

这里写图片描述

Locality Preserving Projections

In PCA, the structure of clusters in origin training set may be changed, which is not true in locality preserving projections. It is another version of linear dimension reduction.
Define the similarity between xi xi and xi xi as Wi,i0 Wi,i0. When they are similar to large degree Wi,i Wi,i is of a large value and vice versa. Since similarity is symmetric, we require Wi,i=Wi,i Wi,i=Wi,i . There are several normal forms of similarity, such as the Gaussian Similarity:

Wi,i=exp(xixi22t2)
Wi,i=exp(xixi22t2)
where t>0 t>0 is a tunable parameter.
For the purpose of holding the structure of clusters, it is necessary to hypothesis that similar xi xi would be transformed to similar zi zi. That is to say, we ought to minimize:
12i,i=1nWi,iTxiTxi2
12ni,i=1Wi,iTxiTxi2
However, to avoid the solution T=0 T=0, we require
TXDXTTT=Im
TXDXTTT=Im
where X=(x1,,xn)Rd×n X=(x1,,xn)Rd×n, D D is a diagonal matrix:
Di,i=i′′=1nWi,i′′0(i=i)(ii)
Di,i={ni=1Wi,i(i=i)0(ii)
If we set L=DW L=DW, then we can represent our optimization goal as
minTRm×dtr(TXLXTTT)s.t.TXDXTTT=Im
minTRm×dtr(TXLXTTT)s.t.TXDXTTT=Im
So how to solve it? Consider the method we use in PCA:
XLXTξ=λXDXTξ
XLXTξ=λXDXTξ
Then define the generalized eigenvalues and eigen vectors as λ1λ2λd0 λ1λ2λd0 and ξ1,,ξd ξ1,,ξd respectively. Therefore
T=(ξd,ξd1,,ξdm+1)T
T=(ξd,ξd1,,ξdm+1)T.
n=100;
%x=[2*randn(n,1) randn(n,1)];
x=[2*randn(n,1) 2*round(rand(n,1))-1+randn(n,1)/3];
x=x-repmat(mean(x),[n,1]);
x2=sum(x.^2,2);
W=exp(-(repmat(x2,1,n)+repmat(x2',n,1)-2*x*x'));
D=diag(sum(W,2)); L=D-W;
z=x'*D*x;
z=(z+z')/2;
[t,v]=eigs(x'*L*x,z,1,'sm');

figure(1); clf; hold on; axis([-6 6 -6 6]);
plot(x(:,1),x(:,2),'rx');
plot(9*[-t(1) t(1)], 9*[-t(2) t(2)]);
AI 代码解读

这里写图片描述

Kernalized PCA

Let us turn to methods of nonlinear dimension reduction. Due to the time limit, we may not analyze it as deep as the linear one.
When it comes to nonlinearity, kernal functions are sure to be highlighted. Take the Gaussian Kernal function for example:

K(x,x)=exp(xx22h2)
K(x,x)=exp(xx22h2)
Here we will not take the eigenvalues of C C into account as we did in PCA, but the eigenvalues of kernal matrix Kα=λα Kα=λα, where the (i,i) (i,i)th element is K(xi,xi) K(xi,xi). Hence KRn×n KRn×n . Note that dimension of the kernal matrix K K depends only on the number of samples.
However, centralization is necessary:
KHKH
KHKH
where
H=In1n×n/n
H=In1n×n/n
1n×n 1n×n is a matrix with all the elements to be one. The final outcome of kernalized PCA is:
(z1,.zn)=(1λ1α1,,1λmαm)THKH
(z1,.zn)=(1λ1α1,,1λmαm)THKH
where α1,,αm α1,,αm are m m eigen vectors corresponded with m m largest eigenvalues of HKH HKH.
目录
打赏
0
0
0
0
1
分享
相关文章
JavaOpenCV相似度计算基础教程
JavaOpenCV是一个基于开放源代码的计算机视觉库,它可以实现许多计算机视觉任务,如图像处理、物体识别和图像相似度计算等。本教程旨在向您介绍JavaOpenCV中的相似度计算基础,帮助您理解如何使用该库计算图像之间的相似度
451 0
获取后端接口请求中的参数(@PathVariable,@RequestParam,@RequestBody区别,使用postman请求
获取后端接口请求中的参数(@PathVariable,@RequestParam,@RequestBody区别,使用postman请求
467 1
【AI系统】TVM 实践案例
本文探讨了如何利用AI编译器在新硬件上部署神经网络,重点介绍了TVM的工作流程,包括模型导入、转换为Relay、TE和TIR、自动调优、编译为机器码等步骤。文章还讨论了算法层面上的算子优化、量化技术,以及编译层面对量化模型的解析和计算图优化。此外,还介绍了TVM的BYOC框架,允许硬件加速器供应商通过即插即用的方式集成代码工具,实现高效编译和优化。最后,文章提及了算子和网络仿真的重要性,确保新硬件平台上的模型正确性和性能。
159 2
深度探索微服务架构中的容器化技术
在现代软件开发中,微服务架构因其模块化和可扩展性而广受欢迎。而容器化技术,尤其是Docker,成为了支持微服务架构的核心工具。本文将探讨容器化在微服务架构中的作用,包括其如何提升开发效率、简化部署过程以及解决传统方法中的问题。通过具体实例和最佳实践的分析,读者将了解如何有效利用容器化技术来优化微服务架构。
(十二)彻悟并发之JUC分治思想产物-ForkJoin分支合并框架原理剖析下篇
在《(十二)彻悟并发之JUC分治思想产物-ForkJoin分支合并框架原理剖析上篇》中,我们曾初步了解了ForkJoin分支合并框架的使用,也分析框架的成员构成以及任务提交和创建工作的原理实现,在本篇则会对框架的任务执行、任务扫描、线程挂起、结果合并以及任务窃取的源码实现进行分析。
基于springboot的校园二手交易平台(程序+数据库+文档)
基于springboot的校园二手交易平台(程序+数据库+文档)
北大推出全新机器人多模态大模型!面向通用和机器人场景的高效推理和操作
【6月更文挑战第29天】北京大学研发的RoboMamba是新型机器人多模态大模型,融合Mamba SSM的高效推理与视觉编码器,提升复杂任务处理能力。通过微调策略,仅用少量参数即可快速习得操作技能,实现在通用及机器人场景的高效运行,推理速度提升7倍。尽管面临泛化和可解释性挑战,RoboMamba展示了多模态模型的新潜力。[论文链接:](https://arxiv.org/abs/2406.04339)
241 1
新商业模式创新下AIGC的发展
【1月更文挑战第13天】新商业模式创新下AIGC的发展
167 2
新商业模式创新下AIGC的发展
AI助理

你好,我是AI助理

可以解答问题、推荐解决方案等

登录插画

登录以查看您的控制台资源

管理云资源
状态一览
快捷访问