留言板

 引用本文: 雷大江, 唐建烊, 李智星, 吴渝. 基于中心对齐多核学习的稀疏多元逻辑回归算法[J]. 电子与信息学报.
Dajiang LEI, Jianyang TANG, Zhixing LI, Yu WU. Sparse Multinomial Logistic Regression Algorithm Based on Centered Alignment Multiple Kernels Learning[J]. Journal of Electronics and Information Technology. doi: 10.11999/JEIT190426
 Citation: Dajiang LEI, Jianyang TANG, Zhixing LI, Yu WU. Sparse Multinomial Logistic Regression Algorithm Based on Centered Alignment Multiple Kernels Learning[J]. Journal of Electronics and Information Technology.

• 中图分类号: TP181

Sparse Multinomial Logistic Regression Algorithm Based on Centered Alignment Multiple Kernels Learning

Funds: The Chongqing Innovative Project of Overseas Study(cx2018120), The National Social Science Foundation of China(17XFX013), The Natural Science Foundation of Chongqing(cstc2015jcyjA40018)
• 摘要: 稀疏多元逻辑回归(SMLR)作为一种广义的线性模型被广泛地应用于各种多分类任务场景中。SMLR通过将拉普拉斯先验引入多元逻辑回归(MLR)中使其解具有稀疏性，这使得该分类器可以在进行分类的过程中嵌入特征选择。为了使分类器能够解决非线性数据分类的问题，该文通过核技巧对SMLR进行核化扩充后得到了核稀疏多元逻辑回归(KSMLR)。KSMLR能够将非线性特征数据通过核函数映射到高维甚至无穷维的特征空间中，使其特征能够充分地表达并最终能进行有效的分类。此外，该文还利用了基于中心对齐的多核学习算法，通过不同的核函数对数据进行不同维度的映射，并用中心对齐相似度来灵活地选取多核学习权重系数，使得分类器具有更好的泛化能力。实验结果表明，该文提出的基于中心对齐多核学习的稀疏多元逻辑回归算法在分类的准确率指标上都优于目前常规的分类算法。
•  [1] ZHOU Changjun, WANG Lan, ZHANG Qiang, et al. Face recognition based on PCA and logistic regression analysis[J]. Optik, 2014, 125(20): 5916–5919. [2] WARNER P. Ordinal logistic regression[J]. Journal of Family Planning and Reproductive Health Care, 2008, 34(3): 169–170. [3] LIU Wu, FOWLER J E, and ZHAO Chunhui. Spatial logistic regression for support-vector classification of hyperspectral imagery[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(3): 439–443. [4] ABRAMOVICH F and GRINSHTEIN V. High-dimensional classification by sparse logistic regression[J]. IEEE Transactions on Information Theory, 2019, 65(5): 3068–3079. [5] CARVALHO C M, CHANG J, LUCAS J E, et al. High-dimensional sparse factor modeling: Applications in gene expression genomics[J]. Journal of the American Statistical Association, 2008, 103(484): 1438–1456. [6] GALAR M, FERNÁNDEZ A, BARRENECHEA E, et al. An overview of ensemble methods for binary classifiers in multi-class problems: Experimental study on one-vs-one and one-vs-all schemes[J]. Pattern Recognition, 2011, 44(8): 1761–1776. [7] 曾志强, 吴群, 廖备水, 等. 一种基于核SMOTE的非平衡数据集分类方法[J]. 电子学报, 2009, 37(11): 2489–2495. ZENG Zhiqiang, WU Qun, LIAO Beishui, et al. A classfication method for imbalance data set based on kernel SMOTE[J]. Acta Electronica Sinica, 2009, 37(11): 2489–2495. [8] CAO Faxian, YANG Zhijing, REN Jinchang, et al. Extreme sparse multinomial logistic regression: A fast and robust framework for hyperspectral image classification[J]. Remote Sensing, 2017, 9(12): 1255. [9] LIU Tianzhu, GU Yanfeng, JIA Xiuping, et al. Class-specific sparse multiple kernel learning for spectral–spatial hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(12): 7351–7365. [10] FANG Leyuan, WANG Cheng, LI Shutao, et al. Hyperspectral image classification via multiple-feature-based adaptive sparse representation[J]. IEEE Transactions on Instrumentation and Measurement, 2017, 66(7): 1646–1657. [11] OUYED O and ALLILI M S. Feature weighting for multinomial kernel logistic regression and application to action recognition[J]. Neurocomputing, 2018, 275: 1752–1768. [12] 徐金环, 沈煜, 刘鹏飞, 等. 联合核稀疏多元逻辑回归和TV-L1错误剔除的高光谱图像分类算法[J]. 电子学报, 2018, 46(1): 175–184. XU Jinhuan, SHEN Yu, LIU Pengfei, et al. Hyperspectral image classification combining kernel sparse multinomial logistic regression and TV-L1 error rejection[J]. Acta Electronica Sinica, 2018, 46(1): 175–184. [13] SCHÖLKOPF B and SMOLA A J. Learning With Kernels: Support Vector Machines, Regularization, Optimization, and Beyond[M]. Cambridge: MIT Press, 2002. [14] 汪洪桥, 孙富春, 蔡艳宁, 等. 多核学习方法[J]. 自动化学报, 2010, 36(8): 1037–1050. WANG Hongqiao, SUN Fuchun, CAI Yanning, et al. On multiple kernel learning methods[J]. Acta Automatica Sinica, 2010, 36(8): 1037–1050. [15] GÖNEN M and ALPAYDIN E. Multiple kernel learning algorithms[J]. Journal of Machine Learning Research, 2011, 12: 2211–2268. [16] GU Yanfeng, LIU Tianzhu, JIA Xiuping, et al. Nonlinear multiple kernel learning with multiple-structure-element extended morphological profiles for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(6): 3235–3247. [17] RAKOTOMAMONJY A, BACH F R, CANU S, et al. SimpleMKL[J]. Journal of Machine Learning Research, 2008, 9: 2491–2521. [18] LOOSLI G and ABOUBACAR H. Using SVDD in SimpleMKL for 3D-Shapes filtering[C]. CAp - Conférence D'apprentissage, Saint-Etienne, 2017. doi:  10.13140/2.1.3091.3605. [19] JAIN A, VISHWANATHAN S V N, and VARMA M. SPF-GMKL: Generalized multiple kernel learning with a million kernels[C]. The 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Beijing, China, 2012: 750–758. doi:  10.1145/2339530.2339648. [20] BAHMANI S, BOUFOUNOS P T, and RAJ B. Learning model-based sparsity via projected gradient descent[J]. IEEE Transactions on Information Theory, 2016, 62(4): 2092–2099. [21] CORTES C, MOHRI M, and ROSTAMIZADEH A. Algorithms for learning kernels based on centered alignment[J]. Journal of Machine Learning Research, 2012, 13(28): 795–828. [22] CHENG Chunyuan, HSU C C, and CHENG Muchen. Adaptive kernel principal component analysis (KPCA) for monitoring small disturbances of nonlinear processes[J]. Industrial & Engineering Chemistry Research, 2010, 49(5): 2254–2262. [23] YANG Hongjun and LIU Jinkun. An adaptive RBF neural network control method for a class of nonlinear systems[J]. IEEE/CAA Journal of Automatica Sinica, 2018, 5(2): 457–462. [24] BECK A and TEBOULLE M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems[J]. SIAM Journal on Imaging Sciences, 2009, 2(1): 183–202. [25] KRISHNAPURAM B, CARIN L, FIGUEIREDO M A T, et al. Sparse multinomial logistic regression: Fast algorithms and generalization bounds[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(6): 957–968. [26] CHEN Xi, LIN Qihang, KIM S, et al. Smoothing proximal gradient method for general structured sparse regression[J]. The Annals of Applied Statistics, 2012, 6(2): 719–752. [27] LECUN Y, BENGIO Y and HINTON G. Deep learning[J]. Nature, 2015, 521(7553): 436–444. [28] PÉREZ-ORTIZ M, GUTIÉRREZ P A, SÁNCHEZ-MONEDERO J, et al. A study on multi-scale kernel optimisation via centered kernel-target alignment[J]. Neural Processing Letters, 2016, 44(2): 491–517.
•  [1] 张颖, 姚雨丰.  基于快速贝叶斯匹配追踪优化的海上稀疏信道估计方法, 电子与信息学报. 2020, 42(2): 534-540. doi: 10.11999/JEIT190102 [2] 陈志坤, 杜康, 彭冬亮, 朱新挺.  基于混合三角变异差分进化算法的平面稀疏阵列约束优化, 电子与信息学报. 2020, 42(4): 895-901. doi: 10.11999/JEIT190705 [3] 刘明, 孟宪辉, 熊鹏, 刘秀玲.  基于核稀疏编码的阵发性房颤检测, 电子与信息学报. 2020, 42(7): 1743-1749. doi: 10.11999/JEIT190582 [4] 陈怡君, 李开明, 张群, 罗迎.  稀疏线性调频步进信号ISAR成像观测矩阵自适应优化方法, 电子与信息学报. 2018, 40(3): 509-516. doi: 10.11999/JEIT170554 [5] 金艳, 田田, 姬红兵.  基于稀疏贝叶斯学习的码元速率估计, 电子与信息学报. 2018, 40(7): 1598-1603. doi: 10.11999/JEIT170906 [6] 宋虎, 蒋迺倜, 刘溶, 李洪涛.  基于稀疏采样阵列优化的APG-MUSIC算法, 电子与信息学报. 2018, 40(6): 1390-1396. doi: 10.11999/JEIT170807 [7] 李佩佳, 石勇, 汪华东, 牛凌峰.  基于有序编码的核极限学习顺序回归模型, 电子与信息学报. 2018, 40(6): 1287-1293. doi: 10.11999/JEIT170765 [8] 刘大千, 刘万军, 费博雯.  局部感知下的稀疏优化目标跟踪方法, 电子与信息学报. 2018, 40(2): 272-281. doi: 10.11999/JEIT170473 [9] 施孝盼, 洪涛.  基于凸优化的稀疏阵列方向调制信号综合算法研究, 电子与信息学报. 2017, 39(11): 2563-2570. doi: 10.11999/JEIT170391 [10] 宋文青, 王英华, 时荔蕙, 刘宏伟, 保铮.  基于多特征融合词包模型的SAR目标鉴别算法, 电子与信息学报. 2017, 39(11): 2705-2715. doi: 10.11999/JEIT170086 [11] 张凯, 于宏毅, 胡赟鹏, 沈智翔.  稀疏信道下基于稀疏贝叶斯学习的精简星座盲均衡算法, 电子与信息学报. 2016, 38(9): 2255-2260. doi: 10.11999/JEIT151307 [12] 黄宏图, 毕笃彦, 高山, 查宇飞, 侯志强.  基于局部敏感核稀疏表示的视频跟踪, 电子与信息学报. 2016, 38(4): 993-999. doi: 10.11999/JEIT150785 [13] 蒋成龙, 赵曜, 张柘, 张冰尘, 洪文.  基于相关准则的稀疏微波成像方位向采样优化方法, 电子与信息学报. 2015, 37(3): 580-586. doi: 10.11999/JEIT140613 [14] 戚晨皓, 吴乐南, 朱鹏程.  认知无线电中的稀疏信道估计与导频优化, 电子与信息学报. 2014, 36(4): 763-768. doi: 10.3724/SP.J.1146.2013.00971 [15] 刘鲁锋, 杜新鹏, 成礼智.  一种基于粒子群优化的稀疏恢复算法, 电子与信息学报. 2013, 35(11): 2733-2738. doi: 10.3724/SP.J.1146.2012.01397 [16] 胡恒, 贺亚鹏, 庄珊娜, 朱晓华.  高频地波雷达稀疏频率波形优化设计, 电子与信息学报. 2012, 34(6): 1291-1296. doi: 10.3724/SP.J.1146.2011.00666 [17] 杨清山, 郭成安, 金明录.  基于Gabor多通道加权优化与稀疏表征的人脸识别方法, 电子与信息学报. 2011, 33(7): 1618-1624. doi: 10.3724/SP.J.1146.2010.01280 [18] 郑建炜, 王万良, 蒋一波, 陈伟杰.  概率型稀疏核Logistic多元分类机, 电子与信息学报. 2011, 33(7): 1632-1638. doi: 10.3724/SP.J.1146.2010.01237 [19] 陈客松, 何子述, 唐海红.  对称线阵的优化稀疏研究, 电子与信息学报. 2009, 31(6): 1490-1492. doi: 10.3724/SP.J.1146.2008.00977 [20] 陈客松, 何子述, 韩春林.  利用GA实现非对称稀疏线阵旁瓣电平的优化, 电子与信息学报. 2007, 29(4): 987-990. doi: 10.3724/SP.J.1146.2005.00723
• 点击查看大图
表ll (4)
计量
• 文章访问数:  107
• HTML全文浏览量:  37
• PDF下载量:  13
• 被引次数: 0
出版历程
• 收稿日期:  2019-06-11
• 修回日期:  2020-03-28
• 网络出版日期:  2020-08-27

目录

/

• 分享
• 用微信扫码二维码

分享至好友和朋友圈

官方微信，欢迎关注