高级搜索

基于分类误差一致性准则的自适应知识迁移

梁爽 杭文龙 冯伟 刘学军

引用本文: 梁爽, 杭文龙, 冯伟, 刘学军. 基于分类误差一致性准则的自适应知识迁移[J]. 电子与信息学报, doi: 10.11999/JEIT181054 shu
Citation:  Shuang LIANG, Wenlong HANG, Wei FENG, Xuejun LIU. Adaptive Knowledge Transfer Based on Classification-error Consensus Regularization[J]. Journal of Electronics and Information Technology, doi: 10.11999/JEIT181054 shu

基于分类误差一致性准则的自适应知识迁移

    作者简介: 梁爽: 女,1987年生,讲师,研究方向为机器学习、信号处理;
    杭文龙: 男,1988年生,讲师,研究方向为机器学习、模式识别;
    冯伟: 男,1995年生,硕士生,研究方向机器学习、模式识别;
    刘学军: 男,1970年生,教授,硕士生导师,研究方向为数据挖掘、大数据分布式处理;
    通讯作者: 杭文龙, wlhang@njtech.edu.cn
  • 基金项目: 国家自然科学基金(61802177),江苏省高校自然科学研究面上项目(18KJB520020),南京邮电大学引进人才科研启动基金资助项目(NY219034),江苏省重点研发计划(BE2015697)

摘要: 目前大多数迁移学习方法在利用源域数据辅助目标域数据建模时,通常假设源域中的数据均与目标域数据相关。然而在实际应用中,源域中的数据并非都与目标域数据的相关程度一致,若基于上述假设往往会导致负迁移效应。为此,该文首先提出分类误差一致性准则(CCR),对源域与目标域分类误差的概率分布积分平方误差进行最小化度量。此外,该文提出一种基于CCR的自适应知识迁移学习方法(CATL),该方法可以快速地从源域中自动确定出与目标域相关的数据及其权重,以辅助目标域模型的构建,使其能在提高知识迁移效率的同时缓解负迁移学习效应。在真实图像以及文本数据集上的实验结果验证了CATL方法的优势。

English

    1. [1]

      DENG Zhaohong, JIANG Yizhang, CHOI K S, et al. Knowledge-leverage-based TSK fuzzy system modeling[J]. IEEE Transactions on Neural Networks and Learning Systems, 2013, 24(8): 1200–1212. doi: 10.1109/TNNLS.2013.2253617

    2. [2]

      DAI Wenyuan, YANG Qiang, XUE Guirong, et al. Boosting for transfer learning[C]. Proceedings of the 24th International Conference on Machine Learning, Corvalis, USA, 2007: 193–200.

    3. [3]

      JIANG Yizhang, DENG Zhaohong, CHUNG F L, et al. Recognition of epileptic EEG signals using a novel multiview TSK fuzzy system[J]. IEEE Transactions on Fuzzy Systems, 2017, 25(1): 3–20. doi: 10.1109/TFUZZ.2016.2637405

    4. [4]

      ZHUANG Fuzhen, LUO Ping, DU Changying, et al. Triplex transfer learning: Exploiting both shared and distinct concepts for text classification[J]. IEEE Transactions on Cybernetics, 2014, 44(7): 1191–1203. doi: 10.1109/TCYB.2013.2281451

    5. [5]

      PAN S J, NI Xiaochuan, SUN Jiantao, et al. Cross-domain sentiment classification via spectral feature alignment[C]. Proceedings of the 19th International Conference on World Wide Web, Raleigh, USA, 2010: 751–760.

    6. [6]

      ZANG Shaofei, CHENG Yuhu, WANG Xuesong, et al. Semi-supervised transfer discriminant analysis based on cross-domain mean constraint[J]. Artificial Intelligence Review, 2018, 49(4): 581–595. doi: 10.1007/s10462-016-9533-3

    7. [7]

      WANG Guanjin, ZHANG Guangquan, CHOI K S, et al. Deep additive least squares support vector machines for classification with model transfer[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2017. doi: 10.1109/TSMC.2017.2759090

    8. [8]

      YANG Jun, YAN Rong, and HAUPTMANN A G. Adapting SVM classifiers to data with shifted distributions[C]. Proceedings of the Seventh IEEE International Conference on Data Mining Workshops, Omaha, USA, 2007: 69–76.

    9. [9]

      JIANG Yizhang, DENG Zhaohong, CHUNG F L, et al. Realizing two-view TSK fuzzy classification system by using collaborative learning[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2017, 47(1): 145–160. doi: 10.1109/TSMC.2016.2577558

    10. [10]

      CHU Wensheng, DE LA TORRE F, and COHN J F. Selective transfer machine for personalized facial action unit detection[C]. Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, USA, 2013: 3515–3522.

    11. [11]

      GRETTON A, SMOLA A, HUANG Jiayuan, et al. Covariate shift by kernel mean matching[M]. QUIÑONERO-CANDELA J, SUGIYAMA M, SCHWAIGHOFER A, et al. Dataset Shift in Machine Learning. Cambridge, USA: MIT Press, 2009: 131–160.

    12. [12]

      CHENG Yuhu, WANG Xuesong, and CAO Ge. Multi-source tri-training transfer learning[J]. IEICE Transactions on Information and Systems, 2014, E97-D(6): 1668–1672. doi: 10.1587/transinf.e97.d.1668

    13. [13]

      WANG Yunyun, ZHAI Jie, LI Yun, et al. Transfer learning with partial related " instance-feature” knowledge[J]. Neurocomputing, 2018, 310: 115–124. doi: 10.1016/j.neucom.2018.05.029

    14. [14]

      CHEN Minmin, XU Zhixiang, WEINBERGER K Q, et al. Marginalized denoising autoencoders for domain adaptation[C]. Proceedings of the 29th International Conference on Machine Learning, Edinburgh, Scotland, 2012: 1627–1634.

    15. [15]

      ZHOU J T, PAN S J, TSANG I W, et al. Hybrid heterogeneous transfer learning through deep learning[C]. Proceedings of the 28th AAAI Conference on Artificial Intelligence, Québec City, Canada, 2014: 2213–2219.

    16. [16]

      GLOROT X, BORDES A, and BENGIO Y. Domain adaptation for large-scale sentiment classification: A deep learning approach[C]. Proceedings of the 28th International Conference on Machine Learning, Bellevue, Washington, USA, 2011: 513–520.

    17. [17]

      LONG Mingsheng, WANG Jianmin, CAO Yue, et al. Deep learning of transferable representation for scalable domain adaptation[J]. IEEE Transactions on Knowledge and Data Engineering, 2016, 28(8): 2027–2040. doi: 10.1109/TKDE.2016.2554549

    18. [18]

      PARZEN E. On estimation of a probability density function and mode[J]. The Annals of Mathematical Statistics, 1962, 33(3): 1065–1076. doi: 10.1214/aoms/1177704472

    19. [19]

      DENG Zhaohong, CHUNG F L, and WANG Shitong. FRSDE: Fast reduced set density estimator using minimal enclosing ball approximation[J]. Pattern Recognition, 2008, 41(4): 1363–1372. doi: 10.1016/j.patcog.2007.09.013

    20. [20]

      TOMMASI T, ORABONA F, and CAPUTO B. Learning categories from few examples with multi model knowledge transfer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(5): 928–941. doi: 10.1109/TPAMI.2013.197

    21. [21]

      LACOSTE-JULIEN S, SCHMIDT M, and BACH F. A simpler approach to obtaining an O(1/t) convergence rate for the projected stochastic subgradient method[J]. arXiv:1212.2002, 2012.

    22. [22]

      LONG Mingsheng, WANG Jianmin, DING Guiguang, et al. Transfer learning with graph co-regularization[J]. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(7): 1805–1818. doi: 10.1109/TKDE.2013.97

    23. [23]

      SUYKENS J A K and VANDEWALLE J. Least squares support vector machine classifiers[J]. Neural Processing Letters, 1999, 9(3): 293–300. doi: 10.1023/a:1018628609742

    24. [24]

      BART E and ULLMAN S. Cross-generalization: Learning novel classes from a single example by feature replacement[C]. Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, USA, 2005: 672–679.

    25. [25]

      GU Xiaoqing, CHUNG F L, and WANG Shitong. Bayesian Takagi-Sugeno-Kang fuzzy classifier[J]. IEEE Transactions on Fuzzy Systems, 2017, 25(6): 1655–1671. doi: 10.1109/TFUZZ.2016.2617377

    1. [1]

      张小恒李勇明王品曾孝平颜芳张艳玲承欧梅. 基于语音卷积稀疏迁移学习和并行优选的帕金森病分类算法研究. 电子与信息学报, doi: 10.11999/JEIT180792

    2. [2]

      王斐吴仕超刘少林张亚徽魏颖. 基于脑电信号深度迁移学习的驾驶疲劳检测. 电子与信息学报, doi: 10.11999/JEIT180900

    3. [3]

      王进王科闵子剑孙开伟邓欣. 基于迁移权重的条件对抗领域适应. 电子与信息学报, doi: 10.11999/JEIT190115

    4. [4]

      葛芸马琳江顺亮叶发茂. 基于高层特征图组合及池化的高分辨率遥感图像检索. 电子与信息学报, doi: 10.11999/JEIT190017

    5. [5]

      李世宝王升志刘建航黄庭培张鑫. 基于接收信号强度非齐性分布特征的半监督学习室内定位指纹库构建. 电子与信息学报, doi: 10.11999/JEIT180599

    6. [6]

      唐伦杨恒马润琳陈前斌. 基于5G接入网络的多优先级虚拟网络功能迁移开销与网络能耗联合优化算法. 电子与信息学报, doi: 10.11999/JEIT180906

    7. [7]

      陈曦张坤. 一种基于树增强朴素贝叶斯的分类器学习方法. 电子与信息学报, doi: 10.11999/JEIT180886

    8. [8]

      陆潞高梅国. 分布式阵列雷达基线位置和相位误差的卫星标校方法. 电子与信息学报, doi: 10.11999/JEIT181152

    9. [9]

      赵建高海英胡斌. 基于容错学习的属性基加密方案的具体安全性分析. 电子与信息学报, doi: 10.11999/JEIT180824

    10. [10]

      田子建贺方圆. 一种基于分布式压缩感知的矿井目标指纹数据库建立方法. 电子与信息学报, doi: 10.11999/JEIT180857

    11. [11]

      张艳陈建华唐猛. 多层中继网络上的分布式LT码. 电子与信息学报, doi: 10.11999/JEIT180804

    12. [12]

      胡长雨汪玲朱栋强. 结合字典学习技术的ISAR稀疏成像方法. 电子与信息学报, doi: 10.11999/JEIT180747

    13. [13]

      陈鸿昶谢天高超李邵梅黄瑞阳. 候选标记信息感知的偏标记学习算法. 电子与信息学报, doi: 10.11999/JEIT181059

    14. [14]

      蒋莹王冰切韩俊何翼. 基于分布式压缩感知的宽带欠定信号DOA估计. 电子与信息学报, doi: 10.11999/JEIT180723

    15. [15]

      王宗原周卫东. 学生t量测分布下鲁棒粒子滤波算法的设计与实现. 电子与信息学报, doi: 10.11999/JEIT190144

    16. [16]

      陈莹许潇月. 基于双向参考集矩阵度量学习的行人再识别. 电子与信息学报, doi: 10.11999/JEIT190159

    17. [17]

      李瑞张群苏令华梁佳罗迎. 基于稀疏贝叶斯学习的双基雷达关联成像. 电子与信息学报, doi: 10.11999/JEIT180933

    18. [18]

      兰巨龙于倡和胡宇翔李子勇. 基于深度增强学习的软件定义网络路由优化机制. 电子与信息学报, doi: 10.11999/JEIT180870

    19. [19]

      刘彩霞胡鑫鑫刘树新游伟赵宇. 基于Lowe分类法的5G网络EAP-AKA\begin{document}$ ' $\end{document}协议安全性分析. 电子与信息学报, doi: 10.11999/JEIT190063

    20. [20]

      赵勇胜胡德秀刘智鑫赵拥军赵闯. 基于相邻互相关函数-参数化中心频率-调频率分布-Keystone变换的无源雷达机动目标相参积累方法. 电子与信息学报, doi: 10.11999/JEIT180858

  • 图 1  6种对比算法在文本数据集上的分类精度

    表 1  图像数据集USPS以及MNIST中源域数据与目标域数据的详细设置

    任务源域数据目标域数据
    正类负类正类负类
    1USPS7USPS9MNIST7MNIST9
    2USPS4USPS9MNIST4MNIST9
    3USPS0USPS6MNIST0MNIST6
    下载: 导出CSV

    表 2  文本数据集20-Newsgroups中源域数据与目标域数据的详细设置

    任务源域数据目标域数据
    正类负类正类负类
    1comp.graphicsrec.autoscomp.os.ms-windows.miscrec.motorcycles
    2comp.sys.ibm.pc.hardwarerec.sport.baseballcomp.sys.mac.hardwarerec.sport.hokey
    3sci.crypttalk.politics.gunssci.electronicstalk.politics.mideast
    4sci.medtalk.politics.miscsci.spacetalk.religion.misc
    5rec.autostalk.politics.gunsrec.motorcyclestalk.politics.mideast
    6rec.sport.baseballtalk.politics.miscrec.sport.hokeytalk.religion.misc
    下载: 导出CSV

    表 3  各种算法在图像任务上的分类精度

    任务已标注样本LSSVMCDSVMASVMTrAdaBoostSTMPRIFCATL2
    140.52870.56110.59130.57990.60180.62450.6359
    60.55200.58000.60940.61330.62980.63840.6477
    80.58970.61120.62660.60070.63190.64210.6528
    100.60300.63920.65020.62130.64870.65390.6672
    120.63810.64610.63830.65880.66430.67530.6791
    140.65410.65870.67540.66820.69010.69820.7014
    240.53540.57430.59980.58870.59830.62230.6133
    60.58970.59920.62930.59030.64260.64780.6520
    80.62760.63870.64920.66900.68030.68930.6927
    100.65080.66410.68430.69050.70670.70290.7168
    120.68920.66980.69880.71230.72340.73260.7387
    140.70980.71560.72070.70760.72660.73910.7421
    340.65780.69030.70260.68730.72350.74720.7492
    60.70130.74450.75290.73540.75410.76320.7726
    80.74520.76950.77210.74550.7618077260.7829
    100.77620.78030.77890.78360.79280.79180.8193
    120.79230.79440.80340.79940.82880.81720.8301
    140.82340.82130.81780.81450.83970.82630.8452
    下载: 导出CSV
  • 加载中
图(1)表(3)
计量
  • PDF下载量:  12
  • 文章访问数:  209
  • HTML全文浏览量:  127
  • 引证文献数: 0
文章相关
  • 通讯作者:  杭文龙, wlhang@njtech.edu.cn
  • 收稿日期:  2018-11-20
  • 录用日期:  2019-04-30
  • 网络出版日期:  2019-05-16
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

/

返回文章