高级搜索

基于分类误差一致性准则的自适应知识迁移

梁爽 杭文龙 冯伟 刘学军

引用本文: 梁爽, 杭文龙, 冯伟, 刘学军. 基于分类误差一致性准则的自适应知识迁移[J]. 电子与信息学报, 2019, 41(11): 2736-2743. doi: 10.11999/JEIT181054 shu
Citation:  Shuang LIANG, Wenlong HANG, Wei FENG, Xuejun LIU. Adaptive Knowledge Transfer Based on Classification-error Consensus Regularization[J]. Journal of Electronics and Information Technology, 2019, 41(11): 2736-2743. doi: 10.11999/JEIT181054 shu

基于分类误差一致性准则的自适应知识迁移

    作者简介: 梁爽: 女,1987年生,讲师,研究方向为机器学习、信号处理;
    杭文龙: 男,1988年生,讲师,研究方向为机器学习、模式识别;
    冯伟: 男,1995年生,硕士生,研究方向机器学习、模式识别;
    刘学军: 男,1970年生,教授,硕士生导师,研究方向为数据挖掘、大数据分布式处理
    通讯作者: 杭文龙,wlhang@njtech.edu.cn
  • 基金项目: 国家自然科学基金(61802177),江苏省高校自然科学研究面上项目(18KJB520020),南京邮电大学引进人才科研启动基金资助项目(NY219034),江苏省重点研发计划(BE2015697)

摘要: 目前大多数迁移学习方法在利用源域数据辅助目标域数据建模时,通常假设源域中的数据均与目标域数据相关。然而在实际应用中,源域中的数据并非都与目标域数据的相关程度一致,若基于上述假设往往会导致负迁移效应。为此,该文首先提出分类误差一致性准则(CCR),对源域与目标域分类误差的概率分布积分平方误差进行最小化度量。此外,该文提出一种基于CCR的自适应知识迁移学习方法(CATL),该方法可以快速地从源域中自动确定出与目标域相关的数据及其权重,以辅助目标域模型的构建,使其能在提高知识迁移效率的同时缓解负迁移学习效应。在真实图像以及文本数据集上的实验结果验证了CATL方法的优势。

English

    1. [1]

      DENG Zhaohong, JIANG Yizhang, CHOI K S, et al. Knowledge-leverage-based TSK fuzzy system modeling[J]. IEEE Transactions on Neural Networks and Learning Systems, 2013, 24(8): 1200–1212. doi: 10.1109/TNNLS.2013.2253617

    2. [2]

      DAI Wenyuan, YANG Qiang, XUE Guirong, et al. Boosting for transfer learning[C]. The 24th International Conference on Machine Learning, Corvalis, USA, 2007: 193–200.

    3. [3]

      JIANG Yizhang, DENG Zhaohong, CHUNG F L, et al. Recognition of epileptic EEG signals using a novel multiview TSK fuzzy system[J]. IEEE Transactions on Fuzzy Systems, 2017, 25(1): 3–20. doi: 10.1109/TFUZZ.2016.2637405

    4. [4]

      ZHUANG Fuzhen, LUO Ping, DU Changying, et al. Triplex transfer learning: Exploiting both shared and distinct concepts for text classification[J]. IEEE Transactions on Cybernetics, 2014, 44(7): 1191–1203. doi: 10.1109/TCYB.2013.2281451

    5. [5]

      PAN S J, NI Xiaochuan, SUN Jiantao, et al. Cross-domain sentiment classification via spectral feature alignment[C]. Proceedings of the 19th International Conference on World Wide Web, Raleigh, USA, 2010: 751–760.

    6. [6]

      ZANG Shaofei, CHENG Yuhu, WANG Xuesong, et al. Semi-supervised transfer discriminant analysis based on cross-domain mean constraint[J]. Artificial Intelligence Review, 2018, 49(4): 581–595. doi: 10.1007/s10462-016-9533-3

    7. [7]

      WANG Guanjin, ZHANG Guangquan, CHOI K S, et al. Deep additive least squares support vector machines for classification with model transfer[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2019, 49(7): 1527–1540. doi: 10.1109/TSMC.2017.2759090

    8. [8]

      YANG Jun, YAN Rong, and HAUPTMANN A G. Adapting SVM classifiers to data with shifted distributions[C]. The Seventh IEEE International Conference on Data Mining Workshops, Omaha, USA, 2007: 69–76.

    9. [9]

      JIANG Yizhang, DENG Zhaohong, CHUNG F L, et al. Realizing two-view TSK fuzzy classification system by using collaborative learning[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2017, 47(1): 145–160. doi: 10.1109/TSMC.2016.2577558

    10. [10]

      CHU Wensheng, DE LA TORRE F, and COHN J F. Selective transfer machine for personalized facial action unit detection[C]. 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, USA, 2013: 3515–3522.

    11. [11]

      GRETTON A, SMOLA A, HUANG Jiayuan, et al. Covariate Shift by Kernel Mean Matching[M]. QUIÑONERO-CANDELA J, SUGIYAMA M, SCHWAIGHOFER A, et al. Dataset Shift in Machine Learning. Cambridge, USA: MIT Press, 2009: 131–160.

    12. [12]

      CHENG Yuhu, WANG Xuesong, and CAO Ge. Multi-source tri-training transfer learning[J]. IEICE Transactions on Information and Systems, 2014, E97-D(6): 1668–1672. doi: 10.1587/transinf.e97.d.1668

    13. [13]

      WANG Yunyun, ZHAI Jie, LI Yun, et al. Transfer learning with partial related " instance-feature” knowledge[J]. Neurocomputing, 2018, 310: 115–124. doi: 10.1016/j.neucom.2018.05.029

    14. [14]

      CHEN Minmin, XU Zhixiang, WEINBERGER K Q, et al. Marginalized denoising autoencoders for domain adaptation[C]. The 29th International Conference on Machine Learning, Edinburgh, Scotland, 2012: 1627–1634.

    15. [15]

      ZHOU J T, PAN S J, TSANG I W, et al. Hybrid heterogeneous transfer learning through deep learning[C]. The 28th AAAI Conference on Artificial Intelligence, Québec City, Canada, 2014: 2213–2219.

    16. [16]

      GLOROT X, BORDES A, and BENGIO Y. Domain adaptation for large-scale sentiment classification: A deep learning approach[C]. The 28th International Conference on Machine Learning, Bellevue, Washington, USA, 2011: 513–520.

    17. [17]

      LONG Mingsheng, WANG Jianmin, CAO Yue, et al. Deep learning of transferable representation for scalable domain adaptation[J]. IEEE Transactions on Knowledge and Data Engineering, 2016, 28(8): 2027–2040. doi: 10.1109/TKDE.2016.2554549

    18. [18]

      PARZEN E. On estimation of a probability density function and mode[J]. The Annals of Mathematical Statistics, 1962, 33(3): 1065–1076. doi: 10.1214/aoms/1177704472

    19. [19]

      DENG Zhaohong, CHUNG F L, and WANG Shitong. FRSDE: Fast reduced set density estimator using minimal enclosing ball approximation[J]. Pattern Recognition, 2008, 41(4): 1363–1372. doi: 10.1016/j.patcog.2007.09.013

    20. [20]

      TOMMASI T, ORABONA F, and CAPUTO B. Learning categories from few examples with multi model knowledge transfer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(5): 928–941. doi: 10.1109/TPAMI.2013.197

    21. [21]

      LACOSTE-JULIEN S, SCHMIDT M, and BACH F. A simpler approach to obtaining an O(1/t) convergence rate for the projected stochastic subgradient method[J]. arXiv:1212.2002, 2012.

    22. [22]

      LONG Mingsheng, WANG Jianmin, DING Guiguang, et al. Transfer learning with graph co-regularization[J]. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(7): 1805–1818. doi: 10.1109/TKDE.2013.97

    23. [23]

      SUYKENS J A K and VANDEWALLE J. Least squares support vector machine classifiers[J]. Neural Processing Letters, 1999, 9(3): 293–300. doi: 10.1023/a:1018628609742

    24. [24]

      BART E and ULLMAN S. Cross-generalization: Learning novel classes from a single example by feature replacement[C]. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, USA, 2005: 672–679.

    25. [25]

      GU Xiaoqing, CHUNG F L, and WANG Shitong. Bayesian Takagi-Sugeno-Kang fuzzy classifier[J]. IEEE Transactions on Fuzzy Systems, 2017, 25(6): 1655–1671. doi: 10.1109/TFUZZ.2016.2617377

    1. [1]

      周牧, 李垚鲆, 谢良波, 蒲巧林, 田增山. 基于多核最大均值差异迁移学习的WLAN室内入侵检测方法. 电子与信息学报, 2020, 42(5): 1149-1157.

    2. [2]

      陈前斌, 管令进, 李子煜, 王兆堃, 杨恒, 唐伦. 基于深度强化学习的异构云无线接入网自适应无线资源分配算法. 电子与信息学报, 2020, 42(6): 1468-1477.

    3. [3]

      陈卓, 冯钢, 何颖, 周杨. 运营商网络中基于深度强化学习的服务功能链迁移机制. 电子与信息学报, 2020, 42(0): 1-7.

    4. [4]

      晋守博, 魏章志, 李耀红. 基于大通讯时滞的2阶多智能体系统的一致性分析. 电子与信息学报, 2020, 42(0): 1-6.

    5. [5]

      张惊雷, 厚雅伟. 基于改进循环生成式对抗网络的图像风格迁移. 电子与信息学报, 2020, 42(5): 1216-1222.

    6. [6]

      黄静琪, 胡琛, 孙山鹏, 高翔, 何兵. 一种基于异步传感器网络的空间目标分布式跟踪方法. 电子与信息学报, 2020, 42(5): 1132-1139.

    7. [7]

      王一宾, 裴根生, 程玉胜. 基于标记密度分类间隔面的组类属属性学习. 电子与信息学报, 2020, 42(5): 1179-1187.

    8. [8]

      夏士超, 姚枝秀, 鲜永菊, 李云. 移动边缘计算中分布式异构任务卸载算法. 电子与信息学报, 2020, 41(0): 1-8.

    9. [9]

      张坤, 水鹏朗, 王光辉. 相参雷达K分布海杂波背景下非相干积累恒虚警检测方法. 电子与信息学报, 2020, 41(0): 1-9.

    10. [10]

      张斌, 吴浩明. 一种面向连接的快速多维包分类算法. 电子与信息学报, 2020, 42(6): 1526-1533.

    11. [11]

      蒋瀚, 刘怡然, 宋祥福, 王皓, 郑志华, 徐秋亮. 隐私保护机器学习的密码学方法. 电子与信息学报, 2020, 42(5): 1068-1078.

    12. [12]

      张文明, 姚振飞, 高雅昆, 李海滨. 一种平衡准确性以及高效性的显著性目标检测深度卷积网络模型. 电子与信息学报, 2020, 42(5): 1201-1208.

    13. [13]

      刘坤, 吴建新, 甄杰, 王彤. 基于阵列天线和稀疏贝叶斯学习的室内定位方法. 电子与信息学报, 2020, 42(5): 1158-1164.

    14. [14]

      李骜, 刘鑫, 陈德运, 张英涛, 孙广路. 基于低秩表示的鲁棒判别特征子空间学习模型. 电子与信息学报, 2020, 42(5): 1223-1230.

    15. [15]

      唐伦, 曹睿, 廖皓, 王兆堃. 基于深度强化学习的服务功能链可靠部署算法. 电子与信息学报, 2020, 42(0): 1-8.

    16. [16]

      魏宏安, 吴小清, 张昂. 基于能量误差的人体有限元模型网格剖分优化研究. 电子与信息学报, 2020, 42(0): 1-6.

    17. [17]

      赵国繁, 唐伦, 胡彦娟, 赵培培, 陈前斌. 面向可靠性的5G网络切片重构及映射算法. 电子与信息学报, 2020, 42(6): 1478-1485.

    18. [18]

      柳娟, 谢文彬, 汪改英, 汤敏丽. 基于DNA和限制性核酸内切酶的基本逻辑门设计. 电子与信息学报, 2020, 42(6): 1332-1339.

    19. [19]

      刘焕淋, 杜理想, 陈勇, 王展鹏. 基于灾难预测多区域故障的虚拟光网络生存性映射. 电子与信息学报, 2020, 42(7): 1710-1717.

    20. [20]

      王璐慧, 王越, 钱梦瑶, 董亚非. 基于氧化石墨烯与金属离子的逻辑模型设计与可控性验证. 电子与信息学报, 2020, 42(6): 1410-1419.

  • 图 1  6种对比算法在文本数据集上的分类精度

    表 1  图像数据集USPS及MNIST中源域数据与目标域数据的详细设置

    任务源域数据目标域数据
    正类负类正类负类
    1USPS7USPS9MNIST7MNIST9
    2USPS4USPS9MNIST4MNIST9
    3USPS0USPS6MNIST0MNIST6
    下载: 导出CSV

    表 2  文本数据集20-Newsgroups中源域数据与目标域数据的详细设置

    任务源域数据目标域数据
    正类负类正类负类
    1comp.graphicsrec.autoscomp.os.ms-windows.miscrec.motorcycles
    2comp.sys.ibm.pc.hardwarerec.sport.baseballcomp.sys.mac.hardwarerec.sport.hokey
    3sci.crypttalk.politics.gunssci.electronicstalk.politics.mideast
    4sci.medtalk.politics.miscsci.spacetalk.religion.misc
    5rec.autostalk.politics.gunsrec.motorcyclestalk.politics.mideast
    6rec.sport.baseballtalk.politics.miscrec.sport.hokeytalk.religion.misc
    下载: 导出CSV

    表 3  各种算法在图像任务上的分类精度

    任务已标注样本LSSVMCDSVMASVMTrAdaBoostSTMPRIFCATL2
    140.52870.56110.59130.57990.60180.62450.6359
    60.55200.58000.60940.61330.62980.63840.6477
    80.58970.61120.62660.60070.63190.64210.6528
    100.60300.63920.65020.62130.64870.65390.6672
    120.63810.64610.63830.65880.66430.67530.6791
    140.65410.65870.67540.66820.69010.69820.7014
    240.53540.57430.59980.58870.59830.62230.6133
    60.58970.59920.62930.59030.64260.64780.6520
    80.62760.63870.64920.66900.68030.68930.6927
    100.65080.66410.68430.69050.70670.70290.7168
    120.68920.66980.69880.71230.72340.73260.7387
    140.70980.71560.72070.70760.72660.73910.7421
    340.65780.69030.70260.68730.72350.74720.7492
    60.70130.74450.75290.73540.75410.76320.7726
    80.74520.76950.77210.74550.7618077260.7829
    100.77620.78030.77890.78360.79280.79180.8193
    120.79230.79440.80340.79940.82880.81720.8301
    140.82340.82130.81780.81450.83970.82630.8452
    下载: 导出CSV
  • 加载中
图(1)表(3)
计量
  • PDF下载量:  28
  • 文章访问数:  1145
  • HTML全文浏览量:  696
文章相关
  • 通讯作者:  杭文龙, wlhang@njtech.edu.cn
  • 收稿日期:  2018-11-20
  • 录用日期:  2019-04-30
  • 网络出版日期:  2019-05-16
  • 刊出日期:  2019-11-01
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

/

返回文章