高级搜索

基于分类误差一致性准则的自适应知识迁移

梁爽 杭文龙 冯伟 刘学军

引用本文: 梁爽, 杭文龙, 冯伟, 刘学军. 基于分类误差一致性准则的自适应知识迁移[J]. 电子与信息学报, doi: 10.11999/JEIT181054 shu
Citation:  Shuang LIANG, Wenlong HANG, Wei FENG, Xuejun LIU. Adaptive Knowledge Transfer Based on Classification-error Consensus Regularization[J]. Journal of Electronics and Information Technology, doi: 10.11999/JEIT181054 shu

基于分类误差一致性准则的自适应知识迁移

    作者简介: 梁爽: 女,1987年生,讲师,研究方向为机器学习、信号处理;
    杭文龙: 男,1988年生,讲师,研究方向为机器学习、模式识别;
    冯伟: 男,1995年生,硕士生,研究方向机器学习、模式识别;
    刘学军: 男,1970年生,教授,硕士生导师,研究方向为数据挖掘、大数据分布式处理
    通讯作者: 杭文龙,wlhang@njtech.edu.cn
  • 基金项目: 国家自然科学基金(61802177),江苏省高校自然科学研究面上项目(18KJB520020),南京邮电大学引进人才科研启动基金资助项目(NY219034),江苏省重点研发计划(BE2015697)

摘要: 目前大多数迁移学习方法在利用源域数据辅助目标域数据建模时,通常假设源域中的数据均与目标域数据相关。然而在实际应用中,源域中的数据并非都与目标域数据的相关程度一致,若基于上述假设往往会导致负迁移效应。为此,该文首先提出分类误差一致性准则(CCR),对源域与目标域分类误差的概率分布积分平方误差进行最小化度量。此外,该文提出一种基于CCR的自适应知识迁移学习方法(CATL),该方法可以快速地从源域中自动确定出与目标域相关的数据及其权重,以辅助目标域模型的构建,使其能在提高知识迁移效率的同时缓解负迁移学习效应。在真实图像以及文本数据集上的实验结果验证了CATL方法的优势。

English

    1. [1]

      DENG Zhaohong, JIANG Yizhang, CHOI K S, et al. Knowledge-leverage-based TSK fuzzy system modeling[J]. IEEE Transactions on Neural Networks and Learning Systems, 2013, 24(8): 1200–1212. doi: 10.1109/TNNLS.2013.2253617

    2. [2]

      DAI Wenyuan, YANG Qiang, XUE Guirong, et al. Boosting for transfer learning[C]. The 24th International Conference on Machine Learning, Corvalis, USA, 2007: 193–200.

    3. [3]

      JIANG Yizhang, DENG Zhaohong, CHUNG F L, et al. Recognition of epileptic EEG signals using a novel multiview TSK fuzzy system[J]. IEEE Transactions on Fuzzy Systems, 2017, 25(1): 3–20. doi: 10.1109/TFUZZ.2016.2637405

    4. [4]

      ZHUANG Fuzhen, LUO Ping, DU Changying, et al. Triplex transfer learning: Exploiting both shared and distinct concepts for text classification[J]. IEEE Transactions on Cybernetics, 2014, 44(7): 1191–1203. doi: 10.1109/TCYB.2013.2281451

    5. [5]

      PAN S J, NI Xiaochuan, SUN Jiantao, et al. Cross-domain sentiment classification via spectral feature alignment[C]. Proceedings of the 19th International Conference on World Wide Web, Raleigh, USA, 2010: 751–760.

    6. [6]

      ZANG Shaofei, CHENG Yuhu, WANG Xuesong, et al. Semi-supervised transfer discriminant analysis based on cross-domain mean constraint[J]. Artificial Intelligence Review, 2018, 49(4): 581–595. doi: 10.1007/s10462-016-9533-3

    7. [7]

      WANG Guanjin, ZHANG Guangquan, CHOI K S, et al. Deep additive least squares support vector machines for classification with model transfer[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2017. doi: 10.1109/TSMC.2017.2759090

    8. [8]

      YANG Jun, YAN Rong, and HAUPTMANN A G. Adapting SVM classifiers to data with shifted distributions[C]. The Seventh IEEE International Conference on Data Mining Workshops, Omaha, USA, 2007: 69–76.

    9. [9]

      JIANG Yizhang, DENG Zhaohong, CHUNG F L, et al. Realizing two-view TSK fuzzy classification system by using collaborative learning[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2017, 47(1): 145–160. doi: 10.1109/TSMC.2016.2577558

    10. [10]

      CHU Wensheng, DE LA TORRE F, and COHN J F. Selective transfer machine for personalized facial action unit detection[C]. 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, USA, 2013: 3515–3522.

    11. [11]

      GRETTON A, SMOLA A, HUANG Jiayuan, et al. Covariate shift by kernel mean matching[M]. QUIÑONERO-CANDELA J, SUGIYAMA M, SCHWAIGHOFER A, et al. Dataset Shift in Machine Learning. Cambridge, USA: MIT Press, 2009: 131–160.

    12. [12]

      CHENG Yuhu, WANG Xuesong, and CAO Ge. Multi-source tri-training transfer learning[J]. IEICE Transactions on Information and Systems, 2014, E97-D(6): 1668–1672. doi: 10.1587/transinf.e97.d.1668

    13. [13]

      WANG Yunyun, ZHAI Jie, LI Yun, et al. Transfer learning with partial related " instance-feature” knowledge[J]. Neurocomputing, 2018, 310: 115–124. doi: 10.1016/j.neucom.2018.05.029

    14. [14]

      CHEN Minmin, XU Zhixiang, WEINBERGER K Q, et al. Marginalized denoising autoencoders for domain adaptation[C]. The 29th International Conference on Machine Learning, Edinburgh, Scotland, 2012: 1627–1634.

    15. [15]

      ZHOU J T, PAN S J, TSANG I W, et al. Hybrid heterogeneous transfer learning through deep learning[C]. The 28th AAAI Conference on Artificial Intelligence, Québec City, Canada, 2014: 2213–2219.

    16. [16]

      GLOROT X, BORDES A, and BENGIO Y. Domain adaptation for large-scale sentiment classification: A deep learning approach[C]. The 28th International Conference on Machine Learning, Bellevue, Washington, USA, 2011: 513–520.

    17. [17]

      LONG Mingsheng, WANG Jianmin, CAO Yue, et al. Deep learning of transferable representation for scalable domain adaptation[J]. IEEE Transactions on Knowledge and Data Engineering, 2016, 28(8): 2027–2040. doi: 10.1109/TKDE.2016.2554549

    18. [18]

      PARZEN E. On estimation of a probability density function and mode[J]. The Annals of Mathematical Statistics, 1962, 33(3): 1065–1076. doi: 10.1214/aoms/1177704472

    19. [19]

      DENG Zhaohong, CHUNG F L, and WANG Shitong. FRSDE: Fast reduced set density estimator using minimal enclosing ball approximation[J]. Pattern Recognition, 2008, 41(4): 1363–1372. doi: 10.1016/j.patcog.2007.09.013

    20. [20]

      TOMMASI T, ORABONA F, and CAPUTO B. Learning categories from few examples with multi model knowledge transfer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(5): 928–941. doi: 10.1109/TPAMI.2013.197

    21. [21]

      LACOSTE-JULIEN S, SCHMIDT M, and BACH F. A simpler approach to obtaining an O(1/t) convergence rate for the projected stochastic subgradient method[J]. arXiv:1212.2002, 2012.

    22. [22]

      LONG Mingsheng, WANG Jianmin, DING Guiguang, et al. Transfer learning with graph co-regularization[J]. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(7): 1805–1818. doi: 10.1109/TKDE.2013.97

    23. [23]

      SUYKENS J A K and VANDEWALLE J. Least squares support vector machine classifiers[J]. Neural Processing Letters, 1999, 9(3): 293–300. doi: 10.1023/a:1018628609742

    24. [24]

      BART E and ULLMAN S. Cross-generalization: Learning novel classes from a single example by feature replacement[C]. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, USA, 2005: 672–679.

    25. [25]

      GU Xiaoqing, CHUNG F L, and WANG Shitong. Bayesian Takagi-Sugeno-Kang fuzzy classifier[J]. IEEE Transactions on Fuzzy Systems, 2017, 25(6): 1655–1671. doi: 10.1109/TFUZZ.2016.2617377

    1. [1]

      张小恒, 李勇明, 王品, 曾孝平, 颜芳, 张艳玲, 承欧梅. 基于语音卷积稀疏迁移学习和并行优选的帕金森病分类算法研究. 电子与信息学报,

    2. [2]

      王斐, 吴仕超, 刘少林, 张亚徽, 魏颖. 基于脑电信号深度迁移学习的驾驶疲劳检测. 电子与信息学报,

    3. [3]

      王进, 王科, 闵子剑, 孙开伟, 邓欣. 基于迁移权重的条件对抗领域适应. 电子与信息学报,

    4. [4]

      葛芸, 马琳, 江顺亮, 叶发茂. 基于高层特征图组合及池化的高分辨率遥感图像检索. 电子与信息学报,

    5. [5]

      唐伦, 周钰, 谭颀, 魏延南, 陈前斌. 基于强化学习的5G网络切片虚拟网络功能迁移算法. 电子与信息学报,

    6. [6]

      刘浩, 孙晓帆, 张鑫生, 吴乐明, 况奇刚. 面向图像集的置信区间内一致性增强质量评价准则. 电子与信息学报,

    7. [7]

      黄开枝, 潘启润, 袁泉, 游伟. 一种侧信道风险感知的虚拟节点迁移方法. 电子与信息学报,

    8. [8]

      田俊峰, 屈雪晴, 何欣枫, 李珍. 基于哈希图的虚拟机实时迁移方法. 电子与信息学报,

    9. [9]

      李世宝, 王升志, 刘建航, 黄庭培, 张鑫. 基于接收信号强度非齐性分布特征的半监督学习室内定位指纹库构建. 电子与信息学报,

    10. [10]

      董书琴, 张斌. 一种面向流量异常检测的概率流抽样方法. 电子与信息学报,

    11. [11]

      唐伦, 赵培培, 赵国繁, 陈前斌. 基于深度信念网络资源需求预测的虚拟网络功能动态迁移算法. 电子与信息学报,

    12. [12]

      唐伦, 杨恒, 马润琳, 陈前斌. 基于5G接入网络的多优先级虚拟网络功能迁移开销与网络能耗联合优化算法. 电子与信息学报,

    13. [13]

      陈曦, 张坤. 一种基于树增强朴素贝叶斯的分类器学习方法. 电子与信息学报,

    14. [14]

      陆潞, 高梅国. 分布式阵列雷达基线位置和相位误差的卫星标校方法. 电子与信息学报,

    15. [15]

      蒋俊正, 李杨剑, 赵海兵, 欧阳缮. 一种大规模传感器网络节点分布式定位算法. 电子与信息学报,

    16. [16]

      王鑫, 李可, 宁晨, 黄凤辰. 基于深度卷积神经网络和多核学习的遥感图像分类方法. 电子与信息学报,

    17. [17]

      吴超, 李雅倩, 张亚茹, 刘彬. 用于表示级特征融合与分类的相关熵融合极限学习机. 电子与信息学报,

    18. [18]

      刘江义, 王春平. 基于双马尔科夫链的势概率假设密度滤波. 电子与信息学报,

    19. [19]

      赵建, 高海英, 胡斌. 基于容错学习的属性基加密方案的具体安全性分析. 电子与信息学报,

    20. [20]

      田子建, 贺方圆. 一种基于分布式压缩感知的矿井目标指纹数据库建立方法. 电子与信息学报,

  • 图 1  6种对比算法在文本数据集上的分类精度

    表 1  图像数据集USPS以及MNIST中源域数据与目标域数据的详细设置

    任务源域数据目标域数据
    正类负类正类负类
    1USPS7USPS9MNIST7MNIST9
    2USPS4USPS9MNIST4MNIST9
    3USPS0USPS6MNIST0MNIST6
    下载: 导出CSV

    表 2  文本数据集20-Newsgroups中源域数据与目标域数据的详细设置

    任务源域数据目标域数据
    正类负类正类负类
    1comp.graphicsrec.autoscomp.os.ms-windows.miscrec.motorcycles
    2comp.sys.ibm.pc.hardwarerec.sport.baseballcomp.sys.mac.hardwarerec.sport.hokey
    3sci.crypttalk.politics.gunssci.electronicstalk.politics.mideast
    4sci.medtalk.politics.miscsci.spacetalk.religion.misc
    5rec.autostalk.politics.gunsrec.motorcyclestalk.politics.mideast
    6rec.sport.baseballtalk.politics.miscrec.sport.hokeytalk.religion.misc
    下载: 导出CSV

    表 3  各种算法在图像任务上的分类精度

    任务已标注样本LSSVMCDSVMASVMTrAdaBoostSTMPRIFCATL2
    140.52870.56110.59130.57990.60180.62450.6359
    60.55200.58000.60940.61330.62980.63840.6477
    80.58970.61120.62660.60070.63190.64210.6528
    100.60300.63920.65020.62130.64870.65390.6672
    120.63810.64610.63830.65880.66430.67530.6791
    140.65410.65870.67540.66820.69010.69820.7014
    240.53540.57430.59980.58870.59830.62230.6133
    60.58970.59920.62930.59030.64260.64780.6520
    80.62760.63870.64920.66900.68030.68930.6927
    100.65080.66410.68430.69050.70670.70290.7168
    120.68920.66980.69880.71230.72340.73260.7387
    140.70980.71560.72070.70760.72660.73910.7421
    340.65780.69030.70260.68730.72350.74720.7492
    60.70130.74450.75290.73540.75410.76320.7726
    80.74520.76950.77210.74550.7618077260.7829
    100.77620.78030.77890.78360.79280.79180.8193
    120.79230.79440.80340.79940.82880.81720.8301
    140.82340.82130.81780.81450.83970.82630.8452
    下载: 导出CSV
  • 加载中
图(1)表(3)
计量
  • PDF下载量:  14
  • 文章访问数:  354
  • HTML全文浏览量:  246
文章相关
  • 通讯作者:  杭文龙, wlhang@njtech.edu.cn
  • 收稿日期:  2018-11-20
  • 录用日期:  2019-04-30
  • 网络出版日期:  2019-05-16
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

/

返回文章