高级搜索

基于高层特征图组合及池化的高分辨率遥感图像检索

葛芸 马琳 江顺亮 叶发茂

引用本文: 葛芸, 马琳, 江顺亮, 叶发茂. 基于高层特征图组合及池化的高分辨率遥感图像检索[J]. 电子与信息学报, doi: 10.11999/JEIT190017 shu
Citation:  Yun GE, Lin MA, Shunliang JIANG, Famao YE. The Combination and Pooling Based on High-level Feature Map for High-resolution Remote Sensing Image Retrieval[J]. Journal of Electronics and Information Technology, doi: 10.11999/JEIT190017 shu

基于高层特征图组合及池化的高分辨率遥感图像检索

    作者简介: 葛芸: 女,1983年生,博士,讲师,研究方向为遥感图像处理与机器学习;
    马琳: 女,1996年生,硕士生,研究方向为遥感图像处理与机器学习;
    江顺亮: 男,1966年生,博士,教授,博士生导师,研究方向为算法设计与分析、计算机模拟与仿真、机器视觉;
    叶发茂: 男,1978年生,博士,副教授,研究方向为遥感图像处理、计算机图形学、机器学习;
    通讯作者: 葛芸, geyun@nchu.edu.cn
  • 基金项目: 国家自然科学基金(41801288, 41261091, 61662044, 61663031, 61762067)

摘要: 高分辨率遥感图像内容复杂,提取特征来准确地表达图像内容是提高检索性能的关键。卷积神经网络(CNN)迁移学习能力强,其高层特征能够有效迁移到高分辨率遥感图像中。为了充分利用高层特征的优点,该文提出一种基于高层特征图组合及池化的方法来融合不同CNN中的高层特征。首先将高层特征作为特殊的卷积层特征,进而在不同输入尺寸下保留高层输出的特征图;然后将不同高层输出的特征图组合成一个更大的特征图,以综合不同CNN学习到的特征;接着采用最大池化的方法对组合特征图进行压缩,提取特征图中的显著特征;最后,采用主成分分析(PCA)来降低显著特征的冗余度。实验结果表明,与现有检索方法相比,该方法提取的特征在检索效率和准确率上都有优势。

English

    1. [1]

      DEMIR B and BRUZZONE L. A novel active learning method in relevance feedback for content-based remote sensing image retrieval[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(5): 2323–2334. doi: 10.1109/TGRS.2014.2358804

    2. [2]

      ÖZKAN S, ATEŞ T, TOLA E, et al. Performance analysis of state-of-the-art representation methods for geographical image retrieval and categorization[J]. IEEE Geoscience and Remote Sensing Letters, 2014, 11(11): 1996–2000. doi: 10.1109/LGRS.2014.2316143

    3. [3]

      陆丽珍, 刘仁义, 刘南. 一种融合颜色和纹理特征的遥感图像检索方法[J]. 中国图象图形学报, 2004, 9(3): 328–333. doi: 10.3969/j.issn.1006-8961.2004.03.013
      LU Lizhen, LIU Renyi, and LIU Nan. Remote sensing image retrieval using color and texture fused features[J]. Journal of Image and Graphics, 2004, 9(3): 328–333. doi: 10.3969/j.issn.1006-8961.2004.03.013

    4. [4]

      WANG Yuebin, ZHANG Liqiang, TONG Xiaohua, et al. A three-layered graph-based learning approach for remote sensing image retrieval[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(10): 6020–6034. doi: 10.1109/TGRS.2016.2579648

    5. [5]

      郭智, 宋萍, 张义, 等. 基于深度卷积神经网络的遥感图像飞机目标检测方法[J]. 电子与信息学报, 2018, 40(11): 2684–2690. doi: 10.11999/JEIT180117
      GUO Zhi, SONG Ping, ZHANG Yi, et al. Aircraft detection method based on deep convolutional neural network for remote sensing images[J]. Journal of Electronics &Information Technology, 2018, 40(11): 2684–2690. doi: 10.11999/JEIT180117

    6. [6]

      YE Famao, SU Yanfei, XIAO Hui, et al. Remote sensing image registration using convolutional neural network features[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(2): 232–236. doi: 10.1109/LGRS.2017.2781741

    7. [7]

      KRIZHEVSKY A, SUTSKEVER I, and HINTON G E. ImageNet classification with deep convolutional neural networks[C]. Proceedings of the 25th International Conference on Neural Information Processing Systems, Nevada, USA, 2012: 1097–1105.

    8. [8]

      CHATFIELD K, SIMONYAN K, VEDALDI A, et al. Return of the devil in the details: Delving deep into convolutional networks[C]. Proceedings of the 25th British Machine Vision Conference, Nottingham, UK, 2014.

    9. [9]

      SIMONYAN K and ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]. Proceedings of the 3rd International Conference on Learning Representations, San Diego, USA, 2015.

    10. [10]

      SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]. Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015: 1–9.

    11. [11]

      HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778.

    12. [12]

      CASTELLUCCIO M, POGGI G, SANSONE C, et al. Land use classification in remote sensing images by convolutional neural networks[J]. Acta Ecologica Sinica, 2015, 28(2): 627–635.

    13. [13]

      ALIAS B, KARTHIKA R, and PARAMESWARAN L. Content based image retrieval of remote sensing images using deep learning with different distance measures[J]. Journal of Advanced Research in Dynamical and Control Systems, 2018, 10(3): 664–674.

    14. [14]

      NAPOLETANO P. Visual descriptors for content-based retrieval of remote-sensing Images[J]. International Journal of Remote Sensing, 2018, 39(5): 1343–1376. doi: 10.1080/01431161.2017.1399472

    15. [15]

      ZHOW Weixun, NEWSAM S, LI Congmin, et al. Learning low dimensional convolutional neural networks for high-resolution remote sensing image retrieval[J]. Remote Sensing, 2017, 9(5): 489. doi: 10.3390/rs9050489

    16. [16]

      HU Fan, TONG Xinyi, XIA Guisong, et al. Delving into deep representations for remote sensing image retrieval[C]. Proceedings of the IEEE 13th International Conference on Signal Processing, Chengdu, China, 2016: 198–203.

    17. [17]

      SHELHAMER E, LONG J, and DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640–651. doi: 10.1109/TPAMI.2016.2572683

    18. [18]

      VEDALDI A and LENC K. MatConvNet: Convolutional neural networks for MATLAB[C]. Proceedings of the 23rd ACM International Conference on Multimedia, Brisbane, Australia, 2015: 689–692.

    19. [19]

      HU Fan, XIA Guisong, HU Jingwen, et al. Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery[J]. Remote Sensing, 2015, 7(11): 14680–14707. doi: 10.3390/rs71114680

    20. [20]

      ZOU Qin, NI Lihao, ZHANG Tong, et al. Deep learning based feature selection for remote sensing scene classification[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(11): 2321–2325. doi: 10.1109/LGRS.2015.2475299

    1. [1]

      张小恒李勇明王品曾孝平颜芳张艳玲承欧梅. 基于语音卷积稀疏迁移学习和并行优选的帕金森病分类算法研究. 电子与信息学报, doi: 10.11999/JEIT180792

    2. [2]

      王斐吴仕超刘少林张亚徽魏颖. 基于脑电信号深度迁移学习的驾驶疲劳检测. 电子与信息学报, doi: 10.11999/JEIT180900

    3. [3]

      王肖锋孙明月葛为民. 基于图像协方差无关的增量特征提取方法研究. 电子与信息学报, doi: 10.11999/JEIT181138

    4. [4]

      唐伦魏延南马润琳贺小雨陈前斌. 虚拟化云无线接入网络下基于在线学习的网络切片虚拟资源分配算法. 电子与信息学报, doi: 10.11999/JEIT180771

    5. [5]

      陈光武程鉴皓杨菊花刘昊张琳婧. 基于改进神经网络增强自适应UKF的组合导航系统. 电子与信息学报, doi: 10.11999/JEIT181171

    6. [6]

      李世宝王升志刘建航黄庭培张鑫. 基于接收信号强度非齐性分布特征的半监督学习室内定位指纹库构建. 电子与信息学报, doi: 10.11999/JEIT180599

    7. [7]

      胡长雨汪玲朱栋强. 结合字典学习技术的ISAR稀疏成像方法. 电子与信息学报, doi: 10.11999/JEIT180747

    8. [8]

      杨文哲杨宏雷王学运张升康赵环杨军冯克明. 高精度光纤时间频率一体化传递. 电子与信息学报, doi: 10.11999/JEIT180807

    9. [9]

      臧鸿雁黄慧芳柴宏玉. 一类2次多项式混沌系统的均匀化方法研究. 电子与信息学报, doi: 10.11999/JEIT180735

    10. [10]

      陈莹许潇月. 基于双向参考集矩阵度量学习的行人再识别. 电子与信息学报, doi: 10.11999/JEIT190159

    11. [11]

      涂开辉黄志洪侯峥嵘杨海钢. 基于配置模式匹配和层次化映射结构的高效FPGA码流生成系统研究. 电子与信息学报, doi: 10.11999/JEIT190143

    12. [12]

      杨宏宇王峰岩. 基于深度卷积神经网络的气象雷达噪声图像语义分割方法. 电子与信息学报, doi: 10.11999/JEIT190098

    13. [13]

      周洋吴佳忆陆宇殷海兵. 面向三维高效视频编码的深度图错误隐藏. 电子与信息学报, doi: 10.11999/JEIT180926

    14. [14]

      丛雯珊余岚沃江海. 基于粒子群算法的宽带真延时方向图栅瓣抑制方法. 电子与信息学报, doi: 10.11999/JEIT180719

    15. [15]

      陈鸿昶明拓思宇刘树新高超. 基于整数线性规划重构抽象语义图结构的语义摘要算法. 电子与信息学报, doi: 10.11999/JEIT180720

    16. [16]

      潘洁王帅李道京卢晓春. 基于方向图和多普勒相关系数的天基阵列SAR通道相位误差补偿方法. 电子与信息学报, doi: 10.11999/JEIT181061

    17. [17]

      潘一苇彭华李天昀王文雅. 一种新的时分多址信号射频特征及其在特定辐射源识别中的应用. 电子与信息学报, doi: 10.11999/JEIT190163

  • 图 1  图像检索流程

    图 2  融合高层特征

    图 3  UC-Merced中不同特征检索结果比较

    图 4  WHU-RS中不同特征检索结果比较

    图 5  不同CoP特征PCA降维结果

    表 1  不同输入图像尺寸下高层CNN特征的输出值

    输入图像尺寸fcG-pool5R-pool5
    默认尺寸1×1×40961×1×10241×1×2048
    256×256×3(UC-Merced)2×2×40962×2×10242×2×2048
    600×600×3(WHU-RS)13×13×409612×12×102413×13×2048
    下载: 导出CSV

    表 2  UC-Merced中特征的相关系数

    特征AM1619G
    M–0.0037
    160.00060.0028
    19–0.00230.00530.4817
    G0.0012–0.0063–0.0086–0.0100
    R–0.01000.0008–0.0060–0.00210.1175
    下载: 导出CSV

    表 3  WHU-RS中特征的相关系数

    特征AM1619G
    M–0.0080
    16–0.00090.0027
    190.00010.00510.4762
    G–0.0024–0.0038–0.0110–0.0093
    R–0.00450.0084–0.0069–0.00220.1138
    下载: 导出CSV

    表 4  不同输入尺寸CoP特征检索结果比较

    数据集特征默认尺寸原始尺寸
    ANMRRmAPANMRRmAP
    UC-MercedCoP(16_G)0.28980.64110.28800.6446
    CoP(16_G_M)0.28340.64850.28320.6504
    CoP(16_G_M_19)0.28340.64960.28050.6544
    WHU-RSCoP(A_16)0.20070.74660.23300.7116
    CoP(A_16_G)0.18910.75820.23190.7125
    CoP(A_16_G_19)0.18750.76100.23180.7124
    下载: 导出CSV

    表 5  UC-Merced中微调融合特征检索结果比较

    数据集特征默认尺寸原始尺寸
    ANMRRmAPANMRRmAP
    UC-MercedCoP(16_G)-FT0.27380.66020.27770.6566
    CoP(16_G_M)-FT0.26420.67160.26780.6683
    CoP(16_G_M_19)-FT0.26040.67670.25610.6822
    WHU-RSCoP(A_16)-FT0.17230.78090.19750.7501
    CoP(A_16_G)-FT0.15820.79710.19240.7559
    CoP(A_16_G_19)-FT0.15190.80480.18790.7615
    下载: 导出CSV

    表 6  UC-Merced中融合特征与其他特征检索结果比较

    特征ANMRR维数
    浅层特征VLAD[2]0.460416384
    3层图[4]0.4317
    CNN特征VGGM-fc[14]0.37804096
    VGGM-conv5-IFK[15]0.4580102400
    VGG16-fc[15]0.39404096
    VGG16-conv5-IFK[15]0.4070102400
    LDCNN[15]0.439030
    GoogLeNet-MultiPatch-FT[16]0.31401024
    GoogLeNet-MultiPatch-FT-PCA[16]0.285032
    CoP(16_G)0.28804096
    CoP(16_G_M_19)0.28054096
    CoP(16_G_M_19)-FT0.25614096
    CoP(16_G_M_19)-PCA0.257732
    下载: 导出CSV
  • 加载中
图(5)表(6)
计量
  • PDF下载量:  5
  • 文章访问数:  95
  • HTML全文浏览量:  65
  • 引证文献数: 0
文章相关
  • 通讯作者:  葛芸, geyun@nchu.edu.cn
  • 收稿日期:  2019-01-09
  • 网络出版日期:  2019-06-25
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

/

返回文章