高级搜索

面向三维高效视频编码的深度图错误隐藏

周洋 吴佳忆 陆宇 殷海兵

引用本文: 周洋, 吴佳忆, 陆宇, 殷海兵. 面向三维高效视频编码的深度图错误隐藏[J]. 电子与信息学报, 2019, 41(11): 2760-2767. doi: 10.11999/JEIT180926 shu
Citation:  Yang ZHOU, Jiayi WU, Yu LU, Haibing YIN. Depth Map Error Concealment for 3D High Efficiency Video Coding[J]. Journal of Electronics and Information Technology, 2019, 41(11): 2760-2767. doi: 10.11999/JEIT180926 shu

面向三维高效视频编码的深度图错误隐藏

    作者简介: 周洋: 男,1979年生,副教授,硕士生导师,研究方向为视频分析和三维视频编解码;
    吴佳忆: 男,1994年生,硕士生,研究方向为视频差错控制技术;
    陆宇: 男,1977年生,讲师,研究方向为视频压缩技术;
    殷海兵: 男,1974年生,教授,研究方向为视频编码与压缩芯片设计
    通讯作者: 周洋,zhouyang_hz@126.com
  • 基金项目: 浙江省自然科学基金(LY17F020027),国家自然科学基金(61401132, 61572449)

摘要: 基于多视点视频序列视点内、视点间存在的相关性,并结合视点间运动矢量共享技术,该文提出一种面向3维高效视频编码中深度序列传输丢包的错误隐藏算法。首先,根据3D高效视频编码(3D-HEVC)的分层B帧预测(HBP)结构和深度图纹理特征,将深度图丢失块分成运动块和静止块;然后,对于受损运动块,使用结合纹理结构的外边界匹配准则来选择相对最优的运动/视差矢量进行基于位移矢量补偿的错误掩盖,而对受损静止块采用参考帧直接拷贝进行快速错误隐藏;最后,使用参考帧拆分重组来获取新的运动/视差补偿块对修复质量较差的重建块进行质量提升。实验结果表明:相较于近年提出的对比算法,该文算法隐藏后的深度帧平均峰值信噪比(PSNR)能提升0.25~2.03 dB,结构相似度测量值(SSIM)能提升0.001~0.006,且修复区域的主观视觉质量与原始深度图更接近。

English

    1. [1]

      LI Yufeng and CHEN Ruining. Motion vector recovery for video error concealment based on plane fitting[J]. Computer Engineering &Applications, 2018, 54(2): 182–187. doi: 10.3778/j.issn.1002-8331.1608-0085

    2. [2]

      XU Dawen and WANG Rangding. Two-dimensional reversible data hiding-based approach for intra-frame error concealment in H.264/AVC[J]. Journal of Visual Communication& Image Representation, 2016, 47(2): 369–379. doi: 10.1016/j.jvcir.2013.12.008

    3. [3]

      ZABIHI S M, GHANEI Y H, and MEHRSHAD N. Adaptive temporal error concealment method based on the MB behavior estimation in the video[C]. The 7th International Conference on Computer and Knowledge Engineering. Mashhad, Iran, 2017: 193–198.

    4. [4]

      EL-SHAFAI W, EL-RABAIE S, EL-HALAWANY M M, et al. Recursive Bayesian filtering-based error concealment scheme for 3D video communication over severely lossy wireless channels[J]. Circuits Systems & Signal Processing, 2018(20): 1–32. doi: 10.1007/s00034-018-0786-8

    5. [5]

      周洋, 蒋刚毅, 郁梅, 等. 面向HBP编码格式的立体视频B帧整帧丢失分层错误隐藏算法[J]. 电子与信息学报, 2014, 36(2): 377–383. doi: 10.3724/SP.J.1146.2013.00528
      ZHOU Yang, JIANG Gangyi, YU Mei, et al. A hierarchical error concealment algorithm for entire B-frame loss in stereoscopic video coding with hierarchical B pictures[J]. Journal of Electronics &Information Technology, 2014, 36(2): 377–383. doi: 10.3724/SP.J.1146.2013.00528

    6. [6]

      SONG K, CHUNG T, OH Y, et al. Error concealment of multi-view video sequences using inter-view and intra-view correlations[J]. Journal of Visual Communication & Image Representation, 2009, 20(4): 281–292. doi: 10.1016/j.jvcir.2009.02.002

    7. [7]

      PENG Yantsung and COSMAN P C. Weighted boundary matching error concealment for HEVC using block partition decisions[C]. The 48th Asilomar Conference on Signals, Systems & Computers, Grove, USA, 2014: 921–925. doi: 10.1109/ACSSC.2014.7094587.

    8. [8]

      KHATTAK S, MAUGEY T, HAMZAOUI R, et al. Temporal and inter-view consistent error concealment technique for multiview plus depth video[J]. IEEE Transactions on Circuits & Systems for Video Technology, 2016, 26(5): 829–840. doi: 10.1109/TCSVT.2015.2418631

    9. [9]

      LIU Shujie, CHEN Ying, WANG Yekui, et al. Frame loss error concealment for multiview video coding[C]. IEEE International Symposium on Circuits and Systems (ISCAS 2008), Washington, USA, 2008: 3470–3473.

    10. [10]

      ZHOU Yuan, XIANG Wei, and WANG Gengkun. Frame loss concealment for multiview video transmission over wireless multimedia sensor networks[J]. IEEE Sensors Journal, 2015, 15(3): 1892–1901. doi: 10.1109/JSEN.2014.2366511

    11. [11]

      HUANG Pincheng, LIN Jieru, LI Gwolong, et al. Improved depth-assisted error concealment algorithm for 3D video transmission[J]. IEEE Transactions on Multimedia, 2017, 19(11): 2625–2632. doi: 10.1109/TMM.2017.2694218

    12. [12]

      TAI Shenchuan, WANG Chuenching, HONG Chienshiang, et al. An efficient full frame algorithm for object-based error concealment in 3D depth-based video[J]. Multimedia Tools & Applications, 2016, 75(16): 9927–9947. doi: 10.1007/s11042-015-2899-4

    13. [13]

      时琳, 刘荣科, 李君烨. 基于深度信息的立体视频错误隐藏方法[J]. 电子与信息学报, 2012, 34(7): 1678–1684. doi: 10.3724/SP.J.1146.2011.01107
      SHI Ling, LIU Rongke, and LI Junye. Error concealment based on depth information for stereoscopic video coding[J]. Journal of Electronics &Information Technology, 2012, 34(7): 1678–1684. doi: 10.3724/SP.J.1146.2011.01107

    14. [14]

      YANG Meng, LAN Xuguang, ZHENG Nanning, et al. Depth-assisted temporal error concealment for intra frame slices in 3-D video[J]. IEEE Transactions on Broadcasting, 2014, 60(2): 385–393. doi: 10.1109/TBC.2014.2321676

    15. [15]

      YAN Bo and ZHOU Jie. Efficient frame concealment for depth image-based 3-D video transmission[J]. IEEE Transactions on Multimedia, 2012, 14(3): 936–941. doi: 10.1109/TMM.2012.2184743

    16. [16]

      MERKLE P, MULLER K, MARPE D, et al. Depth intra coding for 3D video based on geometric primitives[J]. IEEE Transactions on Circuits & Systems for Video Technology, 2016, 26(3): 570–582. doi: 10.1109/TCSVT.2015.2407791

    17. [17]

      HEWAGE C T E R, WORRALL S, DOGAN S, et al. Frame concealment algorithm for stereoscopic video using motion vector sharing[C]. 2008 IEEE International Conference on Multimedia and Expo, Hannover, Germany, 2008: 485–488. doi: 10.1109/ICME.2008.4607477.

    18. [18]

      ASSUNCAO P, MARCELINO S, SOARES S, et al. Spatial error concealment for intra-coded depth maps in multiview video-plus-depth[J]. Multimedia Tools &Applications, 2017, 76(12): 13835–13858. doi: 10.1007/s11042-016-3766-7

    19. [19]

      HTM Reference Software 14.0. http://hevc.kw.bbc.co.uk/git/w/jctvc-3de.git, 2016.

    20. [20]

      WANG Zhou, BOVIK A C, SHEIKH H R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600–612. doi: 10.3969/j.issn.1006-8961.2006.12.00.003

    1. [1]

      王莉, 曹一凡, 杜高明, 刘冠宇, 王晓蕾, 张多利. 一种低延迟的3维高效视频编码中深度建模模式编码器. 电子与信息学报, 2019, 41(7): 1625-1632.

    2. [2]

      吴培良, 杨霄, 毛秉毅, 孔令富, 侯增广. 一种视角无关的时空关联深度视频行为识别方法. 电子与信息学报, 2019, 41(4): 904-910.

    3. [3]

      涂开辉, 黄志洪, 侯峥嵘, 杨海钢. 基于配置模式匹配和层次化映射结构的高效FPGA码流生成系统研究. 电子与信息学报, 2019, 41(11): 2585-2591.

    4. [4]

      郭晨, 简涛, 徐从安, 何友, 孙顺. 基于深度多尺度一维卷积神经网络的雷达舰船目标识别. 电子与信息学报, 2019, 41(6): 1302-1309.

    5. [5]

      申滨, 吴和彪, 崔太平, 陈前斌. 基于最优索引广义正交匹配追踪的非正交多址系统多用户检测. 电子与信息学报, 2019, 41(0): 1-8.

    6. [6]

      沈璇, 孙兵, 刘国强, 李超. 数字视频广播通用加扰算法的不可能差分分析. 电子与信息学报, 2019, 41(1): 46-52.

    7. [7]

      孙怡峰, 吴疆, 黄严严, 汤光明. 一种视频监控中基于航迹的运动小目标检测算法. 电子与信息学报, 2019, 41(11): 2744-2751.

    8. [8]

      吴志军, 李常亮, 李荣. 一种基于基音预测过程的语音信息隐藏方法. 电子与信息学报, 2019, 41(0): 1-9.

    9. [9]

      邹晓军, 廉保旺. 矢量GNSS接收机自主完好性监控性能研究. 电子与信息学报, 2019, 41(8): 1966-1973.

    10. [10]

      梁爽, 杭文龙, 冯伟, 刘学军. 基于分类误差一致性准则的自适应知识迁移. 电子与信息学报, 2019, 41(11): 2736-2743.

    11. [11]

      李保珠, 关键, 董云龙. 基于航迹矢量检测的雷达与电子支援设施抗差关联算法. 电子与信息学报, 2019, 41(1): 123-129.

    12. [12]

      潘一苇, 杨司韩, 彭华, 李天昀, 王文雅. 基于矢量图的特定辐射源识别方法. 电子与信息学报, 2019, 41(0): 1-9.

    13. [13]

      李保珠, 张林, 董云龙, 关键. 基于航迹矢量分级聚类的雷达与电子支援措施抗差关联算法. 电子与信息学报, 2019, 41(6): 1310-1316.

    14. [14]

      刘浩, 孙晓帆, 张鑫生, 吴乐明, 况奇刚. 面向图像集的置信区间内一致性增强质量评价准则. 电子与信息学报, 2019, 41(1): 219-225.

    15. [15]

      张颖, 高灵君. 基于格拉布斯准则和改进粒子滤波算法的水下传感网目标跟踪. 电子与信息学报, 2019, 41(10): 2294-2301.

    16. [16]

      张顺外, 魏琪. 多信源多中继编码协作系统准循环LDPC码的联合设计与性能分析. 电子与信息学报, 2019, 41(10): 2325-2333.

    17. [17]

      张红旗, 黄睿, 常德显. 一种基于匹配博弈的服务链协同映射方法. 电子与信息学报, 2019, 41(2): 385-393.

    18. [18]

      牛燕雄, 陈梦琪, 张贺. 基于尺度不变特征变换的快速景象匹配方法. 电子与信息学报, 2019, 41(3): 626-631.

    19. [19]

      崔维嘉, 张鹏, 巴斌. 基于循环匹配追踪的稀疏重构时延估计算法. 电子与信息学报, 2019, 41(3): 523-529.

    20. [20]

      田增山, 王阳, 周牧, 未平. 基于自适应渐消记忆的蓝牙序列匹配定位算法. 电子与信息学报, 2019, 41(6): 1381-1388.

  • 图 1  算法总体流程图

    图 2  丢失块周围邻块位移矢量的选择

    图 3  PSNR逐帧测试结果曲线图

    图 4  Poznanstreet第3视点第3帧错误隐藏结果

    图 5  Kendo第3视点第3帧主观测试结果

    表 1  组合参考块对应表

    参考列表参考帧号L0L1
    R00R01R0KR10R11R1K
    L0R00P0B00B01B0iB0K
    R01P1B10B11B1iB1K
    PiBi0Bi1BiiBiK
    R0KPKBK0BK1BKiBKK
    L1R10B00B10Bi0BK0PK+1
    R11B01B11Bi1BK1PK+2
    B0iB1iBiiBKiPK+i
    R1KB0KB1KBiKBKKP2K
    下载: 导出CSV

    表 2  QP=30, 25时各测试序列重建图像平均PSNR计算结果(dB)

    QP序列原始错误率50%错误率25%错误率10%
    文献[6]文献[7]MVC本文算法文献[6]文献[7]MVC本文算法文献[6]文献[7]MVC本文算法
    30Kendo42.76136.40636.12235.89437.10337.89637.35936.78638.31039.44938.92538.02539.833
    Bookarrival39.92336.93235.91135.65637.52537.86537.17136.52638.28638.33837.69937.38938.747
    Poznanstreet44.02839.59338.83436.66240.86441.15541.02840.79941.89041.45041.21940.98342.171
    Balloon40.57438.71338.51237.69139.26939.51439.45638.34739.83939.70639.63838.48539.958
    25Kendo45.87436.73636.54836.09637.49939.22738.05837.28739.65541.36839.68839.13341.501
    Bookarrival41.62537.98136.73336.35438.27038.41137.48836.94739.92139.72839.19638.46340.187
    Poznanstreet46.12239.95538.92837.06641.56141.72341.17840.80242.31942.11341.79741.20742.622
    Balloon44.02240.82939.43439.15641.24741.89941.27740.89442.14942.23442.15841.08442.612
    下载: 导出CSV

    表 3  QP=30, 25时各测试序列重建图像平均SSIM计算结果

    QP序列原始错误率50%错误率25%错误率10%
    文献[6]文献[7]MVC本文算法文献[6]文献[7]MVC本文算法文献[6]文献[7]MVC本文算法
    30Kendo0.98670.97080.96880.96470.97210.97800.97760.97330.97850.98190.98160.97660.9822
    Bookarrival0.95650.94820.94570.94220.94950.95030.94810.94520.95100.95250.95180.94770.9534
    Poznanstreet0.97950.97320.97200.97140.97590.97460.97400.97380.97670.97510.97500.97440.9769
    Balloon0.97670.97380.97330.97200.97430.97450.97420.97360.97510.97470.97460.97430.9757
    25Kendo0.98740.97220.96960.96600.97510.97890.97830.97650.97960.98280.98240.97950.9832
    Bookarrival0.97010.95770.95470.95390.95980.96340.96270.96130.96410.96430.96420.96330.9655
    Poznanstreet0.98780.98180.98030.97960.98240.98250.98210.98160.98480.98430.98410.98240.9849
    Balloon0.98550.98100.98030.97960.98210.98340.98330.98280.98390.98410.98360.98350.9844
    下载: 导出CSV

    表 4  各测试序列解码运行时间(s)

    序列原解码文献[6]文献[7]MVC本文
    Kendo188.71231.45198.83192.07194.19
    Balloon172.35216.14178.13173.82174.69
    Bookarrival175.86219.35183.19177.18179.13
    Poznanstreet483.67558.15498.68484.68486.07
    下载: 导出CSV
  • 加载中
图(5)表(4)
计量
  • PDF下载量:  10
  • 文章访问数:  469
  • HTML全文浏览量:  328
文章相关
  • 通讯作者:  周洋, zhouyang_hz@126.com
  • 收稿日期:  2018-09-30
  • 录用日期:  2019-04-05
  • 网络出版日期:  2019-05-21
  • 刊出日期:  2019-11-01
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

/

返回文章