高级搜索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

具有聚类结构相似性的非参数贝叶斯字典学习算法

董道广 芮国胜 田文飚 张洋 刘歌

董道广, 芮国胜, 田文飚, 张洋, 刘歌. 具有聚类结构相似性的非参数贝叶斯字典学习算法[J]. 电子与信息学报, 2020, 42(11): 2765-2772. doi: 10.11999/JEIT190496
引用本文: 董道广, 芮国胜, 田文飚, 张洋, 刘歌. 具有聚类结构相似性的非参数贝叶斯字典学习算法[J]. 电子与信息学报, 2020, 42(11): 2765-2772. doi: 10.11999/JEIT190496
Daoguang DONG, Guosheng RUI, Wenbiao TIAN, Yang ZHANG, Ge LIU. A Nonparametric Bayesian Dictionary Learning Algorithm with Clustering Structure Similarity[J]. Journal of Electronics and Information Technology, 2020, 42(11): 2765-2772. doi: 10.11999/JEIT190496
Citation: Daoguang DONG, Guosheng RUI, Wenbiao TIAN, Yang ZHANG, Ge LIU. A Nonparametric Bayesian Dictionary Learning Algorithm with Clustering Structure Similarity[J]. Journal of Electronics and Information Technology, 2020, 42(11): 2765-2772. doi: 10.11999/JEIT190496

具有聚类结构相似性的非参数贝叶斯字典学习算法

doi: 10.11999/JEIT190496
基金项目: 国家自然科学基金(41606117, 41476089, 61671016)
详细信息
    作者简介:

    董道广:男,1990年生,博士,研究方向为贝叶斯统计学习、压缩感知与大气波导

    芮国胜:男,1968年生,博士,教授,博士生导师,研究方向为现代通信理论及信号处理

    田文飚:男,1987年生,博士,副教授,研究方向为大气波导与压缩感知

    张洋:男,1983年生,博士,讲师,研究方向为混沌通信技术

    刘歌:女,1991年生,博士,研究方向为压缩感知与大气波导

    通讯作者:

    董道广 sikongyu@yeah.net

  • 中图分类号: TN911.73; TP301.6

A Nonparametric Bayesian Dictionary Learning Algorithm with Clustering Structure Similarity

Funds: The National Natural Science Foundation of China (41606117, 41476089, 61671016)
  • 摘要: 利用图像结构信息是字典学习的难点,针对传统非参数贝叶斯算法对图像结构信息利用不充分,以及算法运行效率低下的问题,该文提出一种结构相似性聚类beta过程因子分析(SSC-BPFA)字典学习算法。该算法通过Markov随机场和分层Dirichlet过程实现对图像局部结构相似性和全局聚类差异性的兼顾,利用变分贝叶斯推断完成对概率模型的高效学习,在确保算法收敛性的同时具有聚类的自适应性。实验表明,相比目前非参数贝叶斯字典学习方面的主流算法,该文算法在图像去噪和插值修复应用中具有更高的表示精度、结构相似性测度和运行效率。
  • 图  1  本文算法的概率图模型

    图  2  本文算法字典学习过程中获得的聚类效果

    图  3  不同算法的图像去噪效果

    图  4  不同信噪比条件下的去噪信误比

    图  5  不同算法的图像修复效果

    图  6  不同缺失率条件下的平均SER结果

    图  7  不同缺失率条件下的平均SSIM结果

    图  8  不同缺失率条件下的平均时间代价

    表  1  模型中隐变量及其变分参数的VB推断更新公式

    隐变量变分分布变分参数的VB推断更新公式隐变量更新公式
    ${{{d}}_k}$$q\left( {{{{d}}_k}} \right) \propto {\rm{Normal}}\left( {{{{d}}_k}|{{{\tilde { m}}}_k},{{{\tilde { \Lambda }}}_k}} \right)$${ {\tilde{ \varLambda } }_k} = P{ {{I} }_P} + \displaystyle\sum\nolimits_{i = 1}^N {\dfrac{ { { {\tilde e}_0} } }{ { { {\tilde f}_0} } }\left( {\tilde \mu _{ik}^2 + \tilde \sigma _{ik}^2} \right){ {\tilde \rho }_{ik} }{ {{I} }_P} }$,

    ${ {\tilde { m} }_k} = {\tilde { \varLambda } }_k^{ - 1} \cdot \left( {\displaystyle\sum\nolimits_{i = 1}^N {\dfrac{ { { {\tilde e}_0} } }{ { { {\tilde f}_0} } }{ {\tilde \mu }_{ik} }{ {\tilde \rho }_{ik} }{{x} }_i^{ - k} } } \right)$
    ${{{d}}_k} = {{\tilde { m}}_k}$
    ${s_{ik}}$$q\left( {{s_{ik}}} \right) \propto {\rm{Normal}}\left( {{s_{ik}}|{{\tilde \mu }_{ik}},\tilde \sigma _{ik}^2} \right)$$\tilde \sigma _{ik}^2 = {\left\{ {\dfrac{ { { {\tilde e}_0} } }{ { { {\tilde f}_0} } }{ {\tilde \rho }_{ik} }\left[ { {\tilde { m} }_k^{\rm{T} }{ { {\tilde { m} } }_k} + {\rm{tr(} }{\tilde { \varLambda } }_k^{ - 1}{\rm{)} } } \right] + \dfrac{ { { {\tilde c}_0} } }{ { { {\tilde f}_0} } } } \right\}^{ - 1} }$,

    ${\tilde \mu _{ik}} = \tilde \sigma _{ik}^2 \cdot \left( {\dfrac{{{{\tilde e}_0}}}{{{{\tilde f}_0}}}{{\tilde \rho }_{ik}}{\tilde { m}}_k^{\rm{T}}{{x}}_i^{ - k}} \right)$
    ${s_{ik}} = {\tilde \mu _{ik}}$
    ${z_{ik}}$$q\left( {{z_{ik}}} \right) \propto {\rm{Bernoulli}}\left( {{z_{ik}}|{{\tilde \rho }_{ik}}} \right)$${\tilde \rho _{ik}} = \dfrac{{{\rho _{ik,1}}}}{{{\rho _{ik,1}} + {\rho _{ik,0}}}}$,

    ${\rho _{ik,0}} = \exp \left\{ \displaystyle\sum\nolimits_{l = 1}^L {{{\tilde \xi }_{il}} \cdot [\psi ({{\tilde b}_{lk}}) - \psi ({{\tilde a}_{lk}} + {{\tilde b}_{lk}})]} \right\} $$\begin{array}{l} {\rho _{ik,1} } = \exp \Bigr\{ - \dfrac{1}{2}\dfrac{ { { {\tilde e}_0} } }{ { { {\tilde f}_0} } }(\tilde \mu _{ik}^2 + \tilde \sigma _{ik}^2)({\tilde { m} }_k^{\rm{T} }{ { {\tilde { m} } }_k} + {\rm{tr} }({\tilde { \varLambda } }_k^{ - 1})) \\\qquad\quad\ + \dfrac{ { { {\tilde e}_0} } }{ { { {\tilde f}_0} } }{ {\tilde \mu }_{ik} }{\tilde { m} }_k^{\rm{T} }{{x} }_i^{ - k} + \displaystyle\sum\nolimits_{l = 1}^L { { {\tilde \xi }_{il} } \cdot [\psi ({ {\tilde a}_{lk} }) - \psi ({ {\tilde a}_{lk} } + { {\tilde b}_{lk} })]} \Bigr\} \end{array}$
    ${z_{ik}} = {\tilde \rho _{ik}}$
    $\pi _{lk}^ * $$q\left( {\pi _{lk}^ * } \right) \propto {\rm{Beta}}\left( {\pi _{lk}^ * |{{\tilde a}_{lk}},{{\tilde b}_{lk}}} \right)$${\tilde a_{lk} } = { { {a_0} } / K} + \displaystyle\sum\nolimits_{i = 1}^N { { {\tilde \rho }_{ik} }{ {\tilde \xi }_{il} } }$, ${\tilde b_{lk}} = {{{b_0}(K - 1)} / K} + \displaystyle\sum\nolimits_{i = 1}^N {(1 - {{\tilde \rho }_{ik}}){{\tilde \xi }_{il}}} $$\pi _{lk}^ * = \dfrac{{{{\tilde a}_{lk}}}}{{{{\tilde a}_{lk}} + {{\tilde b}_{lk}}}}$
    ${t_i}$$q\left( { {t_i} } \right) \propto {\rm{Multi} }\left( { { {\tilde \xi }_{i1} },{ {\tilde \xi }_{i2} },···,{ {\tilde \xi }_{iL} } } \right)$${\tilde \xi _{il}} = {{{\xi _{il}}} / {\displaystyle\sum\nolimits_{l' = 1}^L {{\xi _{il'}}} }}$,

    $\begin{array}{l} {\xi _{il} } = \exp \Bigr\{ \displaystyle\sum\nolimits_{k = 1}^K { { {\tilde \rho }_{ik} }[\psi ({ {\tilde a}_{lk} }) - \psi ({ {\tilde a}_{lk} } + { {\tilde b}_{lk} })]} \\ \qquad\quad + \displaystyle\sum\nolimits_{k = 1}^K {(1 - { {\tilde \rho }_{ik} })[\psi ({ {\tilde b}_{lk} }) - \psi \left({ {\tilde a}_{lk} } + { {\tilde b}_{lk} }\right)]} \\ \qquad\quad + \psi ({ {\tilde \zeta }_{il} }) - \psi \left(\displaystyle\sum\nolimits_{l' = 1}^L { { {\tilde \zeta }_{il'} } } \right)\Bigr\} \\ \end{array}$
    ${t_i} = \mathop {\arg \max }\limits_l \left\{ {{{\tilde \xi }_{il}},_{}^{}\forall l \in [1,L]} \right\}$
    ${\beta _{il}}$$q\left( {\{ {\beta _{il} }\} _{l = 1}^L} \right) \propto {\rm{Diri} }\left( { { {\tilde \lambda }_{i1} },{ {\tilde \lambda }_{i2} },···,{ {\tilde \lambda }_{iL} } } \right)$${\tilde \lambda _{il}} = {\tilde \xi _{il}} + {\alpha _0}{G_{il}}$${\beta _{il}} = {{{{\tilde \lambda }_{il}}} / {\displaystyle\sum\nolimits_{l' = 1}^L {{{\tilde \lambda }_{il'}}} }}$
    ${\gamma _s}$$q\left( {{\gamma _s}} \right) \propto {\rm{Gamma}}\left( {{\gamma _s}|{{\tilde c}_0},{{\tilde d}_0}} \right)$${\tilde c_0} = {c_0} + \dfrac{1}{2}NK$, ${\tilde d_0} = {d_0} + \dfrac{1}{2}\displaystyle\sum\nolimits_{i = 1}^N {\displaystyle\sum\nolimits_{k = 1}^K {\left( {\tilde \mu _{ik}^2 + \tilde \sigma _{ik}^2} \right)} } $${\gamma _s} = {{{{\tilde c}_0}} / {{{\tilde d}_0}}}$
    ${\gamma _\varepsilon }$$q\left( {{\gamma _\varepsilon }} \right) \propto {\rm{Gamma}}\left( {{\gamma _\varepsilon }|{{\tilde e}_0},{{\tilde f}_0}} \right)$${\tilde e_0} = {e_0} + \dfrac{1}{2}NP$, ${\tilde f_0} = {f_0} + \dfrac{1}{2}\displaystyle\sum\nolimits_{i = 1}^N {\left\| {{{{x}}_i} - {\hat{ D}}\left( {{{{\hat{ s}}}_i} \odot {{{\hat{ z}}}_i}} \right)} \right\|} $${\gamma _\varepsilon } = {{{{\tilde e}_0}} / {{{\tilde f}_0}}}$
     算法1 SSC-BPFA算法
     输入:训练数据样本集$\left\{ {{{{x}}_i}} \right\}_{i = 1}^N$
     输出:字典${{D}}$、权重$\left\{ {{{{s}}_i}} \right\}_{i = 1}^N$、稀疏模式指示向量$\left\{ {{{{z}}_i}} \right\}_{i = 1}^N$及聚类标签$\left\{ {{t_i}} \right\}_{i = 1}^N$
     步骤 1 设置收敛阈值$\tau $和迭代次数上限${\rm{it}}{{\rm{r}}_{\max }}$,初始化原子数目$K$与聚类数目$L$,通过K-均值算法对数据样本进行初始聚类;
     步骤 2 按照式(1)—式(15)完成NPB-DL的概率建模;
     步骤 3 初始化隐变量集${{\varTheta } }$和变分参数集${{{ H}}}$,计算${\hat{ X}} = {{D}}\left( {{{S}} \odot {{Z}}} \right)$,令迭代索引${\rm{itr}}$=1;
     步骤 4 按照表1的VB推断公式对${{\varTheta } }$和${{{ H}}}$进行更新,计算${{\hat{ X}}^{{\rm{new}}}} = {{D}}\left( {{{S}} \odot {{Z}}} \right)$;
     步骤 5 令${\rm{itr}}$值增加1,删除${{D}}$中未使用的原子并更新$K$的值,计算$r = {{\left\| {{{{\hat{ X}}}^{{\rm{new}}}} - {\hat{ X}}} \right\|_{\rm{F}}^2} / {\left\| {{\hat{ X}}} \right\|_{\rm{F}}^2}}$,若$r < \tau $或${\rm{itr}} \ge {\rm{it}}{{\rm{r}}_{\max }}$,删除所含样本
     占全部样本数量比例低于${10^{ - 3}}$的聚类,将被删聚类内的样本分配到剩余聚类中${\tilde \xi _{il}}$最大的那个聚类中,进入步骤6,否则跳回步骤4;
     步骤 6 固定${t_i}$和${\beta _{il}}$及其变分参数的估计结果,将${{{d}}_k}$, ${s_{ik}}$, ${z_{ik}}$, $\pi _{lk}^ * $, ${\gamma _s}$和${\gamma _\varepsilon }$这6个隐变量及其对应的变分参数继续进行迭代优化更新,直至
     重新达到收敛.
    下载: 导出CSV
  • [1] XUAN Junyu, LU Jie, ZHANG Guangquan, et al. Doubly nonparametric sparse nonnegative matrix factorization based on dependent Indian buffet processes[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(5): 1835–1849. doi:  10.1109/TNNLS.2017.2676817
    [2] LI Shaoyang, TAO Xiaoming, and LU Jianhua. Variational Bayesian inference for nonparametric signal compressive sensing on structured manifolds[C]. 2017 IEEE International Conference on Communications, Paris, France, 2017. doi:  10.1109/ICC.2017.7996389.
    [3] DANG H P and CHAINAIS P. Indian buffet process dictionary learning: Algorithms and applications to image processing[J]. International Journal of Approximate Reasoning, 2017, 83: 1–20. doi:  10.1016/j.ijar.2016.12.010
    [4] SERRA J G, TESTA M, MOLINA R, et al. Bayesian K-SVD using fast variational inference[J]. IEEE Transactions on Image Processing, 2017, 26(7): 3344–3359. doi:  10.1109/TIP.2017.2681436
    [5] NGUYEN V, PHUNG D, BUI H, et al. Discriminative Bayesian nonparametric clustering[C]. The 26th International Joint Conference on Artificial Intelligence, Melbourne, Australia, 2017. doi:  10.24963/ijcai.2017/355.
    [6] HUYNH V and PHUNG D. Streaming clustering with Bayesian nonparametric models[J]. Neurocomputing, 2017, 258: 52–62. doi:  10.1016/j.neucom.2017.02.078
    [7] AHARON M, ELAD M, and BRUCKSTEIN A. K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation[J]. IEEE Transactions on Signal Processing, 2006, 54(11): 4311–4322. doi:  10.1109/TSP.2006.881199
    [8] ELAD M and AHARON M. Image denoising via sparse and redundant representations over learned dictionaries[J]. IEEE Transactions on Image Processing, 2006, 15(2): 3736–3745. doi:  10.1109/TIP.2006.881969
    [9] ZHOU Mingyuan, CHEN Haojun, PAISLEY J, et al. Nonparametric Bayesian dictionary learning for analysis of noisy and incomplete images[J]. IEEE Transactions on Image Processing, 2012, 21(1): 130–144. doi:  10.1109/TIP.2011.2160072
    [10] ZHOU Mingyuan, PAISLEY J, and CARIN L. Nonparametric learning of dictionaries for sparse representation of sensor signals[C]. The 3rd IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, Aruba, Netherlands, 2009. doi:  10.1109/CAMSAP.2009.5413290.
    [11] PAISLEY J, ZHOU Mingyuan, SAPIRO G, et al. Nonparametric image interpolation and dictionary learning using spatially-dependent Dirichlet and beta process priors[C]. 2010 IEEE International Conference on Image Processing, Hong Kong, China, 2010. doi:  10.1109/ICIP.2010.5653350.
    [12] JU Fujiao, SUN Yanfeng, GAO Junbin, et al. Nonparametric tensor dictionary learning with beta process priors[J]. Neurocomputing, 2016, 218: 120–130. doi:  10.1016/j.neucom.2016.08.064
    [13] KNOWLES D and GHAHRAMANI Z. Infinite sparse factor analysis and infinite independent components analysis[C]. The 7th International Conference on Independent Component Analysis and Signal Separation, London, UK, 2007: 381–388. doi:  10.1007/978-3-540-74494-8_48.
    [14] ZHANG Linlin, GUINDANI M, VERSACE F, et al. A spatio-temporal nonparametric Bayesian variable selection model of fMRI data for clustering correlated time courses[J]. NeuroImage, 2014, 95: 162–175. doi:  10.1016/j.neuroimage.2014.03.024
    [15] AKHTAR N and MIAN A. Nonparametric coupled Bayesian dictionary and classifier learning for hyperspectral classification[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(9): 4038–4050. doi:  10.1109/TNNLS.2017.2742528
    [16] POLATKAN G, ZHOU Mingyuan, CARIN L, et al. A Bayesian nonparametric approach to image super-resolution[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(2): 346–358. doi:  10.1109/TPAMI.2014.2321404
    [17] 董道广, 芮国胜, 田文飚, 等. 基于结构相似性的非参数贝叶斯字典学习算法[J]. 通信学报, 2019, 40(1): 43–50. doi:  10.11959/j.issn.1000-436x.2019015

    DONG Daoguang, RUI Guosheng, TIAN Wenbiao, et al. Nonparametric Bayesian dictionary learning algorithm based on structural similarity[J]. Journal on Communications, 2019, 40(1): 43–50. doi:  10.11959/j.issn.1000-436x.2019015
    [18] BISHOP C M. Pattern Recognition and Machine Learning (Information Science and Statistics)[M]. New York: Springer, 2006: 461–522.
  • [1] 许红, 谢文冲, 袁华东, 段克清, 王永良.  基于自适应的增广状态-交互式多模型的机动目标跟踪算法, 电子与信息学报. doi: 10.11999/JEIT190516
    [2] 王琦, 张鹏程, 汪剑鸣, 李秀艳, 连志杰, 陈庆良, 陈彤云, 陈晓静, 贺静, 段晓杰, 王化祥.  基于块稀疏的电阻抗成像算法, 电子与信息学报. doi: 10.11999/JEIT170425
    [3] 许红, 袁华东, 谢文冲, 刘维建, 王永良.  角闪烁下基于变分贝叶斯-交互式多模型的目标跟踪, 电子与信息学报. doi: 10.11999/JEIT171025
    [4] 杨真真, 杨震, 李雷, 金正猛.  Alpha稳态噪声下基于Meridian范数的全变分图像去噪算法, 电子与信息学报. doi: 10.11999/JEIT160657
    [5] 田元荣*, 王星, 周一鹏.  一种新的基于稀疏表示的单通道盲源分离算法, 电子与信息学报. doi: 10.11999/JEIT160888
    [6] 薛模根, 刘文琢, 袁广林, 秦晓燕.  基于编码迁移的快速鲁棒视觉跟踪, 电子与信息学报. doi: 10.11999/JEIT160966
    [7] 王伟, 韩纪庆, 郑铁然, 郑贵滨, 陶耀.  基于Fisher判别字典学习的说话人识别, 电子与信息学报. doi: 10.11999/JEIT 150566
    [8] 袁广林, 薛模根.  基于稀疏稠密结构表示与在线鲁棒字典学习的视觉跟踪, 电子与信息学报. doi: 10.11999/JEIT140507
    [9] 薛模根, 朱虹, 袁广林.  基于在线判别式字典学习的鲁棒视觉跟踪, 电子与信息学报. doi: 10.11999/JEIT141325
    [10] 查翔, 倪世宏, 张鹏.  一类非线性信号去噪的奇异值分解有效迭代方法, 电子与信息学报. doi: 10.11999/JEIT141605
    [11] 阮宗利, 李立萍, 钱国兵, 罗明刚.  基于含噪复值ICA信号模型的快速不动点算法, 电子与信息学报. doi: 10.3724/SP.J.1146.2013.00951
    [12] 欧国建, 杨士中, 蒋清平, 曹海林.  一种三阶多项式相位信号去噪的字典学习算法, 电子与信息学报. doi: 10.3724/SP.J.1146.2013.00726
    [13] 霍雷刚, 冯象初.  基于主成分分析和字典学习的高光谱遥感图像去噪方法, 电子与信息学报. doi: 10.3724/SP.J.1146.2013.01840
    [14] 罗亮, 冯象初, 张选德, 李小平.  基于非局部双边随机投影低秩逼近图像去噪算法, 电子与信息学报. doi: 10.3724/SP.J.1146.2012.00819
    [15] 徐定杰, 沈忱, 沈锋.  时变有色观测噪声下基于变分贝叶斯学习的自适应卡尔曼滤波, 电子与信息学报. doi: 10.3724/SP.J.1146.2012.01457
    [16] 朱波, 汶德胜, 王飞, 李华, 宋宗玺.  应用字典学习算法改善Bayer格式图像彩色恢复效果, 电子与信息学报. doi: 10.3724/SP.J.1146.2012.00947
    [17] 李民, 程建, 李小文, 乐翔.  非局部学习字典的图像修复, 电子与信息学报. doi: 10.3724/SP.J.1146.2010.01426
    [18] 杨春玲, 马力.  基于多假设运动补偿去噪的迭代边信息改进算法, 电子与信息学报. doi: 10.3724/SP.J.1146.2011.00355
    [19] 孙韶杰, 吴琼, 李国辉.  基于变分贝叶斯估计的相机抖动模糊图像的盲复原算法, 电子与信息学报. doi: 10.3724/SP.J.1146.2009.01600
    [20] 陆明俊, 王润生.  计算机视觉中的Markov随机场方法, 电子与信息学报.
  • 加载中
  • 图(8) / 表ll (1)
    计量
    • 文章访问数:  121
    • HTML全文浏览量:  57
    • PDF下载量:  39
    • 被引次数: 0
    出版历程
    • 收稿日期:  2019-07-03
    • 修回日期:  2020-02-28
    • 网络出版日期:  2020-09-01
    • 刊出日期:  2020-11-10

    目录

      /

      返回文章
      返回

      官方微信,欢迎关注