基于隐马尔可夫模型的音频检索
|
摘要
作为多媒体媒质之一的音频信号几乎无处不有，它有效的丰富和补充了人们在信息社会的语义感知和获取。但当前人们对多媒体信息检索获取仍是以视觉为主要途径，特别是常以文本方式进行检索，而基于大量音频信息库的检索形式却未能引起人们的足够重视。为此，本文针对音频检索问题展开了讨论，从音频信号特征提取着手，分析了音频信号的时域和频域特征以提取短时能量、短时过零率、短时能频值和Mel系数等为特征数据，把音频信息流分割为广告、主持人介绍、天气预报、足球比赛、音乐或戏曲等六大类。利用具有较强的时间序列结构建模能力的隐马尔可夫模型和具有并行性、强分类能力的BP神经网络实现了广播电视节目音频信息流分类检索。同时考虑到基于梯度下降法的BP算法训练速度慢，为加强检索手段的时效性，我们进行了BP算法加速收敛的分析应用，得到了适用于音频检索的改进BP算法，数值实验结果表明有效性。
As one component in multimedia, audio signals are filled in the world, which greatly enrich our semantic apperception and acquisition in information society. However, the current way to get information is mainly based on the vision, especially the text. The retrieval based on the audio information is ignored. So, an audio retrieval system is presented in this paper. Depending on time-domain and frequency-domain features: short-time energy, short-time zero-crossing rate, short-time energy-frequency value and mel-coeffients, audio streams are segmented into six classes: commercial, anchorperson, weather forecast, football match, music and drama. Then, an audio retrieval system based on HMM and BP neural network is presented since HMM can simulate stochastic time series data quite well and ANN has many advantages such as parallel processing ability, powerful discriminating ability etc. Based on gradient descent, traditional BP algorithm has a slow operating speed, so an improved BP algorithm is presented in thi
s paper to improve the recognition speed. Experimental results showed its validity.
As one component in multimedia, audio signals are filled in the world, which greatly enrich our semantic apperception and acquisition in information society. However, the current way to get information is mainly based on the vision, especially the text. The retrieval based on the audio information is ignored. So, an audio retrieval system is presented in this paper. Depending on time-domain and frequency-domain features: short-time energy, short-time zero-crossing rate, short-time energy-frequency value and mel-coeffients, audio streams are segmented into six classes: commercial, anchorperson, weather forecast, football match, music and drama. Then, an audio retrieval system based on HMM and BP neural network is presented since HMM can simulate stochastic time series data quite well and ANN has many advantages such as parallel processing ability, powerful discriminating ability etc. Based on gradient descent, traditional BP algorithm has a slow operating speed, so an improved BP algorithm is presented in thi
s paper to improve the recognition speed. Experimental results showed its validity.
引文
1．多媒体数据融合研究．国防科技大学学位论文．
2．庄越挺等．网上多媒体信息分析与检索．清华大学出版社．2002．
3．庄越挺，吴飞，潘云鹤等．基于隐马尔可夫链的广播新闻分割分类．计算机研究与发展．2002，39(9)：1057-1063．
4．杨行峻、迟惠生．语音信号数字处理．北京：电子工业出版社．1995．
5．胡广书．数字信号处理．北京：清华大学出版社．1997．
6．李易军，徐近霈，吴枫．用于连续语音识别的RBF-Gamma-HMM组合模型．电子学报．1999，27(9)：81-85．
7．邓伟，赵荣椿．一种基于改进CP网络与HMM相结合的混合音素识别方法．数据采集与处理．2000，15(1)：6-11．
8．梅晓丹，张毅刚，孙圣和．模糊神经网络语音数掘融合算法的研究．控制与决策．2003，18(2)：213-216．
9．李国辉，李恒峰．基于内容的音频检索：概念和方法．小型微型计算机系统．2000，21(11)：1173-1177．
10．吴飞，庄越挺，郑科，刘骏伟，潘云鹤．基于压缩域特征话者识别的电视节目分类检索．模式识别与人工智能．2002，15(1)：21-27．
11．吴飞，庄越挺，张引，潘云鹤．基于隐马尔可夫链的音频语义检索．模式识别与人工智能．2001，14(1)：104-108．
12．卢坚，陈毅松，孙正行，张福炎．基于隐马尔可夫模型的音频自动分类．软件学报．2002，13(8)：1593-1597．
13．刘建波，刘雪峰，周群彪，陈晋．语噪分离技术综合算法及其应用．四川大学学报．2002，39(2)：225-228．
14．郝杰，李星．基于经典隐马尔可夫模型的汉语连续语音识别系统．电子与信息学报．2002，24(7)：944-947．
15．赵力．基于偶数帧段输入隐马尔可夫模型的噪声环境下汉语连续语音识别研究．声学学报．2002，27(1)：59-61．
16．张椿霖，杨玉红，胡瑞敏．音频内容分割与聚类的研究．计算机工程．2002，28(7)：173-174．
17．贾磊，穆向禺，徐波．广播语音的音频分割．中文信息学报．2002，16(1)：37-42．
18．郝杰，李星．汉语连续语音识别中关键词可信度的贝叶斯估计．声学学报．2002，27(5)：393-397．
19．卢坚，陈毅松，孙正行，张福炎．语音／音乐自动分类中的特征分析．计算机辅助设计与图形学学报．2002，14(3)：233-237．
20. Doh-Suk Kim, Soo-Young Lee. "Auditory Processing of Speech Signals for Robust Speech Recognition in Real-World Noisy Environments" IEEE Trans. Speech, audio processing. Vol.7, No.1, pp. 55-68, Jan. 1999.
21. Levent M. Arslan, John H.L. Hansen. "Selective Training for Hidden Markov Models with Applications to Speech Classication" . IEEE Trans. Speech, audio processing. Vol. 7, No.1, pp. 46-54, Jan. 1999.
22. Rivarol Vergin, Douglas O' Shaughnessy, Azarshid Farhat. "Generalized Mel Frequency Cepstral Coefficients for Large-Vocabulary Speaker Independent Continuous-Speech Recognition" IEEE Trans. Speech, audio processing. Vol.7, No. 5, pp. 525-532, Sep. 1999.
23. Zenton Goh, Kah-Chye Tan, B.T.G. Tan. "Kalman-Filtering Speech Enhancement Method Based on a Voiced-Unvoiced Speech Model" IEEE Trans. Speech, audio processing. Vol. 7, No. 5, pp. 510-524, Sep. 1999.
24. Chulhee Lee, Donghoon Hyun, Euisun Choi et. "Optimizing Feature Extraction for Speech Recognition" IEEE Trans. Speech, audio processing. Vol. 11, No.1, pp. 80-87, Jan. 2003.
25. Stan Z.li. "Content-Based Audio Classification and Retrieval Using the Nearest Feature Line Method" IEEE Trans. Speech, audio processing. Vol.8, NO. 5, pp. 619-625, Sep. 2000.
26. Erling Wold, Thom Blum et. "Content-Based Classification, Search, and Retrieval of Audio" IEEE Multimedia. pp. 27-36, Fall,1996.
27. Hahn-Ming Lee, Chih-Ming Chert, Yung-Feng Lu. "A Self-Organizing HCMAC Neural-Network Classifier" IEEE Trans. Neural Networks. Vol.14, No.1, Jan. 2003.
28. L. Rabiner, B.H.Juans. Fundamentals of Speech Recognition. 1st ed. Beijing. Tsinghua University Press, 1999.
29．陈永彬，王仁华．语言信号处理．第1版．合肥：中国科学技术大学出版社，1990．
30. Guodong Guo, Stan Z. Li. "Content-Based Classification and Retrieval by Support Vector Machines" IEEE Trans. Neural Networks. Vol.14, No.1, pp. 209-215, Jan. 2003.
31. Darryl William Purnell, Elizabeth C. Botha. "Improved Generalization of MCE Parameter Estimation with Application to Speech Recognition" IEEE Trans. Speech, audio processing. Vol. 10, No. 4, pp. 232-239, May 2002.
32. Willianm M.Campbell, Khaled T. Assaleh, Charles C. Broun. "Speaker Recognition With Polynomial Classifier". IEEE Trans. Speech, audio processing. Vol.10, No. 4, pp. 205-212, May 2002.
33. Stephen A. Zahorian, Zaki B. Nossair. "A Partitioned Neural Network ipproch for Vowel Classification Using Smoothed Time/Frequency Features" IEEE Trans. Speech, audio processing. Vol. 7, No. 4, pp. 414-425, Jan. 1999.
34．黄峥．浅谈数字音频格式．音响技术．2002，4：8-10．
35．卢坚，毛兵，孙正行，张福炎．一种改进的基于说话者的语音分割算法．软件学报．2002，13(2)：274-279．
36．马祥杰，孟相如，张百生．一种自相关基因检测算法．通信技术．2003(3)：9-11．
37．赵庆卫，肖熙，王作英等．段长信息在连续语音识别中的应用研究．声学学报．2000，25(2)：175-181．
38．张红，黄泰翼，李治．基于半波差分谱的语音信号音节切分．声学学报．2000，25(14)：323-328．
39．卢绪刚，陈道文．听觉计算模型在鲁棒性语音识别中的应用．声学学报．2000，25(6)：492-498．
40．张焱，张杰，黄志同．基于听觉模型的鲁棒性语音识别的研究．模式识别与人工智能．1998，11(3)：341-346．
41．徐济仁，朱俊樵，谢明祥．基于声韵分割的语音信号特征提取技术．小型微型计算机系统．2002，23(2)：172-175．
42．徐建华．一种新型的多媒体检索技术 1)—基于内容的检索．情报学报．2000，19(4)：411-416．
43．邓军华，钟兵．MPEG-4中的音频编码技术．数据通信．2002(3)：6-9．
44．李建良，蒋勇，汪光先．计算机数值方法．第1版．南京：东南大学出版社．
45．陈亚勇．Matlab信号处理详解．第1版．北京：人民邮电出版社．
46．马鸿飞，樊昌信，宋国乡．基于小波变换和音质模型的音频编码算法研究．电子学报．2001，29(8)：pp．1028-1031．
47．陈明．神经网络模型．第1版．大连：大连理工大学出版社．1995．
48．杨行峻，郑君里．人工神经网络模型．第1版．高等教育出版社．1992．
49．李庆扬等．非线性方程组数值解法．科学出版社．1997
50．袁亚湘，孙文瑜．最优化理论与方法．科学出版社．1997．
51. Roy. S Near-optimal Dynamic Learning Rate for Training Back-Propagation Neural Networks. Science of Artificial Neural Networks. Ⅱ Bellingham. WA: Society of Industrial and Applied Mathematics. 1993, pp. 277-283
52. Jacobs. K.L et. Increased Rates of Convergence through Learning Rate Adaptation. Neural Networks, 1(4), pp. 295-307.
53. S.V. Kamarthi, S. Pittner. Accelerating Neural Networks Training Using Weight Extrapolations. Neural Networks. 12, pp. 1285-1299, 1999.
54. Osowski, S. Signal Flow Graphs and Neural Networks. Biological Cybernetics, 70(1993), pp. 387-395.
55．冯果忱．非线性方程组迭代解法．上海科学技术出版社．1989．
56. Selection and Analysis of HMM's State—Number in Speech Recognition. IEEE on Signal Processing. 1998, 10. Beijing China.
57．王伟．人工神经网络原理．第1版．北京：北京航空航天大学
58．朱小燕，王昱，徐伟．基于循环神经网络的语音识别模型．计算机学报．2001，24(2)：213-218．
59．李苇营，易克初，胡征．神经网络与HMM构成的混合网络在语音识别中应用的研究．电子学报．1994，22(10)：73-80．
60．杨福生．小波变换的分析与应用．科学出版社．1999．
2．庄越挺等．网上多媒体信息分析与检索．清华大学出版社．2002．
3．庄越挺，吴飞，潘云鹤等．基于隐马尔可夫链的广播新闻分割分类．计算机研究与发展．2002，39(9)：1057-1063．
4．杨行峻、迟惠生．语音信号数字处理．北京：电子工业出版社．1995．
5．胡广书．数字信号处理．北京：清华大学出版社．1997．
6．李易军，徐近霈，吴枫．用于连续语音识别的RBF-Gamma-HMM组合模型．电子学报．1999，27(9)：81-85．
7．邓伟，赵荣椿．一种基于改进CP网络与HMM相结合的混合音素识别方法．数据采集与处理．2000，15(1)：6-11．
8．梅晓丹，张毅刚，孙圣和．模糊神经网络语音数掘融合算法的研究．控制与决策．2003，18(2)：213-216．
9．李国辉，李恒峰．基于内容的音频检索：概念和方法．小型微型计算机系统．2000，21(11)：1173-1177．
10．吴飞，庄越挺，郑科，刘骏伟，潘云鹤．基于压缩域特征话者识别的电视节目分类检索．模式识别与人工智能．2002，15(1)：21-27．
11．吴飞，庄越挺，张引，潘云鹤．基于隐马尔可夫链的音频语义检索．模式识别与人工智能．2001，14(1)：104-108．
12．卢坚，陈毅松，孙正行，张福炎．基于隐马尔可夫模型的音频自动分类．软件学报．2002，13(8)：1593-1597．
13．刘建波，刘雪峰，周群彪，陈晋．语噪分离技术综合算法及其应用．四川大学学报．2002，39(2)：225-228．
14．郝杰，李星．基于经典隐马尔可夫模型的汉语连续语音识别系统．电子与信息学报．2002，24(7)：944-947．
15．赵力．基于偶数帧段输入隐马尔可夫模型的噪声环境下汉语连续语音识别研究．声学学报．2002，27(1)：59-61．
16．张椿霖，杨玉红，胡瑞敏．音频内容分割与聚类的研究．计算机工程．2002，28(7)：173-174．
17．贾磊，穆向禺，徐波．广播语音的音频分割．中文信息学报．2002，16(1)：37-42．
18．郝杰，李星．汉语连续语音识别中关键词可信度的贝叶斯估计．声学学报．2002，27(5)：393-397．
19．卢坚，陈毅松，孙正行，张福炎．语音／音乐自动分类中的特征分析．计算机辅助设计与图形学学报．2002，14(3)：233-237．
20. Doh-Suk Kim, Soo-Young Lee. "Auditory Processing of Speech Signals for Robust Speech Recognition in Real-World Noisy Environments" IEEE Trans. Speech, audio processing. Vol.7, No.1, pp. 55-68, Jan. 1999.
21. Levent M. Arslan, John H.L. Hansen. "Selective Training for Hidden Markov Models with Applications to Speech Classication" . IEEE Trans. Speech, audio processing. Vol. 7, No.1, pp. 46-54, Jan. 1999.
22. Rivarol Vergin, Douglas O' Shaughnessy, Azarshid Farhat. "Generalized Mel Frequency Cepstral Coefficients for Large-Vocabulary Speaker Independent Continuous-Speech Recognition" IEEE Trans. Speech, audio processing. Vol.7, No. 5, pp. 525-532, Sep. 1999.
23. Zenton Goh, Kah-Chye Tan, B.T.G. Tan. "Kalman-Filtering Speech Enhancement Method Based on a Voiced-Unvoiced Speech Model" IEEE Trans. Speech, audio processing. Vol. 7, No. 5, pp. 510-524, Sep. 1999.
24. Chulhee Lee, Donghoon Hyun, Euisun Choi et. "Optimizing Feature Extraction for Speech Recognition" IEEE Trans. Speech, audio processing. Vol. 11, No.1, pp. 80-87, Jan. 2003.
25. Stan Z.li. "Content-Based Audio Classification and Retrieval Using the Nearest Feature Line Method" IEEE Trans. Speech, audio processing. Vol.8, NO. 5, pp. 619-625, Sep. 2000.
26. Erling Wold, Thom Blum et. "Content-Based Classification, Search, and Retrieval of Audio" IEEE Multimedia. pp. 27-36, Fall,1996.
27. Hahn-Ming Lee, Chih-Ming Chert, Yung-Feng Lu. "A Self-Organizing HCMAC Neural-Network Classifier" IEEE Trans. Neural Networks. Vol.14, No.1, Jan. 2003.
28. L. Rabiner, B.H.Juans. Fundamentals of Speech Recognition. 1st ed. Beijing. Tsinghua University Press, 1999.
29．陈永彬，王仁华．语言信号处理．第1版．合肥：中国科学技术大学出版社，1990．
30. Guodong Guo, Stan Z. Li. "Content-Based Classification and Retrieval by Support Vector Machines" IEEE Trans. Neural Networks. Vol.14, No.1, pp. 209-215, Jan. 2003.
31. Darryl William Purnell, Elizabeth C. Botha. "Improved Generalization of MCE Parameter Estimation with Application to Speech Recognition" IEEE Trans. Speech, audio processing. Vol. 10, No. 4, pp. 232-239, May 2002.
32. Willianm M.Campbell, Khaled T. Assaleh, Charles C. Broun. "Speaker Recognition With Polynomial Classifier". IEEE Trans. Speech, audio processing. Vol.10, No. 4, pp. 205-212, May 2002.
33. Stephen A. Zahorian, Zaki B. Nossair. "A Partitioned Neural Network ipproch for Vowel Classification Using Smoothed Time/Frequency Features" IEEE Trans. Speech, audio processing. Vol. 7, No. 4, pp. 414-425, Jan. 1999.
34．黄峥．浅谈数字音频格式．音响技术．2002，4：8-10．
35．卢坚，毛兵，孙正行，张福炎．一种改进的基于说话者的语音分割算法．软件学报．2002，13(2)：274-279．
36．马祥杰，孟相如，张百生．一种自相关基因检测算法．通信技术．2003(3)：9-11．
37．赵庆卫，肖熙，王作英等．段长信息在连续语音识别中的应用研究．声学学报．2000，25(2)：175-181．
38．张红，黄泰翼，李治．基于半波差分谱的语音信号音节切分．声学学报．2000，25(14)：323-328．
39．卢绪刚，陈道文．听觉计算模型在鲁棒性语音识别中的应用．声学学报．2000，25(6)：492-498．
40．张焱，张杰，黄志同．基于听觉模型的鲁棒性语音识别的研究．模式识别与人工智能．1998，11(3)：341-346．
41．徐济仁，朱俊樵，谢明祥．基于声韵分割的语音信号特征提取技术．小型微型计算机系统．2002，23(2)：172-175．
42．徐建华．一种新型的多媒体检索技术 1)—基于内容的检索．情报学报．2000，19(4)：411-416．
43．邓军华，钟兵．MPEG-4中的音频编码技术．数据通信．2002(3)：6-9．
44．李建良，蒋勇，汪光先．计算机数值方法．第1版．南京：东南大学出版社．
45．陈亚勇．Matlab信号处理详解．第1版．北京：人民邮电出版社．
46．马鸿飞，樊昌信，宋国乡．基于小波变换和音质模型的音频编码算法研究．电子学报．2001，29(8)：pp．1028-1031．
47．陈明．神经网络模型．第1版．大连：大连理工大学出版社．1995．
48．杨行峻，郑君里．人工神经网络模型．第1版．高等教育出版社．1992．
49．李庆扬等．非线性方程组数值解法．科学出版社．1997
50．袁亚湘，孙文瑜．最优化理论与方法．科学出版社．1997．
51. Roy. S Near-optimal Dynamic Learning Rate for Training Back-Propagation Neural Networks. Science of Artificial Neural Networks. Ⅱ Bellingham. WA: Society of Industrial and Applied Mathematics. 1993, pp. 277-283
52. Jacobs. K.L et. Increased Rates of Convergence through Learning Rate Adaptation. Neural Networks, 1(4), pp. 295-307.
53. S.V. Kamarthi, S. Pittner. Accelerating Neural Networks Training Using Weight Extrapolations. Neural Networks. 12, pp. 1285-1299, 1999.
54. Osowski, S. Signal Flow Graphs and Neural Networks. Biological Cybernetics, 70(1993), pp. 387-395.
55．冯果忱．非线性方程组迭代解法．上海科学技术出版社．1989．
56. Selection and Analysis of HMM's State—Number in Speech Recognition. IEEE on Signal Processing. 1998, 10. Beijing China.
57．王伟．人工神经网络原理．第1版．北京：北京航空航天大学
58．朱小燕，王昱，徐伟．基于循环神经网络的语音识别模型．计算机学报．2001，24(2)：213-218．
59．李苇营，易克初，胡征．神经网络与HMM构成的混合网络在语音识别中应用的研究．电子学报．1994，22(10)：73-80．
60．杨福生．小波变换的分析与应用．科学出版社．1999．