基于多传感器数据融合仿真系统的人机交互技术研究
摘要
分布式信息处理、传感器技术、网络技术的发展推动了多传感器数据融合技术的发展,该技术目前已被广泛地应用于军事和民用的各个领域,在各个方面对人们的生产和生活起了积极的作用。目前多传感器数据融合技术的研究主要集中在融合算法上,对该技术的人机交互研究有所不足。人机交互设计的目‘的在于使用户安全、有效地实现各种功能,其设计的好坏直接影响到系统能否得到很好的推广,因此多传感器数据融合仿真系统的人机交互研究具有重要的理论和现实意义。
论文研究了在多传感器数据融合仿真系统这一特殊环境中的人机交互技术,主要研究工作有：
(1)分析了多传感器数据融合仿真系统的具体需求,包括功能、性能、输入输出、数据管理以及安全性等方面的需求；
(2)对一些经典的人机交互模型进行研究,分析经典模型的优势和不足之处,并结合多传感器数据融合仿真系统的具体需求,提出了面向任务的人机交互模型-PITA(Presentation-Interaction-Task Management-Application)模型,该模型将人机交互过程分成表现层、任务层、任务管理层和应用层；
(3)分析PITA模型和MVC模型在结构上的异同,并将两者结合,提出了PITA-MVC实现模型；
(4)基于PITA模型和PITA-MVC实现模型,研究了多传感器数据融合仿真系统的人机交互的设计方法,包括系统结构、系统的显示风格、系统之间的交互以及系统网络传输的设计；
(5)实现了多传感器数据融合仿真系统,在系统中实现了三维仿真界面、提示界面、导航界面和系统界面；考虑到不同终端的异构性,采用了界面自适应技术,实现了系统在不同平台上的移植；实现了各种交互技术,包括双缓冲绘图技术、异或绘图技术、鹰眼技术等。
论文研究工作表明,PITA模型适合多传感器数据融合仿真系统,能够满足系统在功能、性能、安全性和稳定性上的诸多要求。PITA模型支持对话独立性、界面自适应、应用的可移植性、任务的可控制性并能给用户及时的反馈。实现了多传感器数据融合仿真系统,并在系统中采用了多种人机交互技术。
The development of distributed information technology, sensor technology and network technology has contributed to multi-sensor data fusion technology, the technology has been widely used in various fields of military and civilian, play an active role in all aspects of people's production and life. At present, the research of multi-sensor data fusion is focused on the integration algorithm, the technology of human-computer interaction(HCI) is inadequate. Interaction aims to enable users to achieve a variety functions in a safe and effective way. The design will have a direct impact on the spread of the system, so the research of HCI on multi-sensor data fusion system has a strong theoretical and practical significance.
The thesis studies HCI technology under the special environment of multi-sensor data fusion simulation system, and main research work are as follows:
(1)Analyze the specific needs of multi-sensor data fusion simulation system, including function, performance, input and output, data management, security and so on.
(2)Study some classic HCI models, analyze their strengths and weaknesses, combined with the specific needs of multi-sensor data fusion simulation system, task-oriented human-computer interaction model-PITA model (Presentation-Interaction-Task Management-Application) has been proposed, which divides the process of HCI into presentation layer, task layer, task management and application layer;
(3) Analyze the similarities and differences in the structure between PITA model and MVC model, PITA-MVC implementation model is proposed by combining them;
(4)Study the design of HCI in multi-sensor data fusion simulation system, based on PITA model and PITA-MVC implementation model, including system structure, system display style, interaction between the various parts of the system and the network transmission process between systems;
(5)Achieve multi-sensor data fusion simulation system, achieve three dimension-al simulation interface, prompt interface, navigation interface and system interface; Taking into account the heterogeneity of different terminals, the system interface using adaptive technology to ensure the system can be used on different platforms; Achieve a variety of interactive technologies, including double buffering graphics technology, XOR graphics technology, Hawkeye technology and so on.
The research of this thesis show that PITA model fits multi-sensor data fusion simulation system, it can satisfy the needs on function, performance, security and stability. PITA model supports dialogue independence, adaptive interfaces, application portability, controlled task and give user immediate feedback. Achieve multi-sensor data fusion simulation system, and many HCI technologies are used in the system.
论文研究了在多传感器数据融合仿真系统这一特殊环境中的人机交互技术,主要研究工作有：
(1)分析了多传感器数据融合仿真系统的具体需求,包括功能、性能、输入输出、数据管理以及安全性等方面的需求；
(2)对一些经典的人机交互模型进行研究,分析经典模型的优势和不足之处,并结合多传感器数据融合仿真系统的具体需求,提出了面向任务的人机交互模型-PITA(Presentation-Interaction-Task Management-Application)模型,该模型将人机交互过程分成表现层、任务层、任务管理层和应用层；
(3)分析PITA模型和MVC模型在结构上的异同,并将两者结合,提出了PITA-MVC实现模型；
(4)基于PITA模型和PITA-MVC实现模型,研究了多传感器数据融合仿真系统的人机交互的设计方法,包括系统结构、系统的显示风格、系统之间的交互以及系统网络传输的设计；
(5)实现了多传感器数据融合仿真系统,在系统中实现了三维仿真界面、提示界面、导航界面和系统界面；考虑到不同终端的异构性,采用了界面自适应技术,实现了系统在不同平台上的移植；实现了各种交互技术,包括双缓冲绘图技术、异或绘图技术、鹰眼技术等。
论文研究工作表明,PITA模型适合多传感器数据融合仿真系统,能够满足系统在功能、性能、安全性和稳定性上的诸多要求。PITA模型支持对话独立性、界面自适应、应用的可移植性、任务的可控制性并能给用户及时的反馈。实现了多传感器数据融合仿真系统,并在系统中采用了多种人机交互技术。
The development of distributed information technology, sensor technology and network technology has contributed to multi-sensor data fusion technology, the technology has been widely used in various fields of military and civilian, play an active role in all aspects of people's production and life. At present, the research of multi-sensor data fusion is focused on the integration algorithm, the technology of human-computer interaction(HCI) is inadequate. Interaction aims to enable users to achieve a variety functions in a safe and effective way. The design will have a direct impact on the spread of the system, so the research of HCI on multi-sensor data fusion system has a strong theoretical and practical significance.
The thesis studies HCI technology under the special environment of multi-sensor data fusion simulation system, and main research work are as follows:
(1)Analyze the specific needs of multi-sensor data fusion simulation system, including function, performance, input and output, data management, security and so on.
(2)Study some classic HCI models, analyze their strengths and weaknesses, combined with the specific needs of multi-sensor data fusion simulation system, task-oriented human-computer interaction model-PITA model (Presentation-Interaction-Task Management-Application) has been proposed, which divides the process of HCI into presentation layer, task layer, task management and application layer;
(3) Analyze the similarities and differences in the structure between PITA model and MVC model, PITA-MVC implementation model is proposed by combining them;
(4)Study the design of HCI in multi-sensor data fusion simulation system, based on PITA model and PITA-MVC implementation model, including system structure, system display style, interaction between the various parts of the system and the network transmission process between systems;
(5)Achieve multi-sensor data fusion simulation system, achieve three dimension-al simulation interface, prompt interface, navigation interface and system interface; Taking into account the heterogeneity of different terminals, the system interface using adaptive technology to ensure the system can be used on different platforms; Achieve a variety of interactive technologies, including double buffering graphics technology, XOR graphics technology, Hawkeye technology and so on.
The research of this thesis show that PITA model fits multi-sensor data fusion simulation system, it can satisfy the needs on function, performance, security and stability. PITA model supports dialogue independence, adaptive interfaces, application portability, controlled task and give user immediate feedback. Achieve multi-sensor data fusion simulation system, and many HCI technologies are used in the system.
引文
[1]Edward Waltz, James Llinas. Multisensor Data Fusion. London:Artech House,1990
[2]Lawrence A. Klein. Sensor and Data Fusion Concepts and Applications. Bellingham, Washington:SPIE Optical Engineering Press,1999
[3]何友,王国宏,陆大金.多传感器信息融合及应用(第二版).北京：电子工业出版社,2007
[4]潘泉,于听,程咏梅.信息融合理论的基本方法与进展.北京：自动化学报,2003,29(4)：599-615
[5]王璇,李春升,周荫清.多传感器信息融合技术.北京：航空航天大学学报,1994,20(4)402-406
[6]孟祥旭,李学庆.人机交互技术.北京：清华大学出版社,2004：1-5,122-123,157-158,166-169
[7]董士海,王衡.人机交互.北京：北京大学出版社,2004：1-2
[8]陈启安.软件人机界面设计.北京：高等教育出版社,2008：3-5
[9]Lizhong Wu, Sharon L. Oviatt, Philip R. Cohen. Multimodal Integration-A Statistical View. IEEE Transactions on Multimedia,1999,1(4):334-341
[10]Edward Ross. Intelligent User Interfaces:Survey and Research Directions. http://www.cs.bris.ac.uk,2000,3
[11]Interactive Systems Labs at CMU. http://www.is.cs.cmu.edu/js/,2010,4
[12]Joelle Coutaz, Laurence Nigay, D. Salber, et al. Four Easy Pieces for Assessing the Usability of Multimodal Interaction:The CARE Properties. Proceedings of INTERACT95 Conference, 1995,6:115-120
[13]HCI at Stanford. http://hci.stanford.edu/,2010,3
[14]MIT Media Lab Research. http://www.media.mit.edu/research/,2010,4
[15]HI-SPACE DEVELOPMENT at Washington University. http://www.hitl.washington.edu/research/hispace/index.html,2010,3
[16]普建涛,董士海.任务制导的多通道分层整合模型及其算法.北京：计算机研究与发展.2001,(8)：966-971
[17]微软亚洲研究院自然用户界面项目.http://www.msra.cn/research/,2010,3
[18]User Interface Technologies projects. http://www.research.ibm.com/compsci/hci/index.html,2010,4
[19]Coutaz J, Paterno F, Faconti G, Nigay L. A comparison of approaches for specifying multimodal interactive systems. Proceedings of the ERCIM Workshop on Multimodal Human-Computer Interaction,1993,11
[20]European Strategic Programmer for Research in Information Technology. http://www.auditmypc.com/acronym/ESPRIT.asp,2010,4
[21]Nonogaki Hajime, Ueda Hirotada. FRIEND21 Project:A Construction of 21st Century Human Interface. Proceedings of Conference on Human Factors in Computing Systems (CHI'91),1991:407-414
[22]Carr B, Goldstein I. P. Overlays:A Theory of Modeling for Computer Aided Instruction (Technical Report AI Memo 406). Cambridge, Massachusetts:AI Laboratory, Massachusetts Institute of Technology,1977
[23]E. Rich. User Modeling via Stereotypes. Cognitive Science,1979,3(4):329-354
[24]Wolfgang Wahlster, Alfred Kobsa. User models in dialog systems. Berlin, Heidelberg: Springer,1989:35-51
[25]Gerhard Weber, Marcus Specht. User Modeling and Adaptative Navigation Support in WWW-Based Tutoring Systems. Proceedings of the 6th International Conference on User Modeling(UM97),1997:289-300
[26]Gerhard Weber, Peter Brusilovsky. ELM-ART:An Adaptive Versatile System for Web-based Instruction. International Journal of Artificial Intelligence in Education,2001,12:351-384
[27]J. Fink, A. Kobsa, A. Nill. User oriented adaptivty and adaptability in the Avanti Project. Redmond Washington:Microsoft Usability Group,1996
[28]Billsus D, M. J. Pazzani. User Modeling for Adaptive News Access. Netherlands:Kluwer Academic Publisher,2000,10
[29]Art Technology Group. http://www.atg.com/,2010,4
[30]人机交互技术与智能信息处理实验室.http://iel.iscas.ae.cn/lab academic/lab_academic_chi.htm,2010,4
[31]智能人机交互课题组.http://www.jdl.ac.cn/project/5.asp,2010,4
[32]北京大学图形与交互技术实验室.http://www.graphics.pku.edu.cn/research.htm,2010,4
[33]清华大学人机交互与媒体集成研究所.http://media.cs.tsinghua.edu.cn/,2010,4
[34]浙江大学计算机辅助设计与图形学国家重点实验室,http://www.cad.zju.edu.cn/web2008/index.jsp,2010,4
[35]Hong Guo, Xiaoqing Ding. Comprehensive recognition method for improving the robustness of recognition of printed Chinese characters. Proceedings of the 1st International Conference on Multimodal Interface(ICMI96),1996,10:221-226
[36]APCHI Conference 2002. http://www.hcirn.com/res/event/apchi.php,2010,3
[37]HCI International 2007. http://www.hci-international.org/index.php?module=conference&CF_op=view&CF_id=5, 2010,3
[38]2010 China Symposium on Human Computer Interaction. http://research.microsoft.com/en-US/events/chinahci2010/default.aspx,2010,4
[39]Trygve Reenskaug. MVC XEROX 1978-79. http://heim.ifi.uio.no/-trygver/mvc/index.html,2010,3
[40]Joelle Coutaz. PAC, an Object Oriented Model for Dialog Design. North Holland, Amsterdam:Elsevier Science Publishers,1987
[41]Berson, Alex. Client/Server Architecture. New York:McGraw-Hill Press.1992
[42]Mikko Korpela. From Legacy Systems via Client/Server to Web Browser Technology in Hospital Informatics in Finland, Proceedings of MEDINFO'98 Conference,1998:222-226
[43]杨培添.软件界面设计.北京：电子l：业出版社,2007
[44]罗仕鉴,朱上上,孙守迁.人机界面设计.北京：机械工业出版社,2002
[45]郭亚军,金先级.人机交互.武汉：华中科技大学出版社,2005
[46]谢希仁.计算机网络(第五版).北京：电子工业出版社,2008
[2]Lawrence A. Klein. Sensor and Data Fusion Concepts and Applications. Bellingham, Washington:SPIE Optical Engineering Press,1999
[3]何友,王国宏,陆大金.多传感器信息融合及应用(第二版).北京：电子工业出版社,2007
[4]潘泉,于听,程咏梅.信息融合理论的基本方法与进展.北京：自动化学报,2003,29(4)：599-615
[5]王璇,李春升,周荫清.多传感器信息融合技术.北京：航空航天大学学报,1994,20(4)402-406
[6]孟祥旭,李学庆.人机交互技术.北京：清华大学出版社,2004：1-5,122-123,157-158,166-169
[7]董士海,王衡.人机交互.北京：北京大学出版社,2004：1-2
[8]陈启安.软件人机界面设计.北京：高等教育出版社,2008：3-5
[9]Lizhong Wu, Sharon L. Oviatt, Philip R. Cohen. Multimodal Integration-A Statistical View. IEEE Transactions on Multimedia,1999,1(4):334-341
[10]Edward Ross. Intelligent User Interfaces:Survey and Research Directions. http://www.cs.bris.ac.uk,2000,3
[11]Interactive Systems Labs at CMU. http://www.is.cs.cmu.edu/js/,2010,4
[12]Joelle Coutaz, Laurence Nigay, D. Salber, et al. Four Easy Pieces for Assessing the Usability of Multimodal Interaction:The CARE Properties. Proceedings of INTERACT95 Conference, 1995,6:115-120
[13]HCI at Stanford. http://hci.stanford.edu/,2010,3
[14]MIT Media Lab Research. http://www.media.mit.edu/research/,2010,4
[15]HI-SPACE DEVELOPMENT at Washington University. http://www.hitl.washington.edu/research/hispace/index.html,2010,3
[16]普建涛,董士海.任务制导的多通道分层整合模型及其算法.北京：计算机研究与发展.2001,(8)：966-971
[17]微软亚洲研究院自然用户界面项目.http://www.msra.cn/research/,2010,3
[18]User Interface Technologies projects. http://www.research.ibm.com/compsci/hci/index.html,2010,4
[19]Coutaz J, Paterno F, Faconti G, Nigay L. A comparison of approaches for specifying multimodal interactive systems. Proceedings of the ERCIM Workshop on Multimodal Human-Computer Interaction,1993,11
[20]European Strategic Programmer for Research in Information Technology. http://www.auditmypc.com/acronym/ESPRIT.asp,2010,4
[21]Nonogaki Hajime, Ueda Hirotada. FRIEND21 Project:A Construction of 21st Century Human Interface. Proceedings of Conference on Human Factors in Computing Systems (CHI'91),1991:407-414
[22]Carr B, Goldstein I. P. Overlays:A Theory of Modeling for Computer Aided Instruction (Technical Report AI Memo 406). Cambridge, Massachusetts:AI Laboratory, Massachusetts Institute of Technology,1977
[23]E. Rich. User Modeling via Stereotypes. Cognitive Science,1979,3(4):329-354
[24]Wolfgang Wahlster, Alfred Kobsa. User models in dialog systems. Berlin, Heidelberg: Springer,1989:35-51
[25]Gerhard Weber, Marcus Specht. User Modeling and Adaptative Navigation Support in WWW-Based Tutoring Systems. Proceedings of the 6th International Conference on User Modeling(UM97),1997:289-300
[26]Gerhard Weber, Peter Brusilovsky. ELM-ART:An Adaptive Versatile System for Web-based Instruction. International Journal of Artificial Intelligence in Education,2001,12:351-384
[27]J. Fink, A. Kobsa, A. Nill. User oriented adaptivty and adaptability in the Avanti Project. Redmond Washington:Microsoft Usability Group,1996
[28]Billsus D, M. J. Pazzani. User Modeling for Adaptive News Access. Netherlands:Kluwer Academic Publisher,2000,10
[29]Art Technology Group. http://www.atg.com/,2010,4
[30]人机交互技术与智能信息处理实验室.http://iel.iscas.ae.cn/lab academic/lab_academic_chi.htm,2010,4
[31]智能人机交互课题组.http://www.jdl.ac.cn/project/5.asp,2010,4
[32]北京大学图形与交互技术实验室.http://www.graphics.pku.edu.cn/research.htm,2010,4
[33]清华大学人机交互与媒体集成研究所.http://media.cs.tsinghua.edu.cn/,2010,4
[34]浙江大学计算机辅助设计与图形学国家重点实验室,http://www.cad.zju.edu.cn/web2008/index.jsp,2010,4
[35]Hong Guo, Xiaoqing Ding. Comprehensive recognition method for improving the robustness of recognition of printed Chinese characters. Proceedings of the 1st International Conference on Multimodal Interface(ICMI96),1996,10:221-226
[36]APCHI Conference 2002. http://www.hcirn.com/res/event/apchi.php,2010,3
[37]HCI International 2007. http://www.hci-international.org/index.php?module=conference&CF_op=view&CF_id=5, 2010,3
[38]2010 China Symposium on Human Computer Interaction. http://research.microsoft.com/en-US/events/chinahci2010/default.aspx,2010,4
[39]Trygve Reenskaug. MVC XEROX 1978-79. http://heim.ifi.uio.no/-trygver/mvc/index.html,2010,3
[40]Joelle Coutaz. PAC, an Object Oriented Model for Dialog Design. North Holland, Amsterdam:Elsevier Science Publishers,1987
[41]Berson, Alex. Client/Server Architecture. New York:McGraw-Hill Press.1992
[42]Mikko Korpela. From Legacy Systems via Client/Server to Web Browser Technology in Hospital Informatics in Finland, Proceedings of MEDINFO'98 Conference,1998:222-226
[43]杨培添.软件界面设计.北京：电子l：业出版社,2007
[44]罗仕鉴,朱上上,孙守迁.人机界面设计.北京：机械工业出版社,2002
[45]郭亚军,金先级.人机交互.武汉：华中科技大学出版社,2005
[46]谢希仁.计算机网络(第五版).北京：电子工业出版社,2008