CADAL新型阅读体验研究
|
摘要
从古到今,人类从未停止过寻求知识和探索真理的脚步；记录和阅读作为求知的一种手段亦是由来长久,从遥远的壁画或是简牍到现在的纸本书卷,阅读已经成为人类学习生活的重要部分。随着技术的发展,尤其是当代电子产业和计算机技术的日新月异,阅读的媒介和方式也发生了翻天覆地的变化。传统的记录媒介通常是笔墨以及纸张,阅读方式是单一的人手一本书地阅读,传统意义上的图书馆也仅仅是大的物理书本的集合以及分类呈现。而随着大型展示屏幕,便携式平板电子产品以及三维动画技术等的使用,阅读方式变得丰富多彩起来；图书不仅文字的表现,更辅以多媒体的表现媒介以及2D,3D的图形模型；阅读和记录的交互方式也多样化起来,不仅可以虚拟化模拟现实阅读,也可以辅助增强阅读以及记录的体验。
本文依靠CADAL已有的电子图书资源,在特有硬件的支持下研究在特定环境下如何增强用户体验以及实现阅读平台,主要工作包括：
第一,对于在大屏幕显示器的依托下的阅读展示平台CReader,设计实现该系统的两个模块,包括设计实现新型的图书内容融合与展示模块,即设计实现跨媒体图书阅读；以及利用Kinect设计实现系统中的姿势交互,并介绍相关关键技术。
第二,在深度摄像头Kinect的支持下,实现基于传统阅读方式的电子笔记自动生成系统D-Note。该系统支持传统的物理书的阅读,通过获取和分析深度图像信息和彩色图像信息,以及获得的指尖轨迹信息生成电子笔记。
Past and present, human never stop their steps heading on seeking for knowledge and employing the truth. As one of the methods for seeking knowledge, writing and reading has a long history too. From fresco and bamboo slips which are old-age to paper books now, reading has become one important part of our studying live. With the development of technology, especially the rapid development of contemporary electronic industry and computer technology, the medium and way of reading have great change too. Traditional recording mediums include pen and paper, the traditional reading is just one person keep one book, the library in traditional sense is just collections and classification and the finally present of books. But now, by using the large display, pad, and phone, reading becomes rich and colorful, not only including the present of text, but also including the present of multimedia and 2D images,3D models. The interaction becomes rich too, not only including virtual presentation of the traditional books but also including real time interactions which trigger the auxiliary functions.
Using the digital books in CADAL, we research how to implement reading systems under different conditions and different hardware support. We mainly do work in three aspects.
Firstly, perfect the reading system CReader, which under the support of the touched large display, and enhance its user experience. We discuss how to design and implement the cross-media book and pose interaction with the support of Kinect.
Secondly, with the support of Kinect, we design a digital note taking system D-Note. D-Note supports traditional reading with physical books, meanwhile output digital note through analyzing depth image and color image and the tracks of fingertip. Here we will discuss the detail workflow and key technical.
本文依靠CADAL已有的电子图书资源,在特有硬件的支持下研究在特定环境下如何增强用户体验以及实现阅读平台,主要工作包括：
第一,对于在大屏幕显示器的依托下的阅读展示平台CReader,设计实现该系统的两个模块,包括设计实现新型的图书内容融合与展示模块,即设计实现跨媒体图书阅读；以及利用Kinect设计实现系统中的姿势交互,并介绍相关关键技术。
第二,在深度摄像头Kinect的支持下,实现基于传统阅读方式的电子笔记自动生成系统D-Note。该系统支持传统的物理书的阅读,通过获取和分析深度图像信息和彩色图像信息,以及获得的指尖轨迹信息生成电子笔记。
Past and present, human never stop their steps heading on seeking for knowledge and employing the truth. As one of the methods for seeking knowledge, writing and reading has a long history too. From fresco and bamboo slips which are old-age to paper books now, reading has become one important part of our studying live. With the development of technology, especially the rapid development of contemporary electronic industry and computer technology, the medium and way of reading have great change too. Traditional recording mediums include pen and paper, the traditional reading is just one person keep one book, the library in traditional sense is just collections and classification and the finally present of books. But now, by using the large display, pad, and phone, reading becomes rich and colorful, not only including the present of text, but also including the present of multimedia and 2D images,3D models. The interaction becomes rich too, not only including virtual presentation of the traditional books but also including real time interactions which trigger the auxiliary functions.
Using the digital books in CADAL, we research how to implement reading systems under different conditions and different hardware support. We mainly do work in three aspects.
Firstly, perfect the reading system CReader, which under the support of the touched large display, and enhance its user experience. We discuss how to design and implement the cross-media book and pose interaction with the support of Kinect.
Secondly, with the support of Kinect, we design a digital note taking system D-Note. D-Note supports traditional reading with physical books, meanwhile output digital note through analyzing depth image and color image and the tracks of fingertip. Here we will discuss the detail workflow and key technical.
引文
[1]Microsoft Word. [OL] http://office.microsoft.com/
[2]Adobe reader pdf [OL] http://www.adobe.com/
[3]Zmaker. Zinemaker[OL] http://zmaker.zcom.com/
[4]Enrico Costanza, Andreas Kunz and Morten Fjeld. Mixed Reality:A Survey [J], Lecture Notes in Computer Science,2009,Volume 5440/2009:47-68
[5]CADAL管理中心.大学数字图书馆国际合作计划项目 [0L]http://www.cadal.zju.edu.cn
[6]Iphone [OL] www.apple.com/iphone/
[7]ZhenKun Zhou, Jiangqin Wu, Yin Zhang, DaWei Xie and YueTing Zhang. Javelin:an access and manipulation interface for large display [C], ICUDL.2010:893-902
[8]Sangwon Choi, Jaehyun Han, Geehyuk Lee, Narae Lee,Woohun Lee. Remote Touch:touch-screen-like interaction in the tv viewing environment, [C] CHI'l 1.2011
[9]Tan, D.S., Meyers, B., Czerwinski, M. WinCuts:Manipulating Arbitrary Window Regions for More Effective Use of Screen Space [C], Extended Abstract, ACM CHI 2004
[10]George Robertson, Eric Horvitz, Mary Czerwinski, and Patrick Baudisch. Scalable Fabric:flexible task management [C], AVI'04
[11]Hrvoje Benko, Andrew D. Wilson, Patrick Baudisch. Precise selection techniques for multi-touch screens [C], CHI'06
[12]OneNote [OL] http://office.microsoft.com/en-us/onenote/
[13]EverNote [OL] http://www.evernote.com/
[14]MyBase [OL] http://www.wjjsoft.com/mybase.html
[15]Mackay, W. E. The missing link:integrating paper and electronic documents [C]. IHM'03.2003:1-8
[16]DigiMemo [OL] http://www.digimemo.com/
[17]Anoto [OL] http://www.anoto.com
[18]Khan, F. A Survey of Note-Taking Practices [C]. HPL 1993:93-107
[19]P. Brandl, C. Richter, and M. Haller. Nicebook-supporting natural note taking [C], Proc. Human Factors in Computing Systems (CHI10),2010:599-608
[20]Lynn D. Wilcox, Bill N. Schilit, Nitin Sawhney. Dynomite:A dynamically organized ink and audio notebook [C], CHI'97
[21]Mark Billingburst, Hirokazu Kato, and Ivan Poupyrev. The MagicBook:a transitional AR interface [J], Computers&Graphics,2001,25(5):745-753
[22]Raphael Grasset, Andreas Duenser, Hartmut Seichter, and Mark Billinghurst. The mixed reality book:a new multimedia reading experience [C], CHI EA'07
[23]C. Harris and M. Stephens. A combined corner and edge detector [C], Proc. Alvey vision conference (AVC88),1988:147-151
[24]E. Rosten and T. Drummond. Machine learning for high speed corner detection [C], Proc. European Conference on Computer Vision (ECCV06),2006:430-443
[25]D. G. Lowe. Distinctive image features from scale-invariant keypoints [C], Proc. Int. J. Comput. Vision 60,2(IJCV04), Nov.2004:91-110
[26]H. Bay, A. Ess, T. Tuytelaars, and L. V. Gool. SURF:Speeded up robust features [C], Proc. Comput. Vis. Image Underst.110,3(CVIU08), June 2008:346-359
[27]Hideaki Uchiyama, Julien Pilet and Hideo Saito. On-line Document Registering and Retrieving System for AR Annotation Overlay [C], Proc. Augmented Human International Conference (AH10),2010
[28]H. Uchiyama and H. Saito. Augmenting text document by on-line learning of local arrangement of keypoints [C], Proc. International Symposium on Mixed and Augmented Reality (ISMAR09),2009:95-98
[29]Chunyuan Liao, Hao Tang, Qiong Liu, Patrick Chiu, and Francine Chen. FACT: fine-grained cross-media interaction with documents via a portable hybrid paper-laptop interface [C], MM'10
[30]Maribeth Back, Jonathan Cohen, Rich Gold, Steve Harrison, and Scott Minneman. Listen reader:an electronically augumented paper-based book [C], CHI'01
[31]Shaun K. Kane, Daniel Avrahami, Jacob 0. Wobbrock, Beverly Harrison, Adam D. Rea, Matthai Philipose, and Anthony LaMarca. Bonfire:A Nomadic System for Hybrid Laptop-Tabletop Interaction [C], UIST'09
[32]Kinect [OL] http://en.wikipedia.org/wiki/Kinect
[33]OpenNI [OL] http://OpenNI.org/
[34]Khan, F.. A Survey of Note-Taking Practices, Personal Systems Laboratory [R], HP Laboratories Bristol,1993
[35]Hideaki Uchiyama, Julien Pilet and Hideo Saito. On-line Document Registering and Retrieving System for AR Annotation Overlay [C], Proc. Augmented Human International Conference (AH10),2010
[36]H. Uchiyama and H. Saito. Augmenting text document by on-line learning of local arrangement of keypoints [C], Proc. International Symposium on Mixed and Augmented Reality (ISMAR09),2009:95-98
137] Microsoft speech SDK [OL] http://www.microsoft.com/speech/
[38]OpenKinect [OL] https://github.com/OpenKinect/libfreenect
[2]Adobe reader pdf [OL] http://www.adobe.com/
[3]Zmaker. Zinemaker[OL] http://zmaker.zcom.com/
[4]Enrico Costanza, Andreas Kunz and Morten Fjeld. Mixed Reality:A Survey [J], Lecture Notes in Computer Science,2009,Volume 5440/2009:47-68
[5]CADAL管理中心.大学数字图书馆国际合作计划项目 [0L]http://www.cadal.zju.edu.cn
[6]Iphone [OL] www.apple.com/iphone/
[7]ZhenKun Zhou, Jiangqin Wu, Yin Zhang, DaWei Xie and YueTing Zhang. Javelin:an access and manipulation interface for large display [C], ICUDL.2010:893-902
[8]Sangwon Choi, Jaehyun Han, Geehyuk Lee, Narae Lee,Woohun Lee. Remote Touch:touch-screen-like interaction in the tv viewing environment, [C] CHI'l 1.2011
[9]Tan, D.S., Meyers, B., Czerwinski, M. WinCuts:Manipulating Arbitrary Window Regions for More Effective Use of Screen Space [C], Extended Abstract, ACM CHI 2004
[10]George Robertson, Eric Horvitz, Mary Czerwinski, and Patrick Baudisch. Scalable Fabric:flexible task management [C], AVI'04
[11]Hrvoje Benko, Andrew D. Wilson, Patrick Baudisch. Precise selection techniques for multi-touch screens [C], CHI'06
[12]OneNote [OL] http://office.microsoft.com/en-us/onenote/
[13]EverNote [OL] http://www.evernote.com/
[14]MyBase [OL] http://www.wjjsoft.com/mybase.html
[15]Mackay, W. E. The missing link:integrating paper and electronic documents [C]. IHM'03.2003:1-8
[16]DigiMemo [OL] http://www.digimemo.com/
[17]Anoto [OL] http://www.anoto.com
[18]Khan, F. A Survey of Note-Taking Practices [C]. HPL 1993:93-107
[19]P. Brandl, C. Richter, and M. Haller. Nicebook-supporting natural note taking [C], Proc. Human Factors in Computing Systems (CHI10),2010:599-608
[20]Lynn D. Wilcox, Bill N. Schilit, Nitin Sawhney. Dynomite:A dynamically organized ink and audio notebook [C], CHI'97
[21]Mark Billingburst, Hirokazu Kato, and Ivan Poupyrev. The MagicBook:a transitional AR interface [J], Computers&Graphics,2001,25(5):745-753
[22]Raphael Grasset, Andreas Duenser, Hartmut Seichter, and Mark Billinghurst. The mixed reality book:a new multimedia reading experience [C], CHI EA'07
[23]C. Harris and M. Stephens. A combined corner and edge detector [C], Proc. Alvey vision conference (AVC88),1988:147-151
[24]E. Rosten and T. Drummond. Machine learning for high speed corner detection [C], Proc. European Conference on Computer Vision (ECCV06),2006:430-443
[25]D. G. Lowe. Distinctive image features from scale-invariant keypoints [C], Proc. Int. J. Comput. Vision 60,2(IJCV04), Nov.2004:91-110
[26]H. Bay, A. Ess, T. Tuytelaars, and L. V. Gool. SURF:Speeded up robust features [C], Proc. Comput. Vis. Image Underst.110,3(CVIU08), June 2008:346-359
[27]Hideaki Uchiyama, Julien Pilet and Hideo Saito. On-line Document Registering and Retrieving System for AR Annotation Overlay [C], Proc. Augmented Human International Conference (AH10),2010
[28]H. Uchiyama and H. Saito. Augmenting text document by on-line learning of local arrangement of keypoints [C], Proc. International Symposium on Mixed and Augmented Reality (ISMAR09),2009:95-98
[29]Chunyuan Liao, Hao Tang, Qiong Liu, Patrick Chiu, and Francine Chen. FACT: fine-grained cross-media interaction with documents via a portable hybrid paper-laptop interface [C], MM'10
[30]Maribeth Back, Jonathan Cohen, Rich Gold, Steve Harrison, and Scott Minneman. Listen reader:an electronically augumented paper-based book [C], CHI'01
[31]Shaun K. Kane, Daniel Avrahami, Jacob 0. Wobbrock, Beverly Harrison, Adam D. Rea, Matthai Philipose, and Anthony LaMarca. Bonfire:A Nomadic System for Hybrid Laptop-Tabletop Interaction [C], UIST'09
[32]Kinect [OL] http://en.wikipedia.org/wiki/Kinect
[33]OpenNI [OL] http://OpenNI.org/
[34]Khan, F.. A Survey of Note-Taking Practices, Personal Systems Laboratory [R], HP Laboratories Bristol,1993
[35]Hideaki Uchiyama, Julien Pilet and Hideo Saito. On-line Document Registering and Retrieving System for AR Annotation Overlay [C], Proc. Augmented Human International Conference (AH10),2010
[36]H. Uchiyama and H. Saito. Augmenting text document by on-line learning of local arrangement of keypoints [C], Proc. International Symposium on Mixed and Augmented Reality (ISMAR09),2009:95-98
137] Microsoft speech SDK [OL] http://www.microsoft.com/speech/
[38]OpenKinect [OL] https://github.com/OpenKinect/libfreenect