用户名: 密码: 验证码:
一种改进的协同过滤技术在视频推荐中的应用
详细信息    本馆镜像全文|  推荐本文 |  |   获取CNKI官网全文
摘要
随着互联网技术的发展和普及,网络己成为人们获取各种信息和数字化资源的重要途径。然而,大型网站上日益增多的资源在给用户带来更多选择的同时,也使得用户不得不花费更多的时间来查询自己想要的资源。于是,如何帮助用户更快捷地获得他们所感兴趣的资源己成为当前大型资源网站的关键需求之一。
     个性化推荐技术的提出为该需求提供了一个行之有效的解决方案。该技术旨在通过研究个体用户的兴趣喜好,主动而及时地为每个用户提供最符合其个人需求的资源推荐服务,以达到协助用户快速定位在线资源的目的。协同过滤推荐技术(Collaborative FilteringRecommendation Technology)是当前最为成功、使用最为广泛的个性化推荐技术之一。该技术基于用户间兴趣需求的相似性来推荐资源,而无需考虑资源的表示形式。然而,传统的协同过滤推荐技术在推荐的准确性和效率方面仍然存在可改进之处。
     本文在学习和研究协同过滤推荐技术各种已有实施方案的基础上,针对在线视频资源的特点,分别在其推荐准确性和效率方面提出改进。首先,本文提出了一套基于用户观看行为的隐式用户评分获取策略,该策略能够有效地解决严重影响传统协同过滤推荐准确性的评分矩阵稀疏和主观评分不可靠等问题。其次,为了在传统的推荐计算模型中引入时间因素以反映用户兴趣和项目价值随时间的变化规律,进而提高推荐准确性,本文又提出了基于用户兴趣偏差的评分效用衰减函数和基于视频时效性的视频价值衰减函数。最后,本文提出的基于上下文的分布式协同过滤推荐技术通过评分矩阵的分布式存储、上下文信息的获取以及在此基础上的分布式在线推荐,以做到在不降低推荐准确性的前提下,有效地解决协同过滤推荐系统应用于大规模动态数据集时的效率问题。
     实验结果证明,本文所提出的改进方案能够有效的应用于在线视频资源的个性化推荐中,且与传统的协同过滤推荐技术相比,其推荐准确性和推荐效率均有所提高。本文所提出的各项改进方案均可以直接或部分修改后应用于不同类型资源的推荐系统中,因此本研究成果具有一定的普遍意义。
With the development of Internet technology, nowadays the Internet has become such an important channel that makes all sorts of information and digital resources available for people all over the world. However, in spite of offering users nearly all kinds of information, the ever-increasing resources on most large websites force them to spend massive time on searching for what they really need. Therefore, how to help clients obtain what they want swiftly has become one of the key requirements for large resources websites.
     The proposal of the personalization recommendation technology is to bring about an effective solution addressing the above-mentioned requirement. By studying the hobbies and interests of each individual user, this technology can actively provide a kind of much more suitable recommendation service for the individual in due course. As a result, it serves as a mechanism assisting people in locating online resources quickly. The collaborative filtering recommendation technology, currently, is one of the most successful and most widely used personalization recommendation technologies in the world. This technology makes the recommendation of resources based on the similarity of interests between different users without considering the representation of various resources. Nevertheless, the traditional collaborative filtering recommendation technology still has some drawbacks in both accuracy and efficiency that need to be improved.
     Aiming at the features of online video resources, this thesis proposes some improvement methods on both accuracy and efficiency aspects of the collaborative filtering recommendation technology. Firstly, based on users watching behavior, this thesis proposes an implicit user rating collection strategy. This strategy can effectively solve several problems that have negatively influenced the recommendation accuracy of the traditional collaborative filtering technology to a large extent, including the sparsity of the rating matrix and the unreliable of the subjective ratings. Secondly, since the introduction of the time factor into the traditional recommendation model can reflect the changing pattern of user interests and item values along with the time respectively and further enhance its recommendation accuracy, this thesis proposes two reduction functions, namely the rating effectiveness reduction function based on the interest deviation and the video value reduction function based on video timeliness. Finally, this thesis proposes a context-based distributed collaborative filtering technology which includes a series of processing methods such as the rating matrix distributed storage method, the offline context information collection method and the online distributed recommendation algorithm. As a result, it can effectively solve the efficiency problem of the traditional collaborative filtering recommendation while securing its accuracy when it is applied to large and dynamic settings.
     The experimental results show that all the approaches proposed in the thesis can be more effectively applied to the personalization recommendation of online video by enhancing both recommendation accuracy and efficiency in comparison to the traditional collaborative filtering recommendation. In addition, these approaches can also be applied to other various online resources directly or after being partly modified. Thus, it has a wide range of application in practice.
引文
[1]Baeza-Yates R,Ribeiro-Neto B.Modern Information Retrieval[M].New York:Addison-Wesley Publishing Co.,1999
    [2]Shlomo Berkvosky,Tsvi Kuflik,Francesco Ricci.Distributed Collaborative Filtering with Domain Specialization[C].In:Proceedings of the 2007 ACM conference on Recommender systems,2007,33-40
    [3]刘开南.基于兴趣的电子商务数据挖掘技术的研究[J].商场现代化,2007,1.
    [4]Balabanovic M,Shoham Y.Fab:Content-Based Collaborative Recommendation.[J]Communications of the ACM,1997,40(3):66-72.
    [5]Goldberg,D.,Niehols,D.,Oki,B.M.,Terry,D.Using collaborative filtering to weave an information apestry[J].In Communications of the ACM,1992,35(12):61-70.
    [6]Konstan,J.,Miller,B.,Maltz,D.,et al.GroupLens:applying collaborative filtering to usenet news[J].In Communications of the ACM,1997,40(3):77-87.
    [7]Hill W,Stead L,Rosenstein M,Fumas G.Recommending and evaluating choices in a virtual community of use[C].In CHI 95:Conference proceedings on humanfactors in Computing Systems,Denver,CO,1995.194-201.
    [8]Miller BN,Albert I,Lam SK,Konstan JA,Riedl J.MovieLens unplugged:Experiences with an occasionally connected recommender systemiC].In:Proc.of the Int'l Conf.on Intelligent User Interfaces.New York:ACM Press,2003.263-266.
    [9]Shardanand U,Maes P.Social information filtering:Algorithms for automating of mouth[C].In Proceedings of the ACM CHI Conference,1995.210-217
    [10]Resnick Paul,Varianm Hal.Recommender Systems,introduction to special section of Communications of the ACM,March 1997,vol.40(3).
    [11]Schafer J.B.Konstan,J.Riedl,J.E-commerce recommendation applications[J].Data Mining and Knowledge Discovery,2001,1.
    [12]Chickering,D.,and Hecherman,D.Efficient Approximations for the Marginal Likelihood of Bayesian Networks with Hidden Variables[J].Machine Learning,1997.
    [13]Good.N.Schafer,J.B.Konstan,J.A.Borchers,A.Sarwar,B.Herlocker,J.Riedl,[J].Combining collaborative filtering with personal agents for better recommendations.In Proceedings of the 16th National Conference on Artificial Intelligence AAAI-99,1999:439-446.
    [14]Jae Kyeong Kim,Yoon Ho Cho,Woo Ju Kim,Je Ran Kim.A personalized recommendation procedure for Internet shopping support[J].Electronic Commerce Research and Applications,2002,pp.301-313.
    [15]R.J.Kuo,J.L.Liao,C.Tu.Integration of ART2 neural network and genetic k-means algorithm for analyzing web browsing paths in electronic commerce[J].Decision Support Systems,2005,pp.355-374.
    [16]Resnick P,Varian HR.Recommender systems[J].Communications of the ACM,1997,40(3):56-58.
    [17]Adomavicius G,Tuzhilin A.Toward the next generation of recommender systems:A survey of the state-of-the-art and possible extensions[J].IEEE Trans.on Knowledge and Data Engineering,2005,17(6):734-749.
    [18]Ansari,A.,Esseagaier,S.,and Kohli,R.Internet Recommendation Systems[J].Journal of Marketing Research,Vo,37,No.3,2000:363-375.
    [19]Yang,Y.An Evaluation of Statistical Approaches to Text Categorization[J].Information Retrieval,1999 Vol.1,No.1/2,pp.67-88.
    [20]刘柏嵩.信息过滤研究[J].现代图书情报技术,2003(6):23
    [21]Shardanand,U.,and Maes,P.Social Information Filtering:Algorithms for Automating "Word of Mouth"[C].In Proceedings of ACM CHI'95 Conference on Human Factors in Computing System,1995.
    [22]Agrawal,R.,Lmielinski,T.,and Swami,A.,Mining Associations Between Sets of Items in Large Databases[C].Proc.of the ACM SIGMOD Int'l Conference on Management of Data,pp.207-216,Washington D.C.,May,1993.
    [23]韩家炜,数据挖掘,机械工业出版社,p147-149
    [24]Agrawal,R.and Srikant,R.,Fast Algorithms for Mining Association Rules[C].Proc.20th Int.Conf.Very Large Data Bases,VLDB,pp.487-499,Santiago,Chile,Sept.,1994.
    [25]邹显春,谢中,周彦晖.电子商务与web数据挖掘[J].计算机应用,2001,21(5):21-23.
    [26]Resnick,P.,Iacovou,N.,Suchak,M.,Bergstrom,P.and Riedl,Grouplens:An Open Architecture for Collaborative Filtering of Netnews[C].In Proceedings of the Conference on Computer Supported Cooperative Work,Chapel Hill,1994:175-186.
    [27]邓爱林,朱扬勇,施伯乐.基于项目评分预测的协同过滤推荐算法[J].软件 学报,2003.(09):p.1-8.
    [28]Badrul Sarwar,George Karypis,Joseph Konstan,John Riedl.Item-based collaborative filtering recommendation algorithms[C].In:Proceedings of the 10th international conference on World Wide Web,2001,Hong Kong,285-295.
    [29]Goldberg,K.,Roeder,T.,Gupta,D.,and Perkins,C.,Eigentaste:A Constant Time Collaborative Filtering Algorithm[J],Information Retrieval,vol.4(2),pp.133-151,2001.
    [30]孙军,陈峰,ROC曲线分析在放射学中的应用[J].中华发射学杂志,2001.
    [31]SWAMI Collaborative Filtering Evaluation Framework[CP/OL].http://guir.cs.berkeley.edu/projects/swami/,2004.
    [32]Maritza L.Calderon-Benavides,Cristina N.Gonzalez-Caro,Jose de J.Perez-Alcazar,Juan C.Garcia-Diaz,and Joaquin Delgado.A Comparison of Several Predictive Algorithms for Collaborative Filtering on Multi-valued Ratings[C].ACM Symposium on Applied Computing,2004.
    [33]Dhruv Grupta,Mark Digiovanni,Hiro Narita,and Ken Goldberg,Jester 2.0:Evaluation of a New Linear Time Collaborative Filtering Algorithm[C].SIGIR'99,1999.
    [34]Adomavicius G,Tuzhilin A.Toward the next generation of recommender systems:A survey of the state-of-the-art and possible extensions[J].IEEE Trans.on Knowledge and Data Engineering,2005,17(6):734-749.
    [35]Adomavicius G,Tuzhilin A.Multidimensional recommender systems:A data warehousing approach[C].In:Proc.of the 2nd Int'l Workshop on Electronic Commerce.LNCS 2232,London:Springer-Verlag,2001.180-192.
    [36]Canny J.Collaborative filtering with privacy via factor analysis[C].In:Proc.of the 25th Int'l ACM SIGIR Conf.on Research and Development in Information Retrieval.New York:ACM Press,2002.238-245.
    [37]Polat H,Du WL.Privacy-Preserving collaborative filtering using randomized perturbation techniques[C].In:Proc.of the 3rd IEEE Int'l Conf.on Data Mining.Washington:IEEE Computer Society,2003.625-628.
    [38]Mooney RJ,Roy L.Content-Based book recommending using learning for text categorization[C].In:Proc.of the ACM SIGIR'99 Workshop Recommender Systems:Algorithms and Evaluation.New York:ACM,1999.195-204.
    [39]Herlocker JL,Konstan JA,Terveen LG,Riedl J.Evaluating collaborative filtering recommender systems[J].ACM Trans.on Information Systems,2004,22(1):5-53.
    [40]Herlocker,J.,Understanding and Improving Automated Collaborative Filtering System[D].Ph.D.Thesis,Computer Science Dept.,University of Minnesota,2000.
    [41]Sarwar,B.M.m Karypis,G.,Konstan,J.A.,and Riedl,J.,Application of Dimensionality Reduction in Recommender System-A Case Study[C].In ACM Web KDD 2000 Workshop,2000.
    [42]Deereester,S.,Dumais,S.T.,Furnas,G.W.,Landauer,T.K.,and Harshman,R.,Indexing by Latent Semantic Analysis[J],Journal of American Society for Information Science,vol.41(6),pp.391-407,1990.
    [43]Cheung,K.-W.,Tsui,K.-C.,and Liu,L.,Extended latent class modes for collaborative recommendation[J],System,Man and Cybernetics,Part A,IEEE Transactions on System,Man,and Cybernetics,vol.34(1),Jan.,2004.
    [44]Chee,S.H.S.,RecTree:A Linear Collaborative Filtering Algorithm[D].M.Sc.thesis,Computing Science Department,Simon Fraser University,2000.
    [45]Chee,S.H.S.,Han,J.,and Wang,K.,RecTree:A Linear Collaborative Filtering Method[C].Proceedings of Data Warehouse and Knowledge Discovery,Munich,Germany,Sept.,2001.
    [46]QiLin Li and Mingtian Zhou,Research and design of an efficient collaborative filtering predication algorithm[J].Parallel and Distributed Coumputing,Applications and Technologies,pp.171-174,Aug.,2003.
    [47]Tiffany Ya Tang,Gordon McCalla,Mining Implicit Ratings for Focused Collaborative Filtering for Paper Recommendations[C].UM03 Workshop,2003.
    [48]Yi Ding,Xue Li,Mafia E.Orlowska,Recency-Based Collaborative Filtering[C].17th Australasian Database Conference,Vol.49,2006.
    [49]邢春晓,高凤荣,战思南,周立柱,适应用户兴趣变化的协同过滤推荐算法[J].计算机研究与发展,Vol.44(2):296-301,2007.
    [50]Tong Queue Lee,Young Park,Yong-Tae Park,A time-based approach to effective recommender systems using implicit feedback[J].Expert Systems with Applications Vol.34:3055-3062,2008.
    [51]郑先荣,曹先彬,线性逐步遗忘协同过滤算法的研究[J].计算机工程,Vol.33(6),2007.
    [52]Abhinandan Das,Mayur Datar,Ashutosh Garg.Google News Personalization:Scalable Online Collaborative Filtering[C].In:Proceedings of the 16th international conference on World Wide Web,2007,271-280.
    [53]Shlomo Berkvosky,Tsvi Kuflik,Francesco Ricci.Cross-Domain Mediation in Collaborative Filtering[J]. Lecture Notes in Computer Science, Springer Berlin , Vol.4511, 2007, 355-359.

© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号

地址:北京市海淀区学院路29号 邮编:100083

电话:办公室:(+86 10)66554848;文献借阅、咨询服务、科技查新:66554700