负荷预测多维数据模型及实视图研究
|
摘要
本论文根据电力负荷预测系统的历史数据集的特点及各种预测方法、OLAP及数据挖掘的对数据源的需求，研究了多维数据建模及实视图维护的理论，以增强数据源独立性、可维护性和提高联机分析处理和数据挖掘的效率。
本文分为五个部分。第一部分简单介绍了电力负荷预测的意义、现状和存在的问题，提出建立数据仓库支持负荷预测以解决现存问题，然后突出数据仓库研究中有关逻辑建模及效率的两个关键性的理论问题：多维数据建模及实视图的选择与维护，在后面的两个部分分别对它们进行的阐述。
第二部分首先概述了多维数据建模的意义与国内外研究的现状，指出了目前人们已提出的多维数据模型的局限性，然后针对负荷预测系统的特点提出了一个新的模型，该模型解决了不同维公用一个底层层次属性，把系统中不完全的低粒度数据集和完全的粗粒度数据集在逻辑上无缝地结合起来支持联机分析处理的问题，这是以前的多维数据模型所没有解决的。
第三部分首先讨论了国内外实视图选择和维护领域内已有的研究成果，然后根据负荷预测系统OLAP，数据挖掘的特点，提出了一种新的维护算法，该算法能有效地提高数据仓库维护的效率。
第四部分讨论了如何利用本文中提出的多维数据模型及实视图维护算法支持负荷预测系统。
论文最后对全文进行总结，对需进一步研究的问题进行了展望。
Based on the characteristics of the historical data set of load forecasting system, forecasting methods and the requirements of OLAP and DM tools for data sources, this paper researches the theories of multidimensional data model and materialized views maintenance in order to improve the independence and maintainability of the data sources and the efficiency of the OLAP and DM.
The paper has four chapters. In the first chapter, the importance, present situation and problems of load forecasting are introduced simply. To solve the problems, a strategy of building a data warehouse to support the load forecasting system is proposed. To build the data warehouse, there are two important theoretical problems about multidimensional data modeling and materialized views. Both the problems are emphasized at the end of the chapter, and discussed profoundly in the following chapters.
The second chapter begins with the general review of the importance and researching status of multidimensional data modeling, then points out the limitations of the existing multidimensional data models. In order to model the multidimensional structure of the load forecasting system's data set, at the end of the chapter, the paper introduces a new model to solve the problems of different dimensions sharing a common bottom level of hierarchies and incorporating incomplete data set of fine granularity and complete data set of coarse granularity logically to support OLAP which had not been solved by prevenient models.
First, the third chapter talks about the researching achievements on materialized views, and then on the basis of load forecasting, presents a. new materialized maintenance algorithm, which can reduce the space and time cost of the materialized views maintenance.
The fourth chapter discusses how to support the load forecasting using the multidimensional data model and the view maintenance algorithms that the paper presents before.
Finally, the conclusion of the paper is given and the future work is suggested.
本文分为五个部分。第一部分简单介绍了电力负荷预测的意义、现状和存在的问题，提出建立数据仓库支持负荷预测以解决现存问题，然后突出数据仓库研究中有关逻辑建模及效率的两个关键性的理论问题：多维数据建模及实视图的选择与维护，在后面的两个部分分别对它们进行的阐述。
第二部分首先概述了多维数据建模的意义与国内外研究的现状，指出了目前人们已提出的多维数据模型的局限性，然后针对负荷预测系统的特点提出了一个新的模型，该模型解决了不同维公用一个底层层次属性，把系统中不完全的低粒度数据集和完全的粗粒度数据集在逻辑上无缝地结合起来支持联机分析处理的问题，这是以前的多维数据模型所没有解决的。
第三部分首先讨论了国内外实视图选择和维护领域内已有的研究成果，然后根据负荷预测系统OLAP，数据挖掘的特点，提出了一种新的维护算法，该算法能有效地提高数据仓库维护的效率。
第四部分讨论了如何利用本文中提出的多维数据模型及实视图维护算法支持负荷预测系统。
论文最后对全文进行总结，对需进一步研究的问题进行了展望。
Based on the characteristics of the historical data set of load forecasting system, forecasting methods and the requirements of OLAP and DM tools for data sources, this paper researches the theories of multidimensional data model and materialized views maintenance in order to improve the independence and maintainability of the data sources and the efficiency of the OLAP and DM.
The paper has four chapters. In the first chapter, the importance, present situation and problems of load forecasting are introduced simply. To solve the problems, a strategy of building a data warehouse to support the load forecasting system is proposed. To build the data warehouse, there are two important theoretical problems about multidimensional data modeling and materialized views. Both the problems are emphasized at the end of the chapter, and discussed profoundly in the following chapters.
The second chapter begins with the general review of the importance and researching status of multidimensional data modeling, then points out the limitations of the existing multidimensional data models. In order to model the multidimensional structure of the load forecasting system's data set, at the end of the chapter, the paper introduces a new model to solve the problems of different dimensions sharing a common bottom level of hierarchies and incorporating incomplete data set of fine granularity and complete data set of coarse granularity logically to support OLAP which had not been solved by prevenient models.
First, the third chapter talks about the researching achievements on materialized views, and then on the basis of load forecasting, presents a. new materialized maintenance algorithm, which can reduce the space and time cost of the materialized views maintenance.
The fourth chapter discusses how to support the load forecasting using the multidimensional data model and the view maintenance algorithms that the paper presents before.
Finally, the conclusion of the paper is given and the future work is suggested.
引文
[01] 牛东晓，曹树华，赵磊等 电力负荷预测技术及其应用[M] 北京：中国电力出版社，1998年
[02] A. D. P. Lotufo, C. R. Minussi Electric Power Systems Load Forecasting: A Survey, The IEEE Power Tech '99 Conference, Budapest, Hungary, Aug. 29 Sept. 2, 1999
[03] 袁明友，肖先勇，杨洪耕，曾立渊，杨兵 基于灰色理论的供电系统负荷中长期预测模型及其应用 四川大学学报(工程科学版) 2002年04期
[04] 宋魁刚，胡晓光，基于多层神经网络SPDS的电力负荷预测，中国电力2001年06期
[05] 罗仲达，关治洪 人工神经网络在电力系统短期负荷预测中的应用 湖南电力2002年02期
[06] 孙海斌，周淑雄，谢敬东，唐国庆 模糊综合评价在负荷最优组合预测中的应用 电力系统及其自动化学报 2001年04期
[07] 赵希正 中国电力负荷特性分析与预测[M] (第一版) 北京：电力工业出版社 2002年1月
[08] 王珊等，数据仓库技术与联机分析处理[M] 科学出版社 1998．5
[09] 李建中，高宏 一种数据仓库的多维数据模型 软件学报 2000，11(7)pp．908-917．
[10] 李琪，白英彩 数据仓库中维的建模和查询，计算机研究与发展，Vol．39，No．5 May 2002，pp．612-618
[11] 徐海蔚，杨明福 扩充的多维数据模型MDB／MDV及其建模技术 计算机应用与软件 2000，17(11)pp．12-18
[12] Jim Gray, Surajit Chaudhuri, Adam Bosworth, Andrew Layman, Don Reichart, Murali Venkatrao, Frank Pellow, Harold Pirahesh Data Cube: a relational aggregation operator generalizing group-by, cross-tab and sub-totals Microsoft Technical report MSR-TR-95-22, 5 February 1995, Revised 18 November 1995, Expanded June 1996, also appeared in Data Mining and Knowledge Discovery 1(1): 29-53(1997)
[13] 左劫，唐常杰，于中华，李通，殷华蓓，刘欣 一种高效的基于依赖树的Cube算法小型微型计算机系统 第22卷第1期 2001年1月
[14] Marc Gyssens, Laks V.S.Lakeshmanan. A Foundation for multidimensional databases Proceedings of the 23rd Conference on Very Large Databases, 1997 pp.106-115
[15] Anindya Datta, Helen Thomas A Conceptual model and algebra for on-line analytical processing in decision support databases Proceedings of the Seventh Annual Workshop on Information Technologies and Systems, 1997, pp.91-100
[16] 裴健，柴玮，赵畅，唐世渭，杨冬青 联机分析处理数据立方体代数 软件学报 Vol．10，No．6，Jun．1999，pp．561-569
[17] Maurizio RAFANELLI, Arie SHOSHANI STORM: a statistical object representation model Proceedings of the Fifth International Conference on Statistical and Scientific Database Management 1990 pp.14-29
[18] Rakesh Agrawal, Ashish Gupta, Sunita Sarawagi Modeling multidimensional database IBM Technical Report 1995, Proceedings of the 13th International Conference on Data Engineering. 1997 pp.232-243.
[19] Luca Cabibbo, Riccardo Torione Querying multidimensional database Proceedings of the Sixth. International Conference on Database Programming Languages, 1997 pp.319-335.
[20] Wolfgang Lehner Modeling large scale OLAP scenarios Proceedings of the Sixth International Conference on Extending Database Technology, 1998 pp. 153-167.
[21] Panos Vassiliadi Modeling multidimensional databases, cubes, and cube operations Proceedings of the 10th International Conference on Statistical and Scientific Database Management, 1998 pp.53-62.
[22] H.V.Jagadish, Laks V.S.Lakshmanan, Divesh Srivastava What can hierarchies do for data warehouse? Proceedings of the 25th Conference on Very Large Databases, Edinburgh Scotland 1999 pp.530-541
[23] Alberto.O.Mendelzon, Alejandro.A.Vaisman Temporal queries in OLAP
Proceedings of the 26th International Conference on Very Large Databases, 2000 pp.242-253
[24] 陈微，仲萃豪 一种多维数据库和多维视图模型 计算机研究与发展 Vol．36，No．2 Feb．1999 pp．214-218
[25] Microsoft Corporation OLE DB for OLAP Specification Version 1.0 Microsoft Technical Document, 1998.
[26] Patrick O'Neil, Elizabeth O'Neil著 周傲英等译 数据库原理、编程与性能(Database Principles, Programming, and Performance)(第2版)[M] 机械工业出版社2002年
[27] Amit Shukla, Prasad M. Deshpande, Jeffrey F. Naughton Materialized view selection for multidimensional datasets Proceedings of the 24th international VLDB conference. 1998. pp.488-499
[28] 林友芳，黄厚宽，田盛丰 铁路货运数据仓库多维视图的组织及其物化策略 铁道学报 Vol．23 No．2 2001年4月
[29] 张宜红，徐宏炳等实视图选取策略及其实现技术软件学报 1998年12期
[30] 张季、周立柱等 基于抽样的Cube占用空间预测算法 计算机工程与应用 2001年24期
[31] A. Gupta, I. S. Mumick, V. S. Subrahmanian Maintaining views incrementally Proceedings of the ACM SIGMOD International Conference on Management of Data, 1993 pp. 157-166.
[32] Timothy Griffin, Leonid Libkin Incremental maintenance of views with duplicates Proceedings of the ACM SIGMOD International Conference on Management of Data, 1995 pp. 328-339.
[33] Y. Zhuge, H. Gareia-Molina, J. Hammer, J. Widom View maintenance in a warehousing environment Proceedings of the ACM SIGMOD International Conference on Management of Data, 1995 pp. 316-327
[34] 王腾蛟，王海洋等 多物化视图并行增量保持三阶段模式软件学报 1999．11
[35] 高宏，李建中 超大型压缩数据仓库上的CUBE算法 软件学报 2001．12
[36] 励晓健，林友芳，黄厚宽 数据仓库的视图动态物化调整策略 铁道学报
Vol.23 No.4 2001年8月
[37] Timothy Griffin, Leonid Libkin Incremental maintenance of views with duplicates Proceedings of the ACM SIGMOD International Conference on Management of Data, 1995 pp. 328-339.
[38] A. Gupta, V. Harinarayan, D. Quass Aggregate-query processing in data warehousing environments Proceedings of the 21st International Conference on Very Large Data Bases, 1995, pp. 358-369.
[39] A. Y. Levy, Y. Sagiv Queries independent of updates Proceedings of the 19th International Conference on Very Large Data Bases, 1993 pp. 171-181
[40] N. Roussopoulos View indexing in relational databases ACM Trans. Database Systems 7(2) (1982) pp.258-290.
[41] H. Gupta Selection of views to materialize in a data warehouse Proceedings of the Sixth International Conference on Database Theory, 1997 pp. 98-112.
[42] H. Gupta, I. S. Mumick Selection of views to materialize under a maintenance cost constraint Proceedings of the Seventh Internationai Conference on Database Theory, 1999 pp. 453-470.
[43] G. Graefe, W. J. McKenna The volcano optimizer generator: extensibility and efficient search Proceedings of the Ninth International Conference on Data Engineering, 1993 pp. 209-217.
[44] M. O. Akinde, M. H. B. Ohlen Constructing GPSJ view graphs Proceedings of the International Workshop on Design and Management of Data Warehouses, Germany, 1999 pp. 8/1-12.
[45] D. Quass Maintenance expressions for views with aggregation Workshop on Materialized Views: Techniques and Applications, 1996 pp. 110-118.
[46] T. Gri.n, L. Libkin Incremental maintenance of views with duplicates, Proceedings of the ACM SIGMOD International Conference on Management of Data, 1995 pp.328-339.
[47] Harinatayan V., Uajaramn A., Ullman J.D. Implementing data cubes efficiently. Proceedings of the ACM SIGMOD International Conference on Management of Data,
1996 pp.205～216
[48] 陈文伟 决策支持系统及其开发(第二版)[M] 清华大学出版社，广西科学技术出版社 2000年2月
[02] A. D. P. Lotufo, C. R. Minussi Electric Power Systems Load Forecasting: A Survey, The IEEE Power Tech '99 Conference, Budapest, Hungary, Aug. 29 Sept. 2, 1999
[03] 袁明友，肖先勇，杨洪耕，曾立渊，杨兵 基于灰色理论的供电系统负荷中长期预测模型及其应用 四川大学学报(工程科学版) 2002年04期
[04] 宋魁刚，胡晓光，基于多层神经网络SPDS的电力负荷预测，中国电力2001年06期
[05] 罗仲达，关治洪 人工神经网络在电力系统短期负荷预测中的应用 湖南电力2002年02期
[06] 孙海斌，周淑雄，谢敬东，唐国庆 模糊综合评价在负荷最优组合预测中的应用 电力系统及其自动化学报 2001年04期
[07] 赵希正 中国电力负荷特性分析与预测[M] (第一版) 北京：电力工业出版社 2002年1月
[08] 王珊等，数据仓库技术与联机分析处理[M] 科学出版社 1998．5
[09] 李建中，高宏 一种数据仓库的多维数据模型 软件学报 2000，11(7)pp．908-917．
[10] 李琪，白英彩 数据仓库中维的建模和查询，计算机研究与发展，Vol．39，No．5 May 2002，pp．612-618
[11] 徐海蔚，杨明福 扩充的多维数据模型MDB／MDV及其建模技术 计算机应用与软件 2000，17(11)pp．12-18
[12] Jim Gray, Surajit Chaudhuri, Adam Bosworth, Andrew Layman, Don Reichart, Murali Venkatrao, Frank Pellow, Harold Pirahesh Data Cube: a relational aggregation operator generalizing group-by, cross-tab and sub-totals Microsoft Technical report MSR-TR-95-22, 5 February 1995, Revised 18 November 1995, Expanded June 1996, also appeared in Data Mining and Knowledge Discovery 1(1): 29-53(1997)
[13] 左劫，唐常杰，于中华，李通，殷华蓓，刘欣 一种高效的基于依赖树的Cube算法小型微型计算机系统 第22卷第1期 2001年1月
[14] Marc Gyssens, Laks V.S.Lakeshmanan. A Foundation for multidimensional databases Proceedings of the 23rd Conference on Very Large Databases, 1997 pp.106-115
[15] Anindya Datta, Helen Thomas A Conceptual model and algebra for on-line analytical processing in decision support databases Proceedings of the Seventh Annual Workshop on Information Technologies and Systems, 1997, pp.91-100
[16] 裴健，柴玮，赵畅，唐世渭，杨冬青 联机分析处理数据立方体代数 软件学报 Vol．10，No．6，Jun．1999，pp．561-569
[17] Maurizio RAFANELLI, Arie SHOSHANI STORM: a statistical object representation model Proceedings of the Fifth International Conference on Statistical and Scientific Database Management 1990 pp.14-29
[18] Rakesh Agrawal, Ashish Gupta, Sunita Sarawagi Modeling multidimensional database IBM Technical Report 1995, Proceedings of the 13th International Conference on Data Engineering. 1997 pp.232-243.
[19] Luca Cabibbo, Riccardo Torione Querying multidimensional database Proceedings of the Sixth. International Conference on Database Programming Languages, 1997 pp.319-335.
[20] Wolfgang Lehner Modeling large scale OLAP scenarios Proceedings of the Sixth International Conference on Extending Database Technology, 1998 pp. 153-167.
[21] Panos Vassiliadi Modeling multidimensional databases, cubes, and cube operations Proceedings of the 10th International Conference on Statistical and Scientific Database Management, 1998 pp.53-62.
[22] H.V.Jagadish, Laks V.S.Lakshmanan, Divesh Srivastava What can hierarchies do for data warehouse? Proceedings of the 25th Conference on Very Large Databases, Edinburgh Scotland 1999 pp.530-541
[23] Alberto.O.Mendelzon, Alejandro.A.Vaisman Temporal queries in OLAP
Proceedings of the 26th International Conference on Very Large Databases, 2000 pp.242-253
[24] 陈微，仲萃豪 一种多维数据库和多维视图模型 计算机研究与发展 Vol．36，No．2 Feb．1999 pp．214-218
[25] Microsoft Corporation OLE DB for OLAP Specification Version 1.0 Microsoft Technical Document, 1998.
[26] Patrick O'Neil, Elizabeth O'Neil著 周傲英等译 数据库原理、编程与性能(Database Principles, Programming, and Performance)(第2版)[M] 机械工业出版社2002年
[27] Amit Shukla, Prasad M. Deshpande, Jeffrey F. Naughton Materialized view selection for multidimensional datasets Proceedings of the 24th international VLDB conference. 1998. pp.488-499
[28] 林友芳，黄厚宽，田盛丰 铁路货运数据仓库多维视图的组织及其物化策略 铁道学报 Vol．23 No．2 2001年4月
[29] 张宜红，徐宏炳等实视图选取策略及其实现技术软件学报 1998年12期
[30] 张季、周立柱等 基于抽样的Cube占用空间预测算法 计算机工程与应用 2001年24期
[31] A. Gupta, I. S. Mumick, V. S. Subrahmanian Maintaining views incrementally Proceedings of the ACM SIGMOD International Conference on Management of Data, 1993 pp. 157-166.
[32] Timothy Griffin, Leonid Libkin Incremental maintenance of views with duplicates Proceedings of the ACM SIGMOD International Conference on Management of Data, 1995 pp. 328-339.
[33] Y. Zhuge, H. Gareia-Molina, J. Hammer, J. Widom View maintenance in a warehousing environment Proceedings of the ACM SIGMOD International Conference on Management of Data, 1995 pp. 316-327
[34] 王腾蛟，王海洋等 多物化视图并行增量保持三阶段模式软件学报 1999．11
[35] 高宏，李建中 超大型压缩数据仓库上的CUBE算法 软件学报 2001．12
[36] 励晓健，林友芳，黄厚宽 数据仓库的视图动态物化调整策略 铁道学报
Vol.23 No.4 2001年8月
[37] Timothy Griffin, Leonid Libkin Incremental maintenance of views with duplicates Proceedings of the ACM SIGMOD International Conference on Management of Data, 1995 pp. 328-339.
[38] A. Gupta, V. Harinarayan, D. Quass Aggregate-query processing in data warehousing environments Proceedings of the 21st International Conference on Very Large Data Bases, 1995, pp. 358-369.
[39] A. Y. Levy, Y. Sagiv Queries independent of updates Proceedings of the 19th International Conference on Very Large Data Bases, 1993 pp. 171-181
[40] N. Roussopoulos View indexing in relational databases ACM Trans. Database Systems 7(2) (1982) pp.258-290.
[41] H. Gupta Selection of views to materialize in a data warehouse Proceedings of the Sixth International Conference on Database Theory, 1997 pp. 98-112.
[42] H. Gupta, I. S. Mumick Selection of views to materialize under a maintenance cost constraint Proceedings of the Seventh Internationai Conference on Database Theory, 1999 pp. 453-470.
[43] G. Graefe, W. J. McKenna The volcano optimizer generator: extensibility and efficient search Proceedings of the Ninth International Conference on Data Engineering, 1993 pp. 209-217.
[44] M. O. Akinde, M. H. B. Ohlen Constructing GPSJ view graphs Proceedings of the International Workshop on Design and Management of Data Warehouses, Germany, 1999 pp. 8/1-12.
[45] D. Quass Maintenance expressions for views with aggregation Workshop on Materialized Views: Techniques and Applications, 1996 pp. 110-118.
[46] T. Gri.n, L. Libkin Incremental maintenance of views with duplicates, Proceedings of the ACM SIGMOD International Conference on Management of Data, 1995 pp.328-339.
[47] Harinatayan V., Uajaramn A., Ullman J.D. Implementing data cubes efficiently. Proceedings of the ACM SIGMOD International Conference on Management of Data,
1996 pp.205～216
[48] 陈文伟 决策支持系统及其开发(第二版)[M] 清华大学出版社，广西科学技术出版社 2000年2月