基于CWM的企业元数据集成研究
|
摘要
随着信息技术的不断深入,各企业内部逐渐实现了业务的信息化。但是由于业务和功能归属不同,形成了大量分布的异构数据源,“信息孤岛”现象普遍存在。而业务间的关联使得在各个环节之间存在大量数据交换、集成和部门协同的问题,因此有效解决异构数据环境下的数据集成问题具有重要现实意义。
近来,元数据的广泛应用给企业数据集成带来了新的发展动力,元数据集成成为数据集成的一个有效的途径。CWM是OMG组织提出的一个元数据标准,本课题组在CWM基础上,开发了元数据集成管理系统WMMS。在WMMS中,交换的元数据基于公共的元模型CWM,各产品的元数据通过软件适配器转换为CWM元数据进行交互。
本文在概述企业数据集成的发展以及目前研究现状的基础上,分析了元数据集成的重要性和可行性。研究了元数据集成的几种解决方案,讨论了元数据管理策略对元数据集成的重要性,并介绍了基于CWM的元数据集成体系。元仓库是WMMS的中心组件,用于存放CWM元数据。文中对元仓库进行了设计,采用关系数据库实现元仓库。设计了存放元数据的四类表,并给出了类、关联等CWM对象概念到关系型表的映射方法,确保元仓库存储的元数据符合CWM规范。
软件适配器的设计是本文的重点。本适配器主要针对关系数据库,实现RDBMS元数据到CWM元数据的转换,其中包括基本数据类型的转换、对象信息导出、关联信息导出等功能部件。RDBMS的元数据由元数据抽取部件进行抽取,转换后的CWM元数据通过元数据存储部件存入元仓库中。本文给出了各功能部件的详细设计,并以Oracle为例加以实现。
With the increasing demands of information technologies in daily work, more and more enterprises construct their own information systems in recent years. However, because different business entities have different operation models and organization structures, a large amount of heterogeneous data sources are formed, which are commonly called“Information island”. The interconnections between enterprises require solving the problems of data exchange, data integration and departmental cooperation. Therefore how to effectively solve the problem of data integration under heterogeneous data environment has significant practical meaning.
Recently, the widely application of metadata has brought new power to data integration. The metadata integration is effective path of data integration. CWM is a common standard for metadata which is proposed by OMG. We develop a metadata integration management system called as WMMS, which base on CWM specification. In WMMS, the exchanging metadata is base on CWM, so manufacture’s metadata must be transformed into CWM metadata by software adapter in order to interact.
Aiming at the problems mentioned above, the thesis presents the conception of enterprise metadata integration based on the overview of development, and then addressed the importance of metadata integration. Then we present several solutions of metadata integration , research the policy of metadata management and introduce a CWM-based metadata integration system.
Meta-store, the central component of WMMS, is a cheap solution to store CWM metadata. This paper designs the meta-store. Meta-store is based on relational database. By the way of mapping from the objective concepts to relational table, the metadata stored in meta-store is accord with CWM specification.
The design of software adapter is emphases of this paper. The adapter is aim at RDBMS. The function of the adapter is transforming RDBMS metadata into CWM metadata. RDBMS metadata is extracted by extractive tool , and the transformed metadata is stored in meta-store. The paper provides a detailed design of the adapter, and implemented by the example of Oracle.
近来,元数据的广泛应用给企业数据集成带来了新的发展动力,元数据集成成为数据集成的一个有效的途径。CWM是OMG组织提出的一个元数据标准,本课题组在CWM基础上,开发了元数据集成管理系统WMMS。在WMMS中,交换的元数据基于公共的元模型CWM,各产品的元数据通过软件适配器转换为CWM元数据进行交互。
本文在概述企业数据集成的发展以及目前研究现状的基础上,分析了元数据集成的重要性和可行性。研究了元数据集成的几种解决方案,讨论了元数据管理策略对元数据集成的重要性,并介绍了基于CWM的元数据集成体系。元仓库是WMMS的中心组件,用于存放CWM元数据。文中对元仓库进行了设计,采用关系数据库实现元仓库。设计了存放元数据的四类表,并给出了类、关联等CWM对象概念到关系型表的映射方法,确保元仓库存储的元数据符合CWM规范。
软件适配器的设计是本文的重点。本适配器主要针对关系数据库,实现RDBMS元数据到CWM元数据的转换,其中包括基本数据类型的转换、对象信息导出、关联信息导出等功能部件。RDBMS的元数据由元数据抽取部件进行抽取,转换后的CWM元数据通过元数据存储部件存入元仓库中。本文给出了各功能部件的详细设计,并以Oracle为例加以实现。
With the increasing demands of information technologies in daily work, more and more enterprises construct their own information systems in recent years. However, because different business entities have different operation models and organization structures, a large amount of heterogeneous data sources are formed, which are commonly called“Information island”. The interconnections between enterprises require solving the problems of data exchange, data integration and departmental cooperation. Therefore how to effectively solve the problem of data integration under heterogeneous data environment has significant practical meaning.
Recently, the widely application of metadata has brought new power to data integration. The metadata integration is effective path of data integration. CWM is a common standard for metadata which is proposed by OMG. We develop a metadata integration management system called as WMMS, which base on CWM specification. In WMMS, the exchanging metadata is base on CWM, so manufacture’s metadata must be transformed into CWM metadata by software adapter in order to interact.
Aiming at the problems mentioned above, the thesis presents the conception of enterprise metadata integration based on the overview of development, and then addressed the importance of metadata integration. Then we present several solutions of metadata integration , research the policy of metadata management and introduce a CWM-based metadata integration system.
Meta-store, the central component of WMMS, is a cheap solution to store CWM metadata. This paper designs the meta-store. Meta-store is based on relational database. By the way of mapping from the objective concepts to relational table, the metadata stored in meta-store is accord with CWM specification.
The design of software adapter is emphases of this paper. The adapter is aim at RDBMS. The function of the adapter is transforming RDBMS metadata into CWM metadata. RDBMS metadata is extracted by extractive tool , and the transformed metadata is stored in meta-store. The paper provides a detailed design of the adapter, and implemented by the example of Oracle.
引文
[1] Ireland R D,Hitt M A,Vaidynanath D. Alliance Management as a source of Competitive Advantage[J] .Journal of Management,2002,28(3),pp.413-446.
[2] Lancioni R A,Smith M F,Oliva T A.The Role of the Internet in supply chain management[J] .Industrial Marketing Management,2000,29(1),pp.45-56.
[3] 白庆华,崔吉前.网络经济新模式——协同商务[J] .物流技术,2002,3(8):36-38.
[4] 什么是 EAI. http://www.mie168.com/EAI/2005-06/39955.htm.
[5] 徐慧. 元数据集成系统及应用. 江苏大学硕士学位论文,2005,4-5.
[6] 袁磊. 数据集成模型框架探讨. http://searchdatabase.techtarget.com.cn/tips/277/1900777.shtml.
[7] Maurizio L. Data Integration: a Theoretical Perspective. In: Proc.of the ACM SIGACT-SIGMOD –SIGART Symposium on Principles of Database Sytems, 2002.
[8] 靳强勇,李冠,张俊.异构数据集成技术的发展和现状[J].计算机工程与应用,2002,4(11):17-20.
[9] Bicking Barbara,East Russell. Towards Dynamically Integrating Spatial Data And Its metadata[C] . First IEEE Metadata conference April,1996,NOAA Auditorium,Sliver Spring Maryland.
[10] Yoo S B, Kim Y. Web-based Knowledge Management for sharing Product Data in Virtual Enterprises[J] .International Journal of production Economics,2002,75(1-2), pp.173-183.
[11] 韩夏,李秉严.元数据互操作研究 [J].情报科学,2004,22(7):812.
[12] 潘定,沈钧毅.数据仓库元数据互操作性技术研究进展.情报科学,2005,23(10).
[13] 杨芳,陶世群.基于异构数据库系统的集成设计.计算机工程与设计,2005,26(7).
[14] Dr. Daniel T. Chang, CWM Enablement Showcase: Warehouse Metadata Interchange Made Easy Using CWM.TDWI 2000.
[15] Prothman,B, Meta data, IEEE Volume: 19, Issue: 1, Feb. 2 March 2000 pp.20～23.
[16] Tannenbaum, A.Metadata Solutions:Using metamodels, repositories,xml,andenterprise portals to generate information on demand.Addison Wesley Professional,Boston,MA,2001.
[17] (美) W.H Inmon 著,王志海 林友芳等译,数据仓库,北京:机械工业出版社,2003.3.
[18] Vaduva A, Dittrich KR. Metadata Management for data Warehousing [A]. International Symposium on Database Engineering & Applications. 2001: 129~135.
[19] 王强,刘东波,王建新.数据仓库元数据标准研究[J].计算机工程,2002,28(12):123-125.
[20] 聂茹,张虹.数据仓库元数据管理模式的分析与比较[J].计算机应用研究,2005,2:57-61.
[21] Do H H, Rahm E. On Metadata Interoperability in data Warehouses[R] . Technical Report 01- 2000. Leipzig, Germany:University of Leipzig, 2000.
[22] D.Marco. Building and Managing the Meta data Repository:Afull LifeCycle Guide [M] . Wiley,New York, 2000.
[23] Brown Computer Science. Data Integration Service[ EB/ OL ] .http :/ / www. cs. brown. edu/ people/ koa/ 227papers/ chapter. pdf ,2002 - 05.
[24] Open Information Model XML Encoding , Meta Data Coalition[ EB/ OL ] . Version1. 0 Review Draft 2 , December 1999.http :/ / xml. coverpages. org / mdc - oim. Htm.
[25] OMG, OMG Modeling Specifications, http://www.omg.org/technology /documents/formal /omg_modeling_specifications.htm.
[26] Sridhar lyengar. Model Driven Architecture Vision VS Reality. EDOC2001.
[27] John Poole 等著.公共仓库元模型—数据仓库集成标准导论.彭蓉,何璐璐等译.机械工业出版社, 2004.3.
[28] OMG. Common Warehouse Metamodel Specification version1.0. http://www.omg.org /docs/ formal/00-04-03.pdf.
[29] OMG. Unified Modeling Language Specification version1.3a1. http://www.omg.org/cgi-bin/ doc?ptc/02-09-03.pdf.
[30] OMG. Meta Object Facility Specification version1.4. http://www.omg.org/ docs/formal/ 00-04-03.pdf.
[31] OMG. XML Metadata Interchange Specification version1.1. http://www.omg.org/ cgi-bin/apps/doc?formal/00-06-01.pdf.
[32] 吴晓渊.基于 CWM 的企业数据集成研究——元仓库设计及建模工具功能扩展.长沙:国防科技大学硕士学位论文, 2005.
[33] Microsoft, Introduction to Meta Data Service. http: //www.microsoft. com/china/ sql/techinfo/BI/dataware.asp.
[34] Sybase Warehouse Control Center Application Programming Interface Programmer's Guide.http://manuals.sybase.com/onlinebooks /group-wa/wag0200e/ wccapi/@Generic_BookTextView/73.
[35] Automating ETL using Oracle warehouse Builder – OWB architecture http://www.dbasupport.com/oracle/ora9i/ETL02.shtml.
[36] Iryna Kozlova and Martin Husemann and Norbert Ritter and Stefan Witt and Natalia Hanikel: "CWM-based Integration of XML Documents and Object-Relational Data" , Proceedings of the 7th International Conference on Enterprise Information Systems (ICEIS05), 24-28 May 2005, Miami, FL, USA, INSTICC Press.
[37] 周红福,宫学庆,嵇晓,周傲英,基于 CWM 的 ETL 原型系统 METL 的设计与实现,NDBC2004 论文集 A 刊(计算机研究与发展增刊),2004.10.
[38] 丁长松.基于 CWM 的企业元数据管理的研究.长沙:国防科技大学硕士学位论文, 2006.
[39] EAI 技术和概念解析,http://it.sinru.com/planning/html/2004/28.htm
[40] Art Taylor 著.JDBC 数据库编程与 J2EE.李东升等译. 电子工业出版社,2004.4.
[41] John Poole 等著.公共仓库元模型—开发指南.彭蓉,刘进译.机械工业出版社, 2004.9.
[42] David Iseminger. T-SQL Language Reference. 北京大学出版社,March, 2001.
[2] Lancioni R A,Smith M F,Oliva T A.The Role of the Internet in supply chain management[J] .Industrial Marketing Management,2000,29(1),pp.45-56.
[3] 白庆华,崔吉前.网络经济新模式——协同商务[J] .物流技术,2002,3(8):36-38.
[4] 什么是 EAI. http://www.mie168.com/EAI/2005-06/39955.htm.
[5] 徐慧. 元数据集成系统及应用. 江苏大学硕士学位论文,2005,4-5.
[6] 袁磊. 数据集成模型框架探讨. http://searchdatabase.techtarget.com.cn/tips/277/1900777.shtml.
[7] Maurizio L. Data Integration: a Theoretical Perspective. In: Proc.of the ACM SIGACT-SIGMOD –SIGART Symposium on Principles of Database Sytems, 2002.
[8] 靳强勇,李冠,张俊.异构数据集成技术的发展和现状[J].计算机工程与应用,2002,4(11):17-20.
[9] Bicking Barbara,East Russell. Towards Dynamically Integrating Spatial Data And Its metadata[C] . First IEEE Metadata conference April,1996,NOAA Auditorium,Sliver Spring Maryland.
[10] Yoo S B, Kim Y. Web-based Knowledge Management for sharing Product Data in Virtual Enterprises[J] .International Journal of production Economics,2002,75(1-2), pp.173-183.
[11] 韩夏,李秉严.元数据互操作研究 [J].情报科学,2004,22(7):812.
[12] 潘定,沈钧毅.数据仓库元数据互操作性技术研究进展.情报科学,2005,23(10).
[13] 杨芳,陶世群.基于异构数据库系统的集成设计.计算机工程与设计,2005,26(7).
[14] Dr. Daniel T. Chang, CWM Enablement Showcase: Warehouse Metadata Interchange Made Easy Using CWM.TDWI 2000.
[15] Prothman,B, Meta data, IEEE Volume: 19, Issue: 1, Feb. 2 March 2000 pp.20～23.
[16] Tannenbaum, A.Metadata Solutions:Using metamodels, repositories,xml,andenterprise portals to generate information on demand.Addison Wesley Professional,Boston,MA,2001.
[17] (美) W.H Inmon 著,王志海 林友芳等译,数据仓库,北京:机械工业出版社,2003.3.
[18] Vaduva A, Dittrich KR. Metadata Management for data Warehousing [A]. International Symposium on Database Engineering & Applications. 2001: 129~135.
[19] 王强,刘东波,王建新.数据仓库元数据标准研究[J].计算机工程,2002,28(12):123-125.
[20] 聂茹,张虹.数据仓库元数据管理模式的分析与比较[J].计算机应用研究,2005,2:57-61.
[21] Do H H, Rahm E. On Metadata Interoperability in data Warehouses[R] . Technical Report 01- 2000. Leipzig, Germany:University of Leipzig, 2000.
[22] D.Marco. Building and Managing the Meta data Repository:Afull LifeCycle Guide [M] . Wiley,New York, 2000.
[23] Brown Computer Science. Data Integration Service[ EB/ OL ] .http :/ / www. cs. brown. edu/ people/ koa/ 227papers/ chapter. pdf ,2002 - 05.
[24] Open Information Model XML Encoding , Meta Data Coalition[ EB/ OL ] . Version1. 0 Review Draft 2 , December 1999.http :/ / xml. coverpages. org / mdc - oim. Htm.
[25] OMG, OMG Modeling Specifications, http://www.omg.org/technology /documents/formal /omg_modeling_specifications.htm.
[26] Sridhar lyengar. Model Driven Architecture Vision VS Reality. EDOC2001.
[27] John Poole 等著.公共仓库元模型—数据仓库集成标准导论.彭蓉,何璐璐等译.机械工业出版社, 2004.3.
[28] OMG. Common Warehouse Metamodel Specification version1.0. http://www.omg.org /docs/ formal/00-04-03.pdf.
[29] OMG. Unified Modeling Language Specification version1.3a1. http://www.omg.org/cgi-bin/ doc?ptc/02-09-03.pdf.
[30] OMG. Meta Object Facility Specification version1.4. http://www.omg.org/ docs/formal/ 00-04-03.pdf.
[31] OMG. XML Metadata Interchange Specification version1.1. http://www.omg.org/ cgi-bin/apps/doc?formal/00-06-01.pdf.
[32] 吴晓渊.基于 CWM 的企业数据集成研究——元仓库设计及建模工具功能扩展.长沙:国防科技大学硕士学位论文, 2005.
[33] Microsoft, Introduction to Meta Data Service. http: //www.microsoft. com/china/ sql/techinfo/BI/dataware.asp.
[34] Sybase Warehouse Control Center Application Programming Interface Programmer's Guide.http://manuals.sybase.com/onlinebooks /group-wa/wag0200e/ wccapi/@Generic_BookTextView/73.
[35] Automating ETL using Oracle warehouse Builder – OWB architecture http://www.dbasupport.com/oracle/ora9i/ETL02.shtml.
[36] Iryna Kozlova and Martin Husemann and Norbert Ritter and Stefan Witt and Natalia Hanikel: "CWM-based Integration of XML Documents and Object-Relational Data" , Proceedings of the 7th International Conference on Enterprise Information Systems (ICEIS05), 24-28 May 2005, Miami, FL, USA, INSTICC Press.
[37] 周红福,宫学庆,嵇晓,周傲英,基于 CWM 的 ETL 原型系统 METL 的设计与实现,NDBC2004 论文集 A 刊(计算机研究与发展增刊),2004.10.
[38] 丁长松.基于 CWM 的企业元数据管理的研究.长沙:国防科技大学硕士学位论文, 2006.
[39] EAI 技术和概念解析,http://it.sinru.com/planning/html/2004/28.htm
[40] Art Taylor 著.JDBC 数据库编程与 J2EE.李东升等译. 电子工业出版社,2004.4.
[41] John Poole 等著.公共仓库元模型—开发指南.彭蓉,刘进译.机械工业出版社, 2004.9.
[42] David Iseminger. T-SQL Language Reference. 北京大学出版社,March, 2001.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |