基于元数据驱动的多策略通用数据迁移模型

随着企业数据中心的建立,企业需要将不同数据源的数据迁移到数据中心。一种做法是采用商用迁移工具,另一种是针对不同的数据源开发专门迁移软件。随着企业业务及需求的变化必将导致数据源数据模型的改变,这两种迁移方式根本无法满足企业对数据迁移的灵活需求。本文在对数据迁移策略、映射模式、ETL以及元数据等技术研究的基础上提出了一种基于元数据驱动的通用数据迁移模型。当数据源发生变化时,只需要在映射模型中增加新的映射模式就能够适应数据源的变化。该模型的普适性已在大庆油田井下作业数据中心多策略数据迁移中得到了验证。

Multi-strategy Generalized Data Migration Model Based on Metadata-driven

With the construction of the data center,all kinds of data stored in different data sources need to be transferred into the data center.In general,two methods are employed,the one is employed commercial data transferring tools,the other is developing specialized software.With the changes of business of enterprise,it will result in data model of data source to change.All kinds of key techniques are researched,such as the policy for data transferring,the modes for data mapping and ETL,a kind of multi-strategy generalized data migration model based on metadata-driven is proposed.Whenever,data source is changed,the corresponding mapping mode is added,it will adapt these changes.The model has been applied to data transferring for Daqing field down hole job data center,which proves the model proposed is universal.