18-10-2010, 11:32 AM
[attachment=6408]
An Overview of Data Migration Methodology
Joseph R. Hudicka
overview
Most software implementation efforts are conducted to satisfy one of the following initiatives:
1. Deploying a new On Line Transactional Processing (OLTP) system
2. Deploying an On Line Analytical Processing (OLAP) system
Each type of system may replace and/or enhance the functionality currently delivered by one or more legacy systems. Regardless of the type of project, some data conversion must take place. Complications arise when we attempt to take the information currently maintained by the legacy system and transform it to fit into the new system. We refer to the building of this bridge as data migration. Data migration is a common element among most system implementations. It can be performed once, as with a legacy system redesign, or may be an ongoing process as in a data warehouse. Even some legacy system migrations require ongoing data conversion if the incoming data requires continual cleansing.
One would think that any two systems that maintain the same sort of data must be doing very similar things and, therefore, should map from one to another with ease. This is hardly ever the case. Legacy systems have historically proven to be far too lenient with respect to enforcing integrity at the atomic level of data. Fields that should be populated from a list of valid values, such as STATES, tend to require that a value be entered, but seldom validate the value entered by the user.
Another common problem has to do with the theoretical design differences between hierarchical and relational systems. Two of the cornerstones of hierarchical systems, namely de-normalization and redundant storage are strategies that make the relational purist cringe.
For all of the reasons mentioned above, there exists an obvious and desperate need for a sound, methodological approach by which organizations can tackle migration projects. Although there is no way to avoid unpleasant surprises, one can certainly be prepared to confront and resolve them.
