Overcoming the data migration challenge

Data migration is an extremely common, business-critical task. So it may come as a surprise to learn how many companies experience considerable challenges when executing such projects. Recent data quality research commissioned by Experian showed that 9 out of ten businesses engage in data migration projects and as many as 85% experience significant problems managing them.

Often it’s a lack of collaboration that is a major obstacle to achieving success. Facilitating the involvement of the right people at the right time is critical. This view was supported by 38% of respondents in the study. In addition, when we consider that as many as 37% say that a lack of data standardisation or data consistency is a significant problem for their company, the challenge of data migration starts to become less surprising.

These figures illustrate a lack of preparation by companies entering into data migration projects - thus explaining the high failure rates associated with such processes. Many are falling into the same predictable traps, which can be easily avoided with foresight and the right preparation.

Seven stages of a successful data migration

There are a number of resources available to help guide organisations through an effective data migration. One of these is the Data Migration Project Checklist which outlines seven stages that can help you to execute a successful project.

Step 1
The first step is: Pre-migration Planning - an effective migration process should be planned in detail beforehand. This will help with the collaboration issue highlighted earlier, as everything is laid out upfront including viability, cost, structure, security, resources and timescales involved.

Step 2
Next on the list: Project Initiation - to kick off a migration project effectively, a clear plan and policies will need to be defined early on. It is important to set out key tasks at this stage. One in four respondents in the Experian study found that poor interpretation of business data rules was a significant problem, so this step alleviates this challenge. It is particularly important during this phase to agree on data quality and who will be responsible for correcting errors and approving the state of readiness in the project, things which can often be a grey area. Ask representatives of the business a critical question, “what do we need to show you in order to get your approval to switch off the old system?”, as that will focus attention on business context of the project rather than the technical delivery of the data.

Step 3
Landscape Analysis is the third step. This involves the development of a detailed data dictionary, a high-level source-to-target mapping specification, a data quality process and impact report, and a first-cut source-system retirement strategy, which will help to guide the direction of the project.

Step 4
With a third of survey respondents citing poor system design as their main challenge, the fourth step of Solution Design addresses this. At this stage, project leaders should be looking towards their end goals, including the creation of detailed specifications for mapping design, interface design and data quality management.

Step 5
The fifth step is: Build and Test - this phase gives stakeholders an opportunity to put the theoretical model to the test, documenting migration logic reconciliation, testing strategy and data quality management solutions. Fall-back policies, legacy decommissioning strategy and execution training also come under scrutiny.

Step 6
Execute and Validate is the penultimate step. As the process comes to a close, it will be time to independently validate the migration in order to objectively demonstrate to business sponsors that the new system meets business requirements and to auditors that the project has been fully compliant.

Step 7
The final step is: Decommission and Monitor. There will be a number of preconditions that need to be met during a period of use of the target system before a source system can be terminated. Once they have been proven, the source system can be switched off and the project can be closed.

Project managers can get a comprehensive overview of the key activities they should consider within each stage by arming themselves with a full project checklist for data migration. Investing time to overcome the main challenges through simple and achievable steps, and combining it with a suitable technology package such as Experian Pandora to handle the migration, can dramatically reduce the chances of failure which are experienced so often. By planning and collaborating with teams effectively, it ensures that data migration can be a success.