In working for companies of varying size and in different industries over the last 20 years, we saw the same struggles with data conversion repeated over and over. Seems like a purely technical problem, right? We thought so at first too. After suffering through more than one large system implementation where data conversion blew the budget and delayed the timeline, we knew this couldn’t just be a technical problem. The mechanics of moving data from one system to another simply aren’t that hard.
It’s a Business Problem
There are certainly technical challenge points in any data conversion, and there are some great tools available to lessen those. However, what we found after taking a deep look at the sources of struggle was that data conversion is largely a business problem often mistaken for a technical one. In developing a repeatable process to successfully execute data migration projects on time and on budget, we found comprehensive source data analysis and data mapping is the key to addressing core business challenges and ensuring success.
Data Mapping is King
If the analysis of the source data has been thorough and the data mapping is accurate, the technical mechanics of converting the data from one system to another should be fairly straightforward and much of it can be automated with the right tools. The business challenge is that source data analysis and mapping is not easy and there aren’t a lot of tools out there to help. This challenge can be compounded by who is doing the mapping. A job often given to a programmer that really requires the business insight of a business analyst or data analyst who fully understands not just the data structures of source and target, but how the data is used in the target system to complete business tasks.
Master Puzzle Solver
A good data analyst is not just well versed on the data model and the industry usage of the data, they are also an exceptional puzzle solver. Successfully mapping data from legacy system data structures to a new application data structure in a way that ensures the data is fully usable is much like solving a complex puzzle. At first, it seems like none of the pieces fit but you quickly discover the critical border pieces and go from there. A rock star data analyst is able to quickly see the fundamental connections and the pieces that will need some manipulation to fit nicely into the puzzle. They will map the data in stages and gather feedback along the way to ensure everyone is in agreement on what the final picture will look like. If your team does not have someone with this skill set, go find them!
When planning your next data conversion project, remember this is not just a programming activity. Pair your programmer with a seasoned data analyst, support them with the right tools and processes to foster their success, and watch your conversion struggles lessen.