Data migration best practices
Data migration - a process of transfering data from original sources to new destinations - is simple in its foundations, but really tough in practice. The more data and the more crucial to company it is, the more complex data migration processes become. If run not preceded with required preparation, for instance not followed with proper data quality assessment, data migration might be rather destructive than helpful.
Of course, data quality assessment isn't the only step that has to be mentioned right before data migration. All them are identified and described in every data migration methodology. But following step by step one's instruction isn't also a guarantee of success. Once every case is different, it's good to remember about some data migration best practices. They are not the exact instructions, they're just the result of hundreds of migrations and people thinking about them analytically - what can be done next time to make data migration be more accurate, less risky or last shorter. What's been found is gathered in the following ideas.
Before data migration
Data purposes and uses
Paradoxically, data itself is not a value. Even more, data itself could not exist and no one should actually be unhappy because of this reason. So what makes so many people mad about data? What places it in the middle of company managers' interest? And - the most directly - what makes data matter? The answer is: purpose.
Collecting data just to have a lot of data doesn't make sense. It's like you were analyzing the colors of cars that pass you by when thinking of methods for reducing the pollution. In a word, it doesn't matter. And so do companies (do or... should do). They don't focus on data itself but on the way they can use data. About its purpose.
Therefore it's crucial to successful data migration to know what data is used for at the moment and what it can be used for a few days later. Company needs are changing continuously, while responsible data migration allows to be a few steps before the change so that company systems already are prepared for new appliances. It's very important to predict changes in the purpose of data.
Data and business connection
You should know that there is nothing that happens without a reason in a responsibly managed company. The same is to data - it never should be gathered just because it looks good to have a lot of data. Before you approach data migration, think about how your company uses data and what for. That's more or less what's stated above, but now focus on business processes data is linked to. Don't forget to ask data owners for their opinion. Maybe their point of view is totally different than yours and doing data migration your way could catastrophically impede their work.
Shutdown plan
If you're preparing data migration in a large company, you have to be aware that it won't get you sympathy of your colleagues. Prepare yourself rather for many comments in style of "why on earth (yes, it will be "on earth" here...) do you have to do it right now?!". Show your colleagues that you are well prepared and make a system retirement plan, considering all important elements which data migration will impact. Remember about systems, machines, even about buildings.
During data migration
Responsibility for data migration
For most of today's large organizations, data is extremely important and builds a base for all company decisions. Almost everyone across the organization depends on data. Thereupon, it is important not to struggle with data migration on one's own. While it's all people's matter, it shouldn't be a problem to find a set of specialists in technology or management and convince them to help. It seems necessary to instruct them about the need for data migration, its results and circumstances.
Best moments to control data migration process
It's quite a common habit to wait until the job is done to validate its results. In some cases it definitely is a good choice. However, the sooner errors and mistakes are identified, the easier it is to correct them or come up with a hot-fix. Thereupon, one of the best practices in data migration is to keep an eye on the migration during the process itself. This way, it's possible to save a lot of time and - if it says more - really a lot of money.
Tools instead of scripts (especially the hand-written ones)
When you try to do something from the ground, it - indeed - may work better than things prepared by others. It would definitely be tailored, but can you guarantee that it will be free from bugs? Unfortunately not. The best applications developed by worldwide corporations take thousands of hours in testing. Can you afford that with your scripts? Probably not. Thereupon, choose accessible tools instead and enjoy the work someone has done for you. So use whole sets of tools and make a proper use of automation. Sometimes it can save you even weeks of work!
Don't forget the communication
As it's been stated before, data migration is often a very complex process which requires a good cooperation among people from different departments across the organization. Also among the ones who aren't actually IT specialists. Thereupon, don't forget to communicate about your decisions to people involved. It means also the ones who will not be able to do their everyday work for some time because of system downtime that's needed to complete data migration. Don't let anyone be surprised with the fact that "there was data and now there's not".
Right after data migration
Final testing responsibility
Final testing is almost as important as data migration itself. You can never be 100% sure that it's finished with success until you test everything. Preparing an earlier defined sanity check might be helpful.
Moreover, it might be better to lend some testing tasks on data owners and other people involved from the business side. This way, you ensure the most accurate possible control and proper evaluation.
Post-migration report
When you finish data migration with success, don't forget to save it for future generations (or - at least - don't fail to share the joy with your friends). How? Seriously, the best option is to prepare a report which includes all information about the migration. You should mention things like people who were involved in migration, their tasks, dates, and stages of the process in the report. This way, you'll be able to quickly find out what's been done, for instance a year later when someone finally finds a little mistake to repair.
Preparing a strategy, following best practices, and avoiding common problems increase the probability of success in data migration. Once it's needed in almost every organization's systems, thinking about migration should be changed. It no longer should be treated as a theoretical IT's need, but much more as a way to better understanding business through better data delivery. And best practices listed above truly help in achieving this goal.