The Need for Migration
To keep up with current trends as well as industry best practices, a business will find the need to upgrade its enterprise applications to achieve higher ROI by utilizing the latest technologies and features. In these situations, moving data from one system to another may become necessary and will become a major part of the upgrading or integration project.
Surprisingly, data migration is not given the importance it deserves in these projects, and therefore is not planned well enough and given the right resources to guarantee success. Perhaps that’s the reason why the failure rate is so high. According to one survey by Boor Research, about two third of data migration projects end up overrunning the time and the budget, and according to another research by Gartner, 83% of data migrations fail or exceed their allotted budgets.
Common Migration Challenges
Many companies don’t realize the true extent of the deficiencies related to their data management and quality until they engage in a migration project. The absence of a practical data management and governance policy, degradation of data quality and chaos in the organizational structure, access rights, user roles and other governance issues create serious obstacles in a data migration effort.
Low quality of data is the most immediate challenge encountered when starting a migration project. Duplicate data, poor data entry, missing data, misplaced data, data that is not normalized to conform to the system, all are data quality problems that must be corrected through a careful data cleaning process.
In addition to the above, final testing and validation could be a huge effort in large and complex data models, forcing the organization to allocate precious resources and causing unplanned time consumption. Integrating testing and validation of data into every phase of the project takes a huge burden off the final validation phase and could save valuable time and resources.
What Could Go Wrong?
There are many factors at play during a data migration project, and just as many opportunities for error. Two of the biggest issues arise from a lack of proper planning as well as the absence of a qualified data migration team. Existence or absence of a team of experienced data engineers and Salesforce experts handling the migration project could mean the difference between project success or project failure. It is not uncommon to run into serious issues and errors during migration, but with a qualified staff, using best practices and the knowledge they have gained from previous projects, they will find a timely and cost-efficient solution; avoiding delays and any fear of budget overrun.
Additionally, there are problems related to the integrity of the data itself. There always is a risk of data loss or corruption with every operation you perform on your data, and this problem increases exponentially when the data comes from different sources and the sizes are large. The mismatch between the format of the migrating data and the fields in the new system could render the data unusable. Lowering these risks amounts to the careful identification of any errors in the data and difference in the format between corresponding fields during the mapping operation; and then fixing all errors and inconsistencies before any attempt at migration or transfer.
The final hurdle is related to data security and privacy. Depending on the nature of the data that is being transferred, whether it is for compliance purposes regarding personal or healthcare data, or for protecting business data, implementing measures to prevent data privacy and protection violations could become a crucial part of a data migration project. In such cases, allocating necessary resources to methods such as data masking could save the organization time and money.
Planning a Migration Project for Salesforce
It is said that the way you take the first step decides how you will take the last one. A successful migration project begins with preparing a well thought out plan. At Cetrix, we divide every data migration plan into six phases:
- Preparation: discovery, planning, assessment of data and defining the procedures to follow in each step of the project and assigning tasks to project team members.
- Extraction: exporting data from the legacy system, analysis and validation, creating backups, etc.
- Cleaning: data quality definition, data profiling, and data discovery. Error identification and fixing, appending missing data, deduplication, etc.
- Mapping: mapping fields between the two systems and adding missing fields. Creating a mapping template, creating a migration workbook that holds the data mapping for each object involved in the process, completing the mapping based on the workbook, and saving (exporting) the final mapping.
- Loading: importing data into the new org in increments and in a top-down order (e.g. loading master objects before details), testing each import before starting the next increment.
- Testing and Validation: profiling and discovery of the data in the new system, identifying errors and fixing, finding format mismatches and fixing, paying special attention to custom fields, objects, and codes, and finally issuing final approval to close the migration operation.
Table of Contents
- Customer Enterprise Data Integration Best Practices
- Agile MDM Cloud Integration and Enterprise Service Bus Platforms
- Challenges when Introducing an Enterprise Service Bus Platform
Each phase of a migration project is key when measuring overall success , but the Critical Success Factor is the mapping of the data. If the mapping operation is performed well, all other problems can be taken care of with proper assessment and planning. However, if the mapping is not done correctly, everything must be deleted from the new system and the mapping and loading operation repeated, which will cause a considerable delay and cost overrun.
Cetrix places special care on the mapping operation in every migration project it executes. In larger projects, experienced engineers are assigned to this task to avoid trial-and-error methodology often used by some IT staff that do not have the required qualifications and experience.
At the end of the day, the success of a Salesforce migration project depends on proper planning and careful cleaning and mapping of data, all performed by qualified and experienced data engineers.