Integrating Your CRM With a Data Warehouse

Companies accumulate customer data as easily as a beach accumulates sand. As a company adds employees and creates more and better touchpoints to interact with website visitors and customers, data starts to pile up. Soon there are multiple systems in place that hold a multiplicity of customer data, gathered from:

• Website visitor data
• Downloads of trial data
• Requests for product information
• Requests for product documentation
• Sales call notes
• Support call notes
• Internal emails, DLs, and group threads discussing individual cases

Each one of these could easily be in its own format and silo – or several!

In a world where it’s “pics, or it didn’t happen,” it’s great to have separate, tailor-made systems for recording and managing customer relationship data. But if you can’t navigate seamlessly between the different systems, let alone unify that access in order to create meaningful analytics, your data is less useful.

Worse: when you can’t find everything in one single place, it’s easy to create a world of duplicates, unfinished documents, and other content that you just can’t reconcile.

That’s when you know it’s time to start looking for a data warehouse.

What is a data warehouse?

A data warehouse is a federated repository that houses large amounts of data under a single location and architecture. It collects all the data gathered by an enterprise’s various operational systems, using a unifying standard for separate databases.

The collected data is extracted, then transformed – either in a multistage fashion (which happens in a staging area before the data is loaded into the data warehouse), or within the data warehouse itself.

Once all of an enterprise’s salient data is transformed and collected in a data warehouse, it can be accessed from that single location by users across the organization. The data warehouse manages version control, so the version of the data in the data warehouse really does become the single source of truth for your enterprise.

Data warehousing solutions are especially important in customer relationship management (CRM) applications.

CRM applications manage all data related to a customer’s journey, beginning with prospect lead data and progressing throughout the whole customer relationship cycle. CRMs collect data from website forms, emails, texts, and sales meeting notes, and offer visibility into that data to sales reps. Pre-sales personnel can use CRM data to create targeted campaigns to one or several customer prospects with similar needs.

The importance of data warehousing for CRM data

CRMs are great for:

• Cutting down on time-consuming administrative tasks
• Boosting sales via improved lead follow-up
• Tapping into greater personalization via customer segmentation
• Making stakeholder knowledge more transparent throughout the enterprise, creating an improved feedback loop in the process
• Automating data analysis and reporting
• Offering consistent, on-point customer support
• Improving sales forecasting

However, CRMs can’t do everything. For example:

• They can’t build or manage a website. The data you derive from noting the customer journey through your website and other digital channels, can be used to improve an existing website, but not actually build and publish new website content.
• They don’t do Enterprise Resource Planning (ERP). CRMs can broadcast automated reminders for sales or customer service reps to reach out to leads or customers, but they can’t manage internal processes like payroll, supply chain management, or financial services. (For that, there’s ERP software.)
• They don’t do in-depth project management. There are CRMs that play quite nicely with project management software, and certainly project and product managers can gain tons of insight from accessing CRM data, but for now, you can only bring the horse to water. You can’t make it track your customer reps’ work.
• They can’t compel the humans who use them, to keep them updated. If your sales reps don’t take notes on their customer interactions, or don’t enter them in the right place in a timely manner, the CRM won’t be kept up-to-date. And that’s a waste for everyone.

In other words, there’s the customer data, and then there’s all the support structure and personnel that use that customer data.

For this reason, it’s best not to use your CRM as the company’s sole data repository. A better, safer way to safeguard and leverage CRM data is to move it into a data warehouse.

Benefits of moving CRM data to a data warehouse

The advantages of moving CRM data to a data warehouse mainly revolve around keeping CRM data safe and making it easier for other groups in the enterprise to leverage. This creates other opportunities for operational improvement through greatly improved business intelligence, including:

• Improved customer insights and enhanced decision-making, as other groups that are not necessarily on the front lines, have the chance to weigh in.
• Consistent, more accurate data interpretation, as data analysts gain access to CRM data.
• Data standardization across the enterprise, enabling other internal workgroups to interoperate with CRM data and leverage it in their own work.
• A “bird’s eye view” of all company data, enabling CFOs and other decision-makers to make data-backed decisions based on the very latest customer and employee inputs.

How to plan and prepare for your CRM data migration

Once you decide to migrate your CRM data to a data warehouse, you need to start planning. Broadly, this includes:

• Identifying the data you want to move
• Identifying third party integrations that may depend on the inputs or outputs of your current CRM
• Choosing your data warehouse solution
• Developing a data migration plan

Identifying which data and third party integrations will move

Unless you have a lot of legacy data that does not need to be maintained, you probably want to move most or all of your CRM data. This likely includes:

• Contact information
• Demographic information
• Customer interaction logs
• Customer preference data
• Customer purchase history
• File attachments that include miscellaneous project or customer data

You should also identify which third party integrations are currently in place, that depend on the inputs or outputs of your current CRM. Determine whether these integrations can move easily to the new data warehouse.

Choosing your data warehouse solution

The decision to choose and purchase a data warehouse solution is never made in a vacuum. Key considerations for CFOs when considering which data warehouse to implement include:

• Up-front and ongoing costs, including data warehouse hardware, software, software support, and training of in-house personnel.
• Ease of integration with existing data silos. This includes the CRM data and third party integrations you just identified plus any other data or operating systems you may want to take.
• Data security. Ensure that your data warehouse solution supports at least the same level of security that’s in place now, enterprise-wise, both before data transformation and after. Consider what improvements may need to be made once data is available across the enterprise, possibly in geographical locations where data security standards are different.
• Data backup and recovery. Ideally your enterprise already has policies in place for data backup and recovery. Consider what improvements may need to be made to accommodate the new data warehouse hardware and software. Also, consider what changes might be needed once data is available across the enterprise.

Developing a data migration plan

Your chosen data warehouse vendor will work with you to plan the scope of your data migration plan. The precise steps will vary depending on whether your chosen data warehouse onboards data within a staging area (multistage ETL) or within the data warehouse itself, and whether you want to migrate the CRM and its support systems before you migrate the rest of the data silos in your enterprise.

However, your pre-migration steps should always include:

1. Preparing the team. Migrating CRM data is an intricate task. Make sure everyone affected by the migration understands what’s about to happen, and can identify the people who are running the project. Assure them that there will be user acceptance testing and training afterwards. (And then make sure that happens!)
   
   
2. Reviewing the source and target CRM. Conduct a comprehensive review of both your existing CRM and the new CRM, as it will live within the data warehouse. Create a spec of all touchpoints between the old CRM and other systems, such as email and accounting, so that things can be disengaged from each other without breaking valuable corporate systems.
   
   
3. Performing data mapping. Map the names of data fields in the old CRM against the new CRM system to identify areas that require data transformation. For fields that don’t have direct matches in the new CRM, determine how to transform or merge data to fit the new system’s structure. Pay special attention to transactional data such as donations or sales. Decide how historical data will be handled, and identify any archiving needs.
   
   
4. Securing a complete data backup. If anything goes sideways during data migration, it is imperative that you have a way to restore the data. Test this backup to ensure that it’s 100% valid and accurate.

The CRM data migration process

Once you complete your pre-migration checklist, it’s time to begin CRM data migration in earnest. The particular steps your CRM migration will require, will depend on your vendor. However, your migration process will always include running a test migration, and then actually migrating the data.

1. Running a test migration. Conduct a thorough test of the migration process with a subset of your data, to identify any issues and ensure that the data is migrated accurately. This essentially establishes your Proof of Concept (PoC), testing whether the chosen solution meets all of the business and project requirements.
   
   Implement appropriate security measures during the test to ensure the confidentiality and integrity of the data during migration. This can include data encryption at rest and/or data encryption in transit. Based on the results of your tests, refine your ETL workflows and re-test.
   
   
2. Migrating the data. Whether it takes place in a staging area or within the data warehouse itself, data migration entails ETL: extraction, transformation, and loading.

• • Data extraction from the source system, taking into account any data extraction requirements such as data volume, extraction frequency, and data validation.
  • Data transformation as needed to ensure that it is suitable for the target system. This can include any required data formatting, data cleansing, and data validation.
  • Data loading or ingestion into the target system, ensuring that the data is loaded accurately and in a timely manner.

Just as you did when running the test migration, test the migrated data to validate that it was migrated accurately and that the target system is functioning correctly.

Once testing has validated the overall process, post-migration validation begins.

Post-migration activities for your CRM data

Post-migration validation of your CRM data is the first step in onboarding your new data warehouse.

The second step is to perform a data governance audit and security to determine whether sensitive data is appropriately locked down.

Ultimately, once the data is properly secured, it becomes time to take care of the employees who will need to learn the new system and use it on a daily basis. Onboard your employees post-migration by offering in-house documentation and training.

A measure of your new data warehouse’s success is the ease with which it is adopted by your staff. Because as we noted at the beginning of this article, CRMs grow stale if they’re not constantly freshened with new customer data!

Discover CData Sync

If you need to quickly and easily migrate your CRM data (alongside the rest of your enterprise data) to a data warehouse, look no further than CData Sync. With a point-and-click interface that lets you rapidly build automated, continuous replications jobs, CData Sync takes all of the guesswork out of integrating your CRM with your data warehouse. Once warehoused, you are ready to access all of your data for analytics, reporting and more!