by Jerod Johnson | April 10, 2023

Build & Maintain Your Hybrid Cloud with CData Sync

The hybrid cloud allows organizations to operate using some mixture of private cloud, public cloud, and on-premises data storage and IT systems. We've already spoken about the benefits of a hybrid cloud approach, the value of maintaining legacy and modern data stacks, how to create a hybrid cloud strategy, and how the hybrid cloud fits into your organization's digital transformation journey. Now, we think it's time to explore the practical.

In this article, we'll show how to use CData Sync to set up and maintain a hybrid cloud data storage system.

The scenario: A mix of cloud and on-prem

Many organizations are required to store specific information in on-premises or private cloud data stores. This could be for security reasons, to comply with privacy laws and regulations, or both.

In our scenario, we have an organization using MySQL as its transactional database. While MySQL is an excellent option for storing transactional data, organizations may not want to expose the database for direct reporting and analytics. Instead, they adopt a hybrid strategy, where sensitive data is maintained in the MySQL database and a cloud-based warehouse, Snowflake, is used for analytics and reporting.

The problem: Complex, expensive data access

Maintaining a replica of transactional data from one system to another can be an expensive task. Such migrations are often arduous, requiring a manual extract from one system followed by a manual import to the other. Alternatively, organizations can build data pipelines, but the time and energy spent building and maintaining data pipelines can tax overburdened IT resources. If the organization adopts new systems, new manual processes must be adopted, or the pipelines must be rebuilt entirely.

The solution: Location-agnostic data replication

CData Sync is an automated, continuous data replication application that allows you to build data pipeline tasks in a simple point-and-click interface. With CData sync, you can replicate data from more than 250 SaaS applications, relational databases, and NoSQL stores (including MySQL) to any popular database, data warehouse, or data lake (including Snowflake) – in the cloud, on-premises, or a combination of both.

How-to guide

Replicating your on-premises or private data to a cloud storage system takes three basic steps: connecting to the data source, connecting to the data destination, and configuring the replication job.

1. Connect to MySQL

To create a connection to MySQL in CData Sync, navigate to the connections tab in CData Sync, choose MySQL as the data source, and enter the configuration properties (in this case, server, port, database, username and password).

2. Connect to Snowflake

After connecting to the data source, it's time to connect to the data destination. Again, navigate to the connections tab in CData Sync and choose Snowflake as your Destination. Enter the credentials for your Snowflake instance, along with the base URL, warehouse, database, and schema.

3. Create the replication job

Now that both the data source and destination connections are configured, we're ready to create our replication job. Navigate to the Jobs tab and click to create a new job. Select the table(s) to replicate, schedule your replication, and click ‘run.’

Customizing Replications

We mentioned when we introduced this scenario, that the organization was using MySQL as a secure location for sensitive information. Thanks to customization options in CData Sync, we can selectively choose which tables and columns to replicate to Snowflake, ensuring that no sensitive data is ever stored outside of the MySQL database and that line-of-business users only have access to the data they need for analytics and reporting.

Implement your own hybrid cloud

CData Sync is a powerful, user-friendly data integration solution that can help businesses with their hybrid cloud implementation.

Using CData Sync, users can quickly integrate and consolidate organizational data from any source, cloud application, or on-premises system and leverage that data for reporting, operational excellence, customer success, and other important business initiatives.

Ready to quickly build data pipelines to implement and maintain your own hybrid cloud? Try CData Sync for free.