Introducing New dbt Core Integration and Real-Time CDC Sources for CData Sync
We’re excited to announce new capabilities for CData Sync that add support for more real-time sources, a history audit function, and enhanced transformations using dbt Core.
The CData team is dedicated to providing flexible, easy-to-use, and robust data pipeline technology to support our customers’ data integration needs. As enterprises worldwide leverage Sync to move their data from over 250 sources to their desired destinations, these new features allow Sync users to work even more collaboratively within the extended data ecosystem for demanding use cases.
Data Transformation with dbt Core
dbt Core provides new solutions for transformation, adding to Sync’s native transformation capabilities. If you’re unfamiliar with dbt, it’s a popular data transformation tool from dbt Labs.
The tool addresses some of the drawbacks in large organizations where data wrangling is typically the domain of scarce data engineers. dbt provides a workflow environment where analysts write business logic in SQL, the transforms are tested, the code is executed, and documentation is created. It provides a rich environment for building and sharing data transformations that can be used seamlessly within CData Sync as part of any replication job.
Watch this short demo:
Adding dbt to an Existing Sync Replication Job
Change Data Capture for Oracle and MySQL
This latest CData Sync release also adds support for real-time Change Data Capture (CDC) for Oracle and MySQL in addition to SQL Server and PostgreSQL. CDC allows Sync users to replicate data in real-time by automatically tracking updates in source data as they occur. These changes can invoke updates to destinations and trigger other activities. In practice, using Sync with CDC is a great way to feed real-time database changes to data warehouses and keep them up to date.
Watch this short demo:
SQL Server to Snowflake CDC Replication
History Mode for Data Audits
Lastly, we’re introducing a new History Mode for replication jobs with CData Sync. In contrast to traditional replication jobs, the destination in History Mode is appended with a timestamped entry of every change that occurs in the source. This is a great way to audit a source for any creates, updates, or deletes that occur to the data without having to modify existing applications.
Available on-premises or in the cloud via AWS or Azure, CData Sync makes it simple to replicate and transform all the data you need. Interested in trying these new features in Sync? Sign up today for a free trial.
