In general

Most databases nowadays have native capabilities or can work with partner solutions to allow identification of data changes in real-time. This allows someone to create Change Data Capture (CDC) streams and constantly feed cloud data-warehousing systems such as BigQuery, enabling real-time and predictive analytics. Enabling CDC functionality generally requires additional configuration on the source but provides an efficient way to identifying and cascading data changes to downstream systems and applications.

If there is no built-in real-time CDC solution available for the source system in question, identifying what “rows changed” can generally be achieved by executing incremental batch jobs, reading records…

Carlos Augusto

Google Customer Engineer and passionate about Data

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store