Overview

Log Stream enables a dedicated Replicate task to save data changes from the transaction log of a single source database and apply them to multiple targets, thereby eliminating the overhead of reading the logs for each target separately. Data changes from the log are written to a file in the Log Stream Staging folder enabling one or more targets to access them via separate Replication tasks.

Log Stream provides the following enhancements:

• When multiple CDC tasks read changes from large databases that share a single transaction log (e.g. Oracle and IBM DB2 for z/OS), the log is polled for only one target (the Log Stream target) instead of being polled separately for each target. This improves performance while greatly reducing the load on the source database and network.

• Changes are continuously written to the log stream regardless of the availability of the target endpoint. This is especially beneficial in situations where the original source transaction logs are no longer online or cannot be accessed efficiently. Additionally, this approach significantly reduces latency when the target endpoint becomes available again as the logs are read from the staging folder instead of from the source database.
• As the changes are constantly streamed to the Replicate server machine, the Replicate SLA that requires an extended log retention policy to be defined on the source in case the target cannot be accessed is no longer applicable. This allows you to reallocate valuable resources on the source while ensuring that log files will be up-to-date (on the Replicate Server machine) even if the target become unavailable.
• When replicating a single source database to multiple targets, each target can have its own subset of the log stream data (tables, rows, etc.) and be started, stopped, and scheduled independently of the others.

The flow

As mentioned above, a Log Stream setup provides benefits both when replicating to a single target and when replicating to multiple targets. The following diagram illustrates the end-to-end flow of a Log Stream setup with multiple target endpoints:

1. A Log Stream Staging task streams the source logs from the specified source endpoint to a designated staging folder on Replicate Server (specified in the Log Stream target endpoint settings).

Then, at least one Replication task:

2. Performs a Full Load of the source data to the designated target endpoint(s).

3. Transfers the changes from the Log Stream Staging folder to the final target endpoint(s).