Working with views
Compose creates the Storage Zone with both storage tables and storage views. The storage tables are created in the database that you defined in your storage settings while two separate database are created for the views: The exposed views database and the internal views database. The exposed views database is the primary views database and contains all view types. The internal views database is used to store updates to ODS Live Views and HDS Live Views. The exposed views database and the internal views database share the same name as the Storage Zone database, but are appended with a unique suffix (by default, _v and _v_internal respectively), which is set in the project settings’ Naming tab. Consuming applications should be set up to read from the exposed views database, which provide several benefits over tables including better security (requiring read-only access only), data concurrency, and minimizing duplicate records in projects defined with non-ACID storage.
There are four types of view, depending on the project-level or entity-level data store type:
- ODS standard views – Created when the data store type is ODS only. These views will always reflect the same data unless the storage task is run.
- ODS Live Views – Created when the data store type is ODS only. As opposed to standard views, these views always reflect any changes to the Replicate Change Tables and to the Storage tables.
- HDS standard views – Created when the data store type is ODS + HDS. These views contain both current records and historical records and will only be updated if you run storage tasks.
- HDS Live Views – Created when the data store type is ODS + HDS. These views contain both current records and historical records. As opposed to standard views, these views always reflect any changes to the Replicate Change Tables and to the Storage tables.
When using live views, to ensure transactional consistency, it is recommended to turn off Speed partition mode in the Replicate task settings. When set to off, Replicate will close the partition only at the end of each transaction. This might require you to shorten the partition interval in order for the changes to be propagated to Compose in a timely manner. Shortening the partition interval might also require you to increase the partition cleanup frequency to prevent too many files from accumulating on the target and degrading performance.
For information about turning off Speed partition mode, setting partitioning intervals, and partition cleanup, see the Replicate Help.
Tables that were reloaded in Replicate will be automatically reloaded in Compose the next time the task runs. To prevent data inconsistency, Live Views should not be read while the tables are being reloaded.
Standard views contain data that was already applied to the storage tables, with mid to low-level latency. As consuming data from standard views requires less computing resources, this should be the first choice for downstream users. However, if latency is too high for some applications, Live views can be used instead. Although using live views significantly reduces latency, doing so requires greater computational resources. There is also the possibility that the date in live views might be less consistent than the data in standard views as updates may not have been applied to all the storage tables at the same time.
Although the views are in a separate database, you can use the suffixes (specified in the project settings’ Naming tab) to help identify them.