Data for Analysis
The Data for Analysis metric shows how much of your Qlik Cloud capacity is being used. It measures the total volume of data loaded into and stored in your environment. This topic explains what types of data count toward your usage, which are excluded, and how usage is calculated. Understanding these details helps you see what uses up your Qlik Cloud capacity and manage your data more efficiently.
Data included in the metric
The following data is counted:
-
Data loaded into Qlik Cloud from external sources. For reloads, new incremental data increases the data count. If the reload has less data, the data count decreases.
-
Data files uploaded to or created in Qlik Cloud. The file size is counted. If you copy data files within Qlik Cloud, the new data files are included in the count. If you duplicate an app in shared space without reloading, it won't be counted.
The data analyzed metric is calculated as:
-
The volume of external data ingested into Qlik Cloud via a Qlik Sense app.
-
The resulting QVD file size from external data being loaded into Qlik Cloud via Qlik Data Gateway - Data Movement.
-
The file size of data files uploaded to Qlik Cloud.
-
Static byte size of the app.
Data loaded into multiple tenants is counted multiple times, whereas data that is loaded once and used in multiple apps is counted only once.
The following are not counted in the metric:
-
Apps and data that are loaded into a personal space using On-demand app generation (ODAG).
-
Apps in Personal space. Including those using a data connection stored in a shared space.
-
Apps which only binary load another app. The original app being binary loaded is already counted.
-
Apps which only load QVD files. QVDs are already counted separately.
-
Apps and data published to managed spaces.
Information noteReloads of apps in managed spaces are counted in the metric. -
Data that is loaded through Direct Query.
Subscribing to Data for Analysis capacity
You subscribe to data packs based on your Data for Analysis requirements. Each user has a certain amount of Data for Analysis capacity available in their personal space. This personal data does not count toward the total Data for Analysis volume. However, if the user moves the data to a shared space for collaboration, it is included in the total capacity.
The Qlik Cloud Analytics Starter edition has a fixed data capacity. In this edition, subscriptions are based on the number of users.
Moving data into Qlik Cloud
Your options for moving data include:
-
Direct data connections from Qlik Sense
-
Qlik Data Gateway - Direct Access
-
Data movement to Qlik Cloud with Qlik Talend Data Integration
You can move data into Qlik Cloud from any source with the Premium and Enterprise editions of Qlik Cloud Analytics. With Qlik Cloud Analytics Standard, you can move data from any source except SAP, mainframe, and legacy sources.
Calculating Data for Analysis volume
Understanding how Data for Analysis is calculated can help you use your capacity efficiently. This section explains how monthly peak, data loading, app reloads, and data creation are measured.
Monthly peak
When you purchase Data for Analysis capacity, you get a set amount of space for storing and analyzing data. This capacity is a maximum limit, not a consumable credit. For example, if you purchase 250 GB of capacity, you can store and analyze up to 250 GB of data at any time. If you exceed this limit, overages may occur and are tracked separately.
How daily usage is tracked
Your usage is tracked daily. Each day, the system calculates your daily peak as the total of:
-
The total size of files stored
-
The amount of data ingested during app reloads
-
The amount of derived or created data
Carry-forward behavior
If no reload happens on a given day, the system carries forward the ingested data value from the most recent reload. At the start of a new month, the monthly peak resets. If no reload occurs on the first day of the new month, the last reload value from the previous month is applied to that day.
Stored data and derived data are always measured fresh each day, without carry-forward.
Special cases
-
The usage value reflects the last successful reload of each app. If an app is later reloaded with an empty data model (for example, only to create and export QVD files), the reported usage may still show the size from the last successful reload.
-
The usage value updates only when the app is reloaded again. This may cause differences between the data visible in the app and the reported Data for Analysis usage. To reduce reported usage, reload the app with a smaller dataset.
Carry-forward examples
The following examples show how reload values are carried forward when no new reload occurs, and how this behavior interacts with stored and created data.
Example 1 – Regular carry-forward:
-
Day 1: 70 GB datafiles stored, 40 GB app reloads, 30 GB datafiles created. The daily peak is 140 GB.
-
Day 2: 60 GB datafiles stored, 0 GB app reloads, 20 GB datafiles created. The reload value from Day 1 (40 GB) carries forward. The daily peak is 120 GB.
Example 2 – QVD-only app scenario
-
Day 1: Reload the app with 10 GB external data, create QVD files, drop all tables. The final app model is empty. The daily peak is 10 GB (reload portion).
-
Day 2: No reload occurs. The daily peak remains 10 GB (reload portion carried forward), even though the app now has no tables.
Only a new reload updates this value.
This means the daily peak can reflect both carried-forward reload data (even if the app itself is empty, as in the QVD-only scenario) and any new stored or created data for that day.
Monthly peak and high watermark
The monthly peak is the highest daily peak recorded during the month. It acts as a high watermark, showing the single day when your total data usage was highest. The monthly peak is compared against your purchased capacity to determine whether overage charges apply.
Example – Identifying the monthly peak from daily usage
To demonstrate how the monthly peak is determined, this example uses a simplified scenario covering just four days and a purchased capacity of 250 GB. The usage values help illustrate how the highest daily peak becomes the monthly peak.
The table presents data usage over four days, categorized by type:
| Day | Stored | Reload | Created | Daily peak |
|---|---|---|---|---|
| 1 | 50 GB | 30 GB | 15 GB | 95 GB |
| 2 | 70 GB | 40 GB | 30 GB | 140 GB |
| 3 | 60 GB | 0 GB | 20 GB | 120 GB |
| 4 | 80 GB | 50 GB | 40 GB | 170 GB |
Understanding the table data:
-
Day 3 has no data reload. The reload portion from Day 2 (40 GB) is carried forward and contributes to the daily peak (120 GB).
-
Day 4 has the highest daily peak (170 GB), which becomes the monthly peak.
Because the monthly peak is within the purchased capacity of 250 GB, no overage applies. If usage had exceeded 250 GB on any day, overage charges could apply or a capacity upgrade might be required.
Daily usage over four days, showing stored, ingested (reload), and created data. The green line marks the 250 GB purchased capacity.
Loading data into Qlik Cloud
Data loaded into Qlik Cloud from external sources is counted towards the daily peak. When you load data into a tenant, it is counted once and can be analyzed and used multiple times. Data loaded into multiple tenants is counted multiple times.
Data contributing to the daily peak is measured as follows:
-
File-based data loaded via a Qlik Sense app is measured by its file size.
-
App reloads using queries or connectors are counted as the maximum bytes ingested from the data source. When there are multiple reloads happening on the same day, the largest app size is the one that will be counted towards the daily peak. For example, if an app is reloaded during a day with 0.75 GB, 1.25 GB, and 1 GB, respectively, the value used for that day would be 1.25 GB.
As long as an app exists in the Qlik Cloud tenant, the maximum bytes ingested are evaluated for the app.
-
Data loaded into Qlik Cloud via Qlik Data Gateway - Data Movement is measured by the size of the resulting QVD file.
-
Apps that are uploaded or loaded via file import, either in the Analytics activity center or using qlik-cli, are measured by the app's static byte size.
-
Uploaded QVD files are measured by their file size.
Different ways of loading data into Qlik Cloud: via apps, data movement, or imported using the Analytics activity center or qlik-cli.
In the following situations, data is not included in the calculation of the daily peak:
-
Data loaded into a user's personal space is not counted, as long as it is restricted to that space. If the user moves the data to a shared space to collaborate with others, it will be counted.
-
If a reload fails, the bytes ingested are not counted. However, any resulting QVD files are counted.
-
When you load an app with data that already resides in Qlik Cloud, the data load is not counted. For example, copied or binary loaded apps (loading data from another Qlik Sense app) do not impact the daily peak, provided they’re not reloaded from an external source.
App reloads from internal sources have no impact on the daily peak.
Measuring bytes ingested for app reloads
The following applies when you reload a Qlik Sense app from an external source:
-
You can reload an app multiple times from the same source dataset without affecting the daily peak, as long as the data volume remains unchanged.
-
If the source dataset increases in size, the daily peak is affected. Each additional GB of data added to the dataset contributes an equivalent amount to the data ingested during the reload.
-
Conversely, if the source dataset decreases in size, this reduction is also reflected in the daily peak. For example, if the dataset size is reduced by 0.25 GB, the reload size decreases by the same amount. However, if a 1 GB reload occurred earlier in the day, the daily peak for that day would be 1 GB. The reduction would only be reflected in the daily peak for the next day.
-
Changes in the content of the source dataset, without altering its size, do not impact the daily peak. The daily peak is only affected by the data volume.
Measuring bytes ingested when size or content of the source dataset is changed.
-
If you query the same dataset multiple times within a single load script, all those queries are counted separately, and their data volumes are summed up. For example, if you have a load script that includes three queries of 1 GB each from the same dataset, all three of those queries are counted individually. So, the total data counted towards your daily peak is 3 GB.
-
Loading an app and subsequently dropping the table does not reduce the daily peak, as the daily peak is based on the maximum app reload size for the day.
-
If you load an app and then delete it on the same day, it will still contribute to the daily peak for that day. However, it reduces the daily peak for the following day when the app no longer exists.
Measuring bytes ingested for different app reload scenarios.
Measuring data loaded into QVD files with Qlik Talend Data Integration
The following applies when you load data into a QVD file from an external source using Qlik Data Gateway - Data Movement:
-
You can upload, import, or generate a dataset multiple times without affecting the daily peak, as long as the data volume remains unchanged.
-
If the source dataset increases in size, the daily peak is affected. Each additional GB of data added to the dataset contributes an equivalent amount to the size of the resulting QVD file.
-
Conversely, if the source dataset decreases in size, this reduction is also reflected in the daily peak. For example, if the dataset size is reduced by 0.25 GB, the size of the resulting QVD file decreases by the same amount.
-
Changes in the content of the source dataset, without altering its size, do not impact the daily peak. The daily peak is only affected by the data volume.
Measuring QVD file size when the size or content of the source dataset is changed.
Loading apps from external and internal sources
It is important to understand how data loaded into apps affects the daily peak, depending on the data source. Let's consider the following scenarios where data is loaded from different sources.
-
An app is loaded from an external source
When you load data from an external source into an app, it counts as bytes ingested. For example, if you load 10 GB, the contribution to the daily peak is 10 GB.
-
An app is loaded from a QVD in Qlik Cloud
Loading data into an app from a QVD file residing in Qlik Cloud does not count toward the daily peak. If 10 GB of data is loaded into an app from the QVD, no data is counted because there is no ingestion of external data. The contribution to the daily peak is 0 GB.
-
A new QVD file is generated from a QVD in Qlik Cloud
Data loaded into a QVD generator app from a Qlik Cloud-based QVD is not counted toward the daily peak. However, the resulting QVD file generated from the app does count. For example, if a 10 GB QVD file is transformed into a new 5 GB QVD, the contribution to the daily peak is the sum of the two files, which is 15 GB. As there is no external data ingestion, the load of the QVD generator app (a dedicated app that creates a data model and generates a QVD) is not counted.
-
An app is loaded from both external and internal sources
If an app loads 10 GB from an external source and 5 GB from a QVD within Qlik Cloud, the total contribution to the daily peak from the app is 10 GB, as only the data loaded from the external source is counted.
Measuring the total of data files plus data ingested when loading from internal and external sources.
Creating data in Qlik Cloud
When you create new data in Qlik Cloud, whether by copying data files or deriving it through combining and processing existing raw data, it counts toward the daily peak. Data is measured as the total size of the files generated during the data creation process. The created data is only counted once, regardless of how many apps use it.
Consider these examples of data creation:
-
Creating a 1 GB QVD file using the STORE statement adds 1 GB to the daily peak.
-
Copying a 1 GB QVD file adds 1 GB to the daily peak, as both copies contribute to the total.
-
Creating a 0.5 GB QVD file through transformation adds 0.5 GB to the daily peak. Only the resulting QVD file is counted; the QVD generator app isn't counted as it loads data that is already in Qlik Cloud.
Measuring data files created in Qlik Cloud.
Best practices for managing data
Effective data management helps you get the most value out of your Data for Analysis capacity. The recommendations below cover how to optimize data loading, reduce unnecessary storage, and manage older or inactive content in Qlik Cloud.
Creating QVD files for data reuse
Use QVD files when data is shared across multiple apps. Loading external data once and storing it as a QVD reduces repeated ingestion and helps keep the daily peak lower. Creating QVD files for data reuse with Qlik Data Gateway - Data Movement is generally more efficient than reloading data directly into apps.
Example:
If you load 10 GB of external data once and store it as a 5 GB QVD file, the total data counted toward the daily peak is 15 GB. Apps that load the 5 GB QVD do not re-ingesting the original 10 GB. If you instead load the same 10 GB of external data directly into two apps, the ingestion happens twice, resulting in 20 GB counted.
Creating a QVD and loading apps from it is more efficient than loading external data directly into multiple apps.
Using efficient data loading methods
Push as much filtering and transformation as possible to the data source (SQL pushdown). This reduces the amount of data transferred during reloads.
Example:
In this query, the filter is applied at the source so only a subset of the data is transferred.
Using On-demand apps for large datasets
On-demand app generation (ODAG) lets you work with large datasets by loading aggregated data first and retrieving detailed subsets only when needed.
For more information, see On-demand apps.
Handling large datasets with Direct Query and Dynamic Views
Direct Query and Dynamic Views let you query large datasets on demand without loading all data into memory. These approaches reduce ingestion and help keep usage within your capacity.
For more information, see:
Managing old or unused apps
Apps in the tenant continue to contribute to Data for Analysis usage based on their last successful reload. As long as an app exists with its data loaded, it continues to contribute to your capacity reporting.
To avoid unnecessary usage:
Reduce or remove unnecessary data
Remove data from apps that no longer require full datasets or create apps without data when only the structure is needed. Reloading an app with a smaller dataset updates its reported usage.
Avoid unnecessary duplication
Copy apps only when needed. Each copy is treated as a separate app and contributes to capacity usage.
Use shared spaces intentionally
Shared spaces can help control where and how reloads occur.
Delete apps you no longer need
Regular cleanup prevents outdated content from consuming capacity.
-
Identify unused items in Catalog by checking Last updated, Viewed by, and Used in.
-
Use impact analysis and lineage to understand where data files are used.
-
Delete unused apps and data files from activity centers. Administrators can also delete apps from Administration.
For more information, see:
Reducing reload-related consumption
An app’s reported reload size only decreases if the reload produces a smaller dataset. To reduce the reported size, reload the app with less data so the reported size is updated.
App reload behavior
-
If an app is reloaded without new data, the reported size remains the same.
-
When an app is copied to another space, it is treated as a separate app and counted again toward capacity usage.
Approaches to limit reload-related consumption
You can reduce reload-related data consumption using one of the following two methods. Both methods remove data from the app.
-
Stopping the load script early: Add an Exit script; statement at the start of the load script to reload the app without data. When full data is needed, comment out the statement or apply a condition so it only runs in specific scenarios (for example, when the app is in a particular space).
-
Recreating the app without data: Download the app without data and import it as a new app. After validating the new version, delete the original app.