Introducing Qlik Cloud Data Integration

You can deliver data ready for consumption to Qlik Cloud or to cloud data warehouses, such as Snowflake, Google Cloud BigQuery, and Azure Synapse Analytics with Qlik Cloud Data Integration. Data sources can be on-premises or in the cloud. The data can be kept up-to-date without manual intervention using CDC (Change Data Capture) or batch technologies, such as scheduled reloads. You can create a data pipeline and perform fit-for-purpose transformations and create data marts.

You can access Qlik Cloud Data Integration home by selecting Data Integration from the launcher menu ().

For more information about the architecture of Qlik Cloud Data Integration, see Dataset architecture in a cloud data warehouse.

Data spaces

Data spaces are governed areas of your Qlik Cloud tenant that are used to create and store data projects. Inside the space, you can also create new data connections using connectors, and manage access to Data Movement gateways. All data assets will be created in the space of the data project that they belong to.

For more information, see Working in spaces in Qlik Cloud Data Integration.

Data projects

A data project is where you create your data pipeline, using data assets. The data project is associated with a data platform that is used as target for all output. You can create a simple linear pipeline, or a complex pipeline consuming several data sources and generating many outputs.

Data task

A data task is the main unit of work in a data project. You can create data tasks of the following types in a data project. In addition, you can also perform landing and storage with a single task, Onboarding data.

• Landing

  Copy data from a data source to a landing area. Data sources can be on-premises or in the cloud. The landing area can be a cloud target, or an Amazon S3 data bucket (only when creating QVD datasets).

  You can keep data up-to-date without manual intervention by using CDC, or by performing full loads that are scheduled to reload periodically.

  Landing data from data sources

• Registered data

  Register data that already exists on the data platform. This lets you use data that is onboarded with other tools than Qlik Cloud Data Integration, for example, Qlik Replicate.

  Registering data that is already on the data platform

• Storage

  Create ready to consume datasets in a cloud data warehouse, or in Qlik Cloud, from the data copied by the landing data task. The datasets can be kept up-to-date with the landing data without manual intervention.

  Storing datasets

• Transform

  Create reusable data transformations based on rules and custom SQL as a part of your data pipeline. You can perform row-level transformations and create datasets that are either materialized as tables, or created as views that perform transformations on the fly.

  Transforming data

• Data mart

  Create data marts to leverage your Storage data tasks or Transform data tasks. You can create any number of data marts depending on your business needs. Ideally, your data marts should contain repositories of summarized data collected for analysis on a specific section or unit within an organization.

  Creating and managing data marts

Connections

• Data connections are used to let data tasks access data sources, external storage and cloud data platforms for data delivery and push-down transformations. You can connect either through native Qlik Cloud connectivity, or through Data Gateway - Data Movement.

• SaaS application connections are used to let data tasks access SaaS applications as data sources for data delivery.

Connections can only be updated by the owner of the connection.

Viewing your connections

Click Data connections in the left-side menu in Qlik Cloud Data Integration home to view all your data connections.

• You can edit connections that you own.

  Click ... and then Edit.

• You can test a connection.

  Click ... and then Test connection.

• You can delete a connection.

  Click ... and then Delete.

Creating a connection

You create a new connection by:

• Clicking on Add new in the top bar, and then clicking Data connection, or Application connection.

• Clicking on Create new where you select a connection.

• Clicking on Create connection where source connections are listed.

• Clicking on Create connection in the Data connections view.

You will need to select which type of data source, and then enter address and authentication information.

See also:

• Qlik Data Gateway - Data Movement overview.
• Connecting to data sources