Extracting lineage data with Qlik Lineage Connectors

Qlik Lineage Connectors is an add-on module to Qlik Cloud that helps users to visualize, understand, and scale data environments in an intuitive way. The connectors extract lineage and metadata from Qlik on-premises offerings, external BI tools, and data sources.

Source-specific connectors within this utility extract lineage data from Qlik on-premises, external BI tools, and data sources; mapping information and usage between these systems. Source lineage and the data journey displays in your Qlik Cloud tenant. This information assists analysts, database administrators, and data consumers to visualize, understand, and scale data environments in an intuitive way. The mapped information and usage between systems can be displayed by selecting its link in a space you have defined and then drilling in to view Lineage.

Qlik Lineage Connectors are available with Qlik Sense Enterprise SaaS, Qlik Cloud Analytics Premium, and Qlik Cloud Enterprise subscriptions.

How do Qlik Lineage Connectors work?

Qlik Lineage Connectors automatically gather metadata and lineage information from each targeted system. Each connector has its own configuration. Clients configure source-specific fields per connector type and define rules for each connector that tell Qlik Lineage Connectors what to include for mapped lineage in the specified tenant space. Users define how connectors behave and schedule how often to reload the metadata and lineage information.

Qlik Lineage Connectors workflow

To prepare for connection and extraction of lineage metadata from on-premises BI tools and database applications to your Qlik Cloud tenant, refer to the following high-level workflow:

Prerequisites

• You must be assigned the roles of Analytics Admin and Developer role in the target tenant.
• Download and Install Qlik Lineage Connectors. See Installing Qlik Lineage Connectors
• Obtain an API key (JWT token) from your tenant. Go to user profile menu settings> Management: API Keys. Copy and save the key to a place that is easily accessible.
• Create and name an empty managed space in your tenant.
• Open your tenant and Qlik Lineage Connectors utility.

Qlik Lineage Connectors configuration steps

• Project: Add and name a new project.
• Qlik Cloud Target: Define, validate, and save the Qlik Cloud Target.
• Connectors: Define and save a connector.

• Rules: Add at least one rule to each connector in connector settings.

  Information noteFor the QlikView and Qlik Sense connectors, check the box next to Create Link in Qlik Cloud to ensure the assets pulled in by this rule populate as links in Qlik Catalog.
• Scheduler: Select or schedule a job. Once the job completes, navigate to the created space in your tenant to view the imported lineage.

Accessing data lineage in the target tenant

Once the job defined by your connectors populates into Qlik Catalog, navigate to the managed space in your tenant. Extracted lineage sourced by Qlik Lineage Connectors displays with an Open link label when you hover over the tile. Open the lineage graph by clicking on the app, and then selecting Lineage. You can also access the lineage graph from the Details section.

Details provide you with information around ownership and recent activity of this source. On the left side of the panel, IDs display for the job and the connector.

Impact analysis provides a graphical representation of the data lineage across various platforms as well as inputs and dependencies available from the extracted metadata. For example, a QlikView app displays the QVDs and data sources that have produced the resulting QlikView application. When the node is selected in the graph, it displays any outputs that are generated from that QlikView dataset onward, even after it populates in Qlik Cloud.

For more information, see Analyzing the lineage of data.