Preparing to migrate

The first step when preparing to migrate is to do an inventory analysis of your current Stitch environment using the Stitch inventory tool. It is also recommended that you review the available application connectors in Qlik Talend Cloud and learn about the differences in data loading and data extraction between Stitch and Qlik Talend Cloud.

Performing an inventory analysis

Before migrating your replication tasks to Qlik Talend Cloud, it is recommended that you download and run the Stitch inventory tool to create an inventory list of your current Stitch environment. This includes current sources, destinations, and schemas. After you have completed the migration, use the inventory report as a reference to compare and confirm that you have moved all activities Qlik Talend Cloud.

The Stitch inventory tool, is available as part of the Qlik Talend Cloud migration toolkit. Use your Stitch API key to run the tool and generate an inventory report of your Stitch environment.

Do the following:

1. Download and install the Qlik Talend Cloud migration toolkit from the Qlik Talend Cloud migration toolkit User Guide

   • Select the download link that is compatible with your operating system.

   • Unzip the file in a dedicated folder, and then run the start_qtcmt.bat file.
     
     

     

2. On the main page of the Qlik Talend Cloud migration toolkit, select Get Started.

   

3. Sign in or register with your email and password.

4. After you are signed in, select your profile name on the upper right side of your screen, and then select Stitch inventory from the dropdown menu.

   

5. In Stitch: Inventory, under API token, paste your API token, and then select Get inventory. For information about how to generate your Stitch API token, see Generate a new Import API access token.

   

6. Review the generated inventory list of Sources and Destinations to see details about each item, such as name, schema, and dates. You can also do the following:

   • Expand the Connection properties and Table selection sections to see more details.

   • Select the source name link to go to the source settings in Stitch.

   • Select Report in the upper right side of your screen to export your inventory as a generated PDF report.

Source and target connections in Qlik Talend Cloud

Qlik Talend Cloud has many connectors available to land or replicate data from a large number of SaaS application sources and targets. Many of the commonly used integrations in Stitch have been added to Qlik Talend Cloud. However, since Stitch and Qlik Talend Cloud are different platforms, you will notice some differences when you move your pipelines. For a list of available application connectors, see Supported SaaS applications. If you don’t see the connector you need in Qlik Talend Cloud, go to Qlik Ideation and let us know. To submit an idea or upvote an existing suggestion, you must log in to the Qlik Community portal. Use the product filter to select Qlik Talend Cloud.

Differences in data loading

Since Qlik Talend Cloud and Stitch are different platforms, each platform loads data from the same source slightly differently. You will notice differences in the destination schema, data typing, the method used to load semi-structured data, and the replication method used.

Schemas

When loading the same dataset in Stitch and Qlik Talend Cloud, different destination schemas are created. For example, Stitch creates system columns to track different metadata created by the Stitch platform (such as _sdc_created_at, which is a timestamp that tracks when the Stitch pipeline received the extracted record). However, Qlik Talend Cloud does not create system columns.

Example: Schema analysis when loading data from MySQL to Snowflake

The following image highlights the system columns created by Stitch as well as the extracted data in the same schema. In Qlik Talend Cloud these system columns do not exist when loading the same dataset.

In the example images, the from_date and to_date columns show different data typing in Snowflake after being loaded by each platform. Stitch automatically converts the DATE type to Timestamp, while Qlik Talend Cloud keeps the same type (DATE) in the Snowflake target schema. Notice that column names created by Stitch are capitalized, while Qlik Talend Cloud uses the same formatting as the original MySQL source.

For more information about schemas in Qlik Talend Cloud, see Schemas. The following video demonstrates schema differences between the two platforms

Data typing

Stitch and Qlik Talend Cloud have different automatic data normalization to make sure extracted data can be loaded into the specified target destination. This leads to certain changes in the dataset when loaded by each product.

Loading options

In Stitch, you can choose between two loading behaviors: Upsert and Append-only.

When data is loading using Upsert, existing records are updated, and new records are inserted. When data is loading using Append-only, existing rows are not updated. All records are appended to the end of the table as new rows.

In Qlik Talend Cloud, Upsert is the default loading behavior. To append historical data in Qlik Talend Cloud, use the Store changes option, which stores data changes in separate change tables.

Semi-structured (JSON) data

In Stitch, for destinations that natively support JSON data structures, JSON data is stored in the applicable supported type according to the target. However, for destinations that do not support JSON data structures natively, Stitch will flatten JSON objects and arrays into tables and columns.

In Qlik Talend Cloud, all JSON data will be stored as text in the chosen target.

Example - Loading JSON data in Stitch and Qlik Talend Cloud

There are some differences between how Stitch and Qlik Talend Cloud handle JSON data. The following examples highlight these differences using examples that load data to a cloud data warehouse and database.

The following JSON object is used as the sample semi-structured dataset for this example. It contains multiple nested objects and arrays that store information about specific sales reps for an account.

[
    {
        "name": "John Doe",
        "role": "Senior Sales Representative",
        "email": "johndoe@salescorp.com",
        "leads": [
            {
                "status": "Negotiation",
                "company": "Tech Innovations Ltd",
                "contact": {
                    "name": "Sarah Brown",
                    "email": "sarah@techinnovations.com",
                    "phone": "+1 212 555 7890"
                },
                "lead_id": 201,
                "deal_size": 15000,
                "expected_close_date": "2024-03-15"
            },
            {
                "status": "Proposal Sent",
                "company": "AI Solutions Inc",
                "contact": {
                    "name": "David Lee",
                    "email": "david.lee@aisolutions.com",
                    "phone": "+1 646 555 2345"
                },
                "lead_id": 202,
                "deal_size": 22000,
                "expected_close_date": "2024-04-10"
            }
        ],
        "rep_id": 101,
        "territory": {
            "region": "North America",
            "states": [
                "NY",
                "NJ",
                "CT"
            ],
            "target_revenue": 500000
        },
        "performance": {
            "achieved": 115000,
            "win_rate": 0.72,
            "deals_closed": 18,
            "avg_deal_size": 6388.89,
            "quarterly_quota": 125000
        },
        "closed_deals": [
            {
                "value": 35000,
                "company": "Cloud Enterprises",
                "deal_id": 301,
                "closed_date": "2024-01-28",
                "profit_margin": 0.3,
                "sales_cycle_days": 45
            },
            {
                "value": 25000,
                "company": "NextGen AI",
                "deal_id": 302,
                "closed_date": "2024-02-12",
                "profit_margin": 0.35,
                "sales_cycle_days": 30
            }
        ]
    },
    {
        "name": "Jane Smith",
        "role": "Account Executive",
        "email": "janesmith@salescorp.com",
        "leads": [
            {
                "status": "Initial Contact",
                "company": "FinTech Global",
                "contact": {
                    "name": "Emma Wilson",
                    "email": "emma.wilson@fintechglobal.com",
                    "phone": "+44 20 5555 1234"
                },
                "lead_id": 203,
                "deal_size": 18000,
                "expected_close_date": "2024-05-01"
            }
        ],
        "rep_id": 102,
        "territory": {
            "region": "Europe",
            "countries": [
                "UK",
                "Germany",
                "France"
            ],
            "target_revenue": 450000
        },
        "performance": {
            "achieved": 98000,
            "win_rate": 0.68,
            "deals_closed": 15,
            "avg_deal_size": 6533.33,
            "quarterly_quota": 112500
        },
        "closed_deals": [
            {
                "value": 40000,
                "company": "Blockchain Solutions",
                "deal_id": 303,
                "closed_date": "2024-01-22",
                "profit_margin": 0.28,
                "sales_cycle_days": 50
            }
        ]
    },
    {
        "name": "Alice Johnson",
        "role": "Sales Manager",
        "email": "alicej@salescorp.com",
        "leads": [
            {
                "status": "Demo Scheduled",
                "company": "Green Energy Ltd",
                "contact": {
                    "name": "Rajesh Patel",
                    "email": "rajesh@greenenergy.com",
                    "phone": "+91 98765 43210"
                },
                "lead_id": 204,
                "deal_size": 28000,
                "expected_close_date": "2024-04-20"
            }
        ],
        "rep_id": 103,
        "territory": {
            "region": "APAC",
            "countries": [
                "India",
                "Singapore",
                "Australia"
            ],
            "target_revenue": 600000
        },
        "performance": {
            "achieved": 142000,
            "win_rate": 0.75,
            "deals_closed": 22,
            "avg_deal_size": 6454.55,
            "quarterly_quota": 150000
        },
        "closed_deals": [
            {
                "value": 55000,
                "company": "E-Commerce Giants",
                "deal_id": 304,
                "closed_date": "2024-02-05",
                "profit_margin": 0.32,
                "sales_cycle_days": 40
            },
            {
                "value": 47000,
                "company": "Smart Tech Solutions",
                "deal_id": 305,
                "closed_date": "2024-01-18",
                "profit_margin": 0.3,
                "sales_cycle_days": 35
            }
        ]
    }
]

Differences in data extraction

Although many Stitch-certified sources are available in Qlik Talend Cloud, there are still some situations where data extraction will differ between Stitch and Qlik Talend Cloud.

Differences caused by different integration versions

All connections created in Qlik Talend Cloud will use the latest version. If your Stitch pipeline was using an older version, data extracted in Qlik Talend Cloud could be different. For example, you might see the following:

• Some tables and columns might not be available.

• Different data types between different versions.

• Datasets might have different column or table names.