Setting general connection parameters

This section describes how to configure general connection properties. For an explanation of how to configure advanced connection properties, see Setting advanced connection properties.

To add a Snowflake on Azure target endpoint to Qlik Replicate:

1. In the Qlik Replicate Console, click Manage Endpoint Connections to open the Manage Endpoints Connections dialog box.
2. In the Manage Endpoint Connections dialog box, click New Endpoint Connection.
3. In the Name field, type a name for your Snowflake on Azure endpoint.
4. Optionally, in the Description field, enter a description for your Snowflake on Azure target endpoint.

5. Select Target as the role.

6. Select Snowflake on Azure as the Type.

7. Configure the Snowflake on Azure Target settings as follows:

   • Snowflake Account/Host: Your host name for accessing Snowflake on Microsoft Azure.
   • Authentication: Select one of the following:

     • Username and password: Enter the username and password of a user authorized to access the Snowflake database.

       Information noteThis authentication method is not supported when Snowpipe Streaming is the Loading method.

     • OAuth: To use OAuth authentication, your Snowflake database must be configured to use OAuth. The process is described in Snowflake documentation:

       Configure Snowflake OAuth for Custom Clients

       External OAuth

       • Authorize URL: The IdP server for requesting authorization codes. The authorization URL format depends on the IdP.

         For Snowflake:

         https://<yourSnowflakeAccount>/oauth/authorize

          

         For Okta:

         https://<yourOktaDomain>/oauth2/<authorizationServerId>/v1/authorize

       • Token URL: The IdP server used to exchange the authorization code for an access token. The access token URL format depends on the IdP.

         For Snowflake:

         https://<yourSnowflakeAccount>/oauth/token-request

          

         For Okta:

         https://<yourOktaDomain>/oauth2/<authorizationServerId>/v1/token

       • Client ID: The client ID of your application.

       • Client secret: The client secret of your application.

       • Scope: You might be required to specify at least one scope attribute, depending on your IdP configuration. Scope attributes must be separated by a space. Refer to your IdP's online help for information about the available scopes and their respective formats.

       • Use default proxy settings: Select to connect via a proxy server when clicking Generate. Note that the proxy settings must be defined in the server setting's Endpoints tab.

       • Refresh token: The refresh token value. Click Generate to generate a new refresh token. When you click Generate, your IdP will prompt you for your access credentials. Once you have provided the credentials, the Refresh token field will be populated with the token value.

       Warning noteThe IdP must not be configured to rotate the refresh token.
       Information note

       When using Replicate Console, the OAuth redirect URL is https://{hostname}/attunityreplicate/rest/oauth_complete.

       When using Enterprise Manager, the OAuth redirect URL is https://{hostname}/attunityenterprisemanager/rest/oauth_complete.

       The {hostname} part of the URL should be replaced by the domain from which you want to connect (Enterprise Manager, Replicate on Windows, Replicate on Linux, or Replicate on Windows using port 3552).

        

       If you connect to Replicate with a hostname that differs from the hostname in the redirect URL (configured in your IdP), you need to add that name to the end of the <REPLICATE-INSTALL-DIR>\bin\repctl.cfg file in the following format (using localhost as an example):

        

       "address": localhost

        

       Then restart the Qlik Replicate Server service.

     • Key Pair: Select and then provide the following information:

       • Username: The username of a user authorized to access the Snowflake database.

       • Private key file: The full path to your Private key file (in PEM format).

         Example: C:\Key\snow.pem

       • Private key passphrase: If the private key file is encrypted, specify the passphrase.
   • Database name: The name of your Snowflake database.

8. Configure the Data Loading settings as follows:

   • Loading method: Select Bulk Loading (the default) or Snowpipe Streaming.

     Information noteIf you select Snowpipe Streaming, make sure that you are aware of the limitations of this method.

     The main reasons to choose Snowpipe Streaming over Bulk Loading are: 

     • Less costly: As Snowpipe Streaming does not use the Snowflake warehouse, operating costs should be significantly cheaper, although this will depend on your specific use case.

     • Reduced latency: As the data is streamed directly to the target tables (as opposed to via staging), replication from source to target should be faster.

   • When Bulk Loading is selected, the following properties are available:

     • Warehouse: The name of your Snowflake warehouse.
     • Staging type: Select either Snowflake (the default) or Azure Blob storage. When Snowflake is selected, Snowflake's internal storage will be used.
       • When Azure Blob storage is selected, you need to provide the following information:

         • Storage account: The name of an account with write permissions to the container.

           Information note

           To connect to an Azure resource on Government Cloud or China Cloud, you need to specify the full resource name of the storage account. For example, assuming the storage account is MyBlobStorage, then the resource name for China cloud would be MyBlobStorage.dfs.core.chinacloudapi.cn

            

           For information on setting internal parameters, see Setting advanced connection properties

         • Access key: The account access key.
         • SAS token: Your SAS (Shared Access Signature) for accessing the container.
         • Container name: The container name.
         • Folder: The container folder to where the files will be copied.

     • Max file size (MB): Relevant for Full Load and CDC. The maximum size a file can reach before it is loaded to the target. If you encounter performance issues, try adjusting this parameter.

     • Number of file to load in a batch: Relevant for Full Load only. The number of files to load in a single batch. If you encounter performance issues, try adjusting this parameter.

     • Batch load timeout (seconds): If you encounter frequent timeouts when loading the files, try increasing this value.