Using external credentials

Many organizations prefer to keep secrets in a dedicated "vault" as a means of protecting against unauthorized privileged account access, impersonation, fraud, and theft. Storing secrets in a vault also eliminates manually intensive, time consuming and error prone administrative processes.

Replicate can be easily configured to interface with such vaults, thereby adding an additional layer of security while easing administration of endpoints. In Replicate, secret fields are any fields where the input is masked with asterisks. Such fields include database user passwords, user names, Amazon S3 secret keys, and so on.

It is also recommended to use the Replicate Add-ons API which allows you to take advantage of Replicate's memory management and logging capabilities, while eliminating the need to create independent memory and logging routines.

For more information, see Replicate add-ons API.

To facilitate integration with an external vault, Qlik provides the following files:

<INSTALL_DIR>\addons\samples\MyPasswordProvider.c

<INSTALL_DIR>\addons\include\ar_addon_password_provider.h

Limitations and considerations

• Secrets starting with "lookup::" are not supported
• Secret length cannot exceed 4 KB
• Loading more than one passwordProvider will result in a fatal error

Supported endpoints

Endpoints that support both user name and password replacement

The following endpoints support both user name and password replacement:

• Amazon RDS for SQL Server source
• Amazon S3
• Amazon Redshift

• Microsoft Azure SQL (MS-CDC) source

  Information noteOnly when using SQL authentication or Azure Active Directory authentication

• Microsoft Azure SQL Managed Instance source

  Information noteOnly when using SQL authentication or Azure Active Directory authentication
• Microsoft SQL Server source
• Microsoft SQL Server (MS-CDC) source
• Oracle source, except the following:
  • Secret store encryption entries
  • ASM parameters (if the redo logs are stored in ASM)

Endpoints that support password replacement

All endpoints support password replacement, with the following exceptions:

• File source
• File target
• Microsoft SQL Server source, but not for the Replicate has file-level access to the backup log files option.
• ODBC source, but not if the password is specified in the Connection string.
• ODBC with CDC, but not if the password is specified in the Connection string.
• SAP Application (DB) source, but only in the fields supported by the backend endpoint.
• Log Stream target

Changing the default addon name

You can change the default addon name (MyPasswordProvider) simply by renaming the <INSTALL_DIR>\addons\samples\MyPasswordProvider\MyPasswordProvider.dll file as desired. Note that if you rename the DLL, you may also need to specify the new name when editing the addons_def.json.sample file described in Step 3 below (depending on which parameters you use).

Step 1: Edit the MyPasswordProvider.c file

Edit the file MyPasswordProvider.c as follows:

1. Locate the following line:

   static AR_ADDONS_STATUS get_secret(char *name, const char *lookup_key, char *secret, int secret_len, char *principal, int principal_len)

   For a description of these fields, refer to the ar_addon_password_provider.h file.

2. Replace the code below this line (starting AR_AO_LOG and ending AR_ADDONS_STATUS_SUCCESS;) with the code required to access your Secrets Vault.

Step 2: Compile the code

Compile the MyPasswordProvider.c file into a DLL/SO. Note that on Windows the file should be compiled in "Release x64" mode.

Step 3: Edit the addons_def.json.sample file

Edit the file "<INSTALL_DIR>\addons\addons_def.json.sample" as follows:

1. Locate the following section:

   {
   	"addons": [{
   		"name": "MyPasswordProvider",
   		"type": "STARTUP",
   		//"lib_path": "C:\\Program Files\\Attunity\\Replicate\\addons\\samples\\MyPasswordProvider\\MyPasswordProvider.dll",
   		//"lib_path": "/opt/attunity/replicate/addons/samples/MyPasswordProvider.so",
   		"init_function": "ar_addon_init"
   	}]
   }

2. Edit the section as required, where:

   • name is the logical name of the DLL (can be any name). If you do not use the lib_path parameter to specify the DLL path (see below), then the DLL file must reside in <INSTALL_DIR>\addons\<addon_name>. On Linux, the <addon_name> folder needs to be created manually and should either be the default addon name (MyPasswordProvider) or its new name (if you changed it).

   • type is an optional parameter that specifies when to load the DLL. The only value that is currently supported is STARTUP.
   • lib_path is the full path of your DLL (e.g. C:\Vault\MyVaultProvider.dll). This is required only if the DLL does not reside in <INSTALL_DIR>\addons\<addon_name> (as mentioned in the description of the name parameter above).
   • init_function is the function name, as it appears in the C file used to generate the DLL.
3. Save the file as addons_def.json.

Step 4: Configure the endpoints

Any fields where you want the secret to be retrieved from your vault should be configured as described below.

Edit the field as follows:

1. In the appropriate field, enter the following:

   lookup::your_lookup_key

   where your_lookup_key is the Replicate endpoint identifier specified in your vault access code.

2. Save your settings.
3. Click Test Connection to verify that a connection to the endpoint can be established.
4. Run the task.