Azure Flexible Server for Postgres source setup guide

Set up Azure Flexible Server for Postgres as a source for ClickPipes

Azure flexible server for Postgres source setup guide

ClickPipes supports Postgres version 12 and later.

Enable logical replication

You don’t need to follow the below steps if wal_level is set to logical. This setting should mostly be pre-configured if you’re migrating from another data replication tool.

1. Click on the Server parameters section

2. Edit the wal_level to logical

3. This change would require a server restart. So restart when requested.

Creating ClickPipes users and granting permissions

Connect to your Azure Flexible Server Postgres through the admin user and run the below commands:

1. Create a dedicated user for ClickPipes.

   CREATE USER clickpipes_user PASSWORD 'some-password';
   

2. Grant schema-level, read-only access to the user you created in the previous step. The following example shows permissions for the public schema. Repeat these commands for each schema containing tables you want to replicate:

   GRANT USAGE ON SCHEMA "public" TO clickpipes_user;
   GRANT SELECT ON ALL TABLES IN SCHEMA "public" TO clickpipes_user;
   ALTER DEFAULT PRIVILEGES IN SCHEMA "public" GRANT SELECT ON TABLES TO clickpipes_user;
   

3. Grant replication privileges to the user:

   ALTER USER clickpipes_user WITH REPLICATION;
   

4. Create a publication with the tables you want to replicate. We strongly recommend only including the tables you need in the publication to avoid performance overhead.

Any table included in the publication must either have a primary key defined or have its replica identity configured to FULL. See the Postgres FAQs for guidance on scoping.

• To create a publication for specific tables:

  CREATE PUBLICATION clickpipes FOR TABLE table_to_replicate, table_to_replicate2;
  

• To create a publication for all tables in a specific schema:

  CREATE PUBLICATION clickpipes FOR TABLES IN SCHEMA "public";
  

The clickpipes publication will contain the set of change events generated from the specified tables, and will later be used to ingest the replication stream.

Add ClickPipes IPs to Firewall

Please follow the below steps to add ClickPipes IPs to your network.

1. Go to the Networking tab and add the ClickPipes IPs to the Firewall of your Azure Flexible Server Postgres OR the Jump Server/Bastion if you’re using SSH tunneling.

What’s next?

You can now create your ClickPipe and start ingesting data from your Postgres instance into ClickHouse Cloud. Make sure to note down the connection details you used while setting up your Postgres instance as you will need them during the ClickPipe creation process.