Supabase source setup guide

Set up Supabase instance as a source for ClickPipes

This is a guide on how to setup Supabase Postgres for usage in ClickPipes.

ClickPipes supports Supabase via IPv6 natively for seamless replication.

Creating a user with permissions and replication slot

Connect to your Supabase instance as an admin user and execute the following commands:

1. Create a dedicated user for ClickPipes:

   CREATE USER clickpipes_user PASSWORD 'some-password';
   

2. Grant schema-level, read-only access to the user you created in the previous step. The following example shows permissions for the public schema. Repeat these commands for each schema containing tables you want to replicate:

   GRANT USAGE ON SCHEMA "public" TO clickpipes_user;
   GRANT SELECT ON ALL TABLES IN SCHEMA "public" TO clickpipes_user;
   ALTER DEFAULT PRIVILEGES IN SCHEMA "public" GRANT SELECT ON TABLES TO clickpipes_user;
   

3. Grant replication privileges to the user:

   ALTER USER clickpipes_user WITH REPLICATION;
   

4. Create a publication with the tables you want to replicate. We strongly recommend only including the tables you need in the publication to avoid performance overhead.

Any table included in the publication must either have a primary key defined or have its replica identity configured to FULL. See the Postgres FAQs for guidance on scoping.

• To create a publication for specific tables:

  CREATE PUBLICATION clickpipes FOR TABLE table_to_replicate, table_to_replicate2;
  

• To create a publication for all tables in a specific schema:

  CREATE PUBLICATION clickpipes FOR TABLES IN SCHEMA "public";
  

The clickpipes publication will contain the set of change events generated from the specified tables, and will later be used to ingest the replication stream.

Increase max_slot_wal_keep_size

This step will restart your Supabase database and may cause a brief downtime.

You can increase the max_slot_wal_keep_size parameter for your Supabase database to a higher value (at least 100GB or 102400) by following the Supabase Docs

For better recommendation of this value you can contact the ClickPipes team.

Connection details to use for Supabase

Head over to your Supabase Project’s Project Settings -> Database (under Configuration).

Important: Disable Display connection pooler on this page and head over to the Connection parameters section and note/copy the parameters.

The connection pooler isn’t supported for CDC based replication, hence it needs to be disabled.

Note on RLS

The ClickPipes Postgres user must not be restricted by RLS policies, as it can lead to missing data. You can disable RLS policies for the user by running the below command:

ALTER USER clickpipes_user BYPASSRLS;

What’s next?

You can now create your ClickPipe and start ingesting data from your Postgres instance into ClickHouse Cloud. Make sure to note down the connection details you used while setting up your Postgres instance as you will need them during the ClickPipe creation process.