Skip to main content

Documentation Index

Fetch the complete documentation index at: https://private-7c7dfe99-page-updates.mintlify.app/llms.txt

Use this file to discover all available pages before exploring further.

This guide shows how to use Azure Private Link to provide private connectivity via a virtual network between Azure (including customer-owned and Microsoft Partner services) and ClickHouse Cloud. Azure Private Link simplifies the network architecture and secures the connection between endpoints in Azure by eliminating data exposure to the public internet. Azure supports cross-region connectivity via Private Link. This enables you to establish connections between VNets located in different regions where you have ClickHouse services deployed.
Additional charges may be applied to inter-region traffic. Please check the latest Azure documentation.
Please complete the following steps to enable Azure Private Link:
  1. Obtain Azure connection alias for Private Link
  2. Create a Private Endpoint in Azure
  3. Add the Private Endpoint Resource ID to your ClickHouse Cloud organization
  4. Add the Private Endpoint Resource ID to your services allow list
  5. Access your ClickHouse Cloud service using Private Link
ClickHouse Cloud Azure PrivateLink has switched from using resourceGUID to Resource ID filters. You can still use resourceGUID, as it is backward-compatible, but we recommend switching to Resource ID filters. To migrate, simply create a new endpoint using the Resource ID, attach it to the service, and remove the old resourceGUID-based one.

​
Attention

ClickHouse attempts to group your services to reuse the same published Private Link service within the Azure region. However, this grouping isn’t guaranteed, especially if you spread your services across multiple ClickHouse organizations. If you already have Private Link configured for other services in your ClickHouse organization, you can often skip most of the steps because of that grouping and proceed directly to the final step: Add the Private Endpoint Resource ID to your services allow list. Find Terraform examples at the ClickHouse Terraform Provider repository.

​
Option 1: ClickHouse Cloud console

In the ClickHouse Cloud console, open the service that you would like to connect via PrivateLink, then open the Settings menu. Click on the Set up private endpoint button. Make a note of the Service name and DNS name which will be used for setting up Private Link. Make a note of the Service name and DNS name, they will be needed in the next steps.

​
Option 2: API

Before you get started, you’ll need a ClickHouse Cloud API key. You can create a new key or use an existing one. Once you have your API key, set the following environment variables before running any commands: 
REGION=<region code, use Azure format, for example: westus3>
PROVIDER=azure
KEY_ID=<Key ID>
KEY_SECRET=<Key secret>
ORG_ID=<set ClickHouse organization ID>
SERVICE_NAME=<Your ClickHouse service name>
Get your ClickHouse INSTANCE_ID by filtering by region, provider and service name: 
INSTANCE_ID=$(curl --silent --user "${KEY_ID:?}:${KEY_SECRET:?}" \
"https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}/services" | \
jq ".result[] | select (.region==\"${REGION:?}\" and .provider==\"${PROVIDER:?}\" and .name==\"${SERVICE_NAME:?}\") | .id " -r)
Obtain your Azure connection alias and Private DNS hostname for Private Link: 
curl --silent --user "${KEY_ID:?}:${KEY_SECRET:?}" "https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}/services/${INSTANCE_ID:?}/privateEndpointConfig" | jq  .result
{
  "endpointServiceId": "production-westus3-0-0.xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.westus3.azure.privatelinkservice",
  "privateDnsHostname": "xxxxxxxxxx.westus3.privatelink.azure.clickhouse.cloud"
}
Make a note of the endpointServiceId. You’ll use it in the next step.

​
Create a private endpoint in Azure

:::important This section covers ClickHouse-specific details for configuring ClickHouse via Azure Private Link. Azure-specific steps are provided as a reference to guide you on where to look, but they may change over time without notice from the Azure cloud provider. Please consider Azure configuration based on your specific use case. Please note that ClickHouse isn’t responsible for configuring the required Azure private endpoints and DNS records. For any issues related to Azure configuration tasks, contact Azure Support directly. ::: In this section, we’re going to create a Private Endpoint in Azure. You can use either the Azure Portal or Terraform.

​
Option 1: Using Azure Portal to create a private endpoint in Azure

In the Azure Portal, open Private Link Center → Private Endpoints. Open the Private Endpoint creation dialog by clicking on the Create button.
In the following screen, specify the following options:
  • Subscription / Resource Group: Please choose the Azure subscription and resource group for the Private Endpoint.
  • Name: Set a name for the Private Endpoint.
  • Region: Choose a region where the deployed VNet that will be connected to ClickHouse Cloud via Private Link.
After you have completed the above steps, click the Next: Resource button.
Select the option Connect to an Azure resource by resource ID or alias. For the Resource ID or alias, use the endpointServiceId you have obtained from the Obtain Azure connection alias for Private Link step. Click Next: Virtual Network button.
  • Virtual network: Choose the VNet you want to connect to ClickHouse Cloud using Private Link
  • Subnet: Choose the subnet where Private Endpoint will be created
Optional:
  • Application security group: You can attach ASG to Private Endpoint and use it in Network Security Groups to filter network traffic to/from Private Endpoint.
Click Next: DNS button. Click the Next: Tags button.
Optionally, you can attach tags to your Private Endpoint. Click the Next: Review + create button.
Finally, click the Create button. The Connection status of the created Private Endpoint will be in Pending state. It will change to Approved state once you add this Private Endpoint to the service allow list. Open the network interface associated with Private Endpoint and copy the Private IPv4 address(10.0.0.4 in this example), you will need this information in the next steps.

​
Option 2: Using Terraform to create a private endpoint in Azure

Use the template below to use Terraform to create a Private Endpoint: 
resource "azurerm_private_endpoint" "example_clickhouse_cloud" {
  name                = var.pe_name
  location            = var.pe_location
  resource_group_name = var.pe_resource_group_name
  subnet_id           = var.pe_subnet_id

  private_service_connection {
    name                              = "test-pl"
    private_connection_resource_alias = "<data from 'Obtain Azure connection alias for Private Link' step>"
    is_manual_connection              = true
  }
}

​
Obtaining the Private Endpoint Resource ID

In order to use Private Link, you need to add the Private Endpoint connection Resource ID to your service allow list. The Private Endpoint Resource ID is exposed in the Azure Portal. Open the Private Endpoint created in the previous step and click JSON View: Under properties, find id field and copy this value: Preferred method: Using Resource ID Legacy method: Using resourceGUID You can still use the resourceGUID for backward compatibility. Find the resourceGuid field and copy this value: You will need to create a Private DNS zone (${location_code}.privatelink.azure.clickhouse.cloud) and attach it to your VNet to access resources via Private Link.

​
Create Private DNS zone

Option 1: Using Azure portal Please follow this guide to create an Azure private DNS zone using the Azure Portal. Option 2: Using Terraform Use the following Terraform template to create a Private DNS zone: 
resource "azurerm_private_dns_zone" "clickhouse_cloud_private_link_zone" {
  name                = "${var.location}.privatelink.azure.clickhouse.cloud"
  resource_group_name = var.resource_group_name
}

​
Create a wildcard DNS record

Create a wildcard record and point to your Private Endpoint: Option 1: Using Azure Portal
  1. Open the MyAzureResourceGroup resource group and select the ${region_code}.privatelink.azure.clickhouse.cloud private zone.
  2. Select + Record set.
  3. For Name, type *.
  4. For IP Address, type the IP address you see for Private Endpoint.
  5. Select OK.
Option 2: Using Terraform Use the following Terraform template to create a wildcard DNS record: 
resource "azurerm_private_dns_a_record" "example" {
  name                = "*"
  zone_name           = var.zone_name
  resource_group_name = var.resource_group_name
  ttl                 = 300
  records             = ["10.0.0.4"]
}
To link the private DNS zone to a virtual network, you’ll need to create a virtual network link. Option 1: Using Azure Portal Please follow this guide to link the virtual network to your private DNS zone. Option 2: Using Terraform
There are various ways to configure DNS. Please set up DNS according to your specific use case.
You need to point “DNS name”, taken from Obtain Azure connection alias for Private Link step, to Private Endpoint IP address. This ensures that services/components within your VPC/Network can resolve it properly.

​
Verify DNS setup

xxxxxxxxxx.westus3.privatelink.azure.clickhouse.cloud domain should be pointed to the Private Endpoint IP. (10.0.0.4 in this example). 
nslookup xxxxxxxxxx.westus3.privatelink.azure.clickhouse.cloud.
Server: 127.0.0.53
Address: 127.0.0.53#53

Non-authoritative answer:
Name: xxxxxxxxxx.westus3.privatelink.azure.clickhouse.cloud
Address: 10.0.0.4

​
Add the Private Endpoint Resource ID to your ClickHouse Cloud organization

​
Option 1: ClickHouse Cloud console

To add an endpoint to the organization, proceed to the Add the Private Endpoint Resource ID to your services allow list step. Adding the Private Endpoint Resource ID using the ClickHouse Cloud console to the services allow list automatically adds it to organization. To remove an endpoint, open Organization details -> Private Endpoints and click the delete button to remove the endpoint.

​
Option 2: API

Set the following environment variables before running any commands: 
PROVIDER=azure
KEY_ID=<Key ID>
KEY_SECRET=<Key secret>
ORG_ID=<set ClickHouse organization ID>
ENDPOINT_ID=<Private Endpoint Resource ID>
REGION=<region code, use Azure format>
Set the ENDPOINT_ID environment variable using data from the Obtaining the Private Endpoint Resource ID step. Run the following command to add the Private Endpoint: 
cat <<EOF | tee pl_config_org.json
{
  "privateEndpoints": {
    "add": [
      {
        "cloudProvider": "azure",
        "id": "${ENDPOINT_ID:?}",
        "description": "Azure private endpoint",
        "region": "${REGION:?}"
      }
    ]
  }
}
EOF
You can also run the following command to remove a Private Endpoint: 
cat <<EOF | tee pl_config_org.json
{
  "privateEndpoints": {
    "remove": [
      {
        "cloudProvider": "azure",
        "id": "${ENDPOINT_ID:?}",
        "region": "${REGION:?}"
      }
    ]
  }
}
EOF
After adding or removing a Private Endpoint, run the following command to apply it to your organization: 
curl --silent --user "${KEY_ID:?}:${KEY_SECRET:?}" -X PATCH -H "Content-Type: application/json" "https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}" -d @pl_config_org.json

​
Add the Private Endpoint Resource ID to your services allow list

By default, a ClickHouse Cloud service isn’t available over a Private Link connection even if the Private Link connection is approved and established. You need to explicitly add the Private Endpoint Resource ID for each service that should be available using Private Link.

​
Option 1: ClickHouse Cloud console

In the ClickHouse Cloud console, open the service that you would like to connect via PrivateLink then navigate to Settings. Enter the Resource ID obtained from the previous step.
If you want to allow access from an existing PrivateLink connection, use the existing endpoint drop-down menu.

​
Option 2: API

Set these environment variables before running any commands: 
PROVIDER=azure
KEY_ID=<Key ID>
KEY_SECRET=<Key secret>
ORG_ID=<set ClickHouse organization ID>
ENDPOINT_ID=<Private Endpoint Resource ID>
INSTANCE_ID=<Instance ID>
Execute it for each service that should be available using Private Link. Run the following command to add the Private Endpoint to the services allow list: 
cat <<EOF | tee pl_config.json
{
  "privateEndpointIds": {
    "add": [
      "${ENDPOINT_ID:?}"
    ]
  }
}
EOF
You can also run the following command to remove a Private Endpoint from the services allow list: 
cat <<EOF | tee pl_config.json
{
  "privateEndpointIds": {
    "remove": [
      "${ENDPOINT_ID:?}"
    ]
  }
}
EOF
After adding or removing a Private Endpoint to the services allow list, run the following command to apply it to your organization: 
curl --silent --user "${KEY_ID:?}:${KEY_SECRET:?}" -X PATCH -H "Content-Type: application/json" "https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}/services/${INSTANCE_ID:?}" -d @pl_config.json | jq
Each service with Private Link enabled has a public and private endpoint. In order to connect using Private Link, you need to use a private endpoint which will be privateDnsHostnameAPI or DNS nameconsole taken from Obtain Azure connection alias for Private Link.

​
Obtaining the private DNS hostname

​
Option 1: ClickHouse Cloud console

In the ClickHouse Cloud console, navigate to Settings. Click on the Set up private endpoint button. In the opened flyout, copy the DNS Name.

​
Option 2: API

Set the following environment variables before running any commands: 
KEY_ID=<Key ID>
KEY_SECRET=<Key secret>
ORG_ID=<set ClickHouse organization ID>
INSTANCE_ID=<Instance ID>
Run the following command: 
curl --silent --user "${KEY_ID:?}:${KEY_SECRET:?}" "https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}/services/${INSTANCE_ID:?}/privateEndpointConfig" | jq  .result
You should receive a response similar to the following: 
{
  ...
  "privateDnsHostname": "xxxxxxx.<region code>.privatelink.azure.clickhouse.cloud"
}
In this example, connection to the xxxxxxx.region_code.privatelink.azure.clickhouse.cloud hostname will be routed to Private Link. Meanwhile, xxxxxxx.region_code.azure.clickhouse.cloud will be routed over the internet. Use the privateDnsHostname to connect to your ClickHouse Cloud service using Private Link.

​
Troubleshooting

​
Test DNS setup

Run the following command: 
nslookup <dns name>
where “dns name” is the privateDnsHostnameAPI or DNS nameconsole from Obtain Azure connection alias for Private Link You should receive the following response: 
Non-authoritative answer:
Name: <dns name>
Address: 10.0.0.4

​
Connection reset by peer

Most likely, the Private Endpoint Resource ID wasn’t added to the service allow-list. Revisit the Add Private Endpoint Resource ID to your services allow-list step.

​
Private Endpoint is in pending state

Most likely, the Private Endpoint Resource ID wasn’t added to the service allow-list. Revisit the Add Private Endpoint Resource ID to your services allow-list step.

​
Test connectivity

If you have problems with connecting using Private Link, check your connectivity using openssl. Make sure the Private Link endpoint status is Accepted. OpenSSL should be able to connect (see CONNECTED in the output). errno=104 is expected. 
openssl s_client -connect abcd.westus3.privatelink.azure.clickhouse.cloud:9440

# highlight-next-line
CONNECTED(00000003)
write:errno=104
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 335 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)

​
Checking private endpoint filters

Set the following environment variables before running any commands: 
KEY_ID=<Key ID>
KEY_SECRET=<Key secret>
ORG_ID=<please set ClickHouse organization ID>
INSTANCE_ID=<Instance ID>
Run the following command to check Private Endpoint filters: 
curl --silent --user "${KEY_ID:?}:${KEY_SECRET:?}" -X GET -H "Content-Type: application/json" "https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}/services/${INSTANCE_ID:?}" | jq .result.privateEndpointIds

​
More information

For more information about Azure Private Link, please visit azure.microsoft.com/en-us/products/private-link.