Skip to main content
PATCH
/
v1
/
organizations
/
{organizationId}
/
services
/
{serviceId}
/
clickpipes
/
{clickPipeId}
/
scaling
Scaling ClickPipe
curl --request PATCH \
  --url https://api.clickhouse.cloud/v1/organizations/{organizationId}/services/{serviceId}/clickpipes/{clickPipeId}/scaling \
  --header 'Authorization: Basic <encoded-value>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "replicas": 5,
  "concurrency": 17,
  "replicaCpuMillicores": 1062,
  "replicaMemoryGb": 4.25
}
'
{
  "status": 200,
  "requestId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "result": {
    "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
    "serviceId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
    "name": "<string>",
    "state": "<string>",
    "scaling": {
      "replicas": 123,
      "concurrency": 123,
      "replicaCpuMillicores": 1062,
      "replicaMemoryGb": 4.25
    },
    "source": {
      "kafka": {
        "type": "kafka",
        "format": "JSONEachRow",
        "brokers": "<string>",
        "topics": "<string>",
        "consumerGroup": "my-clickpipe-consumer-group",
        "authentication": "PLAIN",
        "iamRole": "arn:aws:iam::123456789012:role/MyRole",
        "offset": {
          "strategy": "from_beginning",
          "timestamp": "2021-01-01T00:00"
        },
        "schemaRegistry": {
          "url": "https://psrc-aa00.us-east-2.aws.confluent.cloud/schemas/ids/100004",
          "authentication": "PLAIN",
          "caCertificate": "<string>"
        },
        "caCertificate": "<string>",
        "reversePrivateEndpointIds": [
          "<string>"
        ]
      },
      "objectStorage": {
        "type": "s3",
        "format": "JSONEachRow",
        "url": "https://datasets-documentation.s3.eu-west-3.amazonaws.com/http/**.ndjson.gz",
        "delimiter": ",",
        "compression": "auto",
        "isContinuous": true,
        "queueUrl": "https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue",
        "authentication": "IAM_ROLE",
        "iamRole": "arn:aws:iam::123456789012:role/MyRole",
        "connectionString": "DefaultEndpointsProtocol=https;AccountName=myaccount;AccountKey=mykey;EndpointSuffix=core.windows.net",
        "path": "data/logs/*.json",
        "azureContainerName": "mycontainer"
      },
      "kinesis": {
        "format": "JSONEachRow",
        "streamName": "my-stream",
        "region": "us-east-1",
        "useEnhancedFanOut": true,
        "iteratorType": "TRIM_HORIZON",
        "timestamp": 1615766400,
        "authentication": "IAM_ROLE",
        "iamRole": "arn:aws:iam::123456789012:role/MyRole"
      },
      "postgres": {
        "host": "<string>",
        "port": 123,
        "database": "<string>",
        "settings": {
          "syncIntervalSeconds": 123,
          "pullBatchSize": 123,
          "publicationName": "<string>",
          "replicationMode": "cdc",
          "replicationSlotName": "<string>",
          "allowNullableColumns": true,
          "initialLoadParallelism": 123,
          "snapshotNumRowsPerPartition": 123,
          "snapshotNumberOfParallelTables": 123,
          "enableFailoverSlots": true
        },
        "tableMappings": [
          {
            "sourceSchemaName": "<string>",
            "sourceTable": "<string>",
            "targetTable": "<string>",
            "excludedColumns": [
              "<string>"
            ],
            "useCustomSortingKey": true,
            "sortingKeys": [
              "<string>"
            ],
            "tableEngine": "MergeTree"
          }
        ]
      }
    },
    "destination": {
      "database": "<string>",
      "table": "<string>",
      "managedTable": true,
      "tableDefinition": {
        "engine": {
          "type": "MergeTree",
          "versionColumnId": "<string>",
          "columnIds": [
            "<string>"
          ]
        },
        "sortingKey": [
          "<string>"
        ],
        "partitionBy": "<string>",
        "primaryKey": "<string>"
      },
      "columns": [
        {
          "name": "<string>",
          "type": "<string>"
        }
      ]
    },
    "fieldMappings": [
      {
        "sourceField": "<string>",
        "destinationField": "<string>"
      }
    ],
    "settings": {
      "streaming_max_insert_wait_ms": 5000,
      "object_storage_concurrency": 1,
      "object_storage_polling_interval_ms": 30000,
      "object_storage_max_insert_bytes": 10737418240,
      "object_storage_max_file_count": 100,
      "clickhouse_max_threads": 8,
      "clickhouse_max_insert_threads": 1,
      "clickhouse_min_insert_block_size_bytes": 1073741824,
      "clickhouse_max_download_threads": 4,
      "clickhouse_parallel_distributed_insert_select": 2,
      "object_storage_use_cluster_function": true,
      "clickhouse_parallel_view_processing": false
    },
    "createdAt": "<string>",
    "updatedAt": "<string>"
  }
}

Documentation Index

Fetch the complete documentation index at: https://private-7c7dfe99-page-updates.mintlify.app/llms.txt

Use this file to discover all available pages before exploring further.

Authorizations

​
Authorization
string
header
required

Use key ID and key secret obtained in ClickHouse Cloud console: https://clickhouse.com/docs/cloud/manage/openapi

Path Parameters

​
organizationId
string<uuid>
required

ID of the organization that owns the service.

​
serviceId
string<uuid>
required

ID of the service that owns the ClickPipe.

​
clickPipeId
string<uuid>
required

ID of the ClickPipe to update scaling settings.

Body

application/json
​
replicas
integer | null

Number of replicas to scale to. Use to scale Kafka pipes.

Required range: 1 <= x <= 10
​
concurrency
integer | null

Number of concurrency to scale to. Use to scale S3 pipes.

Required range: 0 <= x <= 34
​
replicaCpuMillicores
integer | null

CPU in millicores for each replica. Use to scale streaming pipes.

Required range: 125 <= x <= 2000
​
replicaMemoryGb
number | null

Memory in GB for each replica. Use to scale streaming pipes.

Required range: 0.5 <= x <= 8

Response

Successful response

​
status
number

HTTP status code.

Example:

200

​
requestId
string<uuid>

Unique id assigned to every request. UUIDv4

​
result
object