Python

Python for ClickStack - The ClickHouse Observability Stack

ClickStack uses the OpenTelemetry standard for collecting telemetry data (logs and traces). Traces are auto-generated with automatic instrumentation, so manual instrumentation isn’t required to get value out of tracing.

This guide integrates:

• Logs
• Metrics
• Traces

Getting started

Install ClickStack OpenTelemetry instrumentation package

Use the following command to install the ClickStack OpenTelemetry package.

pip install hyperdx-opentelemetry

Install the OpenTelemetry automatic instrumentation libraries for the packages used by your Python application. We recommend that you use the opentelemetry-bootstrap tool that comes with the OpenTelemetry Python SDK to scan your application packages and generate the list of available libraries.

opentelemetry-bootstrap -a install

Configure environment variables

Afterwards you’ll need to configure the following environment variables in your shell to ship telemetry to ClickStack via the OpenTelemetry collector:

Managed ClickStack

OTEL_SERVICE_NAME='<NAME_OF_YOUR_APP_OR_SERVICE>' \
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 

ClickStack Open Source

export HYPERDX_API_KEY='<YOUR_INGESTION_API_KEY>' \
OTEL_SERVICE_NAME='<NAME_OF_YOUR_APP_OR_SERVICE>' \
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 

The OTEL_SERVICE_NAME environment variable is used to identify your service in the HyperDX app, it can be any name you want.

Run the application with OpenTelemetry Python agent

Now you can run the application with the OpenTelemetry Python agent (opentelemetry-instrument).

opentelemetry-instrument python app.py

If you’re using Gunicorn, uWSGI or uvicorn

In this case, the OpenTelemetry Python agent will require additional changes to work.

To configure OpenTelemetry for application servers using the pre-fork web server mode, make sure to call the configure_opentelemetry method within the post-fork hook.

Gunicorn

from hyperdx.opentelemetry import configure_opentelemetry

def post_fork(server, worker):
    configure_opentelemetry()

uWSGI

from hyperdx.opentelemetry import configure_opentelemetry
from uwsgidecorators import postfork

@postfork
def init_tracing():
    configure_opentelemetry()

uvicorn

OpenTelemetry currently doesn’t work with uvicorn run using the --reload flag or with multi-workers (--workers). We recommend disabling those flags while testing, or using Gunicorn.

Advanced configuration

Network capture

By enabling network capture features, developers gain the capability to debug HTTP request headers and body payloads effectively. This can be accomplished simply by setting HYPERDX_ENABLE_ADVANCED_NETWORK_CAPTURE flag to 1.

export HYPERDX_ENABLE_ADVANCED_NETWORK_CAPTURE=1

Troubleshooting

Logs not appearing due to log level

By default, OpenTelemetry logging handler uses logging.NOTSET level which defaults to WARNING level. You can specify the logging level when you create a logger:

import logging

logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)

Exporting to the console

The OpenTelemetry Python SDK usually displays errors in the console when they occur. However, if you don’t encounter any errors but notice that your data is not appearing in HyperDX as expected, you have the option to enable debug mode. When debug mode is activated, all telemetries will be printed to the console, allowing you to verify if your application is properly instrumented with the expected data.

export DEBUG=true

Read more about Python OpenTelemetry instrumentation here: https://opentelemetry.io/docs/instrumentation/python/manual/