Sample logs, traces and metrics

Getting started with ClickStack and a sample dataset with logs, sessions, traces and metrics

ClickStack - Sample logs, traces and metrics

This guide demonstrates both ClickStack Open Source and Managed ClickStack using a sample data set.

Managed ClickStack

The following guide assumes you have completed the Getting Started Guide for Managed ClickStack and have the connection credentials recorded.

1. Select your service

   Select the service with Managed ClickStack from the main ClickHouse cloud landing page.

2. Navigate to the ClickStack UI (HyperDX)

   Select ClickStack from the left menu to navigate to the ClickStack UI, where you will be automatically authenticated.

3. Download sample data

   In order to populate the UI with sample data, download the following file:

   Sample data

   # curl
   curl -O https://storage.googleapis.com/hyperdx/sample.tar.gz
   # or
   # wget https://storage.googleapis.com/hyperdx/sample.tar.gz
   

   This file contains example logs, metrics, and traces from our public OpenTelemetry demo - a simple e-commerce store with microservices. Copy this file to a directory of your choosing.

4. Load sample data

   To load this data, we simply send it to the HTTP endpoint of the deployed OpenTelemetry (OTel) collector.

   Run the following command to send the data to the OTel collector:

   for filename in $(tar -tf sample.tar.gz); do
     endpoint="http://localhost:4318/v1/${filename%.json}"
     echo "loading ${filename%.json}"
     tar -xOf sample.tar.gz "$filename" | while read -r line; do
       printf '%s\n' "$line" | curl -s -o /dev/null -X POST "$endpoint" \
       -H "Content-Type: application/json" \
       -H "authorization: ${CLICKSTACK_API_KEY}" \
       --data-binary @-
     done
   done
   

   This simulates OTLP log, trace, and metric sources sending data to the OTel collector. In production, these sources may be language clients or even other OTel collectors.

   Returning to the Search view, you should see that data has started to load (adjust the time frame to the Last 1 hour if the data doesn’t render):

   Data loading will take a few minutes. Allow for the load to be completed before progressing to the next steps.

5. Explore sessions

   Suppose we have reports that our users are experiencing issues paying for goods. We can view their experience using HyperDX’s session replay capabilities.

   Select Client Sessions from the left menu.

   This view allows us to see front-end sessions for our e-commerce store. Sessions remain Anonymous until users check out and try to complete a purchase.

   Note that some sessions with emails have an associated error, potentially confirming reports of failed transactions.

   Select a trace with a failure and associated email. The subsequent view allows us to replay the user’s session and review their issue. Press play to watch the session.

   The replay shows the user navigating the site, adding items to their cart. Feel free to skip to later in the session where they attempt to complete a payment.

   Any errors are annotated on the timeline in red.

   The user was unable to place the order, with no obvious error. Scroll to the bottom of the left panel, containing the network and console events from the user’s browser. You will notice a 500 error was thrown on making a /api/checkout call.

   Select this 500 error. Neither the Overview nor Column Values indicate the source of the issue, other than the fact the error is unexpected, causing an Internal Error.

6. Explore traces

   Navigate to the Trace tab to see the full distributed trace.

   Scroll down the trace to see the origin of the error - the checkout service span. Select the Payment service span.

   Select the tab Column Values and scroll down. We can see the issue is associated with a cache being full.

   Scrolling up and returning to the trace, we can see logs are correlated with the span, thanks to our earlier configuration. These provide further context.

   We’ve established that a cache is getting filled in the payment service, which is preventing payments from completing.

7. Explore logs

   For further details, we can return to the Search:

   Select Logs from the sources and apply a filter to the payment service.

   We can see that while the issue is recent, the number of impacted payments is high. Furthermore, a cache related to the visa payments appears to be causing issues.

8. Chart metrics

   While an error has clearly been introduced in the code, we can use metrics to confirm the cache size. Navigate to the Chart Explorer view.

   Select Metrics as the data source. Complete the chart builder to plot the Maximum of visa_validation_cache.size (Gauge) and press the play button. The cache was clearly increasing before reaching a maximum size, after which errors were generated.

ClickStack Open Source

The following example assumes you have started Open Source ClickStack using the instructions for the all-in-one image and connected to the local ClickHouse instance.

1. Navigate to the ClickStack UI (HyperDX)

   Visit http://localhost:8080 to access the ClickStack UI.

2. Copy ingestion API key

   Navigate to Team Settings and copy the Ingestion API Key from the API Keys section. This API key ensures data ingestion through the OpenTelemetry collector is secure.

3. Download sample data

   In order to populate the UI with sample data, download the following file:

   Sample data

   # curl
   curl -O https://storage.googleapis.com/hyperdx/sample.tar.gz
   # or
   # wget https://storage.googleapis.com/hyperdx/sample.tar.gz
   

   This file contains example logs, metrics, and traces from our public OpenTelemetry demo - a simple e-commerce store with microservices. Copy this file to a directory of your choosing.

4. Load sample data

   To load this data, we simply send it to the HTTP endpoint of the deployed OpenTelemetry (OTel) collector.

   First, export the API key copied above.

   # export API key
   export CLICKSTACK_API_KEY=<YOUR_INGESTION_API_KEY>
   

   Run the following command to send the data to the OTel collector:

   for filename in $(tar -tf sample.tar.gz); do
     endpoint="http://localhost:4318/v1/${filename%.json}"
     echo "loading ${filename%.json}"
     tar -xOf sample.tar.gz "$filename" | while read -r line; do
       printf '%s\n' "$line" | curl -s -o /dev/null -X POST "$endpoint" \
       -H "Content-Type: application/json" \
       -H "authorization: ${CLICKSTACK_API_KEY}" \
       --data-binary @-
     done
   done
   

   This simulates OTLP log, trace, and metric sources sending data to the OTel collector. In production, these sources may be language clients or even other OTel collectors.

   Returning to the Search view, you should see that data has started to load (adjust the time frame to the Last 1 hour if the data doesn’t render):

   Data loading will take a few minutes. Allow for the load to be completed before progressing to the next steps.

5. Explore sessions

   Suppose we have reports that our users are experiencing issues paying for goods. We can view their experience using HyperDX’s session replay capabilities.

   Select Client Sessions from the left menu.

   This view allows us to see front-end sessions for our e-commerce store. Sessions remain Anonymous until users check out and try to complete a purchase.

   Note that some sessions with emails have an associated error, potentially confirming reports of failed transactions.

   Select a trace with a failure and associated email. The subsequent view allows us to replay the user’s session and review their issue. Press play to watch the session.

   The replay shows the user navigating the site, adding items to their cart. Feel free to skip to later in the session where they attempt to complete a payment.

   Any errors are annotated on the timeline in red.

   The user was unable to place the order, with no obvious error. Scroll to the bottom of the left panel, containing the network and console events from the user’s browser. You will notice a 500 error was thrown on making a /api/checkout call.

   Select this 500 error. Neither the Overview nor Column Values indicate the source of the issue, other than the fact the error is unexpected, causing an Internal Error.

6. Explore traces

   Navigate to the Trace tab to see the full distributed trace.

   Scroll down the trace to see the origin of the error - the checkout service span. Select the Payment service span.

   Select the tab Column Values and scroll down. We can see the issue is associated with a cache being full.

   Scrolling up and returning to the trace, we can see logs are correlated with the span, thanks to our earlier configuration. These provide further context.

   We’ve established that a cache is getting filled in the payment service, which is preventing payments from completing.

7. Explore logs

   For further details, we can return to the Search view:

   Select Logs from the sources and apply a filter to the payment service.

   We can see that while the issue is recent, the number of impacted payments is high. Furthermore, a cache related to the visa payments appears to be causing issues.

8. Chart metrics

   While an error has clearly been introduced in the code, we can use metrics to confirm the cache size. Navigate to the Chart Explorer view.

   Select Metrics as the data source. Complete the chart builder to plot the Maximum of visa_validation_cache.size (Gauge) and press the play button. The cache was clearly increasing before reaching a maximum size, after which errors were generated.