Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
8a199da
Redis logs guide
dhtclk Oct 28, 2025
5c1e911
update curl link
dhtclk Oct 29, 2025
aefe9e5
Merge branch 'main' of https://github.com/ClickHouse/clickhouse-docs …
dhtclk Oct 29, 2025
3c66b79
cleaning up timestamp generation, providing link and note on time ran…
dhtclk Oct 29, 2025
80fc494
guide refinements, nginx guide refinements as I discovered shared issues
dhtclk Oct 29, 2025
449764d
addressing nits
dhtclk Oct 30, 2025
6c9a22d
Merge branch 'main' of https://github.com/ClickHouse/clickhouse-docs …
dhtclk Oct 30, 2025
4f61700
Merge branch 'main' of https://github.com/ClickHouse/clickhouse-docs …
dhtclk Oct 31, 2025
6737724
consistency feedback, polish
dhtclk Oct 31, 2025
357eed2
polishing nginx logs
dhtclk Nov 3, 2025
d2fe3bb
nginx traces polish
dhtclk Nov 3, 2025
64bc2a5
redis guide polish and tracked link update
dhtclk Nov 3, 2025
0850b1f
aspell
dhtclk Nov 3, 2025
628ebc7
Merge branch 'main' into redis-logs-guide
dhtclk Nov 3, 2025
4eab252
new line linting, committing redis logs
dhtclk Nov 3, 2025
ab2cea3
Merge branch 'redis-logs-guide' of https://github.com/ClickHouse/clic…
dhtclk Nov 3, 2025
9100336
Apply suggestion from @Blargian
dhtclk Nov 4, 2025
1a7444f
Apply suggestion from @Blargian
dhtclk Nov 4, 2025
14bd0a2
Apply suggestion from @Blargian
dhtclk Nov 4, 2025
590b054
Apply suggestion from @Blargian
dhtclk Nov 4, 2025
afe536d
Apply suggestion from @Blargian
dhtclk Nov 4, 2025
33dd884
Apply suggestion from @Blargian
dhtclk Nov 4, 2025
e63de8e
Apply suggestion from @Blargian
dhtclk Nov 4, 2025
9eba8a7
Apply suggestion from @Blargian
dhtclk Nov 4, 2025
08eb12f
Apply suggestion from @Blargian
dhtclk Nov 4, 2025
8d4f7a0
newlines
dhtclk Nov 4, 2025
4f01ca6
newline
dhtclk Nov 4, 2025
9825ca9
Update docs/use-cases/observability/clickstack/integration-examples/n…
gingerwizard Nov 5, 2025
504fb30
Apply suggestions from code review
gingerwizard Nov 5, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,6 @@ quick start guides for various log and trace sources.

| Section | Description |
|------|-------------|
| [Nginx Logs](./nginx-logs.md) | Introduction to data ingestion methods and architecture |
| [Nginx Traces](./nginx-traces.md) | Introduction to data ingestion methods and architecture |
| [Nginx Logs](./nginx-logs.md) | Quick start guide for Nginx logs |
| [Nginx Traces](./nginx-traces.md) | Quick start guide for nginx traces |
| [Redis](./redis-logs.md) | Quick start guide for Redis logs |
Original file line number Diff line number Diff line change
@@ -1,48 +1,50 @@
---
slug: /use-cases/observability/clickstack/integrations/nginx
title: 'Monitoring Nginx Logs with ClickStack'
sidebar_label: 'Nginx Logs'
title: 'Monitoring Nginx logs with ClickStack'
sidebar_label: 'Nginx logs'
pagination_prev: null
pagination_next: null
description: 'Monitoring Nginx with ClickStack'
description: 'Monitoring nginx logs with ClickStack'
doc_type: 'guide'
---

import Image from '@theme/IdealImage';
import useBaseUrl from '@docusaurus/useBaseUrl';
import import_dashboard from '@site/static/images/clickstack/import-dashboard.png';
import finish_import from '@site/static/images/clickstack/finish-import.png';
import example_dashboard from '@site/static/images/clickstack/example-logs-dashboard.png';
import finish_import from '@site/static/images/clickstack/finish-nginx-logs-import.png';
import example_dashboard from '@site/static/images/clickstack/nginx-logs-dashboard.png';
import log_view from '@site/static/images/clickstack/log-view.png';
import search_view from '@site/static/images/clickstack/nginx-logs-search-view.png';
import { TrackedLink } from '@site/src/components/GalaxyTrackedLink/GalaxyTrackedLink';

# Monitoring Nginx Logs with ClickStack {#nginx-clickstack}
# Monitoring nginx Logs with ClickStack {#nginx-clickstack}

:::note[TL;DR]
This guide shows you how to monitor nginx with ClickStack by configuring the OpenTelemetry collector to ingest nginx access logs. You'll learn how to:

- Configure nginx to output JSON-formatted logs
- Create a custom OTel collector configuration for log ingestion
- Deploy ClickStack with your custom configuration
- Use a pre-built dashboard to visualize nginx metrics (requests, errors, latency)
- Use a pre-built dashboard to visualize nginx metrics

A demo dataset with 10,000 sample logs is provided to test the integration before connecting your production nginx instances.
A demo dataset with sample logs is available if you want to test the integration before configuring your production nginx.

Time Required: 5-10 minutes.
Time Required: 5-10 minutes
:::

## Prerequisites {#prerequisites}
- ClickStack instance running
- Existing nginx installation
- Access to modify nginx configuration files

## Integration with existing nginx {#existing-nginx}

This section covers configuring your existing nginx installation to send logs to ClickStack by modifying the ClickStack OTel collector configuration.
If you would like to test the integration before configuring your own existing setup, you can test with our preconfigured setup and sample data in the [following section](/use-cases/observability/clickstack/integrations/nginx#demo-dataset).

<VerticalStepper>
##### Prerequisites {#prerequisites}
- ClickStack instance running
- Existing nginx installation
- Access to modify nginx configuration files

## Configure nginx log format {#configure-nginx}
<VerticalStepper headerLevel="h4">

#### Configure nginx log format {#configure-nginx}
First, configure nginx to output logs in JSON format for easier parsing. Add this log format definition to your nginx.conf:

The `nginx.conf` file is typically located at:
Expand Down Expand Up @@ -75,7 +77,7 @@

After making this change, reload nginx.

## Create custom otel collector configuration {#custom-otel}
#### Create custom otel collector configuration {#custom-otel}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#### Create custom otel collector configuration {#custom-otel}
#### Create custom OTel collector configuration {#custom-otel}


ClickStack allows you to extend the base OpenTelemetry Collector configuration by mounting a custom configuration file and setting an environment variable. The custom configuration is merged with the base configuration managed by HyperDX via OpAMP.

Expand Down Expand Up @@ -125,15 +127,15 @@
- The pipelines route data from your receivers to the ClickHouse exporter via the existing processors
:::

## Configure ClickStack to load custom configuration {#load-custom}
#### Configure ClickStack to load custom configuration {#load-custom}

To enable custom collector configuration in your existing ClickStack deployment, you must:

1. Mount the custom config file at /etc/otelcol-contrib/custom.config.yaml
2. Set the environment variable CUSTOM_OTELCOL_CONFIG_FILE=/etc/otelcol-contrib/custom.config.yaml
3. Mount your nginx log directories so the collector can read them

### Option 1: Docker Compose {#docker-compose}
##### Option 1: Docker Compose {#docker-compose}

Update your ClickStack deployment configuration:
```yaml
Expand All @@ -149,7 +151,7 @@
# ... other volumes ...
```

### Option 2: Docker Run (All-in-One Image) {#all-in-one}
##### Option 2: Docker Run (All-in-One Image) {#all-in-one}

If using the all-in-one image with docker run:
```bash
Expand All @@ -165,11 +167,11 @@
Ensure the ClickStack collector has appropriate permissions to read the nginx log files. In production, use read-only mounts (:ro) and follow the principle of least privilege.
:::

## Verifying Logs in ClickStack {#verifying-logs}
#### Verifying Logs in HyperDX {#verifying-logs}
Once configured, log into HyperDX and verify logs are flowing:

1. Navigate to the Logs view
2. Verify you see JSON-parsed log entries with fields like request, request_time, upstream_response_time, etc.
1. Navigate to the search view
2. Set source to Logs, and verify you see log entries with fields like request, request_time, upstream_response_time, etc.

This is an example of what you should see:

Expand All @@ -183,25 +185,22 @@

For users who want to test the nginx integration before configuring their production systems, we provide a sample dataset of pre-generated nginx access logs with realistic traffic patterns.

<VerticalStepper>

## Download the sample dataset {#download-sample}
<VerticalStepper headerLevel="h4">

Download the sample log file and update timestamps to the current time:
#### Download the sample dataset {#download-sample}

```bash
# Download the logs
curl -O https://datasets-documentation.s3.eu-west-3.amazonaws.com/clickstack-integrations/access.log
```

The dataset includes:
- 10,000 log entries with realistic traffic patterns
- Log entries with realistic traffic patterns
- Various endpoints and HTTP methods
- Mix of successful requests and errors
- Realistic response times and byte counts
- Timestamps now distributed over recent time

## Create test collector configuration {#test-config}
#### Create test collector configuration {#test-config}

Create a file named `nginx-demo.yaml` with the following configuration:

Expand Down Expand Up @@ -236,7 +235,7 @@
EOF
```

## Run ClickStack with demo configuration {#run-demo}
#### Run ClickStack with demo configuration {#run-demo}

Run ClickStack with the demo logs and configuration:

Expand All @@ -249,18 +248,18 @@
docker.hyperdx.io/hyperdx/hyperdx-all-in-one:latest
```

## Verify logs in HyperDX {#verify-demo-logs}
#### Verify logs in HyperDX {#verify-demo-logs}

Once ClickStack is running (you may have to create an account and login first):

1. Open [HyperDX](http://localhost:8080/search?from=1760976000000&to=1761062400000&isLive=false&source=690235c1a9b7fc5a7c0fffc7&select=Timestamp,ServiceName,SeverityText,Body&where=&whereLanguage=lucene&filters=[]&orderBy=)
1. Open [HyperDX with demo time range](http://localhost:8080/search?from=1760976000000&to=1761062400000&isLive=false&source=690235c1a9b7fc5a7c0fffc7&select=Timestamp,ServiceName,SeverityText,Body&where=&whereLanguage=lucene&filters=[]&orderBy=)

Here's what you should see in your search view:

:::note
It is important to use the link above to get the correct time range, if you don't use this link set your time range to Oct 20 11:00:00 - Oct 21 11:00:00 to see proper results.
If you don't see logs, ensure the time range is set to 2025-10-20 11:00:00 - 2025-10-21 11:00:00 and 'Logs' is selected as the source. Using the link is important to get the proper time range of results.
:::

Here's what you should see in your search view:

<Image img={search_view} alt="Log view"/>

<Image img={log_view} alt="Log view"/>
Expand All @@ -271,10 +270,11 @@

To help you get started monitoring nginx with ClickStack, we provide essential visualizations for nginx logs.

<VerticalStepper>
## <a href={useBaseUrl('/examples/example-logs-dashboard.json')} download="nginx-logs-dashboard.json">Download</a> the dashboard configuration.
<VerticalStepper headerLevel="h4">

## Import the pre-built dashboard {#import-dashboard}
#### <TrackedLink href={useBaseUrl('/examples/nginx-logs-dashboard.json')} download="nginx-logs-dashboard.json" eventName="docs.nginx_logs_monitoring.dashboard_download">Download</TrackedLink> the dashboard configuration {#download}

#### Import the pre-built dashboard {#import-dashboard}
1. Open HyperDX and navigate to the Dashboards section.
2. Click "Import Dashboard" in the upper right corner under the ellipses.

Expand All @@ -284,7 +284,11 @@

<Image img={finish_import} alt="Finish Import"/>

## The dashboard will be created with all visualizations pre-configured. {#created-dashboard}
#### The dashboard will be created with all visualizations pre-configured {#created-dashboard}

Check warning on line 287 in docs/use-cases/observability/clickstack/integration-examples/nginx-logs.md

View workflow job for this annotation

GitHub Actions / vale

ClickHouse.FutureTense

Instead of future tense 'will be', use present tense.

:::note
Ensure the time range is set to 2025-10-20 11:00:00 - 2025-10-21 11:00:00. The imported dashboard will not have a time range specified by default.
:::

<Image img={example_dashboard} alt="Example Dashboard"/>

Expand Down
Original file line number Diff line number Diff line change
@@ -1,22 +1,23 @@
---
slug: /use-cases/observability/clickstack/integrations/nginx-traces
title: 'Monitoring Nginx Traces with ClickStack'
sidebar_label: 'Nginx Traces'
title: 'Monitoring nginx traces with ClickStack'
sidebar_label: 'Nginx traces'
pagination_prev: null
pagination_next: null
description: 'Monitoring Nginx Traces with ClickStack'
description: 'Monitoring nginx traces with ClickStack'
doc_type: 'guide'
keywords: ['ClickStack', 'nginx', 'traces', 'otel']
---

import Image from '@theme/IdealImage';
import useBaseUrl from '@docusaurus/useBaseUrl';
import import_dashboard from '@site/static/images/clickstack/import-dashboard.png';
import finish_import from '@site/static/images/clickstack/finish-trace-dashboard.png';
import example_dashboard from '@site/static/images/clickstack/example-trace-dashboard.png';
import finish_import from '@site/static/images/clickstack/finish-nginx-traces-dashboard.png';
import example_dashboard from '@site/static/images/clickstack/nginx-traces-dashboard.png';
import view_traces from '@site/static/images/clickstack/nginx-traces-search-view.png';
import { TrackedLink } from '@site/src/components/GalaxyTrackedLink/GalaxyTrackedLink';

# Monitoring Nginx Traces with ClickStack {#nginx-traces-clickstack}
# Monitoring Nginx traces with ClickStack {#nginx-traces-clickstack}

:::note[TL;DR]
This guide shows you how to capture distributed traces from your existing nginx installation and visualize them in ClickStack. You'll learn how to:
Expand All @@ -26,26 +27,29 @@
- Verify traces are appearing in HyperDX
- Use a pre-built dashboard to visualize request performance (latency, errors, throughput)

Time Required: 5-10 minutes.
:::
A demo dataset with sample traces is available if you want to test the integration before configuring your production nginx.

## Prerequisites {#prerequisites}
- ClickStack instance running with OTLP endpoints accessible (ports 4317/4318)
- Existing nginx installation (version 1.18 or higher)
- Root or sudo access to modify nginx configuration
- ClickStack hostname or IP address
Time Required: 5-10 minutes
::::

## Integration with existing nginx {#existing-nginx}

This section covers adding distributed tracing to your existing nginx installation by installing the OpenTelemetry module and configuring it to send traces to ClickStack.
If you would like to test the integration before configuring your own existing setup, you can test with our preconfigured setup and sample data in the [following section](/use-cases/observability/clickstack/integrations/nginx-traces#demo-dataset).

<VerticalStepper>
##### Prerequisites {#prerequisites}
- ClickStack instance running with OTLP endpoints accessible (ports 4317/4318)
- Existing nginx installation (version 1.18 or higher)
- Root or sudo access to modify nginx configuration
- ClickStack hostname or IP address

## Install OpenTelemetry nginx module {#install-module}
<VerticalStepper headerLevel="h4">

#### Install OpenTelemetry nginx module {#install-module}

The easiest way to add tracing to nginx is using the official nginx image with OpenTelemetry support built-in.

### Using the nginx:otel image {#using-otel-image}
##### Using the nginx:otel image {#using-otel-image}

Replace your current nginx image with the OpenTelemetry-enabled version:

Expand All @@ -60,7 +64,7 @@
If you're running nginx outside of Docker, refer to the [OpenTelemetry nginx documentation](https://github.com/open-telemetry/opentelemetry-cpp-contrib/tree/main/instrumentation/nginx) for manual installation instructions.
:::

## Configure nginx to send traces to ClickStack {#configure-nginx}
#### Configure nginx to send traces to ClickStack {#configure-nginx}

Add OpenTelemetry configuration to your `nginx.conf` file. The configuration loads the module and directs traces to ClickStack's OTLP endpoint.

Expand Down Expand Up @@ -132,7 +136,7 @@
- Change **otel_service_name** to match your environment for easier identification in HyperDX
:::

### Understanding the configuration {#understanding-configuration}
##### Understanding the configuration {#understanding-configuration}

**What gets traced:**
Each request to nginx creates a trace span showing:
Expand All @@ -158,7 +162,7 @@
sudo systemctl reload nginx
```

## Verifying Traces in ClickStack {#verifying-traces}
#### Verifying traces in HyperDX {#verifying-traces}

Once configured, log into HyperDX and verify traces are flowing, you should see something like this, if you don't see traces, try adjusting your time range:

Expand All @@ -170,9 +174,9 @@

For users who want to test the nginx trace integration before configuring their production systems, we provide a sample dataset of pre-generated nginx traces with realistic traffic patterns.

<VerticalStepper>
<VerticalStepper headerLevel="h4">

## Start ClickStack {#start-clickstack}
#### Start ClickStack {#start-clickstack}

If you don't have ClickStack running yet, start it with:

Expand All @@ -188,7 +192,7 @@
- Port 4317: OTLP gRPC endpoint (used by nginx module)
- Port 4318: OTLP HTTP endpoint (used for demo traces)

## Download the sample dataset {#download-sample}
#### Download the sample dataset {#download-sample}

Download the sample traces file and update timestamps to the current time:

Expand All @@ -204,7 +208,7 @@
- Latencies ranging from 10ms to 800ms
- Original traffic patterns preserved, shifted to current time

## Send traces to ClickStack {#send-traces}
#### Send traces to ClickStack {#send-traces}

Set your API key as an environment variable (if not already set):

Expand Down Expand Up @@ -232,16 +236,16 @@

You should see a response like `{"partialSuccess":{}}` indicating the traces were successfully sent. All 1,000 traces will be ingested into ClickStack.

## Verify traces in HyperDX {#verify-demo-traces}
#### Verify traces in HyperDX {#verify-demo-traces}

1. Open [HyperDX with demo time range](http://localhost:8080/search?from=1761501600000&to=1761588000000&isLive=false&source=69023d1b4f1d41a964641b09&where=&select=Timestamp,ServiceName,StatusCode,round(Duration/1e6),SpanName&whereLanguage=lucene&orderBy=&filters=[])

1. Open [HyperDX](http://localhost:8080/search?from=1761501600000&to=1761588000000&isLive=false&source=69023d1b4f1d41a964641b09&where=&select=Timestamp,ServiceName,StatusCode,round(Duration/1e6),SpanName&whereLanguage=lucene&orderBy=&filters=[])
Here's what you should see in your search view:

:::note
It is important to use the link above to get the correct time range, if you don't use this link set your time range to Oct 26 13:00:00 - Oct 27 13:00:00 to see proper results.
If you don't see logs, ensure the time range is set to 2025-10-26 13:00:00 - 2025-10-27 13:00:00 and 'Logs' is selected as the source. Using the link is important to get the proper time range of results.
:::

Here's what you should see in your search view:

<Image img={view_traces} alt="View Traces"/>

</VerticalStepper>
Expand All @@ -250,11 +254,11 @@

To help you get started monitoring traces with ClickStack, we provide essential visualizations for trace data.

<VerticalStepper>
<VerticalStepper headerLevel="h4">

## <a href={useBaseUrl('/examples/example-traces.json')} download="example-traces.json">Download</a> the dashboard configuration. {#download}
#### <TrackedLink href={useBaseUrl('/examples/nginx-traces-dashboard.json')} download="nginx-traces-dashboard.json" eventName="docs.nginx_traces_monitoring.dashboard_download">Download</TrackedLink> the dashboard configuration {#download}

## Import the pre-built dashboard {#import-dashboard}
#### Import the pre-built dashboard {#import-dashboard}
1. Open HyperDX and navigate to the Dashboards section.
2. Click "Import Dashboard" in the upper right corner under the ellipses.

Expand All @@ -264,7 +268,11 @@

<Image img={finish_import} alt="Finish Import"/>

## The dashboard will be created with all visualizations pre-configured. {#created-dashboard}
#### The dashboard will be created with all visualizations pre-configured. {#created-dashboard}

Check warning on line 271 in docs/use-cases/observability/clickstack/integration-examples/nginx-traces.md

View workflow job for this annotation

GitHub Actions / vale

ClickHouse.FutureTense

Instead of future tense 'will be', use present tense.

:::note
Ensure the time range is set to 2025-10-26 13:00:00 - 2025-10-27 13:00:00. The imported dashboard will not have a time range specified by default.
:::

<Image img={example_dashboard} alt="Example Dashboard"/>

Expand Down
Loading