mirror of
https://github.com/arc53/DocsGPT.git
synced 2026-05-21 04:45:06 +00:00
111 lines
3.8 KiB
Plaintext
111 lines
3.8 KiB
Plaintext
---
|
||
title: Observability
|
||
description: Send traces, metrics, and logs from DocsGPT to any OpenTelemetry-compatible backend (Axiom, Honeycomb, Grafana, Datadog, Jaeger, etc.).
|
||
---
|
||
|
||
import { Callout } from 'nextra/components'
|
||
|
||
# Observability
|
||
|
||
DocsGPT bundles the OpenTelemetry SDK and auto-instrumentation packages
|
||
in `application/requirements.txt` — they install with the rest of the
|
||
backend deps. Telemetry is **off by default**; opt in by prefixing the
|
||
launch command with `opentelemetry-instrument` and setting OTLP env
|
||
vars.
|
||
|
||
Auto-instrumentation covers Flask, Starlette, Celery, SQLAlchemy,
|
||
psycopg, Redis, requests, and Python logging. LLM/retriever calls are
|
||
not captured at this layer — see *Going further* below.
|
||
|
||
## Enabling
|
||
|
||
Set these env vars in your `.env` (or compose `environment:` block):
|
||
|
||
```bash
|
||
OTEL_SDK_DISABLED=false
|
||
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
|
||
OTEL_EXPORTER_OTLP_ENDPOINT=https://your-collector.example.com
|
||
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer%20<token>
|
||
OTEL_TRACES_EXPORTER=otlp
|
||
OTEL_METRICS_EXPORTER=otlp
|
||
OTEL_LOGS_EXPORTER=otlp
|
||
OTEL_PYTHON_LOG_CORRELATION=true
|
||
OTEL_RESOURCE_ATTRIBUTES=service.name=docsgpt-backend,deployment.environment=prod
|
||
```
|
||
|
||
Then prefix the process command with `opentelemetry-instrument`. The
|
||
simplest way is a compose override (no image rebuild):
|
||
|
||
```yaml
|
||
# deployment/docker-compose.override.yaml
|
||
services:
|
||
backend:
|
||
command: >
|
||
opentelemetry-instrument gunicorn -w 1 -k uvicorn_worker.UvicornWorker
|
||
--bind 0.0.0.0:7091 --config application/gunicorn_conf.py
|
||
application.asgi:asgi_app
|
||
environment:
|
||
- OTEL_SERVICE_NAME=docsgpt-backend
|
||
worker:
|
||
command: opentelemetry-instrument celery -A application.app.celery worker -l INFO -B
|
||
environment:
|
||
- OTEL_SERVICE_NAME=docsgpt-celery-worker
|
||
```
|
||
|
||
For local dev, prepend `dotenv run --` so the `OTEL_*` vars from `.env`
|
||
reach `opentelemetry-instrument` before it boots the SDK:
|
||
|
||
```bash
|
||
dotenv run -- opentelemetry-instrument flask --app application/app.py run --port=7091
|
||
dotenv run -- opentelemetry-instrument celery -A application.app.celery worker -l INFO --pool=solo
|
||
```
|
||
|
||
|
||
<Callout type="info" emoji="ℹ️">
|
||
Logs are exported in-process when `OTEL_LOGS_EXPORTER=otlp` is set —
|
||
`application/core/logging_config.py` detects the flag and preserves
|
||
the OTEL log handler. Without it, `logging` writes only to stdout.
|
||
</Callout>
|
||
|
||
## Backend examples
|
||
|
||
### Axiom
|
||
|
||
```bash
|
||
OTEL_EXPORTER_OTLP_ENDPOINT=https://api.axiom.co
|
||
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer%20xaat-XXXX,X-Axiom-Dataset=docsgpt
|
||
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
|
||
```
|
||
|
||
`%20` is the URL-encoded space between `Bearer` and the token. Create
|
||
the dataset in the Axiom UI before sending.
|
||
|
||
### Self-hosted OTLP collector / Jaeger / Tempo
|
||
|
||
```bash
|
||
OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4317
|
||
OTEL_EXPORTER_OTLP_PROTOCOL=grpc
|
||
```
|
||
|
||
### Honeycomb / Grafana Cloud / Datadog
|
||
|
||
Each vendor publishes a single-line `OTEL_EXPORTER_OTLP_ENDPOINT` plus
|
||
`OTEL_EXPORTER_OTLP_HEADERS` recipe — drop them in alongside the
|
||
service-name override.
|
||
|
||
## Caveats
|
||
|
||
- The Dockerfile uses `gunicorn -w 1`. If you raise worker count, move
|
||
SDK init into a `post_worker_init` hook to avoid one-thread-per-process
|
||
exporter contention.
|
||
- `asgi.py` wraps Flask in Starlette's `WSGIMiddleware`. Both
|
||
instrumentors are installed, so each request produces a Starlette
|
||
span enclosing a Flask span. Drop
|
||
`opentelemetry-instrumentation-flask` from `requirements.txt` if the
|
||
duplication is noisy.
|
||
- OTEL packages add ~50 MB to the image. They install on every build —
|
||
the runtime cost is zero unless you set `opentelemetry-instrument` on
|
||
the command and set the OTLP env vars.
|
||
- The OTEL exporter ecosystem currently caps `protobuf` at `<7`, so the
|
||
backend runs on protobuf 6.x. This will catch up in a future OTEL
|
||
release. |