crmdesk/deploy/docker/readme/readme.logstash.md

Logstash 日志采集（Logstash Log Ingestion）

中文说明

Logstash（compose/compose-logstash.yaml，一镜像一文件）默认不随主栈启动，通过 start.sh 关键字或 compose 组合启用。默认同时采集两类日志：

• Docker 应用容器写入共享卷 bytedesk_log_data（/app/logs/bytedeskim.log）的日志
• 本地源码运行写入 starter/logs 的 bytedeskim.log

# 方式 A（推荐）：启动中间件/应用栈时附带 Logstash
./start.sh all logstash
./start.sh middleware logstash
./stop.sh all logstash
./stop.sh down all logstash

# 方式 B：仅 Logstash（需先确保 bytedesk-network 存在；在 deploy/docker 目录执行）
docker compose --env-file .env -f compose/compose-logstash.yaml up -d

# 方式 C：与现有 compose 组合
docker compose --env-file .env -f compose/compose-elasticsearch.yaml -f compose/compose-logstash.yaml up -d bytedesk-logstash

# 查看 Logstash 运行状态
docker compose --env-file .env -f compose/compose-logstash.yaml ps
docker compose --env-file .env -f compose/compose-logstash.yaml logs -f

# 在 Elasticsearch 中查看日志索引
curl -u elastic:${ELASTIC_PASSWORD} http://127.0.0.1:19200/_cat/indices/bytedesk-logs-*?v

# 查询最近 20 条日志
curl -u elastic:${ELASTIC_PASSWORD} 'http://127.0.0.1:19200/bytedesk-logs-*/_search?size=20&sort=@timestamp:desc'

说明：

• Logstash 监控接口暴露在 19600 端口。
• 日志索引命名为 bytedesk-logs-YYYY.MM.dd。
• 管道会自动合并 Java 异常堆栈；应用文件日志已切换为纯文本格式，便于在 Elasticsearch 中检索。
• 本地源码运行时，默认读取 starter/logs 下的 bytedeskim.log。
• 共享卷 bytedesk_log_data 由 app（写入）与 logstash（只读采集）共同挂载，两个 compose 文件中均显式声明。
• 如果 starter 已经在本地运行，请在更新后重启一次源码进程，让新的纯文本文件日志格式生效；重启后新写入的日志会被稳定解析并按字段入库。

English

Logstash runs in its own file compose/compose-logstash.yaml (one image per file) and does not start with the default stack; enable it via the logstash keyword or compose composition. It collects logs from both of these sources by default:

• Docker app container logs written to the shared volume bytedesk_log_data (/app/logs/bytedeskim.log)
• Local source-run logs written to starter/logs/bytedeskim.log

# Option A (recommended): start the stack with Logstash attached
./start.sh all logstash
./start.sh middleware logstash
./stop.sh all logstash
./stop.sh down all logstash

# Option B: Logstash only (requires bytedesk-network to exist first; run from deploy/docker)
docker compose --env-file .env -f compose/compose-logstash.yaml up -d

# Option C: combine with existing compose files
docker compose --env-file .env -f compose/compose-elasticsearch.yaml -f compose/compose-logstash.yaml up -d bytedesk-logstash

# Inspect Logstash status and logs
docker compose --env-file .env -f compose/compose-logstash.yaml ps
docker compose --env-file .env -f compose/compose-logstash.yaml logs -f

# List the generated Elasticsearch log indices
curl -u elastic:${ELASTIC_PASSWORD} http://127.0.0.1:19200/_cat/indices/bytedesk-logs-*?v

# Fetch the latest 20 log events
curl -u elastic:${ELASTIC_PASSWORD} 'http://127.0.0.1:19200/bytedesk-logs-*/_search?size=20&sort=@timestamp:desc'

Notes:

• The Logstash monitoring API is exposed on port 19600.
• Log indices are named bytedesk-logs-YYYY.MM.dd.
• The pipeline merges Java stack traces, and the application file log is now emitted as plain text for clean Elasticsearch indexing.
• For local source runs, Logstash reads starter/logs/bytedeskim.log by default.
• If starter is already running locally, restart that source-run process once after this update so the new plain-text file logging pattern takes effect; newly written lines will then be parsed and indexed reliably.