90 lines
4.5 KiB
Markdown
90 lines
4.5 KiB
Markdown
<!--
|
||
* @Author: jackning 270580156@qq.com
|
||
* @Description: bytedesk.com https://github.com/Bytedesk/bytedesk
|
||
* Please be aware of the BSL license restrictions before installing Bytedesk IM –
|
||
* selling, reselling, or hosting Bytedesk IM as a service is a breach of the terms and automatically terminates your rights under the license.
|
||
* Business Source License 1.1: https://github.com/Bytedesk/bytedesk/blob/main/LICENSE
|
||
* contact: 270580156@qq.com
|
||
* Copyright (c) 2024 by bytedesk.com, All Rights Reserved.
|
||
-->
|
||
# Logstash 日志采集（Logstash Log Ingestion）
|
||
|
||
## 中文说明
|
||
|
||
Logstash（`compose/compose-logstash.yaml`，一镜像一文件）默认不随主栈启动，通过 `start.sh` 关键字或 compose 组合启用。默认同时采集两类日志：
|
||
|
||
- Docker 应用容器写入共享卷 `bytedesk_log_data`（`/app/logs/bytedeskim.log`）的日志
|
||
- 本地源码运行写入 [starter/logs](../../../starter/logs) 的 `bytedeskim.log`
|
||
|
||
```bash
|
||
# 方式 A（推荐）：启动中间件/应用栈时附带 Logstash
|
||
./start.sh all logstash
|
||
./start.sh middleware logstash
|
||
./stop.sh all logstash
|
||
./stop.sh down all logstash
|
||
|
||
# 方式 B：仅 Logstash（需先确保 bytedesk-network 存在；在 deploy/docker 目录执行）
|
||
docker compose --env-file .env -f compose/compose-logstash.yaml up -d
|
||
|
||
# 方式 C：与现有 compose 组合
|
||
docker compose --env-file .env -f compose/compose-elasticsearch.yaml -f compose/compose-logstash.yaml up -d bytedesk-logstash
|
||
|
||
# 查看 Logstash 运行状态
|
||
docker compose --env-file .env -f compose/compose-logstash.yaml ps
|
||
docker compose --env-file .env -f compose/compose-logstash.yaml logs -f
|
||
|
||
# 在 Elasticsearch 中查看日志索引
|
||
curl -u elastic:${ELASTIC_PASSWORD} http://127.0.0.1:19200/_cat/indices/bytedesk-logs-*?v
|
||
|
||
# 查询最近 20 条日志
|
||
curl -u elastic:${ELASTIC_PASSWORD} 'http://127.0.0.1:19200/bytedesk-logs-*/_search?size=20&sort=@timestamp:desc'
|
||
```
|
||
|
||
说明：
|
||
|
||
- Logstash 监控接口暴露在 `19600` 端口。
|
||
- 日志索引命名为 `bytedesk-logs-YYYY.MM.dd`。
|
||
- 管道会自动合并 Java 异常堆栈；应用文件日志已切换为纯文本格式，便于在 Elasticsearch 中检索。
|
||
- 本地源码运行时，默认读取 [starter/logs](../../../starter/logs) 下的 `bytedeskim.log`。
|
||
- 共享卷 `bytedesk_log_data` 由 app（写入）与 logstash（只读采集）共同挂载，两个 compose 文件中均显式声明。
|
||
- 如果 starter 已经在本地运行，请在更新后重启一次源码进程，让新的纯文本文件日志格式生效；重启后新写入的日志会被稳定解析并按字段入库。
|
||
|
||
## English
|
||
|
||
Logstash runs in its own file `compose/compose-logstash.yaml` (one image per file) and does not start with the default stack; enable it via the `logstash` keyword or compose composition. It collects logs from both of these sources by default:
|
||
|
||
- Docker app container logs written to the shared volume `bytedesk_log_data` (`/app/logs/bytedeskim.log`)
|
||
- Local source-run logs written to [starter/logs](../../../starter/logs)/bytedeskim.log
|
||
|
||
```bash
|
||
# Option A (recommended): start the stack with Logstash attached
|
||
./start.sh all logstash
|
||
./start.sh middleware logstash
|
||
./stop.sh all logstash
|
||
./stop.sh down all logstash
|
||
|
||
# Option B: Logstash only (requires bytedesk-network to exist first; run from deploy/docker)
|
||
docker compose --env-file .env -f compose/compose-logstash.yaml up -d
|
||
|
||
# Option C: combine with existing compose files
|
||
docker compose --env-file .env -f compose/compose-elasticsearch.yaml -f compose/compose-logstash.yaml up -d bytedesk-logstash
|
||
|
||
# Inspect Logstash status and logs
|
||
docker compose --env-file .env -f compose/compose-logstash.yaml ps
|
||
docker compose --env-file .env -f compose/compose-logstash.yaml logs -f
|
||
|
||
# List the generated Elasticsearch log indices
|
||
curl -u elastic:${ELASTIC_PASSWORD} http://127.0.0.1:19200/_cat/indices/bytedesk-logs-*?v
|
||
|
||
# Fetch the latest 20 log events
|
||
curl -u elastic:${ELASTIC_PASSWORD} 'http://127.0.0.1:19200/bytedesk-logs-*/_search?size=20&sort=@timestamp:desc'
|
||
```
|
||
|
||
Notes:
|
||
|
||
- The Logstash monitoring API is exposed on port `19600`.
|
||
- Log indices are named `bytedesk-logs-YYYY.MM.dd`.
|
||
- The pipeline merges Java stack traces, and the application file log is now emitted as plain text for clean Elasticsearch indexing.
|
||
- For local source runs, Logstash reads starter/logs/bytedeskim.log by default.
|
||
- If starter is already running locally, restart that source-run process once after this update so the new plain-text file logging pattern takes effect; newly written lines will then be parsed and indexed reliably.
|