当前位置： 首页 > news >正文

OpenObserve API Usage Guide for Log Management

news 来源：原创 2025/4/30 9:20:22

OpenObserve API Usage Guide for Audit Log Management

1. 概述

1.1 目标

本文档旨在详细介绍 OpenObserve 的 API 使用方法，帮助用户通过 API 实现日志管理功能，包括日志摄入、查询、模糊匹配（类似 SQL 的 LIKE）、stream 管理等。文档基于 test stream 的审计日志场景，解决之前遇到的模糊匹配问题。

1.2 OpenObserve API 简介

OpenObserve 是一个开源的 observability 平台，支持日志、指标和追踪数据。API 是与 OpenObserve 交互的主要方式，支持以下功能：

  • 日志摄入：通过 API 批量摄入日志数据。
  • 日志查询：支持 SQL-like 语法查询日志，支持模糊匹配。
  • Stream 管理：创建、更新、删除 stream，以及管理 stream 设置（如 full_text_search_keys）。
  • 用户和组织管理：管理组织、用户和权限。
  • 告警和仪表盘：配置告警规则和创建仪表盘。

1.3 认证

OpenObserve API 使用 HTTP Basic Authentication，所有请求必须包含 Authorization 头。认证信息是用户 ID 和密码的 Base64 编码。

生成 Authorization 头
  • 用户 ID：admin@admin.com
  • 密码：admin
  • Base64 编码：echo -n "admin@admin.com:admin" | base64 -> YWRtaW5AYWRtaW4uY29tOmFkbWlu
示例
-H "Authorization: Basic YWRtaW5AYWRtaW4uY29tOmFkbWlu"

2. API 端点和使用方法

以下是 OpenObserve 的主要 API 端点及其使用方法，基于 OpenObserve 文档。

2.1 日志摄入 API

端点
  • POST /api/{organization}/{stream}/_json
功能

批量摄入日志数据到指定 stream。OpenObserve 会根据数据自动推断 schema。

请求
  • Content-Type：application/json
  • Body：JSON 数组，每个元素是一条日志记录。
示例

摄入 3 条审计日志到 test stream：

curl -X POST \-H "Authorization: Basic YWRtaW5AYWRtaW4uY29tOmFkbWlu" \-H "Content-Type: application/json" \-d '[{"user": "test_user_create","path": "/test/path","category": "File Management","event": "Create Document","date": "2025-04-28 17:01:00","comment": "Created a new document","ip": "192.168.1.1"},{"user": "admin_user","path": "/admin/path","category": "User Management","event": "Update User","date": "2025-04-28 17:02:00","comment": "Updated user profile","ip": "192.168.1.2"},{"user": "create_team_lead","path": "/team/path","category": "Team Management","event": "Create Team","date": "2025-04-28 17:03:00","comment": "Created a new team","ip": "192.168.1.3"}]' \http://localhost:5080/api/default/test/_json
响应
{"code": 200,"status": [{"name": "test","successful": 3,"failed": 0}]
}
注意事项
  • 如果 test stream 不存在，OpenObserve 会自动创建。
  • 确保字段类型一致（例如 date 应始终为字符串格式）。

2.2 日志查询 API

端点
  • POST /api/{organization}/{stream}/_search
功能

使用 SQL-like 语法查询日志，支持模糊匹配、聚合等操作。

请求
  • Content-Type：application/json
  • Body：
    • query.sql：SQL 查询语句。
    • query.start_time 和 query.end_time：查询时间范围（微秒）。
    • from 和 size：分页参数。
示例 1：基本查询

查询 test stream 的最新 10 条记录：

curl -X POST \-H "Authorization: Basic YWRtaW5AYWRtaW4uY29tOmFkbWlu" \-H "Content-Type: application/json" \-d '{"query": {"sql": "SELECT * FROM test","start_time": 0,"end_time": 999999999999999,"from": 0,"size": 10}}' \http://localhost:5080/api/default/test/_search
响应
{"took": 10,"hits": {"total": 3,"hits": [{"user": "test_user_create","path": "/test/path","category": "File Management","event": "Create Document","date": "2025-04-28 17:01:00","comment": "Created a new document","ip": "192.168.1.1"},{"user": "admin_user","path": "/admin/path","category": "User Management","event": "Update User","date": "2025-04-28 17:02:00","comment": "Updated user profile","ip": "192.168.1.2"},{"user": "create_team_lead","path": "/team/path","category": "Team Management","event": "Create Team","date": "2025-04-28 17:03:00","comment": "Created a new team","ip": "192.168.1.3"}]}
}
示例 2：模糊匹配查询

使用 str_match 实现模糊匹配（类似 LIKE '%create%'），查找 user 字段包含 “create” 的记录：

curl -X POST \-H "Authorization: Basic YWRtaW5AYWRtaW4uY29tOmFkbWlu" \-H "Content-Type: application/json" \-d '{"query": {"sql": "SELECT * FROM test WHERE str_match(user, '\''create'\'')","start_time": 0,"end_time": 999999999999999,"from": 0,"size": 10}}' \http://localhost:5080/api/default/test/_search
响应
{"took": 10,"hits": {"total": 2,"hits": [{"user": "test_user_create","path": "/test/path","category": "File Management","event": "Create Document","date": "2025-04-28 17:01:00","comment": "Created a new document","ip": "192.168.1.1"},{"user": "create_team_lead","path": "/team/path","category": "Team Management","event": "Create Team","date": "2025-04-28 17:03:00","comment": "Created a new team","ip": "192.168.1.3"}]}
}
示例 3：全局全文搜索

使用 match_all 在所有字段中查找 “create”：

curl -X POST \-H "Authorization: Basic YWRtaW5AYWRmiW4uY29tOmFkbWlu" \-H "Content-Type: application/json" \-d '{"query": {"sql": "SELECT * FROM test WHERE match_all('\''create'\'')","start_time": 0,"end_time": 999999999999999,"from": 0,"size": 10}}' \http://localhost:5080/api/default/test/_search
响应
{"took": 10,"hits": {"total": 3,"hits": [{"user": "test_user_create","path": "/test/path","category": "File Management","event": "Create Document","date": "2025-04-28 17:01:00","comment": "Created a new document","ip": "192.168.1.1"},{"user": "admin_user","path": "/admin/path","category": "User Management","event": "Update User","date": "2025-04-28 17:02:00","comment": "Updated user profile","ip": "192.168.1.2"},{"user": "create_team_lead","path": "/team/path","category": "Team Management","event": "Create Team","date": "2025-04-28 17:03:00","comment": "Created a new team","ip": "192.168.1.3"}]}
}
注意事项
  • str_match 不需要字段在 full_text_search_keys 中，但如果启用全文搜索，查询效率可能更高。
  • match_all 依赖 full_text_search_keys，否则可能无法匹配。

2.3 Stream 管理 API

端点 1：列出所有 Streams
  • GET /api/{organization}/streams
功能

列出组织中的所有 stream，包括 schema 和设置。

示例
curl -X GET \-H "Authorization: Basic YWRtaW5AYWRtaW4uY29tOmFkbWlu" \http://localhost:5080/api/default/streams
响应
[{"name": "test","storage_type": "disk","stream_type": "logs","stats": {"doc_num": 3,"storage_size": 0.0003},"schema": [{"name": "_timestamp", "type": "Int64"},{"name": "category", "type": "Utf8"},{"name": "comment", "type": "Utf8"},{"name": "date", "type": "Utf8"},{"name": "event", "type": "Utf8"},{"name": "ip", "type": "Utf8"},{"name": "path", "type": "Utf8"},{"name": "user", "type": "Utf8"}],"settings": {"full_text_search_keys": [],"data_retention": 0}}
]
端点 2：更新 Stream 设置
  • PUT /api/{organization}/streams/{stream}/settings
功能

更新 stream 的设置，例如 full_text_search_keys。

示例

尝试启用 user 字段的全文搜索：

curl -X PUT \-H "Authorization: Basic YWRtaW5AYWRtaW4uY29tOmFkbWlu" \-H "Content-Type: application/json" \-d '{"full_text_search_keys": ["user"]}' \http://localhost:5080/api/default/streams/test/settings
响应
{"code": 200,"message": "Settings updated successfully for stream test"
}
注意事项
  • 之前尝试更新 full_text_search_keys 失败（Json deserialize error），建议通过 UI 手动设置，或者联系 OpenObserve 社区解决。
端点 3：删除 Stream
  • DELETE /api/{organization}/streams/{stream}
功能

删除指定 stream 及其数据。

示例
curl -X DELETE \-H "Authorization: Basic YWRtaW5AYWRtaW4uY29tOmFkbWlu" \http://localhost:5080/api/default/streams/test
响应
{"code": 200,"message": "Stream test deleted successfully"
}

2.4 告警 API

端点
  • POST /api/{organization}/alerts
功能

创建告警规则，例如当 event 字段包含 “Error” 时触发告警。

示例

创建告警，当 event 包含 “Error” 时触发：

curl -X POST \-H "Authorization: Basic YWRtaW5AYWRtaW4uY29tOmFkbWlu" \-H "Content-Type: application/json" \-d '{"name": "error_alert","stream": "test","query": "SELECT * FROM test WHERE str_match(event, '\''Error'\'')","condition": "num > 0","threshold": 1,"frequency": "5m","destination": "email","email": "admin@example.com"}' \http://localhost:5080/api/default/alerts
响应
{"code": 200,"message": "Alert created successfully"
}

3. 总结

OpenObserve 的 API 提供了强大的日志管理功能，支持摄入、查询、模糊匹配和 stream 管理。通过 str_match 可以实现类似 LIKE 的模糊匹配，满足审计日志场景的需求。建议进一步优化 full_text_search_keys 设置，并关注 OpenObserve 的版本更新以获取更多功能支持（如 REGEXP 或 LIKE）。

相关文章：

  • BT134-ASEMI机器人功率器件专用BT134
  • ＜ 自用文 rclone ＞ 在 Ubuntu 24 访问 Google Drive 网络内容
  • JSX初学习
  • 缺口拼图3.3，某里v2，研究分析
  • 海外社交App的Web3革命：去中心化社交与Token经济实战指南
  • 《Origin画百图》之带颜色映射的3维散点图
  • 使用VS2022开发并部署QT应用
  • GC的查看
  • OCR技术，金融行业的“数字魔法”✨
  • Python列表全面解析：从基础到高阶操作
  • 前端性能优化2:结合HTTPS与最佳实践，全面优化你的网站性能
  • 普通IT的股票交易成长史--20250429午
  • 误在非开发分支上开发解决方案
  • 大语言模型能否替代心理治疗师的深度拓展研究:fou
  • 通信协议——SPI通信协议
  • 【C++编程入门】：基本语法
  • Discord多账号注册登录：如何同时管理多个账户？
  • 模型上下文协议MCP协议（Model Context Protocol）未来应用场景分析（多智能体协作A2A协议）
  • CA添加删除辅小区信令流程
  • React 第三十五节 Router 中useNavigate 的作用及用途详解
  • 笔墨如何“构城”？上海美院城市山水晋京展出
  • 人民日报：在大有可为的时代大有作为
  • 这就是上海！
  • 建行一季度净利833.51亿同比下降3.99%，营收降5.4%
  • 王毅：坚持金砖团结合作，改革完善全球治理
  • 国家发展改革委：我们对实现今年经济社会发展目标任务充满信心