Skip to content

feat(sec-core): add session report command#703

Open
jfeng18 wants to merge 8 commits into
alibaba:mainfrom
jfeng18:feat/session-report
Open

feat(sec-core): add session report command#703
jfeng18 wants to merge 8 commits into
alibaba:mainfrom
jfeng18:feat/session-report

Conversation

@jfeng18

@jfeng18 jfeng18 commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add agent-sec-cli observability report command: per-session debrief aggregating observability and security event data from existing DBs into one view.
  • Supports --last (most recent session), --session-id, and --format json.
  • Graceful degradation with actionable hints when data sources are empty or unavailable.

Example output

Session eb5d7f37-9d1  (2026-06-03 14:06:34 — 14:06:46, 12s, 9 turns)
  LLM calls:       2
  Payload:         99,668 bytes sent, 1,581 bytes received
  Tools used:      run_shell_command(1)
  Security:        code_scan succeeded: 1

Changes

  • New: observability/session_report.py — SessionReport dataclass + build_session_report() aggregator + format_text() renderer
  • Modified: observability/cli.py — added report subcommand with --last/--session-id/--format
  • New: tests/unit-test/observability/test_session_report.py — 9 tests covering empty session, basic aggregation, security verdicts, format rendering, JSON roundtrip, and hint coverage
  • Modified: tests/unit-test/observability/test_cli.py — 8 integration tests for the report CLI subcommand (arg validation, text/JSON output, --last flag, security reader wiring)

Data sources (read-only, no writes)

  1. Observability SQLite (session/run/event timeline, LLM call metrics, tool breakdown)
  2. Security events SQLite via query_correlation_candidates (security verdicts by category)

Test plan

  • 已 E2E: on Agentic ECS (short prompt / large output / multi-turn resume)
    • Observability + security data segments verified with real data
    • Empty-data hints verified (security session_id gap)
    • JSON output validated
    • Session-not-found error message verified
  • 仅单测: 9 unit tests (session_report module) + 8 CLI integration tests, all with mocked readers

🤖 Generated with Claude Code

@jfeng18 jfeng18 requested review from RemindD, edonyzpc and kid9 as code owners June 3, 2026 14:13
@github-actions github-actions Bot added the component:sec-core src/agent-sec-core/ label Jun 3, 2026

@RemindD RemindD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. 目前agent-sec-cli observability更多还是为安全能力的展示。将tokenless数据集成容易有歧义。退一步说,与tokenless的集成也应该使用API接口,避免直接访问后端sqldb
  2. format 问题需要使用make python-code-pretty

@jfeng18

jfeng18 commented Jun 4, 2026

Copy link
Copy Markdown
Contributor Author

Agree on both points. Updated:

  1. Removed all tokenless integration — report now only aggregates observability + security data, scoped to security observability. No more import sqlite3, no direct DB access to tokenless stats.db.
  2. Ran black + isort for formatting.

Thanks for the review.

@jfeng18 jfeng18 force-pushed the feat/session-report branch 3 times, most recently from 9c95888 to 3f0fdea Compare June 4, 2026 23:35
@jfeng18

jfeng18 commented Jun 6, 2026

Copy link
Copy Markdown
Contributor Author

Updated per review feedback:

  1. Stale docstring fixed: removed "compression" from cli.py:149 docstring — tokenless code was already removed, this reference was stale
  2. Formatting applied: ran isort --profile black + black, verified zero diff after
  3. PR scope confirmed: only 2 data sources remain (Observability + Security), all tokenless/sqlite3 code was already removed in previous commit

Changes pushed in 81d59f4. Ready for re-review.

@jfeng18 jfeng18 force-pushed the feat/session-report branch from f4559d8 to 8850e4c Compare June 6, 2026 10:22
@jfeng18

jfeng18 commented Jun 6, 2026

Copy link
Copy Markdown
Contributor Author

Review feedback addressed. Ready for re-review.

@jfeng18

jfeng18 commented Jun 8, 2026

Copy link
Copy Markdown
Contributor Author

Hi @RemindD, gentle ping — I've addressed your earlier feedback. Ready for re-review when convenient.

jfeng18 and others added 7 commits June 10, 2026 11:05
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds 6 tests covering the `observability report` CLI command:
- --session-id and --last flag validation
- invalid --format rejection
- session not found error
- text and JSON output formats
- --last with no recorded sessions

These cover the cli.py:report() function (lines 137-199) that was
missing from incremental coverage, causing CI gate failure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jfeng18 jfeng18 force-pushed the feat/session-report branch from 1d37d28 to cd69153 Compare June 10, 2026 03:06

@RemindD RemindD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add E2E tests as well. @edonyzpc any comment?

7 tests covering: JSON/text output, session-id lookup, unknown
session, missing args, invalid format, and empty database.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jfeng18

jfeng18 commented Jun 11, 2026

Copy link
Copy Markdown
Contributor Author

Added E2E tests (tests/e2e/cli/test_session_report_e2e.py):

  • test_report_last_json — seeds session, verifies JSON fields (session_id, llm_calls, bytes, tool_breakdown)
  • test_report_last_text — verifies text output contains specific values (500 bytes, read_file(1))
  • test_report_session_id_json — lookup by specific session ID
  • test_report_unknown_session_fails — unknown session → exit 1
  • test_report_no_args_fails — missing args → exit 1
  • test_report_invalid_format_fails — invalid format → exit 1
  • test_report_last_empty_db_fails — empty database → exit 1

All 7 tests pass on ECS (Python 3.11, agent-sec-cli 0.5.0). Follows existing tests/e2e/cli/conftest.py patterns (isolated_data_dir fixture, run_cli helper).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component:sec-core src/agent-sec-core/

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants