-
Notifications
You must be signed in to change notification settings - Fork 88
bpu,configs: Add optional BTBTAGE upper-bound mode for kmhv3 #770
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 10 commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
dc5faa5
configs: Disable MGSC in align baseline
jensen-yan a068a5f
cpu,configs: Enable branch-PC index in BTB-TAGE
jensen-yan f3327bd
configs: Scale BTB-TAGE ways by 4x in align
jensen-yan 5bc3d42
configs: Scale BTB-TAGE table capacity by 4x
jensen-yan b44d5b1
configs: Fix BTB-TAGE tableSizes assignment
jensen-yan 9774e83
cpu,configs: Add TAGE position-mixed index experiment
jensen-yan 46a96eb
util: add one skill for CI scores
jensen-yan 54f5b4c
util: update docs
jensen-yan c69f618
bpu: Add BTBTAGE upper-bound probe
jensen-yan 9dbb7cb
cpu: Add UB-P path-hash mode
jensen-yan 705affd
configs: Default kmhv3 UB to path-hash
jensen-yan f95abde
merge: Sync xs-dev and trim UB TAGE config surface
jensen-yan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,149 @@ | ||
| --- | ||
| name: ci-perf-analysis | ||
| description: 用于从 GitHub Actions 的 gem5 性能 CI 中定位 summary、score.txt 和归档目录,并结合本地 gem5_data_proc 对 spec06/spec17 结果做 weighted score、benchmark 子项对比和通用 stats 归因。适用于用户给出 run URL/run id、commit、workflow run,或要求分析 CI 跑分变化来源时。 | ||
| --- | ||
|
|
||
| # CI 性能分析 | ||
|
|
||
| ## 概览 | ||
|
|
||
| 这个 skill 只做一条固定链路: | ||
|
|
||
| 1. 用 `gh` 从 CI run 找到 summary 对应的 `score.txt` 和归档目录。 | ||
| 2. 用本地 `gem5_data_proc/run.py` 把 `spec_all/` 处理成 `csv`、`weighted.csv`、`score.csv`。 | ||
| 3. 对比 benchmark 级收益,并在需要时继续下钻到 `stats.txt` 或其他归档结果。 | ||
|
|
||
| ## 快速开始 | ||
|
|
||
| ### 1. 先拿 summary 和归档目录 | ||
|
|
||
| 优先使用 bundled script: | ||
|
|
||
| ```bash | ||
| python3 .codex/skills/ci-perf-analysis/scripts/ci_perf_info.py \ | ||
| https://github.com/OpenXiangShan/GEM5/actions/runs/<run_id> | ||
| ``` | ||
|
|
||
| 输出会包含: | ||
|
|
||
| - `archive_path` | ||
| - `spec_all` 目录 | ||
| - `score.txt` 的最后 42 行 | ||
|
|
||
| 这和 workflow 在 GitHub summary 里展示的内容一致,因为 CI 本身就是把 `score.txt` 的最后 42 行写进 step summary。 | ||
|
|
||
| ### 2. 优先使用本地 `gem5_data_proc` | ||
|
|
||
| 默认优先使用: | ||
|
|
||
| ```bash | ||
| /nfs/home/yanyue/workspace/gem5_data_proc | ||
| ``` | ||
|
|
||
| 如果用户本地没有,再执行: | ||
|
|
||
| ```bash | ||
| git clone https://github.com/jensen-yan/gem5_data_proc | ||
| # 设置环境变量 | ||
| export $GEM5_DATA_PROC_HOME=xxx | ||
| ``` | ||
|
|
||
|
|
||
| ### 3. 用 `gem5_data_proc` 处理整个归档 | ||
|
|
||
| ```bash | ||
| cd $GEM5_DATA_PROC_HOME | ||
| python3 run.py /nfs/home/share/gem5_ci/performance_data/spec06-0.3c/<archive_dir> \ | ||
| --out-dir /tmp/gem5_proc_runA \ | ||
| --tag runA | ||
| ``` | ||
|
|
||
| 关键输出: | ||
|
|
||
| - `<tag>.csv`:point 级或 benchmark 聚合后的原始统计 | ||
| - `<tag>-weighted.csv`:按权重聚合后的 benchmark 统计 | ||
| - `<tag>-score.csv`:最终 score/time/coverage | ||
|
|
||
| ### 4. 对比两个 run | ||
|
|
||
| 最常见的是比较两次 CI: | ||
|
|
||
| ```bash | ||
| python3 .codex/skills/ci-perf-analysis/scripts/ci_perf_info.py <runA> | ||
| python3 .codex/skills/ci-perf-analysis/scripts/ci_perf_info.py <runB> | ||
|
|
||
| cd $GEM5_DATA_PROC_HOME | ||
| python3 run.py <archiveA> --out-dir /tmp/gem5_proc_A --tag A | ||
| python3 run.py <archiveB> --out-dir /tmp/gem5_proc_B --tag B | ||
| ``` | ||
|
|
||
| 然后用短 Python 片段读取两个 `*-score.csv` / `*-weighted.csv` 做对比。优先关注: | ||
|
|
||
| - 总 score 变化 | ||
| - benchmark 级 `time` / `score` 变化 | ||
| - 用户关心的 stats 指标变化 | ||
|
|
||
| ## 下钻分析 | ||
|
|
||
| ### 1. 看 benchmark 级收益 | ||
|
|
||
| - 对两个归档分别运行 `run.py` | ||
| - 比较 `*-score.csv` 里的 `time` 和 `score` | ||
| - 按 `score_delta_pct` 或 `time_delta_pct` 排序 | ||
|
|
||
| ### 2. 看 stats 指标变化 | ||
|
|
||
| - 先看 `*-weighted.csv` 里的通用统计 | ||
| - 如果用户已经给出重点指标,直接围绕这些指标对比 | ||
| - 如果用户没有指定,优先从 `time`、`cpi`、前端、后端、内存、分支等大类里挑变化最明显的项 | ||
| - 归因时优先描述“哪些 stats 在变”,再解释这些变化更像支持哪类根因 | ||
|
|
||
| ### 3. 需要时再读归档里的其他文件 | ||
|
|
||
| - `stats.txt`:看原始统计,位置在 | ||
| /nfs/home/share/gem5_ci/performance_data/spec06-0.3c/<archive_dir>/<spec_bmk>/m5out/stats.txt | ||
| - `score.txt`:对照 summary | ||
| - 其他 CSV 或日志:按用户问题决定是否下钻 | ||
|
|
||
| ## 常用命令 | ||
|
|
||
| ### 已知 run URL,直接拿 archive path | ||
|
|
||
| ```bash | ||
| python3 .codex/skills/ci-perf-analysis/scripts/ci_perf_info.py <run_url_or_id> | ||
| ``` | ||
|
|
||
| ### 已知 archive path,直接处理 | ||
|
|
||
| ```bash | ||
| cd $GEM5_DATA_PROC_HOME | ||
| python3 run.py <archive_dir> --out-dir /tmp/gem5_proc --tag run | ||
| ``` | ||
|
|
||
| ## 输出组织建议 | ||
|
|
||
| 回答这类问题时,优先按下面的顺序组织: | ||
|
|
||
| 1. commit / run / workflow / 配置差异 | ||
| 2. summary 里的总分变化 | ||
| 3. benchmark 级主要收益和回退项 | ||
| 4. 相关 stats 指标变化 | ||
| 5. 对根因的判断 | ||
|
|
||
| 结论要尽量区分: | ||
|
|
||
| - “哪个 benchmark 涨了” | ||
| - “哪些 stats 在变” | ||
| - “这些 stats 更像支持哪类根因” | ||
|
|
||
| ## 资源 | ||
|
|
||
| ### scripts/ | ||
|
|
||
| - `ci_perf_info.py` | ||
| - 输入 run URL 或 run id | ||
| - 输出 archive path、spec_all 路径和 `score.txt` tail | ||
|
|
||
| ### references/ | ||
|
|
||
| 当前不需要额外参考文件。后续如果这套流程扩展到更多 workflow 或更多统计口径,再新增参考文档。 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| #!/usr/bin/env python3 | ||
| """Locate archived CI performance data and print the score summary.""" | ||
|
|
||
| import argparse | ||
| import json | ||
| import os | ||
| import re | ||
| import subprocess | ||
| import sys | ||
| from pathlib import Path | ||
|
|
||
|
|
||
| REPO = "OpenXiangShan/GEM5" | ||
| ARCHIVE_RE = re.compile( | ||
| r"Archiving performance data to " | ||
| r"(/nfs/home/share/gem5_ci/performance_data/\S+)" | ||
| ) | ||
|
|
||
|
|
||
| def run_cmd(cmd: list[str]) -> str: | ||
| result = subprocess.run(cmd, capture_output=True, text=True) | ||
| if result.returncode != 0: | ||
| raise SystemExit(result.stderr.strip() or f"command failed: {' '.join(cmd)}") | ||
| return result.stdout | ||
|
|
||
|
|
||
| def parse_run_id(text: str) -> str: | ||
| match = re.search(r"/actions/runs/(\d+)", text) | ||
| if match: | ||
| return match.group(1) | ||
| if text.isdigit(): | ||
| return text | ||
| raise SystemExit(f"cannot parse run id from: {text}") | ||
|
|
||
|
|
||
| def get_job_id(run_id: str) -> str: | ||
| output = run_cmd(["gh", "api", f"repos/{REPO}/actions/runs/{run_id}/jobs"]) | ||
| data = json.loads(output) | ||
| jobs = data.get("jobs", []) | ||
| if not jobs: | ||
| raise SystemExit(f"no jobs found for run {run_id}") | ||
| return str(jobs[0]["id"]) | ||
|
|
||
|
|
||
| def get_archive_path(job_id: str) -> str: | ||
| log_text = run_cmd(["gh", "api", f"repos/{REPO}/actions/jobs/{job_id}/logs"]) | ||
| match = ARCHIVE_RE.search(log_text) | ||
| if not match: | ||
| raise SystemExit(f"cannot find archive path in logs for job {job_id}") | ||
| return match.group(1) | ||
|
|
||
|
|
||
| def print_score(score_path: Path, tail_lines: int) -> None: | ||
| if not score_path.is_file(): | ||
| print(f"score.txt not found: {score_path}", file=sys.stderr) | ||
| return | ||
|
|
||
| lines = score_path.read_text().splitlines() | ||
| print(f"score.txt: {score_path}") | ||
| print(f"--- tail -n {tail_lines} ---") | ||
| for line in lines[-tail_lines:]: | ||
| print(line) | ||
|
|
||
|
|
||
| def main() -> None: | ||
| parser = argparse.ArgumentParser( | ||
| description="Print CI archive path and score summary for a GEM5 perf run." | ||
| ) | ||
| parser.add_argument("run", help="GitHub Actions run URL or run id") | ||
| parser.add_argument( | ||
| "--tail-lines", | ||
| type=int, | ||
| default=42, | ||
| help="how many trailing lines of score.txt to print", | ||
| ) | ||
| args = parser.parse_args() | ||
|
|
||
| run_id = parse_run_id(args.run) | ||
| job_id = get_job_id(run_id) | ||
| archive_path = Path(get_archive_path(job_id)) | ||
|
|
||
| print(f"run_id: {run_id}") | ||
| print(f"job_id: {job_id}") | ||
| print(f"archive_path: {archive_path}") | ||
| print(f"spec_all: {archive_path / 'spec_all'}") | ||
|
|
||
| score_path = archive_path / "score.txt" | ||
| print_score(score_path, args.tail_lines) | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| main() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.