Skip to content

Commit 6cc70c4

Browse files
jensen-yanjensen-yan
authored
bpu,configs: Add optional BTBTAGE upper-bound mode for kmhv3 (#770)
* configs: Disable MGSC in align baseline Change-Id: I3a357ccab4ae078edb30bd4ffe5e01b5c6d9ac7f * cpu,configs: Enable branch-PC index in BTB-TAGE Change-Id: I0e029d43a10487cc7090e15973db1fd84a731375 * configs: Scale BTB-TAGE ways by 4x in align Change-Id: Ibdd30c6049dca43e022e5bd4e94a89e493397c33 * configs: Scale BTB-TAGE table capacity by 4x Change-Id: If9f2e2a8099ac58c8f9ad5b86b00292eeebc48d4 * configs: Fix BTB-TAGE tableSizes assignment Change-Id: I06630ca8db53f4fd944355fd03a166203c5dca11 * cpu,configs: Add TAGE position-mixed index experiment Change-Id: Ib8b67365d8196e4b2ec0044a8883e6983b88b2f6 * util: add one skill for CI scores Change-Id: If39f7384f40450cae4c7dbb285bc8ea71ef0486b * util: update docs Change-Id: I8ccbe4c2e97c9e87274dabc61f41b4d4398b79d0 * bpu: Add BTBTAGE upper-bound probe Change-Id: I6d73773fcaa021a7d2a0e7c45eb5f9d61aeddda5 * cpu: Add UB-P path-hash mode Add a path-hash-backed exact-history mode to BTBTAGEUpperBound. This keeps the UB implementation aligned with the current RTL-friendly PHR semantics and exposes a kmhv3 flag for direct comparison against UB-S. Change-Id: I9bb7832a5770833562a48dcf7116d55ef8db8b15 * configs: Default kmhv3 UB to path-hash Make kmhv3 default to the UB-P path-hash mode when BTBTAGEUpperBound is enabled, and keep an explicit flag to fall back to outcome-history mode for comparisons. Change-Id: I98cade975baa98cbddfde28bda281cc97b384ec6
1 parent dbc3782 commit 6cc70c4

14 files changed

Lines changed: 2075 additions & 5 deletions

File tree

‎.codex/skills/ci-perf-analysis/SKILL.md‎

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
---
2+
name: ci-perf-analysis
3+
description: 用于从 GitHub Actions 的 gem5 性能 CI 中定位 summary、score.txt 和归档目录，并结合本地 gem5_data_proc 对 spec06/spec17 结果做 weighted score、benchmark 子项对比和通用 stats 归因。适用于用户给出 run URL/run id、commit、workflow run，或要求分析 CI 跑分变化来源时。
4+
---
5+
6+
# CI 性能分析
7+
8+
## 概览
9+
10+
这个 skill 只做一条固定链路：
11+
12+
1. 用 `gh` 从 CI run 找到 summary 对应的 `score.txt` 和归档目录。
13+
2. 用本地 `gem5_data_proc/run.py` 把 `spec_all/` 处理成 `csv`、`weighted.csv`、`score.csv`。
14+
3. 对比 benchmark 级收益，并在需要时继续下钻到 `stats.txt` 或其他归档结果。
15+
16+
## 快速开始
17+
18+
### 1. 先拿 summary 和归档目录
19+
20+
优先使用 bundled script：
21+
22+
```bash
23+
python3 .codex/skills/ci-perf-analysis/scripts/ci_perf_info.py \
24+
https://github.com/OpenXiangShan/GEM5/actions/runs/<run_id>
25+
```
26+
27+
输出会包含：
28+
29+
- `archive_path`
30+
- `spec_all` 目录
31+
- `score.txt` 的最后 42 行
32+
33+
这和 workflow 在 GitHub summary 里展示的内容一致，因为 CI 本身就是把 `score.txt` 的最后 42 行写进 step summary。
34+
35+
### 2. 优先使用本地 `gem5_data_proc`
36+
37+
默认优先使用：
38+
39+
```bash
40+
/nfs/home/yanyue/workspace/gem5_data_proc
41+
```
42+
43+
如果用户本地没有，再执行：
44+
45+
```bash
46+
git clone https://github.com/jensen-yan/gem5_data_proc
47+
# 设置环境变量
48+
export $GEM5_DATA_PROC_HOME=xxx
49+
```
50+
51+
52+
### 3. 用 `gem5_data_proc` 处理整个归档
53+
54+
```bash
55+
cd $GEM5_DATA_PROC_HOME
56+
python3 run.py /nfs/home/share/gem5_ci/performance_data/spec06-0.3c/<archive_dir> \
57+
--out-dir /tmp/gem5_proc_runA \
58+
--tag runA
59+
```
60+
61+
关键输出：
62+
63+
- `<tag>.csv`：point 级或 benchmark 聚合后的原始统计
64+
- `<tag>-weighted.csv`：按权重聚合后的 benchmark 统计
65+
- `<tag>-score.csv`：最终 score/time/coverage
66+
67+
### 4. 对比两个 run
68+
69+
最常见的是比较两次 CI：
70+
71+
```bash
72+
python3 .codex/skills/ci-perf-analysis/scripts/ci_perf_info.py <runA>
73+
python3 .codex/skills/ci-perf-analysis/scripts/ci_perf_info.py <runB>
74+
75+
cd $GEM5_DATA_PROC_HOME
76+
python3 run.py <archiveA> --out-dir /tmp/gem5_proc_A --tag A
77+
python3 run.py <archiveB> --out-dir /tmp/gem5_proc_B --tag B
78+
```
79+
80+
然后用短 Python 片段读取两个 `*-score.csv` / `*-weighted.csv` 做对比。优先关注：
81+
82+
- 总 score 变化
83+
- benchmark 级 `time` / `score` 变化
84+
- 用户关心的 stats 指标变化
85+
86+
## 下钻分析
87+
88+
### 1. 看 benchmark 级收益
89+
90+
- 对两个归档分别运行 `run.py`
91+
- 比较 `*-score.csv` 里的 `time` 和 `score`
92+
- 按 `score_delta_pct` 或 `time_delta_pct` 排序
93+
94+
### 2. 看 stats 指标变化
95+
96+
- 先看 `*-weighted.csv` 里的通用统计
97+
- 如果用户已经给出重点指标，直接围绕这些指标对比
98+
- 如果用户没有指定，优先从 `time`、`cpi`、前端、后端、内存、分支等大类里挑变化最明显的项
99+
- 归因时优先描述“哪些 stats 在变”，再解释这些变化更像支持哪类根因
100+
101+
### 3. 需要时再读归档里的其他文件
102+
103+
- `stats.txt`：看原始统计，位置在
104+
/nfs/home/share/gem5_ci/performance_data/spec06-0.3c/<archive_dir>/<spec_bmk>/m5out/stats.txt
105+
- `score.txt`：对照 summary
106+
- 其他 CSV 或日志：按用户问题决定是否下钻
107+
108+
## 常用命令
109+
110+
### 已知 run URL，直接拿 archive path
111+
112+
```bash
113+
python3 .codex/skills/ci-perf-analysis/scripts/ci_perf_info.py <run_url_or_id>
114+
```
115+
116+
### 已知 archive path，直接处理
117+
118+
```bash
119+
cd $GEM5_DATA_PROC_HOME
120+
python3 run.py <archive_dir> --out-dir /tmp/gem5_proc --tag run
121+
```
122+
123+
## 输出组织建议
124+
125+
回答这类问题时，优先按下面的顺序组织：
126+
127+
1. commit / run / workflow / 配置差异
128+
2. summary 里的总分变化
129+
3. benchmark 级主要收益和回退项
130+
4. 相关 stats 指标变化
131+
5. 对根因的判断
132+
133+
结论要尽量区分：
134+
135+
- “哪个 benchmark 涨了”
136+
- “哪些 stats 在变”
137+
- “这些 stats 更像支持哪类根因”
138+
139+
## 资源
140+
141+
### scripts/
142+
143+
- `ci_perf_info.py`
144+
- 输入 run URL 或 run id
145+
- 输出 archive path、spec_all 路径和 `score.txt` tail
146+
147+
### references/
148+
149+
当前不需要额外参考文件。后续如果这套流程扩展到更多 workflow 或更多统计口径，再新增参考文档。

‎.codex/skills/ci-perf-analysis/scripts/ci_perf_info.py‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
#!/usr/bin/env python3
2+
"""Locate archived CI performance data and print the score summary."""
3+
4+
import argparse
5+
import json
6+
import os
7+
import re
8+
import subprocess
9+
import sys
10+
from pathlib import Path
11+
12+
13+
REPO = "OpenXiangShan/GEM5"
14+
ARCHIVE_RE = re.compile(
15+
r"Archiving performance data to "
16+
r"(/nfs/home/share/gem5_ci/performance_data/\S+)"
17+
)
18+
19+
20+
def run_cmd(cmd: list[str]) -> str:
21+
result = subprocess.run(cmd, capture_output=True, text=True)
22+
if result.returncode != 0:
23+
raise SystemExit(result.stderr.strip() or f"command failed: {' '.join(cmd)}")
24+
return result.stdout
25+
26+
27+
def parse_run_id(text: str) -> str:
28+
match = re.search(r"/actions/runs/(\d+)", text)
29+
if match:
30+
return match.group(1)
31+
if text.isdigit():
32+
return text
33+
raise SystemExit(f"cannot parse run id from: {text}")
34+
35+
36+
def get_job_id(run_id: str) -> str:
37+
output = run_cmd(["gh", "api", f"repos/{REPO}/actions/runs/{run_id}/jobs"])
38+
data = json.loads(output)
39+
jobs = data.get("jobs", [])
40+
if not jobs:
41+
raise SystemExit(f"no jobs found for run {run_id}")
42+
return str(jobs[0]["id"])
43+
44+
45+
def get_archive_path(job_id: str) -> str:
46+
log_text = run_cmd(["gh", "api", f"repos/{REPO}/actions/jobs/{job_id}/logs"])
47+
match = ARCHIVE_RE.search(log_text)
48+
if not match:
49+
raise SystemExit(f"cannot find archive path in logs for job {job_id}")
50+
return match.group(1)
51+
52+
53+
def print_score(score_path: Path, tail_lines: int) -> None:
54+
if not score_path.is_file():
55+
print(f"score.txt not found: {score_path}", file=sys.stderr)
56+
return
57+
58+
lines = score_path.read_text().splitlines()
59+
print(f"score.txt: {score_path}")
60+
print(f"--- tail -n {tail_lines} ---")
61+
for line in lines[-tail_lines:]:
62+
print(line)
63+
64+
65+
def main() -> None:
66+
parser = argparse.ArgumentParser(
67+
description="Print CI archive path and score summary for a GEM5 perf run."
68+
)
69+
parser.add_argument("run", help="GitHub Actions run URL or run id")
70+
parser.add_argument(
71+
"--tail-lines",
72+
type=int,
73+
default=42,
74+
help="how many trailing lines of score.txt to print",
75+
)
76+
args = parser.parse_args()
77+
78+
run_id = parse_run_id(args.run)
79+
job_id = get_job_id(run_id)
80+
archive_path = Path(get_archive_path(job_id))
81+
82+
print(f"run_id: {run_id}")
83+
print(f"job_id: {job_id}")
84+
print(f"archive_path: {archive_path}")
85+
print(f"spec_all: {archive_path / 'spec_all'}")
86+
87+
score_path = archive_path / "score.txt"
88+
print_score(score_path, args.tail_lines)
89+
90+
91+
if __name__ == "__main__":
92+
main()

‎configs/common/xiangshan.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -847,6 +847,12 @@ def xiangshan_system_init():
847847
Options.addCommonOptions(parser, configure_xiangshan=True)
848848
Options.addXiangshanFSOptions(parser)
849849
Options.addXiangshanTraceOptions(parser)
850+
parser.add_argument(
851+
"--btb-tage-upper-bound",
852+
action="store_true",
853+
default=False,
854+
help="Use BTBTAGEUpperBound in kmhv3 instead of the default BTBTAGE",
855+
)
850856

851857
# Add the ruby specific and protocol specific args
852858
if '--ruby' in sys.argv:

‎configs/example/kmhv3.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,10 @@ def setKmhV3Params(args, system):
9696
cpu.branchPred.ftq_size = 64
9797
cpu.branchPred.fsq_size = 64
9898

99+
if args.btb_tage_upper_bound:
100+
cpu.branchPred.tage = BTBTAGEUpperBound(
101+
usePathHashHistory=True)
102+
99103
cpu.branchPred.mbtb.resolvedUpdate = True
100104
cpu.branchPred.tage.resolvedUpdate = True
101105
cpu.branchPred.ittage.resolvedUpdate = True

0 commit comments

Comments
 (0)