OhmBench · 21 cases · 9 real HarmonyOS apps
Every case, and what each tool found.
OhmBench hands a tool a signed HAP, the unfixed source tree and the symptom a tester reported — no ground truth, no fix patch, no git history. The tool must name the code that causes the slowness. This page shows, for each of the 21 cases, the defect the benchmark planted, the suspects ohos-perf suspects returned and where it ranked them, and what HapRay produced from the same input.
The same cases, run on a device
The table above scores the source path. OhmBench also ships each case as a signed HAP and a workload HapRay's own PerfTestCase would drive, so the same cases can be built, signed, installed and measured on the emulator. bench/ohmbench/dynamic/ does that, and this table is its record — generated by bench/ohmbench/site-dynamic.mjs from dynamic.json, with no number typed in by hand.
| Case | Its own window | Captured | Metrics | Evidence | Verdict | Stopped at |
|---|
A case that stops in setup leaves no scene. A case that stops in verify or overruns its window keeps one, and it is analyzed anyway — the phase is named so a reader knows what the capture holds. The oracle metric every included case is scored on is hiperf_instructions, and the emulator cannot run hiperf record, which is why this table carries no precision/recall number.
How this was measured
The number on this page is produced by the benchmark's own scorer, not by a scorer written here: bench/ohmbench/score.mjs transcribes OhmBench's docs/evaluation.md matcher. A suspect counts as correct when its file matches the ground-truth location on the last two path segments and its symbol matches, or its line range overlaps the location's scope. Precision is correct over reported in the top five; recall is primary root causes matched over all primaries.
HapRay's column is measured too, not assumed. Its root-cause action was run against all 21 blind trees. It accepts a source tree as --report-dir — the flag's help says “report directory” but nothing validates it — and on every one of the 21 it returns 0 with the same body:
It looks for trace_*.json, perf.db and so_file_load.json, and a source tree holds none of them. No HapRay report names a single .ets file. Its static action takes a HAP and reports SO and resource inventory, so it cannot name a source line either.
Reproduce it:
npm run build -w packages/core && npm run build -w packages/cli
node bench/ohmbench/run.mjs --bench <OhmBench> --cases <trees> --out bench/ohmbench/results
node bench/ohmbench/score.mjs --bench <OhmBench> --results bench/ohmbench/results
# exits 0 only when precision >= 0.80
# HapRay baseline, from its checkout with its venv:
PYTHONPATH=. .venv/Scripts/python.exe bench/ohmbench/hapray/probe.py <tree>... <out.json>
Every miss and its reason is written down in bench/ohmbench/RESULTS.md, including the six ranking variants that were measured and rejected. Where a case is missed, the card above says why.