Troubleshooting & FAQ
“not inside a git repository”
Every command except ablate cost needs to run inside a git
repo — trials run in isolated git worktrees, which need one to check out
from. ablate cost is pure arithmetic on a file and is the one exception.
My config is AGENTS.md, not CLAUDE.md
ablate init, ablate cost, and ablate try auto-detect CLAUDE.md,
AGENTS.md, or .cursorrules (first match wins) when you don’t pass a
path. But each task’s config_path field defaults to "CLAUDE.md" —
if your config lives elsewhere, set config_path explicitly in every task
file, or the agent won’t see the variant being tested.
ablate compare CLAUDE.md some-typo.md silently compared against nothing
CONFIG_B (and CONFIG_A) are plain file paths, with no git-ref syntax. A
path that doesn’t exist on disk is treated the same as /dev/null — “no
config” — rather than raising an error. Double-check the path if a result
looks surprising.
Every verdict comes back INCONCLUSIVE
Below 8 tasks, no confidence interval can be produced at all — this is a
fixed floor, not a setting you can lower. ablate init scaffolds 3 tasks
and ablate try runs 3; neither can produce a verdict other than
INCONCLUSIVE, by design. Add tasks (they buy more statistical power than
trials do) and re-run with ablate compare.
INCONCLUSIVE vs. EQUIVALENT — these look similar but mean opposite things
EQUIVALENT means the interval fits inside the margin — “we looked hard and there is nothing there.” It’s a positive finding, and it’s what licenses deleting a section. INCONCLUSIVE means the interval is too wide to say either way — “we didn’t look hard enough.” It licenses nothing. If you see INCONCLUSIVE, the report tells you the tasks/trials/runs needed to resolve it.
ablate apply says a section id doesn’t exist
Section ids come from ablate sections, and they depend
on --granularity. An id like s07 means something different at
--granularity section than at --granularity bullet. Run apply with
the same --granularity you ran sections with.
uvx ablate / uvx --from ablate-cli ablate doesn’t work
Two separate things can cause this:
- The package isn’t on PyPI yet in this milestone — check the release page for the current status.
- The distribution name is
ablate-cli, notablate—ablateon PyPI is an unrelated deep-learning reporting tool.uvx ablatealone will install the wrong package if it resolves at all; useuvx --from ablate-cli ablate.
Does the Codex adapter report cost?
codex exec --json reports token usage but does not report a dollar
cost. ablate never derives one from token counts — cost_usd stays
None for Codex trials, and --max-spend reports how many trials it
couldn’t account for rather than silently treating them as free.
GitHub Action
- uses: ablate-dev/ablate@v1
with:
config: CLAUDE.md
baseline: origin/${{ github.base_ref }}
max-spend: 15 # required — no default, on purpose
fail-on: never # regression | inconclusive | never
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}baseline accepts a git ref (origin/main) or a literal path; a ref is
materialized with git show <ref>:<config> > tmpfile before being handed
to ablate compare — the CLI itself has no ref syntax (see
ablate compare).
max-spend is required, with no default — a runaway loop in CI bills a
card nobody is watching. fail-on: never is comment-only and the
recommended starting point; most teams’ first finding is that their suite
is underpowered for pass rate, worth knowing before it can block a merge.