How CivicNZ rates value for money

What was withdrawn

vfm-1.0 averaged three signals — open-tender share, budget discipline, and workforce — used min-max scaling against other agencies, and had no coverage penalty. ACC could score 100 from a single $3.3 million contract. Sixty of ninety-five agencies sat on one signal. That method is withdrawn. Public routes do not return those grades.

Who is in the roster

The largest eligible agencies by 2024/25 appropriation, from the Treasury Summary Tables (plus ACC’s published $8.1 billion scheme cost, because ACC is levy-funded). Duplicate Votes are merged (MBIE; Justice and Courts). Councils and defunct agencies (including HUD and the Ministry for the Environment after the MCERT Order 2026) are excluded. The scoring year is 2024/25 — the most recent complete financial year with audited annual reports. Partial later-year figures are not used.

The roster is listed by appropriation size. Size is not a judgement of waste, and the order is not a value-for-money ranking.

These scores are not a league table

A levy-funded social insurer, a tax and transfers department, a benefits ministry, and a prison service do not produce interchangeable results. The same weights and gates are applied so the method is consistent. That does not mean ACC is “better” or “worse” than Corrections. Compare an agency with its own sourced record, not with another agency’s grade.

The 4Es, plus accountability

The weights follow the UK National Audit Office’s 3Es (economy, efficiency, effectiveness), plus equity as used in later public-audit practice, plus the NZ audit opinion as a separate accountability pillar. They are a published judgement, not a statistically fitted model.

• Effectiveness 40%. Outcome series and performance against the agency’s own published targets. Required. No Effectiveness data means no score. Only measures inside the auditor’s stated page range are described as audited.
• Efficiency 25%. Cost per output — treatment-cost growth, cost to collect $100 of tax, or a documented equivalent.
• Economy 15%. Budget variance and funding position take the full 15% when sourced. Procurement competitiveness is at most 5% of the whole model, so one GETS notice cannot dominate — a procurement-only agency contributes 5%, not 15%.
• Equity 10%. Named-group gaps that have both a comparator and a sourced direction this year. A stock figure without a comparator (for example the share of women in custody who identify as Māori) is context, not a score.
• Accountability 10%. Audit opinions and material OAG findings.

Gates and thresholds

A numeric score and letter grade are published only when Effectiveness data exists and at least 60% of the model weight is sourced. Otherwise the agency shows Insufficient evidence — not a zero, not a rank. Public routes, the homepage, and department scorecards do not attach a letter grade in that case.

Thresholds are absolute judgement bands written in config/value_for_money_model.yaml. They are not fitted to this year’s four sourced agencies. We do not stretch an agency against its peers. A 9.3% treatment-cost rise against a 4.1% target is judged on those numbers, not on where other agencies sit.

Grades: A 85+, B 70–84, C 55–69, D 40–54, E below 40. Each published score also carries an evidence grade A/B/C for how complete the citations are, and a sensitivity range after moving each pillar weight by ±10 percentage points and each band score by ±5. If the letter moves, the range is shown.

Target-gaming checks

For each performance measure across years we flag:

1. Lowballed. A year-specific target set easier than last year’s actual by more than the tolerance band (3% relative, 2 percentage points, or 1.5 days). Held standards (“no less than 55%”), prior-year-relative targets (“decrease”, “greater than previous year”), multi-year horizons (“21% by 2025”), survey holds inside a volatile series, and ceiling targets (99.8% availability) are not flagged. ACC’s long-term-claims-pool return-to-independence target of 5,300 sits below every year in the printed series (5,406 / 6,117 / 6,278 / 7,950) on p. 49.
2. Loosened while worse. The target got easier and the result got worse.
3. Dropped or redefined this year. A measure disappeared or changed definition in the scoring year. A next-year retirement, a brand-new measure, or a historical method note (for example “current from 2019/20”) is not a this-year redefine.
4. Met via definition change. Reported as met in the same year the definition moved.
5. Volume instead of outcome. Demand or activity counts (claims lodged, presentations, exits processed) rather than results for the public.

Only medium- and high-confidence flags are published. Each flag states why it survived the exemptions. Flagged targets count at half weight in Effectiveness. The agency page shows both rates: the raw official “X of Y” figure, and the adjusted rate after those flags. Scoring uses the adjusted rate whenever individual measures are sourced. Volume-only flags do not halve the weight. A further agency-level penalty applies only when demand or volume counts make up a large share of the suite — it does not restack the same lowball or redefine already counted at half weight.

Trend, attribution, and double counting

“Improving” means at least two more sourced outcome series moved in the desired direction than against it. A 2–1 split is labelled mixed. Trend is a count of series. It is not a finding that the agency caused the movement. Economy, injury rates, court practice, and levy decisions sit outside an agency’s year.

Effectiveness uses each non-efficiency measure once in the hit rate and, if it is classed as an outcome, again in the outcome-trend share — two views of the same series. Equity uses named-group splits of some of those series (for example Māori vs all-client exits). Official X-of-Y summaries that mix efficiency measures are shown as the raw rate only.

Caveats

• We do not invent figures. Blanks reduce coverage.
• Vote totals mix departmental running costs with benefits, debt, and capital. Rank of size is not a judgement of waste.
• Meeting a soft target is not the same as improving an outcome. That is why gaming flags exist, and why easily rebutted flags are not published.
• This is not an allegation of wrongdoing. It is a reading of the public record.
• No paid data sources and no new hosted services. CSVs are cached on disk.