Methodology

Overview

This system compares actual federal spending patterns to a need-weighted counterfactual: if we had redistributed the same total budget proportional to predicted need, how would the allocation differ?

For each test year, we train a need-prediction model on prior years, generate need-weighted reallocations under the same budget envelope and fairness constraints, and measure how well each allocation aligns with predicted need.

Key Limitations

  • No causal claims: This system predicts need levels, not treatment effects. It does not model the marginal impact of an additional dollar spent in a given county. Need-proportional allocation is only optimal if marginal returns are identical everywhere.
  • Proxy outcomes: Some domains use socioeconomic proxies instead of measured outcomes (e.g., poverty + age as a stand-in for preventable mortality). Results are only as meaningful as the outcome variable. Each domain's data quality is marked.
  • County-level aggregation: Federal programs allocate to different units (LEAs for Title I, PHAs for Section 8, states for HSIP). County aggregation is an approximation of the actual decision unit.
  • Need persistence: Need levels are autocorrelated across years. High need-alignment scores may partly reflect this persistence, not the allocator's responsiveness.

Rolling Backtest Design

  1. Train: Fit gradient boosting model on years 1..t-1 to predict the domain's outcome variable. Uses GroupKFold CV by geography to prevent county fixed-effect leakage.
  2. Predict: Score all decision units (counties) in year t with predicted need.
  3. Budget: Use actual total spending in year t as the budget constraint.
  4. Reallocate: Allocate budget proportional to predicted need, subject to fairness constraints (iterative projection to guarantee feasibility).
  5. Compare: Measure need-alignment (rank correlation between need and spending) under actual vs need-weighted allocation.

Need-Scoring Model

Each domain defines features (poverty rate, population, domain-specific indicators) and an outcome variable. A GradientBoostingRegressor is trained with GroupKFold cross-validation (grouped by county) to prevent leaking county fixed effects. Feature importances are reported for explainability but do not imply causal relationships.

Missing features are imputed with training-set medians (not zeros) to avoid nonsensical predictions.

Fairness Constraints

Each domain defines fairness constraints enforced via iterative projection to prevent need-weighted reallocation from concentrating funds inequitably:

  • Min share: A group must receive at least X% of budget.
  • Max share: A group cannot receive more than X% of budget.
  • Floor: Every unit gets at least $X.
  • Proportional: Group shares must be within tolerance of population shares.

Data Sources

Pluggable Domains

The framework supports any domain that defines: an outcome variable, decision unit, budget constraint, fairness constraints, and a panel-building function. Current domains:

  • Synthetic: Known DGP for framework validation (direct outcome)
  • Traffic Safety: NHTSA FARS + HSIP spending (fabricated outcome without FARS data)
  • Education: Title I + IDEA grants (proxy: 1 - college_rate from ACS)
  • Housing: HUD vouchers + CDBG (proxy: rent-to-income ratio from ACS)
  • Health: HRSA + CDC grants (fabricated: poverty + age composite)