# UCRI CA_2026.10: internal robustness checks **Run by Canopulse. Not an independent review. Tests weight sensitivity only.** Class names: the stored class value "Extreme" is displayed as "Very high screening priority" (top fifth statewide) on the Canopulse website and in its reports. This report calls it the top class; the data files keep the stored value. Generated by `robustness/ucri_robustness.py`, published in this package (standard library Python, fixed seed 20261008). Machine-readable results: `results.json`. Per-tract stability: `tract_stability.csv`. Every number below is filled in from `results.json` by the script. ## What was tested UCRI combines five statewide percentile components with fixed weights (0.25 canopy deficit, 0.25 heat, 0.15 impervious, 0.20 social vulnerability, 0.15 environmental burden). The weights are a judgment, not an estimate from data. These checks ask one question: **if the weights were reasonably different, would the same tracts come out on top?** All checks use the 9,041 fully scored tracts and the released component values. Every alternative score is computed and rounded exactly like the released score, then compared with the published ranking: * **Spearman correlation** of the two rankings (1 means identical order). * **Top 10% and top 20% retention**: of the tracts in the published top 10% (911 tracts) or top 20% (the 1,814 tracts in the top class), the share that stays in the alternative's own top 10% or top 20%. * **Class change**: the share of all tracts whose priority class changes when the alternative score is classified with the published thresholds (30, 44, 56.9, 70.3). * **Median rank change**: the typical move in statewide rank, out of 9,041. ## Results ### Equal weights (0.2 each) | Scenario | Spearman | Top 10% kept | Top 20% kept | Class change | Median rank change | |---|---|---|---|---|---| | Equal weights | 0.9911 | 91.3% | 91.3% | 12.5% | 180 | ### Each weight 20% higher or lower (others rescaled to sum to 1) Across all ten scenarios: Spearman at least 0.9988, top 20% retention at least 97.2%, top 10% retention at least 96.7%, class change at most 4.9%, median rank change at most 70 places. | Scenario | Spearman | Top 10% kept | Top 20% kept | Class change | Median rank change | |---|---|---|---|---|---| | CDS +20% | 0.9992 | 97.6% | 97.9% | 4.2% | 60 | | CDS -20% | 0.9990 | 97.3% | 97.2% | 4.8% | 67 | | HES +20% | 0.9991 | 97.5% | 97.6% | 4.6% | 62 | | HES -20% | 0.9988 | 97.5% | 97.4% | 4.9% | 70 | | ISS +20% | 0.9994 | 97.0% | 97.7% | 3.5% | 50 | | ISS -20% | 0.9994 | 96.7% | 98.0% | 3.5% | 54 | | SVS +20% | 0.9994 | 98.1% | 98.8% | 3.7% | 54 | | SVS -20% | 0.9993 | 98.7% | 98.3% | 4.2% | 59 | | EBS +20% | 0.9995 | 98.1% | 97.9% | 3.2% | 48 | | EBS -20% | 0.9995 | 97.3% | 98.0% | 3.4% | 51 | ### Leave one component out (remaining weights rescaled) | Dropped | Spearman | Top 10% kept | Top 20% kept | Class change | Median rank change | |---|---|---|---|---|---| | CDS | 0.9618 | 82.1% | 84.9% | 29.1% | 449 | | HES | 0.9560 | 80.5% | 83.1% | 29.6% | 440 | | ISS | 0.9831 | 79.1% | 86.9% | 19.6% | 312 | | SVS | 0.9749 | 86.5% | 87.8% | 23.5% | 354 | | EBS | 0.9845 | 84.4% | 87.3% | 19.6% | 296 | Every component matters: removing any one changes 19.6% to 29.6% of priority classes. Dropping one of the two 0.25 components (canopy deficit or heat) moves the most tracts, as expected from their weights. ### 1,000 random weightings Weights were drawn from a Dirichlet distribution centered on the published weights, with concentration 100 (alpha = 100 x published weight). The typical draw moves each weight by 13.4% to 19.7% of its value (mean absolute relative change); the 0.25 weights fall between 0.18 and 0.32 in 90% of draws. | Metric | Worst draw | 5th percentile, worst side | Median | |---|---|---|---| | Spearman | 0.978 | 0.989 | 0.997 | | Top 10% kept | 82.8% | 89.0% | 94.5% | | Top 20% kept | 87.2% | 90.9% | 95.3% | | Class change | 20.8% | 15.2% | 8.5% | | Median rank change | 298 | 217 | 122 | For class change and rank change the worst side is the high end (maximum and 95th percentile). Tract-level stability: * **1,432 of the 1,814 top-class tracts (78.9%)** are in the top 20% in at least 95% of draws; 1,140 are in it in every draw. No tract outside the published top class reaches 95%. * 1,807 of the 1,814 top-class tracts are in the top 20% in at least half of the draws. 7 fall out more often than not, and 2 tracts outside the top class get in more often than not. * 791 tracts sit on the boundary (top 20% in 5% to 95% of draws). For these the top-class label depends on the exact weights and is best read as near the top-class threshold (UCRI 70.3). ### Component correlation (Spearman, 9,041 tracts) | | CDS | HES | ISS | SVS | EBS | |---|---|---|---|---|---| | CDS | 1 | 0.56 | 0.33 | 0.48 | 0.37 | | HES | 0.56 | 1 | 0.26 | 0.45 | 0.34 | | ISS | 0.33 | 0.26 | 1 | 0.33 | 0.46 | | SVS | 0.48 | 0.45 | 0.33 | 1 | 0.47 | | EBS | 0.37 | 0.34 | 0.46 | 0.47 | 1 | No pair exceeds 0.7, so no two components are close to redundant. The strongest link is CDS with HES (0.56). Correlation of each component with UCRI: CDS 0.80, HES 0.77, SVS 0.76, EBS 0.66, ISS 0.58. ### By region Retention is measured statewide (a tract must stay in the statewide top 20%), then broken down by where the tract is. The random-weighting column is the median over draws. | Region | Fully scored | Top class (published) | Top 20% kept, equal weights | Top 20% kept, random weights | Top 20% count, equal weights | |---|---|---|---|---|---| | Southern California coast | 4,104 | 1,222 | 98.3% | 96.6% | 1,305 | | Inland Empire and desert | 1,024 | 296 | 69.9% | 95.3% | 215 | | San Joaquin Valley | 970 | 270 | 83.3% | 95.2% | 236 | | Bay Area | 1,756 | 22 | 95.5% | 95.5% | 44 | | Sacramento region | 601 | 3 | 100% | 100% | 12 | | Northern, Sierra and rural | 338 | 1 | 0% | 100% | 0 | | Central Coast | 248 | 0 | n/a | n/a | 1 | County lists: * Bay Area: Alameda, Contra Costa, Marin, Napa, San Francisco, San Mateo, Santa Clara, Solano, Sonoma. * Southern California coast: Los Angeles, Orange, San Diego, Ventura, Santa Barbara. * Inland Empire and desert: Riverside, San Bernardino, Imperial, Inyo. * San Joaquin Valley: Fresno, Kern, Kings, Madera, Merced, San Joaquin, Stanislaus, Tulare. * Sacramento region: Sacramento, Placer, El Dorado, Yolo, Sutter, Yuba. * Central Coast: Monterey, San Benito, San Luis Obispo, Santa Cruz. * Northern, Sierra and rural: Alpine, Amador, Butte, Calaveras, Colusa, Del Norte, Glenn, Humboldt, Lake, Lassen, Mariposa, Mendocino, Modoc, Mono, Nevada, Plumas, Shasta, Sierra, Siskiyou, Tehama, Trinity, Tuolumne. Inland Empire and desert top-class tracts score high on heat (mean HES 93) and canopy deficit (84) but lower on impervious surface (44). Equal weights lower heat and canopy from 0.25 to 0.20 and raise impervious from 0.15 to 0.20, so this region is the most exposed to that particular change: 69.9% of its top-class tracts stay in the top 20%, against a median of 95.3% under moderate random weightings. These rows rest on very few top-class tracts, so their percentages are not meaningful on their own: Bay Area (22 in the top class); Sacramento region (3 in the top class); Northern, Sierra and rural (1 in the top class); Central Coast (0 in the top class). ## What these checks show * The published ranking is not fragile to moderate weight changes. Changing any one weight by 20% keeps at least 97.2% of the top-class tracts and moves the typical tract by at most 70 places out of 9,041. * 78.9% of top-class tracts are in the top 20% under at least 95% of moderate random weightings. The 791 boundary tracts should be treated as near the threshold. * The five components carry distinct information, and removing any one changes a large share of classes. * Of the scenarios tested, the weighting choice matters most for heat-dominated inland desert tracts compared with dense coastal tracts. ## What these checks do not show * **They do not show that UCRI is correct or that it predicts anything.** They test sensitivity to weights, not whether high-UCRI tracts have worse heat illness, mortality or other outcomes. No outcome data were used. * They do not test the inputs. Errors in the canopy, temperature, impervious, SVI or CalEnviroScreen layers pass straight through every scenario. * They do not test other design choices: percentile scaling instead of raw values, statewide instead of regional percentiles, the tract as the unit, or the quintile class cut points. * Comparing UCRI with Landsat temperature, CDC SVI or CalEnviroScreen is not independent evidence, because those datasets are inputs to UCRI. Agreement with them is built in. * These checks were designed, run and interpreted by Canopulse. They are not an independent review, and no independent review of UCRI has been completed. ## Reproduce From the package folder: `python3 robustness/ucri_robustness.py --out rerun`. It reads `ucri_ca_2026_10_tracts.csv`, needs only the Python standard library (3.8 or later), takes about a minute, and writes `results.json`, `tract_stability.csv` and `REPORT.md` to a new folder `rerun/` for comparison with the published files in `robustness/` (seed 20261008). Without `--out` it overwrites the files in `robustness/`. Random draws are bit-identical on the same Python version; this run used Python 3.14.7.