Post-fix Wave 1: NROY Deep Dive (Design B)

Run: 20260703_0711_postfix_wave1 | Completed: 2026-07-04 03:47 UTC | 10,000 designs | External Vs | Cutoff: χ²(16)/16 = 34.27

Contents: Summary • STPM Fitting • STS Fitting • Jackson Survival • Prevalence • Parameter Space • Q Decomposition • Parameter Values
Plausibility criterion: A design is marked PLAUSIBLE only if it passes all three constraints: (1) Q_STPM < 34.3 (NROY), (2) Jackson survival envelope, (3) falling prevalence 2011→2016. Of the 6 NROY designs, only 2 are plausible (design_4669 and design_0976). The other 4 are shown faded/struck as IMPLAUSIBLE — prevalence rising.
6 → 2
NROY → Plausible
20.4
Best Q_STPM (d9924, implaus.)
31.8
Best Plausible Q (d4669)
46.2
Best Plausible Q_STS (d4669)

Summary

DesignQ_STPMQ_STSJacksonPrev ΔStatus
design_992420.4132.4FAIL (1/3)+0.67 ppIMPLAUSIBLE
design_684026.384.8PASS+0.26 ppIMPLAUSIBLE
design_973530.8153.0PASS+0.10 ppIMPLAUSIBLE
design_466931.846.2PASS-0.08 ppPLAUSIBLE
design_097632.096.4PASS-0.43 ppPLAUSIBLE
design_418232.25228.4FAIL (3/3)+1.24 ppIMPLAUSIBLE

STPM Target Fitting (Natural Units)

Blue ticks = STPM target quit rates; coloured dots = model quit rates (with total SE whiskers). Green-titled panels are plausible; red-titled are implausible (faded background).

STPM target fitting

STS Target Fitting (Natural Units)

Quit-attempt rates. Same visual convention: plausible panels have white backgrounds, implausible panels are faded pink.

STS target fitting

Jackson Survival Constraint

Strict band: 0.5×Placebo to 1.0×Varenicline. Plausible designs have bold black borders.

Jackson survival

Prevalence Trajectories

Solid lines = plausible (prevalence falling); dashed = implausible (prevalence rising).

Prevalence

Parameter Space: All 10,000 Designs with NROY Marked

Grey histograms = all 10k samples. Grey curve = prior N(μ,σ). Solid coloured lines = plausible NROY; dashed faded lines = implausible NROY.

Prior distributions

Q Decomposition: What is "Contribution to Q"?

Explanation: Q_STPM is the multivariate Mahalanobis distance:

Q = rT Σ−1 r

where r = (model − target) is the 16-vector of residuals and Σ = Vobs + Vstoch + Vdiscrepancy.

"Contribution to Q" = output i's fractional share of the total Q:
share_i = (Σ⁻¹r)_i × r_i / Q

These 16 shares sum to exactly 100%. A large share means that subgroup is the main driver of overall misfit. Values in the table are in natural units (quit rates as %) — red-highlighted rows contribute >10% of Q.

Plausible Designs

design_4669 — Q_STPM = 31.8 | Prev Δ = -0.08 pp | PLAUSIBLE

OutputModelTargetResidualTotal SEStd ResidContribution to Q
2014-2016 IMD q37.57%9.77%-2.21 pp1.01 pp-2.1940.4%
2014-2016 IMD q46.67%8.36%-1.68 pp0.94 pp-1.7932.9%
2014-2016 IMD q29.15%11.21%-2.06 pp1.15 pp-1.7831.1%
2011-2013 IMD q19.16%11.54%-2.38 pp1.23 pp-1.9329.8%
2011-2013 IMD q37.54%8.99%-1.46 pp0.80 pp-1.8128.1%
2014-2016 IMD q55.48%6.19%-0.71 pp0.90 pp-0.7915.1%
2014-2016 IMD q110.74%12.53%-1.79 pp1.41 pp-1.2714.9%
2011-2013 IMD q210.07%10.63%-0.57 pp1.10 pp-0.514.5%
2011-2016 M 65-746.57%6.43%+0.14 pp1.41 pp+0.100.9%
2011-2016 F 45-647.04%7.06%-0.02 pp0.66 pp-0.03-0.6%
2011-2013 IMD q48.18%7.93%+0.25 pp0.86 pp+0.29-2.6%
2011-2016 F 65-745.91%6.87%-0.96 pp1.30 pp-0.74-3.8%
2011-2016 M 45-647.16%7.39%-0.23 pp0.63 pp-0.36-8.3%
2011-2013 IMD q57.07%5.80%+1.27 pp0.70 pp+1.83-11.0%
2011-2016 F 25-448.80%9.77%-0.97 pp0.69 pp-1.41-22.1%
2011-2016 M 25-448.09%9.86%-1.77 pp0.59 pp-2.98-49.1%

design_0976 — Q_STPM = 32.0 | Prev Δ = -0.43 pp | PLAUSIBLE

OutputModelTargetResidualTotal SEStd ResidContribution to Q
2011-2016 M 25-4410.95%9.86%+1.09 pp0.59 pp+1.83117.5%
2011-2016 F 25-4410.83%9.77%+1.06 pp0.69 pp+1.5378.6%
2014-2016 IMD q110.16%12.53%-2.37 pp1.41 pp-1.6950.6%
2011-2013 IMD q29.19%10.63%-1.44 pp1.10 pp-1.3139.8%
2014-2016 IMD q39.45%9.77%-0.33 pp1.01 pp-0.3211.7%
2014-2016 IMD q210.80%11.21%-0.41 pp1.15 pp-0.369.7%
2011-2013 IMD q111.00%11.54%-0.54 pp1.23 pp-0.449.6%
2014-2016 IMD q48.40%8.36%+0.04 pp0.94 pp+0.05-1.7%
2011-2013 IMD q39.10%8.99%+0.11 pp0.80 pp+0.13-3.0%
2011-2016 F 65-746.10%6.87%-0.77 pp1.30 pp-0.59-11.1%
2011-2016 M 65-744.02%6.43%-2.40 pp1.41 pp-1.70-21.1%
2011-2016 F 45-646.61%7.06%-0.45 pp0.66 pp-0.68-24.1%
2011-2016 M 45-646.81%7.39%-0.57 pp0.63 pp-0.90-33.4%
2014-2016 IMD q57.02%6.19%+0.83 pp0.90 pp+0.93-34.0%
2011-2013 IMD q56.84%5.80%+1.04 pp0.70 pp+1.50-43.4%
2011-2013 IMD q49.40%7.93%+1.47 pp0.86 pp+1.71-45.7%

Implausible Designs (prevalence rising)

design_9924 — Q_STPM = 20.4 | Prev Δ = +0.67 pp | IMPLAUSIBLE

OutputModelTargetResidualTotal SEStd ResidContribution to Q
2014-2016 IMD q37.36%9.77%-2.41 pp1.01 pp-2.3985.1%
2014-2016 IMD q110.39%12.53%-2.14 pp1.41 pp-1.5234.4%
2011-2013 IMD q37.90%8.99%-1.09 pp0.80 pp-1.3630.0%
2014-2016 IMD q47.29%8.36%-1.07 pp0.94 pp-1.1429.6%
2011-2016 F 65-748.50%6.87%+1.63 pp1.30 pp+1.2527.7%
2014-2016 IMD q56.22%6.19%+0.03 pp0.90 pp+0.03-0.8%
2011-2013 IMD q48.05%7.93%+0.12 pp0.86 pp+0.14-2.4%
2011-2013 IMD q55.89%5.80%+0.09 pp0.70 pp+0.13-2.8%
2011-2016 M 65-745.41%6.43%-1.01 pp1.41 pp-0.72-4.5%
2014-2016 IMD q211.76%11.21%+0.55 pp1.15 pp+0.47-5.6%
2011-2013 IMD q112.34%11.54%+0.80 pp1.23 pp+0.65-5.7%
2011-2016 F 45-646.86%7.06%-0.20 pp0.66 pp-0.30-7.8%
2011-2016 M 25-449.67%9.86%-0.19 pp0.59 pp-0.32-12.7%
2011-2013 IMD q211.97%10.63%+1.33 pp1.10 pp+1.21-14.2%
2011-2016 F 25-449.40%9.77%-0.38 pp0.69 pp-0.55-17.1%
2011-2016 M 45-646.43%7.39%-0.95 pp0.63 pp-1.51-33.3%

design_6840 — Q_STPM = 26.3 | Prev Δ = +0.26 pp | IMPLAUSIBLE

OutputModelTargetResidualTotal SEStd ResidContribution to Q
2014-2016 IMD q46.03%8.36%-2.32 pp0.94 pp-2.48132.1%
2014-2016 IMD q37.71%9.77%-2.07 pp1.01 pp-2.05100.0%
2014-2016 IMD q110.49%12.53%-2.04 pp1.41 pp-1.4549.1%
2011-2013 IMD q38.16%8.99%-0.84 pp0.80 pp-1.0438.7%
2014-2016 IMD q55.51%6.19%-0.68 pp0.90 pp-0.7538.6%
2011-2013 IMD q47.39%7.93%-0.54 pp0.86 pp-0.6327.5%
2014-2016 IMD q210.33%11.21%-0.88 pp1.15 pp-0.7627.0%
2011-2013 IMD q55.46%5.80%-0.35 pp0.70 pp-0.5022.9%
2011-2013 IMD q210.53%10.63%-0.11 pp1.10 pp-0.103.3%
2011-2013 IMD q111.63%11.54%+0.09 pp1.23 pp+0.07-2.1%
2011-2016 M 25-449.85%9.86%-0.02 pp0.59 pp-0.03-2.1%
2011-2016 M 65-744.35%6.43%-2.08 pp1.41 pp-1.47-24.6%
2011-2016 F 65-745.31%6.87%-1.56 pp1.30 pp-1.19-28.0%
2011-2016 F 25-449.27%9.77%-0.50 pp0.69 pp-0.73-45.5%
2011-2016 F 45-645.66%7.06%-1.41 pp0.66 pp-2.13-103.6%
2011-2016 M 45-645.74%7.39%-1.65 pp0.63 pp-2.61-133.4%

design_9735 — Q_STPM = 30.8 | Prev Δ = +0.10 pp | IMPLAUSIBLE

OutputModelTargetResidualTotal SEStd ResidContribution to Q
2011-2016 F 45-648.32%7.06%+1.26 pp0.66 pp+1.91103.7%
2011-2016 M 45-648.34%7.39%+0.95 pp0.63 pp+1.5184.9%
2014-2016 IMD q110.48%12.53%-2.06 pp1.41 pp-1.4651.5%
2011-2013 IMD q47.17%7.93%-0.76 pp0.86 pp-0.8837.9%
2014-2016 IMD q39.16%9.77%-0.61 pp1.01 pp-0.6124.9%
2011-2016 M 65-747.90%6.43%+1.48 pp1.41 pp+1.0523.2%
2011-2016 F 65-747.96%6.87%+1.09 pp1.30 pp+0.8417.6%
2011-2013 IMD q111.13%11.54%-0.41 pp1.23 pp-0.338.2%
2011-2013 IMD q38.91%8.99%-0.09 pp0.80 pp-0.113.6%
2011-2016 F 25-449.70%9.77%-0.07 pp0.69 pp-0.10-5.3%
2014-2016 IMD q48.66%8.36%+0.30 pp0.94 pp+0.32-13.1%
2011-2013 IMD q211.14%10.63%+0.51 pp1.10 pp+0.46-14.5%
2014-2016 IMD q211.80%11.21%+0.58 pp1.15 pp+0.51-16.1%
2011-2016 M 25-449.54%9.86%-0.32 pp0.59 pp-0.55-35.0%
2011-2013 IMD q57.25%5.80%+1.44 pp0.70 pp+2.08-75.2%
2014-2016 IMD q58.38%6.19%+2.20 pp0.90 pp+2.45-96.3%

design_4182 — Q_STPM = 32.2 | Prev Δ = +1.24 pp | IMPLAUSIBLE

OutputModelTargetResidualTotal SEStd ResidContribution to Q
2014-2016 IMD q37.99%9.77%-1.78 pp1.01 pp-1.7769.5%
2011-2016 M 45-648.03%7.39%+0.65 pp0.63 pp+1.0253.6%
2011-2013 IMD q37.78%8.99%-1.22 pp0.80 pp-1.5150.7%
2011-2016 F 45-647.67%7.06%+0.61 pp0.66 pp+0.9244.5%
2014-2016 IMD q110.91%12.53%-1.62 pp1.41 pp-1.1532.6%
2011-2013 IMD q29.46%10.63%-1.17 pp1.10 pp-1.0731.0%
2014-2016 IMD q210.46%11.21%-0.75 pp1.15 pp-0.6520.1%
2011-2013 IMD q110.79%11.54%-0.75 pp1.23 pp-0.6114.9%
2014-2016 IMD q56.37%6.19%+0.18 pp0.90 pp+0.20-8.2%
2014-2016 IMD q48.62%8.36%+0.27 pp0.94 pp+0.29-10.3%
2011-2016 M 65-745.20%6.43%-1.22 pp1.41 pp-0.87-13.9%
2011-2016 F 65-743.89%6.87%-2.98 pp1.30 pp-2.29-25.4%
2011-2013 IMD q48.68%7.93%+0.75 pp0.86 pp+0.87-29.0%
2011-2016 F 25-449.30%9.77%-0.47 pp0.69 pp-0.68-30.8%
2011-2016 M 25-449.40%9.86%-0.46 pp0.59 pp-0.78-44.5%
2011-2013 IMD q57.06%5.80%+1.26 pp0.70 pp+1.81-54.8%

Parameter Values

ParameterPrior μPrior σ d4669
PLAUS		 d0976
PLAUS		 d9924
implaus		 d6840
implaus		 d9735
implaus		 d4182
implaus
attempt.oSocialHousing.beta0.09620.2065 -0.2704 -0.0245 -0.2824 0.0606 0.2266 0.6139
attempt.oReceiptGPAdvice.beta0.07910.1045 0.1720 -0.0573 0.0102 0.0480 0.1046 0.0447
attempt.oNumberOfSmokersInSocialNetwork.beta0.01260.0770 0.0269 0.0040 0.0555 -0.0161 -0.0801 0.0111
attempt.cSelfEfficacy.beta0.28230.1664 0.2533 0.4775 0.3102 0.1838 0.1745 0.1450
attempt.mIntentionToQuit.beta0.20190.0287 0.2751 0.1721 0.1742 0.2106 0.2295 0.1853
attempt.mExpenditure.beta0.00140.0021 -0.0031 -0.0015 0.0028 -0.0029 0.0028 0.0016
attempt.mEnjoymentOfSmoking.beta-0.20680.1827 -0.2854 -0.3175 -0.3328 -0.1698 0.0184 -0.2795
attempt.mUseOfNRT.beta0.31300.1248 0.2950 0.4319 0.4189 0.3556 0.3290 0.2965
attempt.mNumberOfRecentQuitAttempts.beta0.32810.0478 0.3175 0.3763 0.3543 0.2250 0.3025 0.2921
attempt.mAgeGroup30To44.beta-0.17150.1403 -0.0322 0.0365 -0.1119 -0.3754 -0.3363 0.0463
attempt.mAgeGroup45To64.beta-0.21480.1402 -0.1440 -0.3443 -0.2470 -0.2582 -0.2646 -0.1157
attempt.mAgeGroup65Plus.beta-0.10500.1733 -0.1535 -0.0123 -0.3604 -0.3208 -0.1764 -0.0856
maintenance.cAlcoholConsumptionGroupHigherRisk.beta0.28500.3335 -0.0119 0.0208 0.3641 -0.1987 0.6367 0.2631
maintenance.cAlcoholConsumptionGroupIncreasingRisk.beta0.11440.2665 0.4516 0.0362 0.3110 -0.2103 0.1694 0.2017
maintenance.cAlcoholConsumptionGroupLowerRisk.beta0.08630.2333 -0.1531 -0.1756 -0.0117 0.0239 0.0481 -0.2460
maintenance.cECigUse.beta0.44510.6396 0.5063 -0.6408 0.0811 0.7460 0.4422 0.6552
maintenance.cPrescriptionNRT.beta0.27590.1025 0.3954 0.3859 0.4988 0.2187 0.2132 0.2761
maintenance.cCigConsumptionPreQuit.beta-0.02960.0142 -0.0377 -0.0289 -0.0106 -0.0374 -0.0257 -0.0181
maintenance.cVareniclineUse.beta0.71590.2062 0.4574 1.0663 0.8455 0.6768 0.7335 0.5693
maintenance.cAgeGroup30To44.beta0.21720.2795 0.0862 0.1954 0.0432 0.5704 0.3609 0.3390
maintenance.cAgeGroup45To64.beta0.09750.2760 0.4839 0.5494 0.0572 0.1514 0.1958 0.3336
maintenance.cAgeGroup65Plus.beta0.23200.3455 0.5594 -0.1388 0.4981 0.3959 0.4586 -0.0714
maintenance.cMentalHealthCondition.beta-0.07430.2881 -0.0273 -0.0813 -0.5541 -0.3501 -0.6526 -0.3097
maintenance.cCigAddictStrength.beta-0.15480.0950 -0.2703 -0.1262 -0.1973 -0.3190 0.0529 -0.2811
maintenance.cUseOfBehaviouralSupport.beta0.20380.1911 -0.2857 0.2427 0.1539 0.3975 0.1511 0.2772
maintenance.cEducationalLevelBelowDegree.beta-0.17410.3105 0.0800 -0.2660 0.2293 0.2626 0.4208 0.1288
maintenance.cEducationalLevelDegreeOrHigher.beta-0.12310.3549 0.0030 0.5828 0.0704 0.6299 0.2650 0.6129
maintenance.cNumberOfRecentQuitAttempts.beta-0.10450.0821 -0.1909 0.0886 -0.2317 -0.0930 -0.0685 -0.1184
maintenance.oSocialHousing.beta-0.34400.2710 -0.0428 -0.6622 -0.3961 -0.3978 -0.3844 -0.5838
maintenance.oNumberOfSmokersInSocialNetwork.beta-0.05800.0501 -0.1135 -0.0288 -0.0319 -0.0611 -0.1532 -0.0900
maintenance.oSEP.beta-0.10390.2156 -0.5381 -0.1177 -0.6069 -0.7777 -0.1609 -0.3032
maintenance.cCytisineUse.beta0.79300.2920 0.8733 0.4849 0.7325 1.0740 0.4958 -0.1077
maintenance.mNonSmokerSelfIdentity.beta0.71780.9226 1.6296 0.7973 1.1998 1.0329 1.9987 0.3256
attempt.bias-3.64430.4484 -4.2244 -4.1247 -3.7026 -3.9457 -4.0046 -3.0904
maintenance.bias-0.38871.3067 0.7921 1.3228 0.7611 1.1825 -0.2633 0.2558
lambda0.03680.0068 0.0469 0.0379 0.0451 0.0330 0.0470 0.0380
alpha1.13120.3241 1.1752 0.5118 0.9580 1.3426 1.2014 1.3368
beta0.50000.1242 0.7629 0.7214 0.5664 0.5475 0.3848 0.4205

Orange cells in plausible columns are >2σ from prior mean. Implausible columns faded.

Generated by NROY deep-dive v2 | Source: output/calibration_runs/20260703_0711_postfix_wave1 | Design B (geography frozen) | Scoring: external V_s from 20260703_0711_postfix_stoch | Plausibility = NROY + Jackson + prevalence falling