Let \(x_{d,s,o}\) be the value of output \(o\) from the run at parameter point \(d\) with seed \(s\). For each \((d,o)\) we form the running mean and running sample variance over the first \(r\) seeds (\(2 \le r \le R\)):
$$\bar{x}_d(r) \;=\; \frac{1}{r}\sum_{s=1}^{r} x_{d,s,o},\qquad s^{2}_d(r) \;=\; \frac{1}{r-1}\sum_{s=1}^{r}\bigl(x_{d,s,o}-\bar{x}_d(r)\bigr)^{2}.$$
The standard error of the mean estimator from \(r\) replicates — the absolute Monte-Carlo standard error — is
$$\mathrm{MCSE}_{\mathrm{abs}}(r;\,d,o) \;=\; \sqrt{s^{2}_d(r)/r}.$$
It is in the same units as the output (here, probability; reported in percentage points). Dividing by the final mean's absolute value gives the scale-free relative version,
$$\mathrm{MCSE}_{\mathrm{rel}}(r;\,d,o) \;=\; \frac{\mathrm{MCSE}_{\mathrm{abs}}(r;\,d,o)}{|\bar{x}_d(R)|}.$$
Both decay like \(1/\sqrt{r}\). Each is the right metric in some regime: relative MCSE is interpretable when the mean is well away from zero, but it explodes spuriously when the mean is small (a sub-percentage-point wobble around a 2% prevalence is 25% in relative terms but irrelevant for calibration). We therefore use a hybrid pass criterion: an output passes at replicate count \(r\) for parameter set \(d\) if
$$\mathrm{MCSE}_{\mathrm{abs}}(r;d,o) \;<\; \tau_{\mathrm{abs}} \quad\text{or}\quad \mathrm{MCSE}_{\mathrm{rel}}(r;d,o) \;<\; \tau_{\mathrm{rel}}.$$
Equivalently, the output's headroom
$$h(r;\,d,o) \;=\; \min\!\Bigl(\tfrac{\mathrm{MCSE}_{\mathrm{abs}}(r;d,o)}{\tau_{\mathrm{abs}}},\;\tfrac{\mathrm{MCSE}_{\mathrm{rel}}(r;d,o)}{\tau_{\mathrm{rel}}}\Bigr) \;<\; 1.$$
For each output we then take the worst case across parameter points, \(h^{\max}_o(r) = \max_d h(r;d,o)\), and define
$$r^{\star}_o \;=\; \min\!\bigl\{\,r\,:\;h^{\max}_o(r')<1 \;\;\forall\;r'\ge r\,\bigr\},\qquad r^{\star} \;=\; \max_o\, r^{\star}_o,$$
with thresholds \(\tau_{\mathrm{abs}}=0.5\)pp and \(\tau_{\mathrm{rel}}=5\%\). \(r^{\star}_o\) is the smallest replicate budget at which output \(o\) becomes precise enough on at least one of the two scales and stays that way; \(r^{\star}\) is the worst over outputs and is the operational answer.
The "all-subsequent-stay-below" rule and worst-output aggregation follow JASSS Lee et al. (2015, §2.6, §2.11). The hybrid abs/rel pass criterion follows their §2.7 recommendation. MCSE \(=s/\sqrt{r}\) is the standard sizing quantity used in simulation studies (Flegal, Haran & Jones 2008 Statistical Science; Koehler et al. 2009).
Caveat. At small \(r\) (2–4) the running variance \(s^{2}(r)\) is noisy, so MCSE curves jitter near the left edge of each panel. The all-subsequent-stay-below rule plus \(R\ge 20\) anchoring the right end means this does not affect the headline number.
