Executive summary
- Simulation converts a modelling problem into a sampling problem; accuracy improves with the square root of the number of paths.
- Tail statistics converge far more slowly than means, so a scenario count adequate for an average is inadequate for a 99.5th percentile.
- Dependence structure between inputs usually matters more for the tail than the marginal distributions do.
- Variance-reduction techniques can deliver large efficiency gains but require care not to bias the region of interest.
- Reproducibility — seed management, version control and documented configuration — is a governance requirement, not a convenience.
Mechanics and convergence
The procedure is straightforward: specify distributions for the uncertain inputs and their dependence, draw a set of values, compute the outcome, and repeat many times. The empirical distribution of outcomes approximates the true distribution, with accuracy improving as the number of paths increases.
SE(mean) ≈ σ / √n
- SE(mean)
- — standard error of the simulated mean
- σ
- — standard deviation of the outcome variable
- n
- — number of simulated paths
The square-root relationship has an uncomfortable implication: halving the error requires four times the computation. For extreme quantiles the situation is worse, because the estimate depends on the small number of paths that fall in the tail. A 99.5th percentile estimated from 10,000 paths rests on approximately fifty observations, and its sampling variability is correspondingly large.
Dependence between inputs
Simulating each input independently is simple and almost always wrong. Real risk drivers move together, and they move together more strongly in adverse conditions. Linear correlation applied to non-normal marginals does not reproduce this behaviour; it constrains the average co-movement while leaving joint tail behaviour largely unconstrained.
- Copulas separate marginal distributions from the dependence structure and allow tail dependence to be specified explicitly.
- Common-factor models represent dependence through shared drivers, which is often more interpretable than a copula.
- Where dependence is weakly evidenced, run the simulation under several plausible structures and report the range.
- Check the simulated joint extremes for economic plausibility, not only the simulated correlation matrix.
Efficiency and validation
- Use antithetic variates or control variates where a correlated quantity with known expectation is available.
- Consider importance sampling when the quantity of interest depends on rare events, and document the weighting scheme.
- Consider quasi-random sequences for smooth low-dimensional problems, recognising that the standard error formula no longer applies.
- Run convergence diagnostics: plot the statistic of interest against path count and confirm it has stabilised.
- Re-run with different seeds and report the variation in the headline result; if it is material, the path count is insufficient.
- Validate against analytical solutions for simplified cases where these exist; a simulation that cannot reproduce a known answer should not be trusted on an unknown one.
Governance and reporting
- Record the random seed, model version, input parameter set and path count with every reported result.
- Report simulation error alongside the estimate, so recipients can distinguish a genuine movement from sampling noise.
- Establish a policy on nested simulation, which is computationally expensive and frequently approximated in ways that require disclosure.
- Subject the simulation engine to independent validation as a model in its own right, separate from the models it implements.
Practical example
A capital model run with 10,000 paths reports a 99.5th percentile loss of 248 million. Repeating with five different seeds gives 236, 251, 244, 262 and 240 million — a spread of 26 million arising purely from sampling variation, with no change to any assumption.
If the reported capital figure moves from 244 to 251 million between quarters, that movement is well within simulation noise and should not be attributed to a change in the risk profile. Increasing to 200,000 paths reduces the spread substantially. Reporting the estimate with its simulation error prevents an entire category of spurious management commentary.
Limitations and caveats
- Simulation reproduces the assumed model; it cannot correct a misspecified structure.
- Tail estimates carry large sampling error unless path counts are high or importance sampling is used.
- Dependence assumptions dominate tail results and are typically the least well evidenced inputs.
- Variance-reduction techniques can introduce bias if applied without care in the region of interest.
- Computational cost frequently forces approximations whose effect on the result is not always quantified.
Conclusion
Monte Carlo simulation is the workhorse of modern risk quantification, and its principal risks are practical rather than theoretical: too few paths, unexamined dependence and unreported sampling error.
A simulation reported with its convergence diagnostics and its simulation error is a measurement. Reported as a single number, it is an assertion.
References
- Glasserman, P. (2003), Monte Carlo Methods in Financial Engineering — Springer, 2003
- McNeil, A., Frey, R. and Embrechts, P., Quantitative Risk Management — Princeton University Press
- Minimum capital requirements for market risk — Basel Committee on Banking Supervision, 2019
Author bio
Jonas Mohamed Osman Abdelghafour, known as Yonas Osman is an actuary, FRM and financial risk professional specialising in banking, insurance, model risk, capital modelling and quantitative risk management.