Sources of Explanation Uncertainty in Deep Learning Emulators of the Fire Weather Index
Explainable artificial intelligence can help interpret deep learning models of environmental processes, but the reliability of these explanations remains difficult to assess. Models with similar predictive skill may produce different explanations, and agreement across methods does not necessarily indicate accurate recovery of the underlying relationships. We propose using the Canadian Fire Weather Index System as a controlled benchmark to investigate the sources of explanation uncertainty and the role of correlated feature proxies. Its known mathematical structure provides a reference for assessing whether explanations recover the dependencies used to calculate the index.
We will examine how explanation variability changes across training runs, model designs, interpretation methods, and input dependence. The study will test whether training randomness dominates explanation uncertainty and whether stable explanations accurately reflect the reference calculation. The findings will inform how predictive performance, explanation reproducibility, and fidelity should be evaluated when interpreting deep learning models in environmental applications.