Ask a Modeler: How Can I Talk to Clients About the Uncertainty in Models?
How can I talk to clients about the uncertainty in models? I recently had a client who wanted to find the lowest cost solution of several design options. Some of the options performed very similarly, probably within the simulation's margin of error. I want to explain to clients like this that I can't guarantee one "best" option, but I don't want to sound like I lack confidence in my models.
- Uncertain in Urbana
My first instinct is to recommend that you bang loudly on your client’s desk while shouting the aphorism (generally attributed to the statistician George Box), ‘ALL MODELS ARE WRONG BUT SOME ARE USEFUL!!!!! ALL MODELS ARE WRONG BUT SOME ARE USEFUL!!!!!’
In case this approach is a bit heavy-handed, here are some practical suggestions to help bridge the modeling gap and help your client have more faith in the uncertain world of energy simulations:
Know your client
This might seem obvious, but it’s important to be aware of your client’s areas of technical acumen. Are they well-versed in HVAC or are they coming from more of a financial background? How interested are they really in digging into an energy model’s minutiae? The answer might surprise you.
Knowing how to read the room is essential; just as it’s important to not be overly-technical when communicating modeled results, it’s also important not to sound like you’re being dismissive of what could be a well-informed concern.
Consider using ranges for reported results
An ECM table that shows savings of $565, $823, $592, and $763 per year is probably too granular for a mid-to-large-size building. In cases of small relative savings, a range of $500-$1,000 (or even establishing a lower boundary like <$2,000) might be more appropriate.
By presenting savings as appropriately-binned ranges or thresholds, you'll bring the inherent uncertainty of modeling into focus, and you'll also save your client the eye-strain of parsing through a sea of insignificant single-digit values.
Don’t blame everything on uncertainty (aka: “maybe you just screwed up”)
Woe betide the modeler who fell back on the nebulous “statistical uncertainty” excuse when their plant reports showed a chiller peaking at a COP of 42 because of a condenser water controls typo, and their client noticed it before they did.
We’re all used to seeing counterintuitive results, but there’s a big difference between counterintuitive and physically impossible. If your client trusts that you’ll own up to a fat-finger mistake rather than muddying the waters with implausible jargon, they’ll be more likely to trust you when you make an articulate case for a truly unanticipated (and potentially informative) result.
When all else fails: Don’t be afraid to get wonky
Sometimes, the best answer might just be to actually talk to them about uncertainty.
It’s probably not a great idea to kick off a design charrette with a dissertation on sensitivity studies and ASHRAE Standard 140, but it doesn’t hurt to have some higher-level technical literacy in your back pocket if you’re really getting pressed on your results. Talk to your client about variation in weather files or unpredictable infiltration rates, day-two occupancy changes or altered controls sequences. Talk to them about whatever you need to in order to help them understand that you’re working with a model—and that no model will perfectly represent reality. But if you’ve done your job, it will at least be useful.
Michael Sweeney, BEMP, LEED AP
Energy Engineer, AKF Engineers LLP
Do you have anything to add or your own answer to contribute? Please share your thoughts by emailing email@example.com; you are an expert too! Do you have your own question for the expert? Submit your question to be considered for a future column. Note that questions requiring an immediate response should be submitted to the community of experts at unmethours.com. If you are interested in replying to a question as a featured expert or have any other feedback about Ask a Modeler please email firstname.lastname@example.org.