When and How Should I Update My Energy Modeling Software?

Andy: What are the priorities for updates to EnergyPlus from a developer point of view?

Edwin: Our biggest priority is to provide new features and capabilities that meet the ever changing needs of industry. Each year, we solicit feedback from our key users, stakeholders, consultant groups, and lab partners, and prioritize the most important features. Features are selected from this prioritized list based on industry demand, available funding, and especially any features that enable EnergyPlus to support projects that include Diversity, Equity, and Inclusion (DEI) initiatives. Over the last couple of years, I’ve been most involved in developing the Python ecosystem around energy, which includes carving out an API, implementing Python plugins, and making sure EnergyPlus becomes a usable library and tool with many applications.

Andy: What concerns and considerations do you have on the practitioner side for updating software company-wide through the lifetime of projects?

Rashmi: We work on a lot of large-scale healthcare and lab projects. They're not limited to pre-canned systems and configurations, which means we need workflows to generate and test those custom configurations. Most teams or modelers have their own workflows and processes to convert project data into an energy model file. Whenever we upgrade to a new EnergyPlus version, we have to rerun all the testing that we did to be confident in the custom setup that we've created. The lifetime of projects that we work on is long compared to the development software update cycle. So we would then need to update multiple versions of the scripts to reflect the changes in IDF files with each version. It's really impractical for us to run all possible combinations and test each release. It just adds up to a lot if we try to update frequently.

Andy: When is it important to upgrade software?

Rashmi: I think I'll start by discussing when we choose not to upgrade software. If I upgraded my software in the middle of the project when the timeline is challenging, just because I have the feature available, it could cause a lot of changes that I wasn't anticipating. Modelers wouldn't upgrade software at a challenging time when there's a limited scope for testing. There's also a cost to updating the version when there's confidence in the existing stable workflow and its physics. It's better to just assess new features released at the start of a project. When I'm starting a new project, I check if there are any software updates that would help me model my project better and then I stick with this version through the end of that specific project. It really makes sense to invest time in the early stages of a project, when people have an expectation that model numbers and inputs can change. So that's a better time to test rather than towards the end when you're submitting for LEED and an upgrade could change everything. That's not something we like to do. The features that drive us to make these upgrades are usually major bug fixes, any feature additions that would really enhance the quality of our modeling or our confidence in modeling, and speed improvements. Speed improvements can really help when you're running multiple simulations!

Edwin: Easy, every time there is an EnergyPlus release, I need everyone to upgrade to the latest version that day and throw away anything they've done before! [laughs] I'm very glad that Rashmi answered that first because it would be easy for a developer to give that kind of naive answer, right? Trying to put myself in the shoes of an application user, I would agree a good time to upgrade is when there's a need, such as an impactful new feature or bug fix. For EnergyPlus specifically, we are actively working on making the upgrade version change process much smoother. We have supported a JSON input structure for multiple years, and will eventually move away from IDF inputs, making it easier to develop and maintain workflows. We eventually would like to get to the point where EnergyPlus can accept an input file of any number of versions back and run with it. We’re really trying to reduce the maintenance that interface developers and practitioners have to put forth to manage version changes, so hopefully that will get better all the time.

Rashmi: That's really encouraging to hear, thanks for sharing that. We're thinking about using the OpenStudio SDK because with the SDK, we don't have to change the translation with the interface and the SDK. But there are challenges with how much we can customize with the SDK because not everything is surfaced. With EnergyPlus, if I open an IDF and use an IDFEditor or some other text editor, I can connect nodes however I want, but I cannot always do that with the SDK. There are challenges with having too many wrappings around the native EnergyPlus, so it's good to hear about the JSON structure because I would definitely be excited to start using it.

Andy: When you have a new release, how do you envision reconciling differences between the previous and the new version?

Edwin: Testing on EnergyPlus has expanded greatly over the last 10 years. We now have 2000 unit tests, 700 integration tests, and a suite of API tests that run on every commit of EnergyPlus across 3 platforms and multiple configurations checking outputs and identifying regressions on multiple platforms. So when someone makes a commit to our repo, within a couple hours it runs those thousands of tests and we get a report back that says, “These three tests failed and five files have differences”. Before we merge any of those changes in, they have to either get rid of those differences or explain why the result changed. It is expected that some changes to EnergyPlus will cause changes in results: bug fixes can fix problems; features can add physics that just weren't captured before. In conjunction with that, we have a file that goes out with EnergyPlus releases that tracks major output changes. Structural, or major, output changes are called out in that file to help alert users of what might have caused changes in their models. I imagine a user would look at the change log, the output changes file, see if there's anything on the community side, and then reach out to our help desk if you're seeing a difference in your building model that doesn't seem to make sense with what's being seen out there. I think there's a number of steps that I would invite people to use, but I'm curious what Rashmi thinks. Is that tractable, or is that just a silly, naive answer?

Rashmi: No, it’s definitely not naive. I think that that reflects a lot of what we do. I will say that, at AEI, we are more in the weeds of EnergyPlus than many other EnergyPlus users. If I was a user who is primarily consulting, and I wanted to catch up on all this with no knowledge of GitHub or development, it would be challenging to understand what a commit, or a pull request, or an issue is on GitHub when that page comes up. I think it would be useful to have a TL;DR type of documentation that separates what you need to know from what you can know if you want to, before this huge list of bug fixes and things that are really less relevant to the end user, because it just gets overwhelming when considering upgrades on a tighter time frame for the workflows. Other than that, we pretty much do what you said. An example I found is when I was trying to add an air source heat pump configuration that wasn't available in EnergyPlus v8.8, but was available in EnergyPlus v9.3. I upgraded a v8.8 model I had that did not have a heat pump and decided to run both to see what the difference was, and I saw energy change. Like you said, bug fixes and additional physics can cause the results to change, and I saw the outdoor air calculation output was different for the same input. It could be really hard to track down why it's happening, and that's exactly why we wouldn't do it in the middle of a project. It's really hard to explain that type of stuff [to clients], and it harms the credibility of your consulting when you have less confidence about the end results. That's just something that we deal with, and if there's a major change, it would be good to have something more easy for a user who's not quite as involved with the development cycle of EnergyPlus to understand.

Edwin: That's really great to hear, I definitely think we can do that. I think that's a great idea.

Andy: You mentioned engagement with DEI efforts earlier. What does that engagement look like?

Edwin: While EnergyPlus is simply a mathematical physics engine, we have the opportunity to engage DEI experts when selecting features and capabilities in the engine. New features that support DEI could be configurations or system types that were not supported previously. Our approach is to interact with experts in industry, DOE, and NREL, and include them in our feature review prioritization process.. We hope that, now that DEI has become such a focus, case studies start defining needs that we can respond to as a simulation engine development team.

Andy: Finally, is there a good way for practitioners to let the EnergyPlus team know what they need?

Edwin: Every summer, while our project budget is being set, we solicit feature requests from a long list of stakeholders: all our national lab partners, a lot of contractors and small businesses, universities, and a big list of power users that keep coming back and contacting EnergyPlus. We then consolidate that list, look for duplicates, and prioritize them with the help of DOE. If a user has a need that isn't met by that, and if they have the available time, it's also possible that they contribute and partner with the development team to get a new feature implemented. It can be a student who's working on their thesis, and they would like to see this added into EnergyPlus, or it might be a company that just has an interest in getting something into EnergyPlus, but it couldn’t be funded by existing funding streams. If they have time available and someone to work on it, then someone from the EnergyPlus team can help shepherd that feature through the development process, from the proposal design all the way through and get it into EnergyPlus, which could be a more direct route.