Shining a Light on Utility Performance in Hawaii’s Clean Energy Transition

In June 2021, the Hawaii Public Utilities Commission (PUC) approved an extensive portfolio of performance mechanisms, including almost 40 metrics and scorecards that provide visibility into a wide scope of utility operations. These metrics and scorecards are now publicly reported on a new web page from Hawaiian Electric — the state’s investor-owned utility. The page is intended to increase transparency and hold Hawaiian Electric accountable to delivering on a number of prioritized social and policy objectives under a regulatory approach known as performance-based regulation (PBR).

PBR is a collection of regulatory mechanisms aimed at overcoming the incentives in traditional “cost of service” regulation that can deter utilities from investing in the resources and technologies needed to support the clean energy transition. PBR is not a new concept, but it is gaining traction: in the past three years, six states have passed new legislation authorizing the use of PBR to better align utilities’ financial incentives —with clean energy goals and evolving grid and customer needs.

As an advisor to the PUC and facilitator of the ongoing PBR stakeholder process, RMI offers three lessons learned from Hawaii’s experience developing a comprehensive portfolio of metrics and scorecards. These lessons can inform other states’ efforts toward developing metrics and scorecards, and build on another set of takeaways that we shared earlier in the process.

Lesson 1: There is untapped potential in using metrics and scorecards to motivate progress on both traditional and emergent outcomes.

The Hawaii PUC’s PBR process is anchored by a broad set of 12 outcomes, which range from more traditional priorities (e.g., affordability, reliability, cost control) to more emergent areas of importance (e.g., customer engagement, customer equity, and electrification of transportation). While utility progress on each of the 12 outcomes will be important for Hawaii’s clean energy future, several outcomes were identified as areas that could benefit from financial rewards or penalties (i.e., performance incentive mechanisms, or “PIMs”) to encourage specific utility actions or behaviors.

But PIMs are not always appropriate. This is particularly true for more emergent outcomes where there is less data and experience available to help design the incentive. In these cases, publicly reported metrics and scorecards (i.e., metrics with targets or benchmarks) can be useful mechanisms to measure utility performance and motivate improvement without needing to establish a financial incentive. In Hawaii, the PUC prioritized scorecards where regulators saw a clear need to hold utilities accountable to an expectation and had sufficient data to establish a target or benchmark — but also where they saw that further data would be helpful in informing incentive design in the future.

Multiple, complementary metrics are often necessary to comprehensively reflect a utility’s performance in a particular area. Hawaii’s affordability outcome is a prime example. The COVID-19 pandemic has increased many states’ interests in understanding the impacts of energy system costs on the most vulnerable communities. However, a lack of transparency and data can often be a barrier to accurately assessing energy affordability and customer energy burden. The Hawaii PUC adopted three metrics that will illuminate how low- to moderate-income (LMI) customers are faring: a metric to monitor the proportion of customers disconnected for nonpayment over time, a metric that will track the proportion of customers that are participating in payment arrangements for unpaid bill debt, and an LMI Energy Burden metric, which will shed light on how energy costs are impacting LMI customers over time.

Lesson 2: The devil is in the details when establishing metrics and targets.

Hawaii’s PBR process highlighted the many intricacies of developing a portfolio of performance mechanisms. The process involved iterative proposals, analysis, and thoughtful deliberations to ensure that the final portfolio was comprehensive and fostered increased transparency as well as administrative efficiency. Below we break down some of the key steps in the process.

Metric Development

Most outcomes can be measured in more than one way, making it difficult to determine which metrics will best reflect utility performance. This is particularly true for new areas of utility operations, where data might be difficult to collect due to technology limitations or processes not yet being in place.

For example, the intent of the LMI Energy Burden metric is to estimate the proportion of a low-income household’s annual income on each island that is spent on electricity. While calculating this metric for each unique household was not possible in Hawaii, there are a variety of different values that can be used to represent a low-income household’s annual income, such as the area median income or income thresholds for participation in state and federal programs that serve low-income households. The Hawaii PUC ultimately approved the use of 150 percent of the Hawaii Federal Poverty Limit.

To reflect low-income households’ annual spending on electricity, the PUC approved two proxies for annual expenditures: average electricity bills for residential customers and an estimate of the typical bill for average electricity consumption (kWh) by island. Despite not having perfect data, these proxies are already enabling valuable insights: the graph below demonstrates how customers’ energy burden has fluctuated over the years on different islands.

Exhibit 1. LMI Energy Burden by Island, chart recreated here by RMI

Reflecting on all the metrics approved in Hawaii, PUCs evaluating metric proposals should consider:

• what data is available or what is measurable;
• the appropriate treatment of anomalies;
• which estimates or proxies are acceptable in the absence of more precise data points; and
• the best use cases for percentages versus absolute numbers.

Target Setting

Arriving at an appropriate goalpost for a utility’s performance scorecard can be challenging for more emergent outcomes where historical data is scarce. Even when there is a historical perspective, significant discretion may be required to define what constitutes an achievable or a stretch goal.

Hawaii’s experience developing scorecards highlights the following takeaways:

• When nonexistent (or imperfect) historical data hinders target setting, policy goals (e.g., goals for reducing greenhouse gas emissions), utility commitments (e.g. see the Fleet Electrification Scorecard), assumptions used in planning processes, or the performance of peer utilities can anchor targets.
• When it is unclear whether a target is appropriate, build in flexibility to reassess after a certain period.
• For certain outcomes, consider adjusting targets over time to encourage continual improvement.

Establishing a Portfolio

Before finalizing a portfolio of PBR mechanisms, it is important to assess the degree of overlap and complementarity of mechanisms. The Hawaii PUC examined the emerging portfolio of proposed metrics but also looked beyond the PBR docket to metrics and reporting requirements established in other proceedings to assess what would be required of the utility going forward. Where possible, regulators and other stakeholders should consider how to establish metric portfolios that are comprehensive enough to cover the most important areas of utility operations but are streamlined enough to avoid unduly burdening the utility.

Lesson 3: The bigger the tent, the better the performance mechanism portfolio.

Another critical component to the quality and robustness of Hawaii’s portfolio of performance mechanisms was the active engagement of a diverse set of stakeholders. Hawaiian Electric, solar industry representatives, environmental nonprofits, municipal governments, and the state’s consumer advocate all participated in the PBR process. The nonutility stakeholders played an invaluable role by proposing an array of performance mechanisms for the PUC’s consideration, many of which were adopted.

Even without financial incentives, performance metrics and scorecards can have a significant impact on utility investments and decision-making — and begin the collection of data to support the design of future regulations. As a new wave of states launch efforts to design their own PBR frameworks, they should consider metrics and scorecards as important complements to other PBR mechanisms. Metrics and scorecards increase transparency and can help to drive the energy transition in new areas of utility operations, or in areas where tracking performance has value even when it is not explicitly tied to financial incentives.