Is ‘delivering better services’ an outcome or output? Is it a KPI or even something measurable?
Performance Management Specialist Mark Hocknell is working with government to achieve clarity in outcomes measurement and erase some of the misconceptions around ‘traditional’ methods of writing KPIs and assessing outputs.
Mark attributes, at least in part, the transition from measuring outputs to measuring outcomes to the PGPA (Public Governance, Performance and Accountability) Act of 2013.
“There’s much more focus on trying to move to what is the impact or the outcomes or the core results that we’re trying to get to, rather than just the outputs. Because the outputs should connect to the outcomes. So we need to be able to measure the impact we’re having on outcomes, rather than just the activity that produces the outputs.
“So that then ensures that government services can demonstrably measure that they are delivering on the outcome rather than delivering on the output and they may or may not have an impact on the outcome itself.”
Historically, Mark says, the way most Government agencies write KPIs is vague. “They say, ‘yes, we’ll deliver a better range of services for the community’ and they don’t know what that means.”
We need to shift, instead, to identifying the desired benefit. “Australians having better health, for example, would be an outcome. And then you would measure different elements of that.”
Mark previously worked in a corporate environment for 14 years managing a call centre and had completed a Masters in Business which included statistics. He thought he knew a lot about KPIs until he realised there was no common methodology for writing them.
“We tend to learn from past experience or from various managers or through trial and error what works or doesn’t work. So therefore we all have these different approaches to developing KPIs.”
Going back to his Master’s textbooks, he found none of them told him how to write KPIs – just sections asking him to ‘insert KPIs here’ or ‘use KPIs to manage people’.
“So if we look at how KPIs are written in most government agencies, they’re a mismatch of different things. Sometimes they’re milestones: ‘we’ll deliver this project by 30 June’, or we’d have vague things like staff productivity or a survey of the community, but we don’t really write KPIs as we should be doing.
“We don’t get any consistency of understanding what the KPI is. That’s the gap that PuMP fills.”
PuMP has been designed by Stacey Barr, a specialist in strategic performance measurement and evidence-based leadership. Stacey trained as a research statistician and worked with Australian Bureau of Statistics, then with Queensland Rail, trying to answer questions around ‘How do you measure outcomes? How do you figure out what the measures should be? How do you use performance measurement to get continuous improvement?’
Over the course of 10 – 15 years, Stacey developed the PuMP methodology which has now been well established for two decades.
Mark breaks down the 8 key steps which make up the methodology:
- Understanding measurement’s purpose
Mark says it’s important to “establish our purpose for measurement on continuous improvement. A lot of organisations aren’t clear on their KPIs because the purpose of measurement might be a compliance approach, we get these measures because we have to produce them rather than using them to achieve continuous improvement.”
2. Mapping measurable results
“How we actually measure outcomes- there are a couple of key techniques in there about how you get from this action-oriented goal to something more measurable. Figuring out what to measure first is key to the whole problem of measuring outcomes.”
3. Designing meaningful measures
“How we design measures for those outcomes or results and there’s a five step technique for specifically designing measures for those results and outcomes.” This step is about choosing the most feasible and relevant measured that evidence your performance results.
4. Building buy-in to measures
“How we can engage the organisation more and get people more engaged in their KPIs and measures, as well as in their initial development so people feel more comfortable about them and know they’re not going to be used as a rod for their back, but they’ll be used as a method or tool for improvement. It’s all about getting buy-in.”
5. Implementing measures
“How we define measures, how we start to bring them to life.” This step helps detail how each measure should be calculated and implemented.
6. Interpreting signals from Measures
“Making sure we use the right sort of statistical tools and charts that help us get consistent interpretation of what the measure is telling us. One of the challenges we have in reporting these measures is that we often sit around the table and argue about our own interpretations of that measure. Whereas with the right statistical tools, we will get correct interpretation each time. This means we can focus on what the measures are telling us about our performance and what we need to be doing next.”
7. Reporting performance measures
Step 7 involves creating useful and usable performance reports that inspire us to action. Reports should be designed in the most easily digestible way to support decision making.
8. Reaching performance targets
“How we use measures to get continuous improvement against those proven initiatives that we’re trying to get to.” The final step is based on improving businesses processes to move current performance towards where it should be.
Mark Hocknell is facilitating a series of masterclasses in November and December in Sydney, Canberra, Melbourne, Adelaide and Brisbane. Measuring Outcomes Based Approaches in Government will take attendees through the process of enhancing outcomes for citizens by understanding what needs to be achieved, indicators that demonstrate progress, how to integrate outcomes into program & policy and key strategies to measure process.