Unless you understand a problem’s underlying cause, you’re unlikely to nail the solution. That’s why audit quality leaders are using root cause analysis to lift audit standards.
By David Walker
In 1995, a 51-year-old disabled diabetic man named Willie King checked into a Florida hospital to have a diseased leg amputated. Unfortunately, his surgeon cut off the wrong leg.
King accepted the stuff-up with startling equanimity, telling one reporter, “I’ve kind of taken it in stride”. A series of errors by hospital personnel and the diseased condition of both King’s legs were contributing factors, and King got a combined payout of US$1.15 million from the hospital and the surgeon.
But more importantly, the case highlighted flaws in hospital protocol and the mistake helped trigger a search for a new error-reduction system.
In the years following, the US Joint Commission on Accreditation for Hospital Organizations (JCAHO) identified that wrong-site surgeries occurred at a consistent rate right across the country. It used a method known as ‘root cause analysis’ to seek out the underlying, deep-down cause of the problem. It turned out that in often-chaotic operating rooms, communication becomes unreliable.
In 2003, the JCAHO put in place a simple protocol to fix that root cause and avoid another episode like King’s. Key people must now affirm they have the right patient, site and procedure, and the surgeon marks the operation’s site with a permanent marker while the patient is still awake.
Delving into the ‘why’
Modern root cause analysis was popularised in the mid-20th century by investigators of industrial accidents and aviation crashes, and was also adopted in the medical sphere. It’s essentially the process of looking past the immediate problem, for example, beyond ‘the surgeon should have listened to a nurse’s warnings’, to ‘listen to everyone’s warnings’, and onwards to, finally, a new operating process.
More recently, root cause analysis has found a place in external auditing, as the audit profession globally looks for ways to improve the quality of audits.
Root cause analysis starts with defining the problem, gathering the relevant information and identifying the issues that helped create the problem. Members of the ‘quality team’ then talk with the ‘engagement team’, sometimes in groups but usually one-on-one. These conversations repeatedly ask, ‘why?’ Once the underlying causes of any audit quality issues have been identified, the team sets about devising permanent solutions.
The Australian Auditing and Accounting Public Policy Committee (APPC) released two new guidance documents in February 2019 to help individual auditors and audit firms strengthen their root cause analysis frameworks.
Acuity spoke to a number of the experts behind the new guidance, who shared their experiences of implementing root cause analysis at their firms.
The root cause of uneaten sandwiches
Valerie Clifford FCA is the assurance risk and quality partner for PwC Australia. Her team ensures root cause analysis is performed on all of PwC’s audit reviews and inspections, internal and external, and puts in place quality improvements.
A former long-serving member of Australia’s Auditing and Assurance Standards Board, Clifford sees root cause analysis as “fundamental”. You can keep shouting “don’t do it that way” at people, she says, but to get sustainable quality improvements you need to unpick the tough cases and identify the deep underlying causes of issues.
To explain the philosophy behind root cause analysis, Clifford tells a story of everyday parental frustration – that her daughter kept bringing uneaten sandwiches home from school.
“I was saying to her, ‘I’m not going to make you lunches any more. You can make your own’,” she recalls. But her daughter insisted that wouldn’t help.
‘“So then we unpicked the why,” Clifford says. “Why wasn’t she eating her lunch? Because she’s got all these lunchtime activities on. She wants to eat lunch, but she just can’t manage a big heavy sandwich when she’s juggling doing activities.”
After replacing the sandwich with bitesized foods the problem went away.
Clifford notes that often it’s necessary to drill down through five layers of explanations in search of the underlying cause of quality issues in audits.
Root cause analysis encourages you to understand how processes break and then how to go about fixing them.
“Good root cause is about the why of the why of the why,” says Clifford.
An evolving technique
Traditionally, the accounting profession tried to fix audit quality issues with more training, points out Deloitte audit quality and risk partner Gareth Bird CA.
It was the US Public Company Accounting Oversight Board (PCAOB) that first encouraged auditors towards root cause analysis in the early 2010s. It’s now entrenched in major firms’ systems.
Bird says the technique is still evolving. “It’s in its infancy,” he says. “All the firms… are grappling with differences of approach and differences of opinion.”
But EY quality enablement leader Chris Lawton CA is optimistic that the relatively concentrated nature of the audit market gives it an edge when it comes to implementing new best practice measures.
Lawton’s introduction to root cause analysis came when EY first rolled out a formal program globally in the mid-2010s. Lawton believes he was well suited to the work not only because of his technical skills but also his seniority.
“I had some grey hair,” he jokes, and that made it easier for him to have the all-important talks with senior partners.
BDO Australia’s national leader for audit quality, Jane Bowen CA, says the big benefit of root cause analysis is the time it gives participants to understand why things have happened. The process requires discussion between the reviewer and the engagement leaders.
Auditors have told BDO they like the opportunity “to stand back and think about why certain things happen, when you’re not in the midst of it,” she says.
Getting hard conversations right
For all his seniority, Lawton admits he had to learn ways to get people talking during the process. Bird goes so far as to call this the biggest challenge in root cause analysis. Deloitte now steers away from recording initial conversations to encourage people to open up. Bird also likes to match interviewees with interviewers of similar seniority.
Grant Thornton assurance partner Merilyn Gwan CA stresses the importance of getting the dynamics of the conversation right. “You’re not looking to blame anyone; you’re actually trying to understand what the underlying issue is,” she says.
Lawton says a finding may be due to a client’s situation, but it may also result from an event inside the firm, such as a team disruption triggered by a major transaction. Regardless of the cause, he encourages auditors to “embrace the mistakes”.
“I try to say to our people when they’re going through root cause analysis… that this is actually quite a liberating process for them,” he says.
It lets people set out all the pressures, the things they were dealing with at a point in time.
“Everyone’s trying to do a quality job, so if things don’t go well, it’s not a deliberate result; there’s got to be something behind it,” Lawton explains. It’s also a fast track to avoid repeating mistakes.
5 things to do in root cause analysis
1. Keep asking why
The discipline of asking why is at the core of root cause analysis. BDO’s Bowen says the key lesson of her time using the technique is “not to stop too soon”.
Practitioners must “get beneath the surface level reasons,” she says. “If you stop the questioning too soon you don’t actually get to the underlying root cause. You know, what is the reason that they didn’t have time to do that on the engagement? What is the underlying reason that they didn’t realise that they had to document something in response to a particular risk?”
The answers to these questions may be about the firm’s management processes or the running of an audit practice, rather than about an individual engagement.
Gwan from Grant Thornton shares a story to illustrate the technique. Your car gets a flat tyre. Why? You ran over a nail on the garage floor. But that’s not the root cause. You ask ‘why’ again, and find the box of nails on the shelf fell apart. Why? The box got wet. Why? You’ve had a hole in your roof. The lesson: don’t just stop at the first ‘why’; seek the deepest possible cause.
Lawton says EY has built its interview process to avoid stopping the questioning too early. The firm sets up separate discussions with all the relevant team members, but it aims to start with the more junior members.
That way, he says, members of the quality team “don’t get a perspective from the partner straight out of the chute that may tend to blindside you a little bit to any other perspective”.
2. Expect early defensiveness
Lawton recalls that early on in EY’s rollout of root cause analysis, staff saw the technique as just another way to tell them they didn’t do something properly.
“Their first reaction tends to be a bit defensive,” adds Clifford from PwC Australia. “People can be quite… ‘abrupt’ is probably the word – abrupt and defensive.”
She works to help them understand that the process is not a personal attack. At the same time, she presses for the explanation of why events during the audit happened as they did. “When you start doing that,” she says, “people tend to open up more.”
Gwan adds that it’s important to take care with your vocabulary. “You want to be really careful when you describe what you are doing to a team.”
For instance, if a review finds that “leadership wasn’t involved at the right time”, many partners may take offence. But Gwan explains that such wording might simply mean they have had the conversation and not documented it properly in the file – which means an outside reviewer has no evidence that the conversation ever happened. It’s critical for auditors to document such sidebar conversations, she says.
3. Understand what’s going right, as well as wrong
Firms can also learn a lot from applying root cause analysis to audits that went well. “Don’t just look at the bad, look at the good,” is Gwan’s advice.
She recalls one audit for an ASX-listed entity working in a new industry with a focus on new technology. The business was complex enough that the audit team enlisted IT specialists from around the world. The analysis showed that the partner and the manager had also responded to the complex brief by involving themselves more heavily in the audit’s early stages – the risk assessment, the understanding of the client’s systems, the fundamental understanding of their debits and credits, and the fundamentals of the business generally.
“And it led to a better-quality audit,” says Gwan.
But the more successful audit was gained at the expense of partner time – over 30% of time on the audit, rather than the typical 5-15%.
How to balance the benefits gained with the cost of the additional partner time remains an unsolved problem. But Grant Thornton is now experimenting to find ways to exploit this new learning.
4. Analyse your follow-ups
Frequently there is not one root cause of problems but several. Clifford, for instance, has noticed engagements where an incident would not normally have led to a finding, but the normal checking process failed – “senior people up the tree didn’t pick it up”.
In these cases, one root cause may be a high turnover of managers in an audit area. That may lead to action to mitigate the turnover. The common thread in such cases is “other things that happened, often operational in nature, in conjunction with the first problem,” explains Clifford.
The teams at Deloitte and PwC, among others, are making more use of data analysis to identify such confluences of events. Put enough data points together, says Clifford, and you see patterns: specialists providing services outside their area of expertise, say, or an audit with a high number of new recruits.
Audits go wrong in much the same way that other processes go wrong, she notes. People inherently want to do the right thing, but sometimes multiple factors come together to create a different outcome – “a sort of perfect storm”.
5. Work on solutions
Gwan most enjoys the last stage of root cause analysis – implementing solutions. “You can put measures in place, change a policy because it’s not appropriate, look at the training.”
But she also acknowledges the difficulties of this phase. She has been dealing with the results of a root cause analysis that indicated partners should spend more time upfront on certain engagements. But partners’ time is critical; every additional hour they invest takes them away from other work with the client base. Grant Thornton faces a major exercise to make the change a reality.
It’s probably the biggest single impact the firm has seen so far from root cause analysis. But Gwan warns: “It’s not going to be a quick fix.”
Bowen agrees that implementing solutions is the toughest stage. People want to make improvements, she says. But they must also resist the temptation to just add to the firm’s existing processes and build up the workload.
“The biggest time component of root cause analysis is how you’re going to respond to the causes that have been identified,” Bowen says. “That’s where the real time is taken.”
This article was originally published in the June 2019 issue of acuity.