A generally accepted definition of "continuous auditing" remains elusive, and expert practitioners remain rare. Here are some tips from the trenches for getting a program going.

Internal audit departments looking to start a "continuous auditing" program are entering an area that is either quite crowded or sparsely populated. The root of this seeming contradiction is, like a lot of things related to continuous auditing, a matter of definition.

To be sure, continuous auditing is on the minds of many executives. In an ongoing benchmarking survey, 32% of 305 organizations have told the Institute of Internal Auditors over the past year that they perform continuous auditing. In a 2006 survey by PricewaterhouseCoopers, 81% of 392 companies said they at least aspired to continuous auditing, if they hadn't already begun doing it.

But just what is "it"? For the vast majority of practitioners, "continuous" is a malapropism. The term first got traction in the 1990s, used as a contrast to the traditional practices of internally auditing individual business processes every year or every few years, and auditing financial reporting systems annually or quarterly. Any audit activities performed more often than every three months came to be known, by some, as continuous. The IIA still defines continuous auditing simply as "any method used to perform audit-related activities on a more continuous or continual basis," without further defining what "more" means.

That leaves a lot of room for interpretation. And definitions have diverged widely, though continuous auditing is generally held to be an automated approach. And increasingly it is assumed to mean examining all data relevant to the audit being performed, rather than the historical norm of examining supposedly representative samples.

A leading continuous auditing expert, Rutgers University professor Miklos Vasarhelyi, calls it "an audit that happens immediately after or closely after a particular event." But he noted that any definition of the term is a moving target, as technology advances and the way organizations use continuous auditing evolves. Although Vasarhelyi published what is regarded as the first significant paper on the topic in 1991, he says now that, "It will take a few decades for businesses and the public to understand what it is, and for us to develop exactly what the field is."

Today, says Vasarhelyi, "there are huge differences in what is considered continuous auditing. Some companies call it continuous when a particular process fails an audit and it is repeated several times over the next year." But the actual prevalence of the practice as per his definition is "limited," he says. Only a smattering of companies are auditing some business processes in something close to real time.

Whether internal auditing is appraised as having attained "continuous" status can depend on one or more of several factors: the number, timing, and frequency of automated processes, the percentage of the organization's risk profile addressed through a continuous audit approach, and the sophistication of the technology employed.

For virtually all companies engaged in continuous auditing, it is a work in progress. While acknowledging that the term is "subject to interpretation," Richard Chambers, president and CEO of the IIA, says, "We're not familiar with anyone out there that has mastered continuous auditing yet."

Further complicating the definition is coming to grips with how continuous auditing differs from continuous monitoring. Typically, the latter is seen as done by company management to ensure that policies, procedures, and business processes are operating effectively and address management's responsibility to assess the effectiveness of internal controls. Continuous audits are performed by audit departments to evaluate the adequacy of management's monitoring function and, thus, often cover the same or similar ground.

In fact, "some would tell you that there is no distinction between the two," says Chambers. However, where there is a distinction, it can be blurry. For example, at some organizations, the audit department's role is not just to scrutinize management monitoring, but also to hand over the data-analytic scripts it created for auditing specific processes to management for use in its monitoring activities.

Following is a look at how three large companies with long-established programs use continuous auditing and the challenges they confronted in rolling them out.

Power Audit
American Electric Power began dabbling in continuous auditing as a way to better allocate internal audit staff resources. The idea was to identify automatable audit processes and free up staff to perform more subjective audits requiring professional judgment.

For example, notes Jay Hoffman, director of internal audit at the electric company, over the past couple of years data privacy has become a hot-button issue. "I've got eight people on my team," he says. "Do I want to send one of them to go look at the emerging risk related to data privacy and understand that? Or would I rather that person do an accounts payable audit that was created three or five years ago and isn't likely to yield a ton of new issues?" Thus, somewhat counter-intuitively, AEP uses continuous auditing for testing low-risk areas.

But the program, launched five years ago, didn't really achieve much success until two years ago, Hoffman notes. Figuring out how to get started proved to be a big challenge, although he was able to avoid one big misstep early on. That is, Hoffman quickly realized that putting a priority on finding a technology tool would be a mistake. "In my experience, if you don't know what you want to do, you'll never find the right tool to help you do it," he says.

Still, the audit leaders felt overwhelmed by the potential opportunities. "Data exists in so many places, we didn't know what to analyze first, and we didn't know what type of analysis to perform," Hoffman says.

He started by asking his team which audits ate up staff resources because they involved cranking out the same reports every cycle. But he found a resistance to change, as well as a problem in getting the auditors to articulate what specifically should be looked for in tests that would be performed through an automated approach.

Eventually, Hoffman came to believe that he needed a "big win" to drive auditor buy-in. He found a perfect candidate: Sarbanes-Oxley compliance audits. The audit leaders were unhappy about Sarbox eating up so much of their budgets, and the auditors didn't like the work. "We said, if we want to get acceptance on a grand scale, let's make Sarbanes-Oxley testing easier," Hoffman says.

The department painstakingly went through its inventory of SOX controls to determine which ones could be assessed just by analyzing data in company systems. Then data-analytic scripts were written to allow exceptions to be found. Now, for example, the entire population of journal entries can be examined quickly for segregation-of-duty violations. "It took a little while to write the script, but now when we have to test that control every year, it going to take seconds and the financial auditors love it," says Hoffman. He adds, "If I had it to do over again, I would have started with a focus on SOX that would have accelerated the evolution process."

Another challenge in implementing the continuous auditing program was a technology learning curve. The auditors had to be trained to use the new ACL applications, the automated tool the company ended up purchasing. But that was a low hurdle compared to winning the auditors' enthusiasm, according to Hoffman.

Some of the ways AEP uses continuous auditing are common to other companies. Cash-management transactions, such as setting up new bank accounts, are scrutinized closely to make sure supervisors approved the activities, for instance.

Other uses are more company-specific. AEP, which consumes a lot of fuel to power its fleet of repair vehicles, has automated audits that identify excessive fuel usage and improper use of credit cards issued for charging fleet expenses.

The audit department also conducts safety audits, watching to make sure employees don't work excessive overtime. The goal is to avoid having line-crew members fall asleep at the wheel because they've been on the clock for 16 hours, and avoid having power-plant workers on the job for two weeks without a day off.

Meanwhile, the continuous-auditing program helps AEP decided what to include and exclude from its annual audit plan. If the output from an automated routine suggests that there is little-to-no suspicious activity in a stable business process, that is a detailed justification for excluding an audit of that process from the audit plan. Conversely, if a routine indicates a significant number of potential exceptions, that's strong justification for a more traditional, manual investigation.

So far Hoffman's department has automated only 25% of Sarbox internal tests, and many other "business-as-usual" audits still must be addressed, and those remain the priorities for AEP. "Some things will always have to be reviewed manually, but any time you can just click a button to analyze data to get your control-effectiveness answer, it's much better," he says.

Access and Auditor Angst
When Microsoft got its continuous-auditing program going three years ago, PricewaterhouseCoopers, which was advising the company on the project, warned the internal audit department to expect trouble in three areas: Getting its hands on the data, and push back from both the audit staff and the internal stakeholders whose business processes were to be audited.

Internal audit director John Digenan was really only worried about getting the buy-in on both sides, and not about the data issue. But there was little push back from the stakeholders, "and I'm not sure why," he says.

The staff, though, did resist. The cultural change in moving from tests of data samples to tests of entire populations of data was a big issue. "Intuitively it's common sense and something you'd want to drive for, so it's a little surprising that there's a pretty wide variance in how accepting people are of it," Digenan points out. Those in charge of implementing a continuous-audit project should respect the fact that it's a big change for people, that is sure to cause the kinds of bumps that accompany most change-management processes, he adds.

There was no one who outright refused to go along, but there was some obvious reluctance, and it turned out that some people had more of an aptitude for identifying the opportunities in continuous auditing than others. At brainstorming meetings, someone would float a good idea, and someone else might have chimed in, "No, the data is too complex, I don't thing we're going to be using continuous auditing for that." Digenan says it has taken a lot of training and reinforcement to get the team to realize that "the more complex the data is, the better opportunity we have."

It was the data collection that indeed was difficult, as PwC had predicted, even though the audit director had been skeptical that it would be so. The issue was not about being able to get permissions. Rather, at a large, complex company, if you're looking at travel and entertainment costs, for example, the data is not all housed in the same place. The were formidalbe logistics involved in working with the IT department to get data in a readable format and compile it in one database to enable the use of a single set of queries instead of several. "I didn't think getting the data would be so difficult," he says.

The situation was saved by a suggestion from PwC: convincing a member Microsoft's product-development team to move over to internal audit. The person who was hired knew how to gather the desired data, who to talk to about issues that cropped up, and the protocols for setting up the data queries. Digenan considers the hire lucky, because "if you're a developer at Microsoft, you probably want to work on the next product that's going to market."

The internal audit team then commenced the process of writing routines to query data relating to accounts payable, T&E, procurement, payroll, and human resources. Eventually the department reached what Digenan calls the "next level," handing off processes to management that they could use in their continuous monitoring efforts. Now team members are auditing monitoring processes that use scripts they wrote for audits.

But the audit department will not turn over a routine to management unless there is a clear business process for handling exceptions that are discovered, that includes a designated process owner and a clear "escalation" path. That is, if a problem is not handled by one person, it automatically goes to the next highest-level person. That policy redressed an annoying problem: Business units would ask for specific reports to be created, and then do nothing with them. "A lot of good ideas were winding up on the shelf," Digenan says.

The department, meanwhile, continues to audit the access to Microsoft's financial systems, and journal entries are examined for large entries, unusual entries, and unusual reviewers and approvers.

Also, the department keeps watch over financial statement ratios for indicators of fraud. For example, a days sales outstanding index measures the relationship between DSO in two consecutive reporting periods. An inflated ratio could reflect fictitious sales or receivables. Among several other ratios, an asset quality index measures a company's propensity to capitalize costs. Manipulators have a greater deterioration of asset quality, but a very high asset quality index may indicate an inappropriate deferral of costs.

Surgical Strikes
Hospital Corporation of America, which owns 163 hospitals and 105 free-standing surgery centers, is one of the true veterans of continuous audit, having started its program eight years ago. In-house developers have built what internal audit director Chase Whitaker calls a "quiltwork" of audit routines about 50 of them that run on a variety of applications, including ACL, Paisley Consulting's Focus (for Sarbanes-Oxley compliance), and Microsoft's .Net, among others.

The mix of routines employed is constantly evolving. Some have been in use for all eight years, but others turned out to be not effective, and in certain cases system changes made routines moot. But there is no shortage of ideas for new ones. "I've probably got at least 120 items in the hopper, everything from just brainstorms to things that are going through the quality-assurance process and are nearly ready to be deployed," Whitaker says. A steering committee meets three or four times a year and sifts through the list, in some cases seeking feedback from business-process leaders on whether a particular automated audit would provide risk coverage that they didn't already have.

After a decision is made to develop a routine, then comes the big challenge of determining its scope, identifying data sets to be analyzed, and setting failure thresholds. "Then you've got to develop and deploy it, and meanwhile you're wondering what kind of results you're going to get," Whitaker says.

Sometimes routines have to be adjusted. HCA, like AEP, watches employee overtime rates, in this case as a hedge against errors by nurses and other clinicians that could result in medical-malpractice lawsuits. At first the routine was set to flag anyone who worked 30% more than full-time hours. That generated a "voluminous amount" of exceptions, so the audit department contacted its facilities and found that, indeed, that kind of overtime rate was not unusual. "So we set it at 50%," Whitaker says, "and it became more valuable information for our HR directors and CFOs."

Another variable is the frequency of continuous audit routines. The overtime audits are done two or three times a year, in each case looking at the previous 12 months' worth of data. But the audit department looks daily at whether business units hospitals, surgery centers, physician practices, shared service centers have been moved onto or off the list of entities whose assets and operations are reflected in consolidated financial statements. That is to guard against unintentional changes and to determine if a change was made for a valid business reason.

And the audit department watches in nearly real time hourly for local data security administrators granting access to applications for employees or themselves. Normally, even the security administrators are not supposed to have that access. When such changes are detected, facilities are alerted.

Other human resources issues with financial implications are subjected to continuous auditing, as well. These include payroll reviews, looking for invalid social security numbers; and exceptionally low levels of vacation time, considered a warning sign that a fraud could be in progress by someone afraid a scheme might be detected in their absence.

About 15 of HCA's continuous-audit routines look directly for irregularities traceable to the finance operation. Journal entries by senior finance executives or business-unit leaders are flagged, because entries normally are made by lower-level accountants. Likewise, any entries that boost revenue by a certain percentage, particularly those that put the business unit just above its budgeted monthly goal, are flagged. "It might be valid, but at least an auditor should take a look to make sure somebody didn't just pad something to make budget," Whitaker says.

Other journal entries that are watched in the continuous-audit telescope are adjusted estimates for contingencies like malpractice-insurance claims, bad debt allowances, and amortization schedules for intangible assets. Again, the goal is to identify instances of invalid earnings management. "If somebody puts something on the balance sheet as an asset that previously was in expenses, we want to know why," Whitaker asserts. "Someone could do a journal entry just to shave some expenses for the month, thinking that the next month they'll just reverse it and no one will be the wiser."