The Institute of Internal Auditors (IIA) defines Sampling as enabling internal auditors to gain information about a population through examination of a selection, or sample, of items without the need to examine every item within a population. The internal auditor will use sampling as an audit technique to provide factual evidence and a reasonable basis to form their conclusions.
The National Audit Office (NAO) defines sampling as ‘a means of gaining information about the population without the need to examine the whole population in its entirety’.
The International Standard of Auditing (UK) 530 (professional standards for the performance of audits of financial statements, primarily for external audit) defines sampling as ‘The application of audit procedures to less than 100% of items within a population of audit relevance such that all sampling units have a chance of selection in order to provide the auditor with a reasonable basis on which to draw conclusions about the entire population’.
Control design and control performance
Audit testing is broadly based on control design and control performance. It is usual at the planning stages of an audit to perform walk-through testing of each control in scope, via a single sample. The results of the walk-through test will then have a bearing on sample size for the control performance testing.
Defining the population
An audit period for review should first be established. This will usually involve auditor judgement based on a number of factors including when the audit topic was last subject to review, whether there have been changes in key controls, or indeed, in the risk framework. It is then a matter of gaining access to the relevant systems and then either running standard, or tailored, reports for the agreed audit period.
The auditor should also gain comfort as to the credibility of the extracted data, perhaps by reconciliation to the financial statements trial balance whilst recognising that this is no easy task given that the data will be subject to adjustment for financial reporting purposes e.g., accruals and prepayments in respect of purchase ledger invoices.
Agile auditing should always be a consideration where there is a large population of transactions and techniques can be used to review 100% of the transactions for a number of control points, as a supplement to control performance testing on a sampling basis.
Practical examples where a simple internal audit agile approach can be deployed for an internal audit might be for the testing of underwriting controls across a large Lloyd’s syndicate writing multiple classes of business. Data can readily be downloaded into Excel from most underwriting platforms and then the internal auditor can use Excel applications such as pivot tables, basic macros and VLOOKUP to test the whole portfolio of data for a number of factors such as risk written within authority, permitted territory, within set limits (Sum Insured), other coverages etc. These quick cost-free Excel techniques can be applied to other areas where there are a significant number of transactions subject to internal audit such as Delegated Authority bordereaux processing or for Accounts Payable. More sophisticated data analytics, and in particular Computer Assisted Audit Techniques (CAATs), are also an important tool for an agile IT internal auditor.
In brief, through the use of technology, or even some simple Excel applications, an internal auditor can select and analyse full data sets to periodically audit, continuously audit or monitor key organisational data for abnormalities or variances that can be used to enhance more traditional audit sampling techniques which identify and effectively evaluate organisational risk and compliance with control and regulatory requirements.
Sampling methods can be classified as either statistical or non-statistical (judgemental). Statistical sampling makes use of mathematical models whereas judgmental sampling is used by internal auditors who use their own knowledge and experience to determine the sampling size.
Random selection – selection of items at random from a large population. Using this method ensures every item in the population has an equal chance of being selected.
Interval/Systematic sampling – this sampling starts at a random point in the population, and then makes additional selections at predetermined intervals e.g., every 10th item.
Block/Batch selection – where a sequential series of selections is made, e.g., the internal auditor picks a sample of 50 insurance policies with policy number 200015 to 200065.
Monetary Unit sampling – samples are drawn in proportion to their size giving a higher chance of selection to the larger items.
Haphazard selection, in which the auditor selects the sample without following a structured technique. Although no structured technique is used, the auditor would nonetheless avoid any conscious bias or predictability (for example, avoiding difficult to locate items, or always choosing or avoiding the first or last entries on a page) and thus attempt to ensure that all items in the population have a chance of selection. Haphazard selection is not appropriate when using statistical sampling.
According to the NAO in its guide (SamplingGuide NAO), there are five key factors to consider when selecting the sample size:
No estimate taken from a sample is expected to be exact, inference to the population will have an attached margin of error. The better the design, the less the margin of error and the tighter the precision but in most cases the larger the sample size.
The amount of variability in the population i.e., the range of values or opinions, will also affect accuracy and therefore the size of sample required when estimating a value. The more variability the less accurate the estimate and the larger the sample size required.
The confidence level is the likelihood that the results obtained from the sample lie within the associated precision. The higher the confidence level, that is the more certain the auditor wants to be that the results are not atypical, the larger the sample size. The NAO recommends between 95% and 90%.
Population size does not normally affect sample size. In fact, the larger the population size the lower the proportion of that population that needs to be sampled to be representative.
The proportion of the population displaying the attribute that the auditor is looking to identify.
Documentation and reporting
According to the IIA, it is important for the internal auditor to ensure that working papers include sufficient detail (IIA Standard 2330) to describe clearly the sampling objective and the sampling process followed. Additionally, the work papers should include the source of the population (with dates), the sampling method used, sampling parameters (e.g., random start number or method by which random start was obtained and sampling interval), the precision and confidence intervals for the estimated items selected, objectives including details of audit tests performed, and conclusions reached.
In reporting terms, the internal auditor needs to ensure that results of testing and the conclusion reached, contains sufficient information for the reader of the report to understand the basis of the conclusion and that this is based on evidence, judgement and impact.