On this page
Figure 9 — The assurance engagement process: Performing
By this point, key decisions on the approach and procedures to be performed during the engagement must have been made, documented and communicated to the audited body. The success of the performing stage of the assurance engagement depends on the audit team’s ability to perform the planned procedures, document their work and assess the results of work performed on the assurance engagement as a whole.
5.4.1 Documentation
Assurance documentation is a key component of undertaking an assurance engagement. It enables effective review and evaluation of the assurance evidence obtained and conclusions reached before the audit team leader’s assurance engagement report is finalised.4
The assurance engagement documentation must provide:
- a sufficient appropriate record of the basis for the audit team leader’s report, and
- evidence the assurance engagement was performed in accordance with the NGER Audit Determination.
Refer to section 3.9 onwards of the NGER Audit Determination about performing an assurance engagement.
See section 3.22 and 3.23 of the NGER Audit Determination for the full list of required contents of the assurance engagement report.
In addition, documentation should provide evidence of how the work performed has followed any other guidance, which the auditor has decided to use (for example, ASAE 3000 or ISO 14064-3:2006).
The assurance engagement documentation must be prepared, on a timely basis, in a way that is sufficient and appropriate to enable an experienced auditor, having no previous connection with the assurance engagement, to understand:
- the nature, timing, and extent of the assurance procedures performed to comply with the NGER Audit Determination—details must include who performed and reviewed the assurance work and when
- the results of the assurance procedures and assurance evidence obtained, and
- significant matters arising during the assurance engagement and the conclusions reached.
The contents which must be included in the assurance engagement report are listed in the NGER Audit Determination and include components, such as the objective of the assurance engagement, the matters being audited and the type of greenhouse and energy assurance engagement carried out.
Relevant assurance engagement documentation must be kept for five years after the date the assurance engagement report is signed.
Refer to
section 4.2 of this handbook for detailed guidance on record-keeping, including the obligation to allow inspectors access to documentation.
5.4.2 Tests of controls
Section 5.3.3 of this handbook covers the assessment of systems, processes and controls during the risk assessment.
Internal controls are a significant part of management’s comfort over the accuracy and completeness of the reported greenhouse and energy information.
For the auditor to rely on these internal controls several things should be considered:
- Can management’s controls be related to the assurance risks which the auditor has identified and assessed during the risk assessment (that is, are they relevant to the assurance engagement)?
- Can the auditor obtain documented evidence to support the operation of the control during the period being audited?
- What further evidence will the auditor need in order to draw independent assurance conclusions for each relevant risk identified during the risk assessment phase?
Testing can then be performed for the auditor to check the controls are operating effectively.
The diagram below provides some examples of the types of procedures performed in validating controls.
Figure 10 — Different types of control testing
Case study 3: How systems, processes and controls are tested and the flow-on considerations for the assurance engagement
The auditor performed a greenhouse and energy assurance engagement for Company B, a food manufacturer with an abattoir facility. The key greenhouse gas emissions sources are electricity consumption (scope 2 emissions) and gas consumption (scope 1 emissions) for the processing and abattoir facility and boilers respectively.
Due to the high levels of consumption and cost of the utilities, Company B uses its own direct metering system to check the utility providers’ invoices.
Through enquiry and observation, the auditor assessed that the key controls in the greenhouse gas emissions information collection process are the monthly manual reconciliation of Company B’s meters to those of the utility providers, as stated on the utility invoices, and the monthly manual sign-off of the data entered, following the monthly reconciliations by the department supervisors.
The auditor first sought to examine evidence (completed reconciliations and manual sign-offs of printouts of entered data) that the controls had been performed for a sample of three months during the reporting year. Through enquiry, the auditor found the reconciliations had been performed in each month but no sign-offs of entered data had occurred in the months selected for testing or any other months during the reporting year.
The auditor then sought to re-perform the reconciliations using the source data (extracts from Company B’s metering system and utility providers’ meter data as shown on utility invoices). The auditor found there had been a minor manual transposition error in one of the three months re‑performed, resulting in a two per cent misstatement in greenhouse gas emissions for this month. The auditor re-performed the test for another two months and noted no further errors. Re‑performance may include recalculating a reconciliation or calculation, or re-performing a control such as testing a password protection control is operating effectively.
In documenting the results of the testing on the assurance engagement file, the auditor included a table summarising the procedures performed, comfort obtained and the flow-on considerations for the assurance engagement. The auditor concluded that the reconciliation control had been operating effectively but that the manual sign-off control had not.
The auditor judged that the reconciliation control was the key control in the process and reliance would be placed on the effective operation of this control. Therefore, the extent of substantive testing to electricity and gas invoices received from the utility providers would be reduced. However, the substantive testing would not be reduced by as much as originally planned, due to the lack of effective controls over checking data inputted following the completion of the monthly reconciliations.
Common issues and how to address them
| Common issues | Suggested responses | Outcomes |
|---|
| How many instances of a control should be tested? | The size of the sample selected for testing should vary according to the frequency of the control being applied. Where a control is performed daily, or multiple times per day, the sample size would be much greater than for a control that is performed monthly. Similarly, if a control is performed quarterly or annually, the sample size would be less than for a control performed monthly. | The auditor is able to determine the sample size required for controls testing. It should be noted that this is an area of judgement for the audit team leader in terms of how many samples is deemed sufficient. Where the auditor does not have a pre-defined method for such sample sizes, it is recommended the audit team leader is consulted with regard to the selection of the sample size. |
| Insufficient evidence is available to support the effective operation of the control. | Where insufficient evidence exists the auditor must consider the impact on the planned mix of assurance procedures. Typically the auditor would perform additional substantive analytical procedures and tests of details to gain the additional comfort required. | The lack of evidence should not prevent the auditor from completing the assurance engagement, but it will necessitate a change in the mix of assurance procedures performed. |
5.4.3 Analytical procedures
Analytical procedures consist of evaluations of information made by a study of plausible relationships between various data sets5 (that is, greenhouse gas emissions and units of production).
They also encompass the investigation of identified fluctuations and relationships that are inconsistent with other relevant information, established expectations, or deviate significantly from predicted amounts6.
As one of the purposes of the assurance engagement plan is an efficient and effective assurance engagement, analytical procedures are often used as one of the key substantive tests (that is, in preference to inspection of records and documents) where appropriate.
This implies several key concepts:
- Analytical procedures are used to understand and test relationships between reported greenhouse and energy information and other factors (for example, greenhouse gas emissions and production levels).
- Effective analytical procedures involve a comparison of reported information with expectations made independently by the audit team. That is, for an analytical procedure to be valuable, the audit team must first have their own independent expectation of the value of the reported information.
- Both financial and non-financial information can be useful in understanding relevant relationships (for example, financial cost of natural gas use year on year and gigajoule of natural gas consumed) and therefore in forming the audit team’s expectations.
- The relationship between the data used to develop an expectation and the recorded amount needs to be independent, yet plausible (that is, sensible and logical). Data used to develop the expectation should not be derived from the recorded amounts themselves.
Common issues
| 1. Preliminary analytical procedures—risk assessment | The purpose of these analytical procedures is to obtain an understanding of the audited body and its environment. These procedures assist with the assessment of the risk of material misstatement in order to determine the nature, timing and extent of further assurance procedures. |
|---|
| 2. Substantive analytical procedures | The purpose of these procedures is to obtain evidence to support the assurance conclusions. Their use can be more effective or efficient than tests of details in reducing the risk of material misstatements to an acceptably low level. |
|---|
| 3. Final analytical procedures | The purpose of these procedures is to provide an overall review of the greenhouse and energy information at the end of the assurance engagement. This helps assess whether the information is consistent with the audit team leader’s understanding of the audited body. |
|---|
Performing analytical procedures can be done in five steps, as depicted below.
Figure 11 — The five step analytical procedures process
It is important to note that the five-step process is designed to standardise the performance of analytical procedures to improve their quality. The auditor should go through the process as outlined above. Further guidance is given on each step in the table below.
| Analytical procedure step | What should the auditor do in this step? |
|---|
| Step 1: Assess the reliability of the data | The data on which the analytical procedure is based needs to be reliable in order for the auditor to develop a robust expectation. The auditor must consider the source of the data, its comparability, the nature and relevance of the data and any controls that existed over its preparation. |
|---|
| Step 2: Develop an independent expectation | Before analysing the data subject to the analytical procedure, develop an independent expectation. For example, by using available data such as greenhouse and energy information previously published by the audited body, and through enquiries with management. |
|---|
| Step 3: Define a significant difference or threshold | With consideration to materiality, define the level at which the auditor will investigate differences between expectations and the actual results. The rationale for the selection of the particular threshold must be documented. |
|---|
| Step 4: Compute difference | Compute the difference between the expectation and the actual data. |
|---|
| Step 5: Investigate significant differences and draw conclusions | Investigate the reasons for the differences between the expected results and the actual results. Assess how reasonable the explanations provided by management are and corroborate these explanations. Where there appears to be additional risk, perform additional review or testing as appropriate. |
|---|
5.4.4 Evidence gathering techniques
It is expected that inspection of the audited body’s records and documents will be required, at least to some extent, in most assurance engagements performed under the NGER Audit Determination. Such inspections form part of ‘tests of details’ as shown in the assurance engagement process flow diagram (Figures 2 to 5). These procedures typically involve testing reported information back to its source documentation.
Sections 3.13 to 3.15 of the NGER Audit Determination address evidence gathering, types of evidence and methods for gathering evidence.
An expert may be brought into the audit team to review significant or complex areas of greenhouse gas emissions measurement, as part of the audit team’s assurance procedures. It is important to note the expert does so as part of the audit team (refer to section 4.1.1 of this handbook).
Examples of evidence gathering techniques are given in the table below.
| Procedure | Definition | Example of procedure |
|---|
| External confirmation | Obtaining confirmation from a third party. | Checking reported information with a third party. |
|---|
| Inspection | Examining records and documents and physical examination. | Examining and reconciling internal metering systems to check they exist and are functioning appropriately. |
|---|
| Observation | Looking at a process or procedure being performed by the participant. Generally, this is conducted when the particular process ordinarily leaves no audit trail of documents. | Touring facilities or observing the collection and reporting of data. |
|---|
| Enquiry | Asking audited body personnel to explain the bases for their judgements. | Discussing the judgements and the rationale for those judgements, made in performing key calculations. |
|---|
| Re-calculation | Conducting independent calculations to confirm the audited body’s calculations. Checking the arithmetical accuracy or completeness of source documents and records. | Re-computing the audited body’s greenhouse gas emissions calculations. Verifying the accuracy or completeness of a report used in the data aggregation process. |
|---|
5.4.5 Testing method
It is usually impractical to perform tests of details on all of the items making up a population. Therefore, in order to gain the required level of assurance, the auditor needs to determine the appropriate means of selecting items for testing. Determining the appropriate selection method depends on the test objectives, including the assertions being tested, and the characteristics of the population to be tested.
An assertion is a representation by the audited body, explicit or otherwise, that is embodied in the reported greenhouse and energy information, as used by the audit team leader to consider the different types of potential misstatement that may occur.
The methods of selecting items for testing include:
- selecting specific items, and
- sampling the population.
The auditor must first consider if selecting specific items will result in the auditor efficiently collecting evidence for the assertion being tested. By selecting specific items the auditor is focusing the procedure on items they consider are at higher risk of misstatement but is forgoing the ability to extrapolate results across the untested population. Characteristics commonly considered higher risk include:
- high value items
- items that are unusual or suspicious
- items that are of a type that are historically prone to error
- items relating to specific matters, and
- items relating to control activities.
In the event the auditor needs to obtain further evidence they will need to perform sample based testing. Such testing requires the selection of a sample that is representative of the population, so the results of the testing can be extrapolated across the untested items. In order to achieve this, the auditor may consider:
- breaking the population into smaller groups that have common characteristics and conducting sampling over each of these groups
- using computerised random number generation to assist with selecting the sample
- selecting the sample without following a structured technique and without reference to external considerations, and
- using a sampling interval and selecting an item for testing every set number of items.
The size of the sample needs to be large enough to ensure that the sample is representative of the population. The sample size selection method should be reviewed by the audit team leader and be included in the internal quality control review system for the audit.
This is an area for the audit team leader to exercise professional judgement as to how much testing is sufficient. Refer also to the guidance given in section 5.4.2 above.
For engagements to provide assurance over a report or data within a report issued by the audited body under section 19 of the NGER Act, or an offsets report under section 76 of the CFI Act, the audit team leader conducts procedures to test the aggregation process underlying that report. These procedures include:
- agreeing and reconciling such reports with the underlying records, and
- examining material adjustments made during the course of preparing the energy and emissions report or offsets report.
Case study 4: Techniques used in performing the assurance engagement
The auditor is performing an assurance engagement over the reported 2009–10 NGER information for Company C, a large electricity generation company with a portfolio of coal and gas fired power stations.
The auditor is currently looking at the reported greenhouse gas emissions from one of Company C’s gas fired power stations. The greenhouse gas emissions are calculated using activity data (gigajoules or cubic metres of gas) and the power station’s own energy content and emissions factors calculated in accordance with Method 2 per the NGER Measurement Determination.
The auditor used a variety of techniques to gain the necessary assurance over the reported greenhouse gas emissions, including:
- Performing analytical procedures (see section 5.4.3 of this handbook) over the reported greenhouse gas emissions using historical site emissions intensity data to form the expectation. The analytical procedures showed a variance in emissions intensity for the first quarter of 2009–10 which management were unable to explain, therefore the auditor assessed that further testing was required on this quarter.
- Obtaining comfort on the NGER Activity data through substantive testing to source data (gas invoices and onsite metering). Selective testing was performed to obtain coverage over enough items to ensure the untested balance was immaterial. The testing focused on quarter 1 which contained the unexplained variance noted in the analytical procedures performed. It was found that July and August 2009 activity data had been incorrectly entered into the greenhouse gas emissions calculations, producing an error of 10 and 12 percent respectively for these months. The errors were noted on the summary of uncorrected errors (see section 5.5.3 of this handbook). Additionally the auditor considered whether errors noted were pervasive to the data for the remainder of the year. It was found that the errors related to July and August only and therefore no further testing was required.
- Obtaining comfort about the site’s energy content and emissions factor calculations through substantive testing of the source data (to site measurement systems, gas composition results from external laboratories and testing whether the equipment used had been calibrated by examining calibration certificates) and re‑performance of the calculations themselves. No errors were found during the testing to source data; however, a calculation error was noted in the energy content factor calculation which was present through all months of the year, resulting in an error of 15 per cent for the reported greenhouse gas emissions for the year. The error was noted on the summary of uncorrected errors.
- Assessing how substantive testing can be reduced in line with the comfort obtained from the results of controls testing (see section 5.4.3 of this handbook).
The auditor used a table to document a summary of the procedures performed and the evidence obtained on the assurance engagement file, to support the overall conclusions reached for the greenhouse gas emissions of the power station.
The errors noted in the summary of uncorrected errors were material individually as well as in aggregate. Accordingly the auditor communicated to management that the errors included in the summary of uncorrected errors should be remediated prior to the signing of the assurance engagement report. Refer to the guidance given below on discussing findings with management.
5.4.6 Discuss findings with management
Before commencing the reporting stage it is highly recommended that the auditor discusses the findings arising from the procedures performed during the performing stage with management.
The amount and basis for any errors and non-compliance with the relevant legislation must be noted and explained by the auditor to management. A meeting to discuss a summary of uncorrected errors (refer to section 5.5.3 below) prepared by the auditor is often the most efficient method to do so.
For section 73 to 74A audits under the NGER legislation, in discussing findings with management, where misstatements and non-compliance are identified, the Clean Energy Regulator will discuss implications and potential changes or resubmission with the audited body, once the audit has been completed.
Footnotes
5 Auditing and Assurance Handbook 2008, ASA 520 Analytical procedures