Transparency and Quality Report
2012/13
June 2013

Introduction
1.	Audit Scotland carries out financial and performance audits on behalf of the Auditor General for Scotland and the Accounts Commission. In carrying out this work, Audit Scotland seeks to be transparent in its governance and operations and to produce consistent high-quality audits.
2.	Audit Scotland's Quality Framework, which sets out the high-level principles underpinning the quality of our work, was updated in 2013 following publication of the Corporate Plan 2012-15: Priorities for 2013. The revised framework takes account of a number of national and international initiatives and publications including the International Audit and Assurance Standards Board's A Framework for Audit Quality (2013) and the Financial Reporting Council's Professional Scepticism (2012). The revised framework 2013 requires that the annual quality report for 2012/13 onwards takes the form of a Transparency and Quality Report, in line with best practice for auditors of public listed entities in the private sector.
3.	This fourth annual report therefore combines information on the overall structure and governance of Audit Scotland (transparency) with information on the arrangements in place for producing high-quality work, by Audit Scotland and the appointed firms, in the services provided to the Auditor General and the Accounts Commission (quality). The report is prepared for Audit Scotland's Audit Committee to provide assurance to it and to the board prior to the sign-off of the annual report and accounts. 
4.	Audit Scotland is a statutory body established under the Public Finance and Accountability (Scotland) Act 2000 to carry out audits and examinations for the Auditor General for Scotland and the Accounts Commission. The Auditor General audits or appoints the external auditor of most public bodies in Scotland, except for local authorities where the Accounts Commission is responsible for securing audits.
5.	The work undertaken in 2012/13 covers over 200 organisations including:
•	77 central government bodies (Scottish Government, non-departmental public bodies, agencies and others)
•	23 NHS bodies
•	32 councils
•	11 local government pension funds
•	45 joint boards and committees (including police and fire and rescue services)
•	37 further education colleges
•	Scottish Water.
6.	Audit Scotland supports the Auditor General and the Accounts Commission to ensure public money in Scotland is used properly, efficiently and effectively.  It does this by carrying out financial, performance and Best Value audits for the organisations listed above whose total annual spend is approximately £40 billion per annum. 
7.	A glossary has been included at the end of the report to assist the reader in understanding the acronyms and terms used in the report.

Transparency
Vision, values and priorities
Vision 
8.	Audit Scotland's Corporate Plan 2012-15 sets out our vision to be a world-class audit organisation that improves the use of public money. The organisation's key function is to provide independent assurance to the people of Scotland that public money is spent properly and provides value for money.
Values
9.	Audit is at the heart of the work carried out by Audit Scotland staff.  The principles which guide this work are set out in the Code of Audit Practice:
•	Auditors are independent of the organisations that they audit.
•	Public audit is wide in scope to reflect the accountabilities attached to the use of public money.
•	Audits are based on comprehensive risk assessment.
•	Reports are made public.
•	Auditors work in partnership with each other.
•	Auditors work collaboratively with other scrutiny bodies.
•	Auditors work as catalysts to help public bodies improve their performance.
•	Audits are carried out to a high standard using skilled and experienced staff.
•	Audits are informed by assessment of costs and benefits and achievement of Best Value or value for money.
•	Auditors balance confidentiality and information security with public accountability and freedom of information.
10.	During 2012, Audit Scotland refreshed its values following consultation with staff. The core values derived from these discussions were:
•	Independence and integrity: these values are integral to the credibility of the audit and it is important that auditors exhibit appropriate values, ethics and attitudes when carrying out their work.
•	Valuing people: we empower and support our staff to be engaged, highly skilled and perform well.
•	Quality: we systematically improve the quality and impact of our work - our arrangements for securing high quality are described in this report.
•	Cooperation: we work together as one organisation.
•	Great communication: we seek out and manage information and intelligence to deliver excellent audits.
Priorities
11.	The corporate plan priorities for 2013 focus on developing the quality and sustainability of our work, with the launch of specific projects as follows:
•	A review of audit expectations to help maximise the impact of our work.
•	A review of Audit Services Group's (ASG) audit methodology to streamline audit work.
•	Development of an audit intelligence function to improve the use of data and judgements made.
Governance and structure
Governance
12.	Audit Scotland is governed by a five-member board which has overall responsibility for the strategic direction of the organisation and for approving significant policies. The board oversees our work and seeks to ensure high standards of governance and management. The board members are:
•	An independent chair, Ronnie Cleland, and two independent non-executive members, Katharine Bryan and John Maclean, who are appointed by the Scottish Commission for Public Audit (SCPA).
•	The Auditor General for Scotland, Caroline Gardner who is also the Accountable Officer for Audit Scotland. She is independent of the audited bodies and was appointed by the Crown for a fixed term of eight years from 1 July 2012. 
•	The Chair of the Accounts Commission, John Baillie: the Accounts Commission is responsible for local authority audits and is independent of local government; the Chair and Commission members are appointed by Scottish ministers.
13.	The SCPA, a parliamentary body, scrutinises Audit Scotland's budget, annual report and accounts and appoints an external auditor.
14.	The board is supported by two committees:
•	An audit committee which supports the board by reviewing the internal controls, risk management processes and governance arrangements. The committee oversees the appointment of internal auditors and considers the annual report and accounts and reports from both internal and external auditors.
•	A remuneration and human resources committee which supports the board in determining the remuneration of management team members and the remuneration policies for all staff. The remuneration of the Auditor General is set by the Scottish Parliamentary Corporate Body.
15.	A management team of seven oversees Audit Scotland's day-to-day operations. The members are:
•	Caroline Gardner, Auditor General for Scotland.
•	Diane McGiffen, Chief Operating Officer, who chairs the management team meetings and is responsible for Audit Scotland's internal operations and business performance.
•	Russell Frith, Assistant Auditor General, who is responsible for the external strategic and development work of financial audit, as well as auditing and accounting standards, technical support and quality assurance; he is also our designated Ethics Partner.
•	Fraser McKinlay, Director of Best Value and Scrutiny Improvement, who is also Controller of Audit and manages the Best Value and Scrutiny Improvement Group (BVSIG).
•	Fiona Kordiak, Director of Audit Services, who is responsible for the in-house audit services provided by ASG to the health, central, further education and local government sectors; she is also our designated ethical compliance partner.
•	Barbara Hurst, Director of Performance Audit, who manages the Performance Audit Group (PAG).
•	Lynn Bradley, Director of Corporate Programmes, who is responsible for key areas of development including corporate knowledge and information management, and corporate performance reporting.
16.	Audit Scotland's stakeholders are:
•	The Scottish Parliament
•	The people of Scotland
•	The Scottish Government
•	All Scottish public organisations.
Operational structure
17.	During 2012/13, Audit Scotland was structured around five business groups as follows:
•	Audit Services Group (ASG) which carries out annual risk-based audits on a five year appointment cycle.
•	Audit Strategy whose key responsibilities are audit procurement, technical guidance and support, reviewing and reporting on audit quality, coordination of the National Fraud Initiative exercise in Scotland and delivering benefits performance audits for local government.
•	Best Value and Scrutiny Improvement (BVSIG) which carries out Best Value (BV) audits of local authorities, scrutiny improvement including shared risk assessments, Community Planning Partnership (CPP) audits and public and overview reporting. 
•	Corporate Services which includes staff in communications and media, finance, information services, human resources and organisational development, facilities management and business support.
•	Performance Audit Group (PAG) which is responsible for delivering the Auditor General and Accounts Commission's programme of national performance audits.
18.	During 2013/14, BVSIG and PAG will be combined to form a single group to focus on performance-related audit work.
19.	Appointments to carry out annual audits are split between ASG staff and seven private firms; the latter carry out just under half of these engagements. The appointments are made by the Auditor General or the Accounts Commission for a period of five years, with the current appointments ending with the 2015/16 audits. The Appendix contains a break-down of sector appointments by firm.
20.	Audit Scotland has a workforce of approximately 260 whole-time equivalent staff, over half of whom are in ASG. Staff are based at our offices in Edinburgh, East Kilbride (Glasgow from May 2013) and Inverness.
Finances
21.	Audit Scotland submits annual budget proposals to the SCPA for consideration. The SCPA takes evidence in public on the proposals and then makes a report to the Scottish Parliament as part of the annual Budget Act approval process.
22.	Audit Scotland's activities are funded through direct funding from the Scottish Consolidated Fund (2012/13 budget: £6.5 million) plus audit fees paid by the audited bodies, bank interest and miscellaneous income (2012/13 budget: £18.1 million). 
23.	We embarked on a four-year plan in 2010 to reduce the cost of audit by at least 20 per cent in real terms by 2014/15 and over the three years to the end of 2012/13, we have delivered a cumulative real term reduction in fees of 22.6 per cent.

Quality
The components of a quality audit
24.	In keeping with our vision to be a world-class audit organisation, we have revised our Corporate Quality Framework to take account of the five elements of a quality audit as set out in the International Audit and Assurance Standards Board's (IAASB) A Framework for Audit Quality, which was issued in 2013. The framework states that:

 A quality audit is likely to be achieved when the auditor's opinion on the financial statements can be relied upon as it was based on sufficient appropriate audit evidence obtained by an engagement team that:	
Exhibited appropriate values, ethics and attitudes	
Was sufficiently knowledgeable and experienced and had sufficient resources allocated to perform the audit work 	
Applied a rigorous audit process and quality control procedures	
Provided valuable and timely reports and	
Interacted appropriately with a variety of different stakeholders	
Source: IAASB, A Framework for Audit Quality, Consultation Paper, IFAC, January 2013, Para 18
25.	Our revised quality framework sets out how these important components are relevant to Audit Scotland's work. Key quality appraisal activities such as biennual audit quality surveys issued to audited bodies for feedback and internal or peer reviews of our work, provide assurance on these criteria. The remainder of this report describes in more detail how we demonstrate these elements across our business groups, through the quality control arrangements which have been put in place and the results of quality control activities during the year.
Quality arrangements
26.	As was noted in the Transparency section of this report, our staff consider quality as one of the core values underlying the work which they carry out. Our corporate plan confirms our commitment to systematically improve the quality and impact of our work. This commitment is embedded in our quality framework document which sets out the following principles:
•	Quality is the responsibility of everyone in Audit Scotland.
•	Our approach to quality will drive continuous improvement across Audit Scotland.
•	Quality will be assessed in terms of outputs, outcomes and impact, as well as inputs and processes.
27.	The overall quality framework is shown in the diagram below:
28.	Getting it right first time is key to Audit Scotland being a high-quality, efficient and effective audit organisation and requires that quality is embedded in all our thinking, our processes and our activities. Each member of staff in Audit Scotland has a responsibility to get it right first time, and needs to demonstrate a commitment to quality in everything they do.
29.	Quality control refers to quality management during an audit or project. It may involve peer review and in some cases external input. There are four principal characteristics of quality control:
•	Review at regular intervals by staff internal or external to the organisation.
•	Regular internal challenge sessions.
•	Formal assessment of adherence to local quality processes.
•	Proportionality of quality control activities to maximise the quality and efficiency of work and allow staff to retain a sense of ownership.
30.	Quality monitoring and review which ensures that all types of audit work is reviewed and lessons are learned and used to improve our processes and methodologies. The activities undertaken as part of quality control and quality monitoring and review are described in more detail in the sections below on quality arrangements and results of quality control activity.
31.	Our corporate plan priorities for 2013 confirm that we will continue to develop professional and ethical requirements in line with international developments and make sure they inform all of our judgements. We have revised our quality framework to reflect our vision and priorities and focus on professional scepticism, making reports valuable and our use of resources.
32.	Business groups are responsible for establishing the quality arrangements for their activities including the processes for getting it right first time, quality control and quality monitoring and review. Key documents are available to staff on the Audit Scotland intranet.  
33.	The Corporate Quality Group supports the maintenance of the Corporate Quality Framework and the preparation of this Transparency and Quality Report. With representatives from each business group, it coordinates and shares best practice in quality monitoring and review and supports each business group's quality control and monitoring procedures. These arrangements are described in detail in the Appendix to this report. 
34.	The following sections of the report outline the results of our quality control activity during 2012/13 and developments in audit quality arrangements taking place in each business group.
Results of quality assurance activity
Audit Services 
Internal Quality Monitoring Reviews
35.	In 2012, ASG Management Team agreed that a five-year cycle of 'cold' reviews would be developed to align with the current 2011-16 audit portfolio timescales. Cold reviews are comprehensive audit reviews undertaken by the Quality Monitoring Team (QMT) or partner-body (see below) after publication of the annual audit report. A total of eight audits were selected for coverage in this first year of the programme, based on a combination of the risk profile of the audit, planned coverage of engagement leads and a mix of sectors. 
36.	The QMT undertook six of these cold reviews in February/March 2013, with the Wales Audit Office (WAO) and Northern Ireland Audit Office (NIAO) undertaking a review each, as part of the reciprocal arrangements developed in 2012.
37.	For all eight reviews, the overall conclusion was that the audits were completed in compliance with International Standards on Auditing and ASG’s Audit Guide in all material respects. However, a number of areas for improvement were identified by the reviewers across all audits and action plans put in place to implement the improvements.
38.	The main recurring issues which came up in a number of the reviews were:
•	Late approval and closure of files.
•	Improvements required in the team risk discussion process.
•	Improvements required to the process for tracking identified risks.
•	Inadequate documentation of the sampling approach.
39.	The key messages from the reviews will form part of the annual technical update sessions for all ASG staff and will be reinforced within ASG. 'Hot' reviews for 2012/13 commenced in April 2013 and include a focus on the issues identified through the cold review process. A hot review takes place during the live audit process and examines audit judgements and risks.
Engagement Peer Reviews
40.	ASG's Audit Guide sets out the criteria for an audit to be considered for an engagement peer review under ISQC1. Twelve 2011/12 audits met the criteria for a peer review. In each case, the peer reviews were completed in a timely manner and confirmed that the audit opinions were appropriate. 
Audit Strategy
Financial Reporting Council Audit Quality Review Reports
41.	The Financial Reporting Council's (FRC) Audit Inspection Unit (AIU) carries out annual audit quality inspections. The AIU issued public reports in June 2012 for three of the seven firms which undertake audits for the Accounts Commission and the Auditor General:
•	PricewaterhouseCoopers
•	Deloitte
•	KPMG.
42.	The reports cover a review of the firm-wide systems and processes for ensuring audit quality, and a sample of their audits of public interest entities. The AIU did not include Grant Thornton in the 2011/12 review but this firm is scheduled to be reviewed in 2012/13. The AIU has also produced an annual report summarising all of its inspection work. This brings together common findings and examples of good practice.
43.	The three firms' public reports show a similar performance as the whole population inspected by the AIU, with 50 per cent of the audits sampled being assessed as good with limited improvements required, 47.5 per cent as acceptable overall with improvements required, and 7.5 per cent requiring significant improvement. The firms have formally accepted the AIU's recommendations for improvement and the AIU has recommended to the Audit Registration Committees of the relevant professional accountancy bodies that their audit registrations should continue.
44.	The results of the AIU's inspection work provide Audit Strategy with additional assurance over the quality of audit work carried out by three firms who collectively audit approximately one quarter of our audited bodies.
45.	The FRC underwent restructuring in 2012 and the AIU has been replaced by the Audit Quality Review Team which will continue to carry out the audit quality inspections for 2012/13. 
The Institute of Chartered Accountants of Scotland Audit Monitoring
46.	The three appointed firms outwith the AIU inspection scope (Scott-Moncrieff, Wylie & Bisset and Henderson Loggie) are registered by the Institute of Chartered Accountants of Scotland (ICAS) to carry out audit work and thereby fall within its audit monitoring responsibilities (as do the four larger firms covered by the AIU). ICAS's Annual Audit Monitoring Report for 2012 was reviewed and the overall outcomes from the 40 monitoring visits ICAS made to firms were analysed. Sixty-seven per cent of firms (not named) visited had satisfactory outcomes in terms of no or minimal follow-up action being requested. Twenty per cent of firms were required to submit further evidence of action taken and 13 per cent were deemed unsatisfactory (none of our appointed firms fell into this category) and have had conditions and/or restrictions imposed on them. The report reflects a robust monitoring process and provides Audit Strategy with assurance over the quality of audit work provided by all seven firms.
Firms' own quality control results
47.	Firms carry out their own internal audit reviews each year and Audit Strategy request copies of any reports issued from these activities and review them. During 2012/13, three such reports were received relating to public sector audits procured for the Auditor General or Accounts Commission. The findings from the review were satisfactory in terms of the quality of the internal review work carried out.
Review of audit outputs
48.	A sample of annual audit reports was reviewed by Audit Strategy for their compliance with the Code of Audit Practice, and as an assessment of the quality of reporting to the audited bodies.  For the 2011/12 audits, one report for each audit provider for each sector was reviewed.
49.	The key findings from the product read exercise were that reports were on the whole compliant with the Code and were of a good quality. Examples of good and poor practice will be reported back to auditors in time to be addressed for the 2012/13 annual reports.  
Audit service quality surveys
50.	Audit service quality surveys were issued to all health bodies (23) and further education (FE) colleges (31) asking for feedback on the 2011/12 audits. Response rates were 65 per cent and 52 per cent respectively.
51.	The key indicators are:
•	What audited bodies thought of the quality of service provided by the auditors.
•	Whether the audit had made an impact on the bodies in the four areas defined in our corporate impact framework (note that the FE survey was updated to reflect the changes to this framework).
•	Whether relevant national performance reports had had an impact on the body.
52.	A summary of the responses received is shown in the tables below. The feedback was significantly positive in terms of the quality of audit service delivered and the impact of audit and national performance reports. The results on impact are more mixed and are being analysed further to identify areas for attention and improvement. The survey instrument is revised every five years and therefore limited direct comparison can be made to the previous health and FE surveys from 2009/10; for those aspects where comparison is possible, the 2009/10 figures are shown in brackets.
Table 1: Health bodies
	%  positive responses
(09/10 comparators)	
Overall quality of service	100 (94)	
Impact of audit on:		
Assurance & accountability	93 (94)	
Planning & management	73 (87)	
Economy & efficiency	60 (71)	
Effectiveness & quality of services                              	67 (64)	
Overall audit impact	87 (Finance Directors)
93 (Audit Committee Chairs)	
Impact of national performance report (s)	65	
Table 2: FE colleges
	% positive responses 
(09/10 comparators)	
Overall quality of service	88 (100)	
Impact of audit on:		
Financial sustainability	50	
Transparency of reporting (financial & performance)	88	
Value for money	19	
Governance & financial management  	75	
Overall audit impact	75 (Finance Directors) 
94 (Audit Committee Chairs)	
Impact of national performance report (s)	75	

Output monitoring
53.	Table 3 below sets out the key outputs from each sector for the audits of the 2011/12 financial statements:

Table 3: Outputs
	LG	Health	CG	FE	
Audit opinion on time	30 September 12 86/88 (98%)	30 June 12  
20/23 (87%)	31 October 12
65/73 (89%)	31 December 12 1/37 (3%)	
Annual audit report on time	31 October 12  77/88 (88%)	31 July 12
23/23 (100%)	30 November 12 65/73 (89%)	31 December 12 1/37 (3%)	
Accounts sent for laying on time	N/A	31 December 12
23/23 (100%)	31 December 12
73/73 (100%)	30 April 13      35/37 (95%)	
54.	The delays in submitting audit certificates and annual audit reports in the FE sector was due to a regulatory query regarding the colleges' financial year-end which was not resolved until January 2013. The statutory deadline for laying accounts was met in all but two cases.
55.	During 2012/13, PAG delivered 14 reports against a target of 10-12 and BVSI delivered 8 BV reports against a target of 5-8.
Oversight of acceptance of non-audit work
56.	Audit Strategy oversees any requests by auditors to carry out additional work outwith the external audit. These requests must be made to Audit Strategy in writing and be accompanied by express assurance that the designated Ethics Partner has reviewed the proposed work and does not consider it to pose any threats to the auditors' independence.
57.	During the year, Audit Strategy has given approval to a small number of such requests, having assessed them against current Ethical Standards and obtaining assurance that the independence of the external auditors would not be compromised.
Respond proportionately to complaints about auditors and audit work
58.	A new complaints process was introduced by Audit Scotland during 2012/13 which conforms to the complaint handling procedures set out by the Scottish Public Services Ombudsman. Complaints are systematically analysed, investigated and reported to Management Team and the board. Two formal complaints were considered during the year: following due consideration, neither was upheld. There have been no complaints by audited bodies about the quality of work undertaken by auditors this year. This is an important element of our overall assurance on the quality of auditors' work. 
Consideration of conflicting audit judgements between auditors
59.	There have been no significant conflicting judgements between auditors this year. Guidance on key technical matters to inform auditors' judgement has been provided by the Technical Services Unit (TSU).  In addition, in the course of the year, there have been regular sectoral meetings and technical forums involving auditors from each of the four sectors where emerging or contentious technical issues were discussed.
Independent reviews of audits by Audit Strategy
60.	Monitoring of the quality of audit provision during 2011/12 has not highlighted any audits requiring an independent review by Audit Strategy. Two specific issues have emerged from events occurring in 2012/13, which will be reviewed in 2013/14 to assess whether any improvements in audit quality can be made.
Best Value and Scrutiny Improvement 
61.	The review of quality assurance (QA) arrangements for the main audit work in BVSI has consolidated the QA arrangements during 2012/13. 
62.	The existing QA arrangements for BV audits in councils, police and fire were refined to support the delivery of the audits of the three CPPs carried out during 2012/13. The lessons learned reviews, which are carried out at the end of each piece of audit work, are used to inform future audit work. 
63.	During 2012/13, members of the Accounts Commission met with representatives of ten audited bodies following the publication of audit reports. The meetings focus on key audit findings, the audited bodies’ response to those issues and feedback from the audited bodies on the audit process itself. Feedback from the audited bodies has been positive and constructive. 
Performance Audit 
64.	Partner audit agencies completed reviews of three of PAG's published reports in the year. The findings were generally positive, with particular emphasis on the effective use of clear language, the flow of recommendations from the audit findings and efforts to apply the findings and recommendations to the wider public sector. Areas for potential improvement included whether particular findings could have been given more emphasis and possible improvements to structure.
65.	PAG held one review of project reviews during the year. This looked at overall findings from all project reviews completed since PAG introduced the process. The review indicated that there had been improvements in many areas over time, but that some issues persisted – many of the persistent issues had already been identified through the 'lean' review discussed under 'developments' below, and actions identified to address them. In other cases, the review found some issues would always be present, and that awareness of them, and appropriate risk mitigation, was the best way to address these. 
66.	External expert review of three questionnaires issued in the period led to improvements in the approach used and the final questionnaire design. In all cases, the audit teams reported that responses had been secured more easily and were more complete. PAG’s management team considered the results and agreed PAG should undertake similar reviews, using external experts, for a further eight questionnaires.
Developments in audit quality arrangements
Audit Services: developments
67.	ASG has a well-established framework for delivering audits which complies with relevant auditing standards. In 2013, the opportunity is being taken to develop a lean improvement approach, aimed at improving workflow and eliminating waste in the planning and risk assessment process, while maintaining full compliance with relevant auditing standards and meeting stakeholder expectations. A short-life working group has been put in place, consisting of members of the Business Improvement Unit (BIU), Professional Standards Group (PSG), QMT and other staff providing a broad representation from across ASG, to roll out the lean review. A series of roadshows are being delivered to staff to encourage feedback on any changes to the approach before adoption of a revised methodology.
68.	During 2012, the BIU successfully adopted some changes to the overall audit approach through a review of electronic audit programmes. These changes, together with the adoption of the leaner planning and risk assessment process will be incorporated into a revised Audit Guide, with a target date for revision of October 2013.
69.	The QMT is continuing to develop its reciprocal arrangements with the NIAO and WAO and will be maintaining on-going dialogue to learn lessons from these arrangements and to explore further development opportunities. 
70.	During 2013, the QMT will review ASG's quality monitoring framework and programme of work to move beyond process to culture and judgements. ASG aims to ensure an appropriate focus and coverage of audit judgements and professional scepticism and will develop training to cover essential auditing skills. 
Audit Strategy: developments
71.	Audit quality surveys for the health and FE sectors were revised during the year. Survey questions were varied to incorporate both positive and negative phrasing and extended to elicit comments on PAG reports.
72.	Transparency reports published by the larger firms were obtained and reviewed to ensure the information within them was consistent with our own understanding of quality arrangements within the firms which provide external audit work for the Auditor General and the Accounts Commission.
73.	ASG extended its audit output monitoring to cover audit plans for 2012/13; all audit plans for the local government sector were reviewed and analysed to identify examples of good and poor practice with findings being provided to auditors.
74.	Technical guidance and practical assistance to inform the judgement of auditors was provided by the TSU. Visits to ASG super teams were arranged by Audit Strategy in order to inform audit staff about the quality work being undertaken and to disseminate initial findings from that work. Feedback from this work is also to be provided to the firms.
Performance and Best Value Audits: developments
75.	During 2013/14, the group will be reviewing a range of its core areas of work and this will include an assessment of the quality assurance arrangements as follows:  
•	For BV audits, a review of the BV audit methodology in the light of the introduction of the CPP audits.
•	For CPP audits, an independent review of the new audit approach will be completed by May 2013. This report will inform any improvements in audit approach for the next series of CPP audits. 
•	For the shared risk assessment (SRA) process, a strategic review including consideration of locality based scrutiny, information and intelligence, capacity building and the core risk assessment process.
•	An independent review of correspondence procedures will be completed by May 2013.  This report will inform any improvements in how the group deals with the changing nature of correspondence it receives.
•	For statutory performance indicators (SPIs), there will be development of the quality assurance framework for verifying public performance reporting (PPR) and SOLACE benchmarking data from 2013/14 onwards.
76.	More generally, the group will be strengthening its audit review framework during 2013/14 to provide a more consistent and comprehensive approach to learning lessons from its work to support continuous improvement.  
77.	During 2012/13, PAG undertook work on a number of improvement projects. The most significant projects were: 
•	'Lean' training and review – PAG used an external expert to facilitate a review of its performance management framework in January 2012. The purpose of the review was to identify where there may be scope for improvements in the process used by PAG to carry out its performance audits. PAG developed an action plan for 2012/13 to facilitate a number of changes and improvements. The outcome will be evaluated in the first quarter of 2013/14.
•	Data and statistical analysis – following awareness training for all staff, PAG developed specific guidance on using statistics and data, to support staff in their efforts to make effective use of both existing data and data gathered during audits.
•	Benchmarking – PAG has agreed with the other UK audit agencies a small set of indicators to benchmark performance on costs and processes. Information for 2012/13 will be collated and shared during 2013/14. 

Conclusion
78.	Overall, on the basis of the arrangements in place and activity for the year, it is reasonable to conclude that Audit Scotland and the private firms continue to provide the Auditor General and the Accounts Commission with high-quality work.
79.	In particular, the quality appraisal work carried out by Audit Strategy on the work of the firms and ASG has confirmed that each auditor:
•	understands and complies with the Ethical Standards in force during the course of the audit and is independent of the audited body
•	has the required professional competence to carry out the audit in accordance with relevant standards and the Code of Audit Practice
•	operates in a regulatory environment that actively oversees auditors
•	has delivered audit opinions that can be relied upon
•	delivered timeous reports which have met the needs of a majority of key stakeholders.
The audits undertaken by ASG and the firms would therefore satisfy IAASB's definition of a quality audit (as per section 24 above).
80.	The report demonstrates that arrangements across Audit Scotland's working groups are continuing to develop, with the aim being to ensure that our quality monitoring framework remains effective.

Appendix: Quality arrangements
This appendix summarises the quality arrangements in place across the organisation.
Audit Services: quality arrangements
ASG has had a quality framework in place since 2005, which reflects the corporate quality framework. ASG’s framework covers financial audit, ethical and quality standards as required by the FRC. The foundation of ASG’s quality framework is the ASG Audit Guide, which incorporates the application of professional auditing, quality and ethical standards together with the Code of Audit Practice, into an audit methodology which is used across all audits in ASG. The audit guide is supported by a number of guidance notes on specific issues such as audit testing guidance.
The PSG, which consists of staff members from across ASG and reports directly to ASG’s management team, oversees the development of the Audit Guide and the integration of any new standards into ASG’s audit approach. 
In addition to complying with the Audit Guide, auditors are required to complete audit engagements using a software tool called MK Insight, an electronic working paper package which allows auditors to document their work to provide evidence of compliance with relevant auditing standards. The package also incorporates appropriate levels of supervision and management. 
International Standard on Quality Control (UK and Ireland) 1 (ISQC1) requires that a system of quality control is established, as part of financial audit procedures, to provide reasonable assurance that professional standards and regulatory and legal requirements are being complied with and that the independent auditor’s report or opinion is appropriate in the circumstances. An updated ISQC1 arrangements questionnaire, describing the quality control and quality monitoring arrangements in place within ASG, was completed and submitted to Audit Scotland’s Audit Strategy Group in 2012, in support of its quality appraisal process.  
As part of the system of quality control, ISQC1 states that an engagement quality control review, known in ASG as a Peer Review, should be performed for all audits meeting certain criteria. ASG management team has set out the criteria, which include among other things the size of fee, risk, and previous or anticipated qualified opinions on the financial statements. 
Peer reviews involve discussion with the appointed auditors, a review of the financial statements and supporting information, and consideration of whether the proposed opinion is appropriate. ASG management team nominate peer reviewers from the assistant directors group to carry out the reviews – peer reviewers have no involvement with the audit in the current or recent financial years, in line with ASG’s rotation policy. 
ASG operates a rotation policy which complies with the Ethical Standards. The policy is designed to reinforce auditor independence by rotating key senior staff every five years and thus protect against threats to independence such as over-familiarity with management at an audited body.  ASG staff must complete an annual fit and proper declaration which covers time spent on particular audits as well as compliance with Audit Scotland's Code of Conduct.
The QMT, which is led by experienced senior staff, work closely with the PSG. Each year the team undertakes ‘hot’ and ‘cold’ reviews of audits. Hot reviews are carried out during the live audit process, focusing on judgements and risks and ensuring that audits are carried out in accordance with ASG’s Audit Guide. Cold reviews are undertaken after the issue of the audit report and cover the same issues, but with additional work which looks more broadly at the wider conduct of the audit including the soundness of the audit opinion and the impact on the public body. A programme of cold reviews has been developed which aims to ensure that all financial audit engagement leads are subject to review over the portfolio appointment period.  The QMT reviews enable the team to report on areas for improvement, training needs and good practice. 
A programme of external monitoring is also in place, which includes cold reviews of audits, as well as considering compliance with quality control standards. The last external cold review programme was undertaken by ICAS and was completed in February 2012. As part of this review, ICAS concluded that ASG’s quality control arrangements, in conjunction with the Audit Guide and the Code of Conduct, were considered generally effective and appropriate for the organisation. ICAS is due to conduct a further review in 2014.
For the 2011/12 cold reviews external monitoring was developed further, with reciprocal review arrangements put in place with both the NIAO and the WAO. These arrangements allowed for two of the audits selected as part of the 2011/12 cold review programme to be subjected to an independent review.  
The work of the PSG and the QMT, along with findings from external monitoring, feeds into the annual learning and development plan which incorporates mandatory annual practitioner updates for all ASG staff. These annual update sessions provide training on changes to the Audit Guide and developments in auditing and professional standards. 
Appointed firms: quality arrangements
At the start of the five year audit cycle in 2011/12, all the appointed audit firms were also required to complete a detailed questionnaire setting out how their quality arrangements complied with ISQC1. As part of this initial process, the firms and ASG submitted details of their internal quality monitoring activity for the audits which they would carry out under their appointment by the Auditor General for Scotland and the Accounts Commission. 
Where firms plan to undertake any non-audit work for the audited bodies to which they have been appointed, they must declare to Audit Strategy that they consider such work permissible under Ethical Standards. Audit Strategy reviews any such assertions and permits non-audit work only where it agrees that the work is consistent with Ethical Standards. This review enhances the independence and ethical conduct of the audits.
The firms involved in auditing bodies under appointment from the Auditor General and the Accounts Commission in 2012/13 are:

No of audit appointments:
Firm	Central Government	Local Government	Health	Further Education	Total	
KPMG LLP	9	9	0	6	24	
PWC LLP	10	7	4	0	21	
Scott-Moncrieff	1	2	7	7	17	
Grant Thornton UK LLP	5	6	0	4	15	
Deloitte LLP	1	6	3	0	10	
Henderson Loggie	0	0	0	6	6	
Wylie & Bisset LLP	0	0	0	6	6	
Total	26	30	14	29	99	
Audit Strategy: quality arrangements
Audit Strategy carries out a quality appraisal function for all financial audits conducted under appointment from the Auditor General for Scotland and the Accounts Commission by the firms and ASG. This work is described in detail in Audit Strategy's Quality Appraisal Framework which was approved in May 2012.
The principal objectives of the quality appraisal work are to:
•	provide assurance to the Auditor General and the Accounts Commission on the quality of audit work undertaken
•	promote improvements and good practices in auditing.
Audit Strategy carries out a number of activities in its monitoring role:
•	Reviews firms' and ASG's quality arrangements including Transparency reporting by firms.
•	Reviews external assurance, in particular reports issued by the FRC's Audit Quality Review team (formerly the AIU) on professional firms as well as monitoring reports produced by ICAS and ICAEW.
•	Reviews and assesses audit outputs, giving feedback to firms on examples of good and bad practice.
•	Issues and reports on audit quality surveys, audited bodies in each sector being asked for their views every two years and findings being relayed to auditors and to the management team.
•	Monitoring audit outputs for timeliness and completeness.
•	Oversight of acceptance by firms of non-audit work.
•	Responding proportionately to complaints about auditors.
•	Considering the impact of conflicting audit judgements between auditors.
•	Independently review audits.
Each of these activities is expanded upon in the 'Results of Quality Control Activity' section in the main body of the report.
None of the measures in place provides absolute assurance for any of the elements of the quality appraisal framework.  However, absolute assurance cannot be gained, nor is it an aim of the framework to do so.
Best Value and Scrutiny Improvement: quality arrangements
Best Value and Scrutiny Improvement's (BVSI) work is managed under a Best Value Quality Framework. The framework covers the wide range of work covered by the group including:
•	Shared Risk Assessment (SRA) process
•	Best Value (BV) audit work 
•	Audits of Community Planning Partnerships (CPPs)
•	Joint Accounts Commission/Auditor General for Scotland reports
•	Statutory reports
•	The local government overview report
•	Reports in the ‘How Councils Work’ series
•	Statutory performance reporting
•	Correspondence.
The framework is supported by a range of processes, procedures and guidance, including BV toolkits, which provide guidance and direction for staff carrying out BV audit work. The BV toolkits are also available on the Audit Scotland website to ensure openness and transparency and help support improvement activity within audited bodies.
During 2012/13, BVSI oversaw the production of a national scrutiny plan and 32 assurance and improvement plans. The group also delivered eight BV audit reports (including councils, police boards and CPPs), five overview/ thematic reports and three statutory reports. All of these complied with the quality framework.
Much of the audit work is risk based and is informed by the SRA process. This process is carried out by Local Area Networks comprised of officers from Audit Scotland, the appointed auditor, Education Scotland, the Care Inspectorate and the Scottish Housing Regulator. The SRA process has an established QA process which involves a centralised review, factual accuracy checking by the audited body and referral to a QA panel where appropriate. The SRA results in a rolling three-year Assurance and Improvement Plan for each council which sets out the risks and the scrutiny response to those risks.
All audit work is scoped in advance and is subject to QA review, both internally by senior managers, the Accounts Commission and/ or the Auditor General and external peer reviewers where appropriate. Audit work is carried out in accordance with set procedures and audit judgements and draft reports are subject to review and challenge using the same process used for agreeing the scope of the work. Audited bodies always have the opportunity to check draft audit reports for factual accuracy. National and thematic reports are subject to review and sign off by the Accounts Commission and/or the Auditor General for Scotland as appropriate at the scoping stage and prior to publication. 
During 2011/12, the Cabinet Secretary for Finance, Employment and Sustainable Growth invited the Accounts Commission to oversee the development of an audit to assess the effectiveness of CPPs. In response, Audit Scotland developed a CPP audit framework and published audit reports on three CPPs and a thematic report on what the three audits found.  
There are robust quality assurance measures built into the CPP audit work which follow the well-established arrangements for the BV audit work. These include a Quality and Consistency Review Panel at the scoping stage and report review stage. These panels involve review and challenge from senior Audit Scotland managers who are external to the audit team and by senior managers/external consultants who are external to Audit Scotland. 
The audit teams carry out a ‘lessons learned review’ following the audit work to identify potential improvements to the audit process for future work. These are discussed at BVSI meetings. Members of the Accounts Commission meet with representatives of councils and CPPs two to three months after the publication of the audit reports. This provides another opportunity for further feedback to inform future audit work. 
Corporate Services: Quality arrangements
All areas of Corporate Services are subject to internal audit as part of a programme of audit approved by the board. Actions for improvement are recommended by the auditors and these are tracked to ensure implementation.
Benchmarking has been used in Corporate Services for several years: the performance indicators used are based on the sets published in the UK Audit Agencies' joint report 'Value for Money in Public Sector Corporate Services'. In addition, since 2007/08 Audit Scotland has taken part in the Scottish Government benchmarking for Central Government and Non Departmental Public Bodies.
A style guide ensures that all reports that are published follow Audit Scotland's brand style.  Established procedures cover all aspects of the desk top publication process.  Internal guides set out media processes with media and parliamentary monitoring taking place.
Performance Audit: quality arrangements
Performance Audit Group (PAG) seeks to ensure the quality of its work through three primary documents: Performance Audit Standards, the Project Management Framework, and the Performance Audit Manual. These support and complement each other, together providing a quality assurance framework for performance audit work in PAG. They are supported by further specific guidance, all of which are under regular review.
The Performance Audit Standards, which comply with INTOSAI standards and guidance for performance auditing, set out the expectations for all PAG projects and provide a summary of good practice for specific project stages (eg, writing and delivering reports). These standards include a specific standard on quality assurance.
The Project Management Framework is designed to support consistent standards of project management in PAG. It outlines the key stages of a performance audit and includes a number of actions and outputs that provide quality assurance. One of the requirements is for each project team to carry out internal peer reviews at two key stages of the audit process. These peer reviews provide an opportunity for robust challenge by other staff (who are not involved in the audit), and reviews are undertaken of both the project brief and the draft report of each audit. Each project team also carries out a post-project review for all audits, with the assistant director and director, where they discuss the key project stages with a view to identifying potential good practice and/or lessons.
The Performance Audit Manual sets out the basic principles for performance audit work (eg, what is a value for money audit), and provides practical guidance and support for implementing key stages of the Framework.  PAG is currently reviewing the manual, drawing on user experience, to determine whether and how it can be improved. 
In addition to these core tools, PAG also operates two further processes to support the effective application of guidance and to support continuous improvement. PAG uses external experts to review the questionnaires that it sometimes uses to gather information, prior to them being issued. The reviews have led to improvements, including better questions and shorter surveys. Secondly, PAG hold regular “reviews of project reviews”, where issues identified in recent post-project reviews are discussed by the group, along with potential improvements and/or solutions.
PAG has arrangements, established in 2010 with the National Audit Office (NAO), WAO and NIAO, to review three to four of PAG’s published audit reports (cold reviews) each year. PAG reviews reports from each of the other audit agencies as part of the reciprocal arrangements. 

Glossary
AIU - Audit Inspection Unit, part of the FRC's Professional Oversight Board, responsible for monitoring the audits of all listed and other major public interest entities. Following restructuring of the FRC in July 2012, the work of the AIU has been passed to the Audit Quality Review Team.
APB - The Auditing Practices Board which operated under the Financial Reporting Council and was responsible for producing auditing and ethical standards for audit practice in the UK and Northern Ireland. The FRC was restructured in July 2012 and the APB has been replaced by an Audit and Assurance Council.
ASG - Audit Services Group, part of Audit Scotland with responsibility for carrying out audits of public bodies falling within the remit of the Auditor General and the Accounts Commission.
BIU - Business Improvement Unit, part of Audit Services Group, responsible for managing a wide range of ASG projects and initiatives such as MK Insight (our electronic working paper package).
BV - A duty of audited bodies or accountable officers; it is defined in statute for local authorities as continuous improvement in the performance of functions. In securing Best Value local authorities are required to balance issues of quality and cost, have regard to efficiency, effectiveness, economy and the need to meet equal opportunity requirements, and contribute to the achievement of sustainable development.  A BV audit is one which examines BV and Community Planning.
BVSIG - Best Value and Scrutiny Improvement Group, part of Audit Scotland with responsibility for BV, scrutiny co-ordination and overview reports across all sectors.
CPP - Community Planning Partnerships, a process by which councils and other public bodies work together, with local communities, businesses and voluntary groups, to plan and deliver better services and improve the lives of people who live in Scotland. It was given a statutory basis by the Local Government in Scotland Act 2003. BVSI conducts audits on the effectiveness of CPPs. 
Ethical Standards - ethical standards originally set by the APB and applicable to auditors of financial statements. The standards are a set of basic principles and essential procedures together with related guidance in the form of explanatory and other material covering the integrity, objectivity and independence of auditors.
FRC - Financial Reporting Council, the UK’s independent regulator responsible for promoting high quality corporate governance and reporting to foster investment. It monitors and enforces accounting and auditing standards and oversees the regulatory activities of the professional accountancy bodies and operates independent disciplinary arrangements for public interest cases involving accountants and actuaries.
IAASB - The International Audit and Assurance Standards Board which is an independent standard-setting body that serves the public interest by setting high-quality international standards for auditing, assurance, and other related standards, and by facilitating the convergence of international and national auditing and assurance standards.
ICAEW - The Institute of Chartered Accountants in England and Wales, a professional accountancy body established in 1880. ICAEW regulates members and firms.
ICAS - The Institute of Chartered Accountants of Scotland, the first professional accountancy body, it was established by Royal Charter in 1854. ICAS regulates members and firms.
INTOSAI - The International Organisation of Supreme Audit Institutions operated as an umbrella organisation for the external government audit community.
ISA - International Standards on Auditing, the professional standards for the performance of financial audit of financial information. ISAs are issued by the International Federation of Accountants (IFAC) through IAASB and are approved in the UK by the FRC.
ISQC1 - International Standard on Quality Control 1, the professional standard for quality control, as with the ISAs, issued by OFAC through IAASB.
NAO - The National Audit Office is responsible for auditing the accounts of all Westminster led government departments and a wide range of other public sector bodies, and has statutory authority to report to Parliament on the economy, efficiency and effectiveness with which departments and other bodies have used their resources.
NIAO - Northern Ireland Audit Office, responsible for auditing central government and local government in Northern Ireland; it also carries out value for money audits, reporting to the Northern Ireland Assembly.
PAG - Performance Audit Group, part of Audit Scotland with responsibility for carrying out performance audits across the public sector.
PPR - Public Performance Reporting, the duty on local government bodies to report SPI information on an annual basis.
PSG - Professional Standards Group, part of Audited Services Group responsible for overseeing the development of the Audit Guide and the integration of new standards into ASG's audit approach.
QA - Quality appraisal.
QMT - Quality Monitoring Team, part of Audited Services Group responsible for carrying out internal hot and cold reviews of ASG audit work.
SCPA - Scottish Commission for Public Audit, a Parliamentary body which is responsible for scrutinising Audit Scotland's budget, annual report and accounts and for appointing an external auditor.
SOLACE - the Society of Local Authority Chief Executives and Senior Managers, tasked with developing a comparative benchmarking framework for Scottish local government. 
SPI - Statutory Performance Indicators for local government, underpinned by the Local Government Act 1992. Some SPIs are prescribed, others are chosen by local authorities as relevant benchmarks to be monitored and disclosed. Publication of SPIs is monitored by auditors.
SRA - Shared Risk Assessment, a process involving a joint approach using key information about a body to plan scrutiny activity that is proportionate and based on risk. SRA is undertaken by a joint scrutiny network of senior officers from a range of audit and inspection agencies including Audit Scotland, Education Scotland, The Care Inspectorate, The Scottish Housing Regulator and the appointed auditors, leading to the preparation of an assurance and improvement plan (AIP), part of which may be the conduct of a BV audit.
TSU - Technical Services Unit, part of Audit Strategy Group, responsible for providing authoritative guidance and practical assistance to external auditors appointed by the Accounts Commission and Auditor General to support them in carrying out their responsibilities under the Code of audit practice. TSU's support enhances auditor judgement in technical matters and thereby improves the quality of the audit delivered.
WAO - Wales Audit Office, either directly audits Welsh public bodies, such as the Welsh Assembly and the NHS or, as in the case of local government, appoints auditors to do so.