Scottish Executive: supporting new initiatives
How Government Works series

Prepared for the Auditor General for Scotland
November 2005

Auditor General for Scotland
The Auditor General for Scotland is the Parliament's watchdog for ensuring propriety and value for money in the spending of public funds.

He is responsible for investigating whether public spending bodies achieve the best possible value for money and adhere to the highest standards of financial management.

He is independent and not subject to the control of any member of the Scottish Executive or the Parliament.

The Auditor General is responsible for securing the audit of the Scottish Executive and most other public sector bodies except local authorities and fire and police boards.

The following bodies fall within the remit of the Auditor General:

• departments of the Scottish Executive eg, the Health Department

• executive agencies eg, the Prison Service, Historic Scotland

• NHS boards 

• further education colleges

• Scottish Water

• NDPBs and others eg, Scottish Enterprise.

Audit Scotland is a statutory body set up in April 2000 under the Public Finance and Accountability (Scotland) Act 2000. It provides services to the Auditor General for Scotland and the Accounts Commission. Together they ensure that the Scottish Executive and public sector bodies in Scotland are held to account for the proper, efficient and effective use of public funds. 

Foreword
In November 2002, the Auditor General published How Government Works in Scotland. The report explained the organisation of government in Scotland, the responsibilities of public servants and how they are held accountable, and the role of public audit in both  holding public bodies to account and supporting democratic scrutiny.  The report did not analyse or assess accountability mechanisms in place but was to be seen as a source of reference.

This report is one of a series which builds on the original reference work. Each of the reports in the series will comment, in detail, on specific aspects of the business of the public sector in Scotland which cut across a range of organisations.  The reports comment on the way the cross-cutting issues are handled by different parts of the public sector and the impact they have on the delivery of services.

Part 1. Introduction

Summary of key messages

1. This report sets out the findings of an examination, undertaken by Audit Scotland, of the arrangements the Scottish Executive has in place to manage and monitor funding allocated to initiatives. It is based on a sample of 20 initiatives for which funding was announced during 2004, representing total expenditure of more than £1.4 billion.

2. Half of the initiatives complied with most important aspects of good practice and most of the initiatives complied with the majority of aspects. But there is scope to improve arrangements in a number of areas, most notably setting objectives, agreements with delivery partners, and monitoring and evaluation arrangements. Guidance may help to bring about these improvements.

3. Delivery partner experiences of arrangements varied substantially. They cited a number of specific concerns with arrangements, including the adequacy of consultation, difficulties with the short-term nature of funding and uncertainty around long-term funding arrangements.

Background

4. During 2004, the Executive announced funding for 74 projects or schemes to address specific policy objectives. These projects or schemes are not necessarily identified separately in the Executive’s spending plans as set out in the annual Budget Act. Total expenditure for these initiatives will exceed £1.4 billion, although the level of funding and the length of time over which it will be distributed varies between individual projects.
    
5. The objectives of the initiatives varied widely and included supporting community regeneration, developing sport facilities, preventing youth crime and supporting new businesses. Most of the funding is distributed through a range of partners, including local authorities and the voluntary sector, who are responsible for achieving the outcomes which meet the policy objectives. In some cases, more than one tier of partner organisation is involved.

6. The number of initiatives, and the significant funding associated with them, signals the importance the Executive attaches to this type of arrangement to deliver specific policy objectives. The money allocated is additional to funding provided to local authorities and others for core services, and is frequently ring-fenced. It is therefore important that appropriate arrangements are in place to control the use of funds involved, to manage any risks associated with the projects and schemes, and to monitor the results achieved against the policy objectives. It is also important that partners are effectively engaged and provided with an appropriate level of support. 

7. Initiatives are frequently targeted at national priorities but delivery partners also work to meet local needs. Groups such as Community Planning Partnerships (CPPs) use a range of funding, including that associated with initiatives, to address local priorities. Unless national and local priorities are clearly matched, it may be difficult for partnerships to deliver their vision for local communities. 
  
The purpose of this report

8. This report examines the arrangements to secure economy, efficiency and effectiveness from funds associated with a sample of the projects and schemes for which funding was initiated in 2004. It does not evaluate the efficiency and effectiveness of individual projects, but focuses instead on the extent to which the arrangements put in place by the Executive provide assurance about the progress and success of the projects. It also looks at delivery partners’ experience of the process.   
      
9. The funding for these initiatives is often distributed to local authorities, who may work with other partners to deliver the objectives. Audit Scotland is currently undertaking work on behalf of the Accounts Commission to both examine how local authorities control funds passed to other delivery partners and assess the progress made by CPPs. These studies will complement the findings of this report. 

Approach and methodology

10. We compared the arrangements for a sample of 20 initiatives against good practice identified by Audit Scotland, taking account of potential risk areas. The areas of good practice we examined were: 

• project strategy, objectives and planning 
• development of costed project proposals 
• delivery arrangements 
• formal agreement or contract 
• monitoring and evaluation.

11. The sample of 20 initiatives was selected to reflect the wide range of funding associated with individual projects and to represent the distribution of projects across Executive departments. 

12. We also asked the Executive’s delivery partners for their views on six important areas for partnership:

• Consultation on the project. 
• The application process.
• Roles, responsibilities and expected outcomes.
• Reporting arrangements.
• Timing of funding.
• Impact for their organisation.
13. We contacted 107 different organisations, including local authorities, health organisations, Non-Departmental Public Bodies (NDPBs), universities and community and voluntary organisations. We received responses from 76 organisations,1 a response rate of 70%. 

1. In some cases, local authorities’ views were sought on more than one project.

Part 2. Findings

Introduction

14. This part of the report gives an overview of the extent to which the sample initiatives complied with good practice and met partners’ expectations.

Overview

Compliance with good practice
15. All of the initiatives complied with most aspects of good practice, and none of the failures to comply gave rise to significant unmanaged risk. However, for nine of the 20 projects, weaknesses in the arrangements left some unmanaged risk. The nine projects comprised:

• one of the five projects with associated expenditure exceeding £40 million
• three of the six projects with associated expenditure between £10 million and £40 million
• five of the nine projects with associated expenditure below £10 million.
16. The degree of control applied to a project needs to take account of the expenditure and the strategic priority involved, and most of the cases with some unmanaged risk are lower value. Nevertheless, four projects with expenditure above £10 million are exposed to risk beyond that normally controlled by the good practice we identified.
  
17. The Community Regeneration Fund (CRF) is the highest value initiative in our sample and illustrates a number of the challenges the Executive faces in developing appropriate controls. The fund replaces three previous funding streams and has been established to bring improvements to Scotland’s most deprived areas and to help individuals and families escape poverty. Effective community regeneration is a complex process involving many different types of projects and many agencies. To secure its CRF funding, each Community Planning Partnership must submit a three-year Regeneration Outcome Agreement (ROA) for approval by Scottish ministers. The ROA must demonstrate clear links between local outcomes and the five national priorities for community regeneration, and set out how the CPP will use the CRF alongside partner resources, through a strategic, needs-based approach, rather than a project-led approach.

18. The arrangements the Executive is using to control the fund build on those previously used. However, our review indicates scope for greater clarity in setting objectives and measures of success. We also identified a need to put in place arrangements for an evaluation that will provide evidence of the extent to which local delivery has contributed to the national priorities.

19. Scottish Executive staff involved in some projects felt the need for more guidance, particularly in the following areas:

• An index of all available guidance. 
• Some guidance on what works well/does not work well in particular circumstances.
• Signposting/Where to go for help.
• A checklist of the main documents required for the various stages in a project eg, consultation, submission to ministers, bidding process, etc.
• Templates for offer letters and agreements/contracts.
Responses from delivery partners
20. Delivery partners were generally more positive than negative about roles, responsibilities and expected outcomes, reporting arrangements, and the impact for their organisation. But more were negative about consultation, the application process and timing of funding. Responses varied by type of delivery partner, with 
local authorities responding more negatively than other organisations. 

21. Just over half of respondents offered a positive comment about the impact of the project on either the organisation, or on the services the organisation provided. But a third of responses expressed concerns about the impact on the organisation. Of these, half felt that the costs outweighed the benefits. This concern covered the process of applying, monitoring and reporting, and the implementation of the project, suggesting that the Executive should ensure an appropriate balance between sufficient controls to provide assurance and the potential burden on delivery partners.

22. Five local authorities indicated that the overall number of initiatives announced by the Executive caused difficulties. 

Part 3. Specific aspects of control

Introduction

23. This part of the report examines the arrangements in place under the main headings of our good practice principles, and summarises partners’ views on related aspects of the arrangements.
 
Project strategy, objectives and planning

24. Initiatives need to fit clearly with the overall strategic priorities of the funding organisation. If the project is to be successful, clear, specific objectives should be set and implementation should be planned effectively. Effective consultation with stakeholders, to gather views and to gain support, is an important part of the planning process as it should help to ensure smooth implementation.

25. The initiatives we examined did not always have a clear basis for monitoring progress. Objectives had 
been set for all of the cases we examined and were linked to the Executive’s policy. In 14 cases, the measures of success were defined in a way which would allow progress against objectives to be monitored. But in the remaining six cases, the basis on which the initiative’s future success would be judged was not clear . 

26. In all cases, there was evidence that the Executive had undertaken consultation with stakeholders prior to the announcement of the initiative. Consultation was undertaken to: establish the level of support for projects, identify areas of concern, and identify potential methods of implementation. We also found evidence that the Executive had considered both the skills likely to be required of delivery partners and the types of organisation that were likely to be well-placed to deliver the required outcomes in each case.
  
27. However, a third of the responses indicated that the partner had not been consulted or that they felt consultation was inadequate, because it was limited or because there was insufficient time to respond. Some delivery partners also commented on time constraints on the consultation process, particularly where they had to consult with other groups or organisations before they could finalise responses.
 
28. Where respondents were positive about the Executive’s consultation, the most common contributing factors were:

• conducting workshops and/or face-to-face meetings with delivery partners
• allowing sufficient time for delivery partners to consult with third parties or other partners eg, where a project was likely to require a multi-agency approach.   
Development of cost proposals

29. There is not a single process for allocating funds to initiatives. The allocation in each case reflects a mix of factors, including the availability of funds at any particular time and the extent to which it has been possible to determine in advance the cost of meeting objectives. The majority of the initiatives we examined 
had their funding determined before full proposals were developed.

30. The Executive requires new spending proposals to be supported by an assessment of the expected impacts and value for money. It has developed a ‘Pre-expenditure assessment’ (PEA) guide for staff to use. Although the PEA was not introduced until December 2004, we found that approval submissions for the projects we examined generally included information which complied with PEA expectations. However, some project managers were not aware of the PEA and others were unclear as to when it should be applied.      

31. We found five cases where project managers had consulted the Executive’s Finance Department and also found examples of advice being sought from other specialists, such as economists, statisticians, the legal team and the internal audit team. 

Delivery arrangements

32. Delivery arrangements cover the process from announcement of the project and provision of guidance on the application or funding distribution process through to funding. A clear and consistent approach to alert potential partners and to assess partners’ funding requirements is necessary to ensure appropriate levels of funding are distributed to those most likely to deliver results. 

33. All projects examined were announced or publicised using the Executive’s news release system. Because potential delivery partners had been identified at the planning stage, announcements were accompanied, or preceded, by direct contact with delivery partners. However, one in seven responses expressed concerns 
about delays in issuing guidance or slow responses to applicants.

34. Providing appropriate support 
can help ensure delivery partners are well-prepared to deliver project objectives. Examples of good practice 
in providing support included comprehensive guidance – delivered through meetings, workshops and the 
internet – examples of outcome measures, and good feedback systems to improve the quality of applications for funding.

35. Funds were distributed to partners in several ways. In the majority of cases, the Executive set 
up an application process which required potential partners to make a case based on the level of contribution 
the partner expected to make to the policy objectives, while in others, the Executive developed models which allocated funding by reference to predetermined criteria such as deprivation levels. In a few cases, the funding distribution process was a combination of these.
   
36. Guidance and support was made available to potential delivery partners where there was an application process for funding, though in two cases the guidance was not available when the fund was launched. As a result, partners were unable to develop applications for the Executive to assess on timescales consistent with project objectives. 

37. In all cases, the Executive had developed a clear process for assessing bids. We noted, however, that bid assessment and scoring models appear to be developed separately for each project. This suggests that staff may be ‘reinventing the wheel’ and duplicating efforts. Given that processes will be similar for most funding arrangements, there is likely to be scope to develop standard models, which could improve both efficiency and consistency.

38. Nearly half of the responses were positive about the application process, citing the clear and straightforward process or the support provided by Executive staff. Most negative comments related to the amount of time available to prepare and submit an application for funding, deadlines being changed mid-process, and the level of detail or bureaucracy associated with the process. 

39. A third of the responses were positive about the timing of funding, and particularly the relatively new ability to carry over funding.

40. Concerns about the timing of funding included: 

• Difficulties in trying to spend the funding in the required timescale.
 One respondent commented:
	
“...we had to spend our first year’s funding by March 2005. Since this was only six months rather than a year we tried to find out from the Scottish Executive if any under-spend could be carried into next year, however we didn’t really feel confident in the response that we received. So we decided to just try and spend the money.”
• Problems fitting applications for funding into the organisation’s internal funding or approval processes and/or alignment with internal budget-setting. 
• The relationship with other funding streams for similar projects. 
41. Concerns were also expressed about the short-term nature of funding, particularly where the objectives were to address long-term problems. Where new funding introduces or enhances services, creating an expectation among service users, difficulties can arise if funding runs out and the service is discontinued or the service provider has to re-prioritise other work. Uncertainty about future funding can also cause difficulties for delivery partners, for example, when trying to recruit staff.  

Agreement or contract

42. For the cases we examined, the funding provided to individual delivery partners ranged from awards of a few thousand pounds to some well in excess of £1 million. A formal agreement or contract between the funding provider and the delivery partner, clearly setting out roles, responsibilities and expected outcomes, provides an important reference point for both parties. It can provide a basis to monitor progress and, should something go wrong, it could help to avoid unnecessarily protracted disputes.

43. A formal agreement was in place for 16 of the 20 projects we reviewed. Of these, ten met all aspects of good practice, while the other six had an agreement that met only some aspects of good practice. Of the remaining four, three did not have a formal agreement or contract, and instead used a signed offer and acceptance letter as the basis of an agreement with delivery partners. These agreements did not include detailed terms and conditions for the funding arrangement, and relied upon separate documents such as the initial bid or bid guidance. The risk that conditions are missed out or not met is greater if important information is distributed across different documents. The remaining project had not yet reached the stage where a formal agreement was required.  

44. Nearly half of the delivery partner responses were positive about roles, responsibilities and outcomes, with most indicating that these aspects were clear or well-defined. However, just under a quarter of the responses mentioned that roles, responsibilities and expected outcomes were unclear, unrealistic or inappropriate.

Monitoring and evaluation

45. It is important that funds are used for their intended purpose, and that the success of each initiative is evaluated. Undertaking evaluation at the end of a project leaves a risk that problems are identified too late to take corrective action, or to plan for modifications to achieve objectives. Monitoring can provide assurance that projects are proceeding to plan and can identify where projects may need remedial action. Effective evaluation provides assurance that projects have achieved the intended objectives and can help inform future plans. 

46. Requirements for monitoring were contained either within offer letters or within agreements/contracts issued to delivery partners. Twelve of the projects had a monitoring framework that complied with good practice and a further four had some monitoring in place, but the arrangements did not comply with all aspects of good practice. Three projects had not finalised arrangements at the time funds were paid out. In the remaining case, the initiative was not yet at the stage where monitoring was required because no funds had been released. 

47. The Executive’s finance manual recognises the benefit of specific audits of payments under projects or schemes delivered by partners to provide assurance on financial control. However, the manual is not prescriptive about the circumstances or risk conditions where such audit should be commissioned. We found that, although audit had been considered as a monitoring tool in each of the five high-value initiatives we examined, audit arrangements were in place for only two. In one of these cases, inspection by EU auditors was a condition of European funding and was not optional.
 
48. Nine of the projects we examined had a clear evaluation framework in place. A further seven had a framework partially in place. For the remaining cases, no clear plans existed. These projects were at a relatively early stage of implementation and the Executive expected to develop evaluation plans as the schemes progressed. 

49. Six of the 20 projects examined had in place, or were developing, outcome-based agreements. In these 
cases, where delivery partners and the Executive agree outcomes, measures and targets as part of the application process, this provides a clear baseline against which to judge success. However, it is important to ensure that the success of individual projects can be aggregated in such a way as to enable the overall success of the initiative to be measured.

50. Two-fifths of respondents commented positively on reporting arrangements, with most of these indicating that the arrangements had been agreed and were clear.

51. A third of responses included a negative comment, with around half indicating they were unclear about what and when they had to report. Others expressed concerns about the level of detail or information required or the format in which it was to be provided. Concerns were raised not only where delivery partners felt the requirements to be too detailed, but also where they felt arrangements were insufficient. Other responses suggested that the timescales for submission of monitoring information were too short, and affected the quality of reports. 

Part 4. Conclusions and recommendations
52. The arrangements in place for most of the 20 projects and schemes we examined offer assurance that funds will be directed to projects that will contribute to the overall objectives. In most cases, the arrangements will also provide a basis for judging the efficiency and effectiveness of individual initiatives. However, a few projects could be at risk because of inadequate controls. The Executive should require project managers to ensure that:

• measures of success are defined in a way that will allow monitoring of progress against objectives
• formal agreements with delivery partners, where appropriate, cover all important aspects of delivery, including a clear indication about both the level and the duration of funding
• plans for monitoring and evaluation, including milestones, are clearly defined at the point delivery partners are engaged.    
53. There is scope to improve corporate guidance, and compliance with the guidance, specifically on setting objectives, formal agreements and monitoring. The Executive should:

• review and, where appropriate, improve guidance for staff to encourage consistency and good practice
• raise staff awareness of guidance and promote compliance
• encourage staff to make better use of individual departments’ existing experience of projects, and to share good practice. 
54. Funding for the projects and schemes we examined are generally welcomed by delivery partners, but 
some of the arrangements for managing and monitoring can cause difficulties. The Executive should consider the following actions to ensure consistent, effective engagement with delivery partners: 
    
• Consultation on new projects should be effectively undertaken with potential delivery partners, as part of stakeholder consultation. Sufficient time should be allocated to allow delivery partners to consult with other partners.
• Guidance for delivery partners should be provided in sufficient time to allow robust applications to be made.
• As with consultation, sufficient time should be provided to allow delivery partners to effectively involve other partners.   
• The implications of individual funding arrangements for the core work of delivery partners should be carefully considered.  
• The implications of short-term funding for both delivery of overall objectives and project sustainability should be carefully considered. 
Appendix 1. Principles of good practice for managing and monitoring funding arrangements
Project strategy, objectives and planning

• A clear strategy and objectives. 
• Success criteria which link objectives to desired outcomes.
• Identifying stakeholders and ensuring that an appropriate level of consultation has taken place and their support obtained.
• Evaluating the initiative for strategic fit, feasibility and identifying required skills for delivery.
• Project planning arrangements for managing the initiative’s progress.
Development of cost proposals

• A clear methodology determining the costs of achieving the initiatives’ objectives.
Delivery arrangements

• Clear and effective announcement of the initiative to potential delivery partners.
• Providing clear guidance to delivery partners so they can align the project and funding claims to fit the objectives.
• A clear methodology for assessing bids to ensure they meet agreed criteria.
• Clear allocation of funds to delivery partners.
• Assurance obtained over delivery partners' ability to deliver and their project planning.
• Documented approval of successful bids.
Agreement or contract

• A formal agreement or contract has been setup between the Executive (funder) and the delivery partners who are in receipt of funding and are responsible for delivery against established objectives. This sets out:
• objectives and purpose of funding
• expected outcomes
• terms and conditions of funding
• roles and responsibilities
• payment arrangements and whether funds are ring-fenced
• monitoring arrangements and outcome assessment
• actions in event of a breach of conditions and penalties
• requirements for audit and/or need for records and expenditure controls.
• A signed agreement is received from the delivery partners.
Monitoring and evaluation

• An agreed framework setting out monitoring and reporting arrangements, including financial, activity and progress against objectives and milestones (at both initiative and project level).
• Efficient use of audit as a method for obtaining assurance that funds are managed robustly and spent appropriately.
• A clear set of outcomes for the initiative and individual projects, and a framework for measuring both their attainment and the impact of the initiative overall. 

If you require this publication in an alternative format and/or language, please contact us to discuss your needs. It is also available on our website: www.audit-scotland.gov.uk

Audit Scotland 
110 George Street 
Edinburgh EH2 4LH 

Telephone 
0131 477 1234 
Fax 
0131 477 4567
 
www.audit-scotland.gov.uk 

ISBN 1 904651 92 5

AGS/2005/7