Informed to care
Managing IT to deliver information in the NHS in Scotland

Prepared for the Auditor General for Scotland
November 2006

Auditor General for Scotland
The Auditor General for Scotland is the Parliament’s watchdog for ensuring propriety and value for money in the spending of public funds.

He is responsible for investigating whether public spending bodies achieve the best possible value for money and adhere to the highest standards of financial management.

He is independent and not subject to the control of any member of the Scottish Executive or the Parliament.

The Auditor General is responsible for securing the audit of the Scottish Executive and most other public sector bodies except local authorities and fire and police boards.

The following bodies fall within the remit of the Auditor General:

•	departments of the Scottish Executive eg, the Health Department
•	executive agencies eg, the Prison Service, Historic Scotland
•	NHS boards
•	further education colleges
•	Scottish Water
•	NDPBs and others eg, Scottish Enterprise.

Summary
1. The planning and delivery of NHS services need to be underpinned by good information to ensure that patients get the best possible care within the resources available. Better information supports better care. The NHS in Scotland is undergoing large scale change: new structures such as NHS boards and Community Health Partnerships (CHPs) have been put in place; new staff contracts are being implemented; and the way forward for service delivery has been set out in Delivering for Health.1 The need for the NHS in Scotland to work in partnership with others, including local authorities, to plan and deliver joint services where appropriate is an additional challenge. It is important that the arrangements for providing information through Information Management and Technology (IM&T) to support these changes are fit for purpose.2 In the NHS, IM&T and e-Health are often used interchangeably – for the purposes of this report, e-Health (patient related systems) is a subset of IM&T.

Key findings

•	Delivering for Health signals a more corporate approach for IM&T in future where “... previous freedoms to procure 
and implement systems locally will be curtailed to ensure that local systems align with the move to Electronic Health Records.”3  This is a significant cultural shift in the way in which IT is managed within the NHS in Scotland and will take time to plan and implement.
•	At the time of the review the Scottish Executive Health Department (SEHD) did not have in place fully developed arrangements to demonstrate IM&T leadership, stakeholder involvement and project and programme management to meet internationally recognised good practice standards. In the main body of the report we highlight areas where work is in hand to address these issues.
•	The SEHD recognises the need to review the governance and management arrangements for IM&T and is currently taking steps to improve them. Its proposals are in line with good practice but it is too early to say whether they are sufficient to address the issues raised during the course of our review.4 
•	There has been limited strategic planning for IM&T at a national level. For example, there is not a national information framework or strategy which is aligned to the Scottish Executive’s overall strategy for health, although we have been advised that this is being undertaken as a matter of priority by the Information Services Division (ISD). Existing IM&T plans and structures focus on e-Health and it is difficult to get a picture of IM&T as a whole, including operational and support services such as finance and human resources systems.  
•	There is considerable room for improvement in other aspects of leadership and governance of IM&T so that in future:
–	it is clear who is in charge and where accountabilities lie
–	the NHS in Scotland’s IM&T strategy is aligned to its overall health strategy, and the expected contribution of each IM&T programme and project to the delivery of the strategy is clear and its performance monitored.5 
•	Funding should be based on a sound business case which clearly specifies the justification for the investment over the whole lifetime of the project, and the benefits that the investment will deliver. Currently this only happens where the capital spend of a project is over £2 million.6 In addition, a ‘stage gate’ approach should be adopted whereby revenue and capital funds are released on a phased basis once success criteria have been achieved. This is not routinely 
in place.
•	The NHS in Scotland does not know exactly how much it spends on IM&T overall, but the estimated national IT revenue budget of £65 million and £35 million capital in 2006/07 falls well short of the Wanless target of 
3-4 per cent of total health spend.7 8 This would be over £373 million for 2006/07. Even so the growth in investment is substantial and will continue into 2007/08, when the revenue budget is expected to be over £100 million and the capital budget £40 million. The challenge is to ensure that it represents value for money and delivers the information that people need to do their jobs. 
•	The SEHD and NHS boards recognise the importance of user or stakeholder engagement. However, we did not find evidence of a formal, structured approach to ensure that stakeholders are identified and engaged and their needs met through IM&T programmes 
and projects.
•	There is reasonable compliance with generally recognised programme and project management standards according to the evidence from our case studies. PRINCE2, a formal project management method, is often applied but the rigour with which standards are applied varies.9 There are also limited arrangements for sharing best practice and lessons learnt. In particular, there is little evidence that expected benefits are identified, monitored and delivered.
About the study

2. In this report we provide a high-level overview of the national picture covering: 

•	the background to IM&T in the NHS in Scotland
•	how IM&T is being led
•	the nature and extent of stakeholder involvement 
•	how programmes and projects are being managed. 
3. Specifically we examined whether:

•	there are clear, unambiguous leadership and governance arrangements in place, ensuring that IM&T investment is focused on achieving better services for patients
•	stakeholders are properly engaged and their information needs are identified and addressed in accordance with agreed priorities
•	programme and project management is effective so that stakeholders’ needs are met and they have appropriate buy-in. The aim is to ensure that systems are not only implemented to specification, on time and within budget but also that they deliver the expected benefits. 
4. In carrying out the study we commissioned Pricewaterhouse-Coopers (PwC) to:

•	identify and assess the NHS in Scotland’s performance against internationally accepted good practice in terms of the planning, management and delivery of IM&T solutions within complex environments. These are from sources including the Office of Government Commerce (OGC),10 the National Audit Office (NAO)11 and PwC’s global network12.
•	review documentation at a national level relating to, or with implications for, IM&T  
•	interview key individuals covering both IM&T users and providers at the SEHD, NHS National Services Scotland (NHS NSS)13 and NHS boards
•	carry out reviews of four national IM&T projects as case studies to provide further evidence to support findings from the review of documentation and interviews. The four case studies were Accident & Emergency; Best Procurement Implementation (BPI) of eProcurement Scotl@nd; Emergency Care Summary; and Scottish Care Information. These were selected to cover clinical and non-clinical topics; and long-running and more recent projects. 
5. The fieldwork was completed in March 2006. However, we recognise that this is an area where work is in progress; for example, the SEHD has made some changes to the high-level governance for national IM&T projects. We reflect these in the main body of the report. 

6. Findings and recommendations from the audit should help the SEHD in improving the effectiveness of IM&T, and the assessment against good practice can be used as a mechanism for monitoring progress and identifying areas of greatest risk. Given the level and importance of the investment we will follow up progress against the SEHD’s targets and plans.
1	Delivering for Health, SEHD, November 2005.
2	IM&T is also sometimes known as Information and Communications Technology (ICT).
3	http://www.scotland.gov.uk/Publications/2005/11/02102635/26380
4	Good practice applies to all aspects of IM&T management in both commercial and public sector organisations. See Appendix 3.
5	A programme is a management framework for coordinating related projects to deliver desired outcomes.
6	This figure is the threshold for undertaking a Full Business Case. 
7	Securing our Future Health: taking a long-term view, HM Treasury, April 2002.
8	Draft Budget 2006/07, Scottish Executive. http://www.scotland.gov.uk/Publications/2005/09/06112356/24114; capital figures supplied by SEHD, October 2006.
9	Prince2 is the designated programme and project management methodology for all large scale projects within the NHS in Scotland.
10	http://www.ogc.gov.uk 
11	http://www.nao.org.uk/
12	www.pwc.com 
13	NHS NSS has direct responsibility for the ongoing management of a number of national projects and programme management responsibility for many of the national IM&T solutions.  

Part 1. Setting the scene
Key findings

People in the NHS in Scotland need information to do their job and to support the improvements to patient care and service performance envisaged in Delivering for Health.  Given the size and complexity of the health service, information technology will increasingly be required to meet information needs efficiently and effectively. 

But providing IT technical solutions is not enough – they need to be tailored to the needs of the end user and often require changes in working practices.  

Information is vital to improving services for patients but any  investment in IM&T needs to be managed well

7. Both the Audit and Health Committees of the Scottish Parliament have been critical of the quality and coverage of information in the NHS in Scotland.15 Audit Scotland has also drawn attention to poor information and the slow implementation of information systems in its reports.16 

8. IM&T in the NHS needs to support a range of people with diverse information needs. (These people are commonly known as stakeholders). For example, the NHS needs IT systems to provide the right information at the right time for:

•	clinicians, to support the current and future delivery of individual patient care in a range of settings
•	managers, to support the operation and continuous improvement of their service 
•	policy makers and planners, to support decisions on future services.
9. These stakeholders need to make decisions based on a wide range of data including activity, cost and quality. These data come from different information systems, so looking at one IM&T programme or project in isolation is unlikely to be an effective strategy. There needs to be clarity about how each IT programme or project fits with, and supports, the NHS in Scotland’s corporate policies, strategies and plans. This is even more challenging given the requirement to work in partnership with others, including local authorities, to plan and deliver joint services where appropriate. 
	
10. Effective IM&T can benefit patients and staff in a number of ways. To achieve this it needs to be viewed as a central part of change and innovation, not as a back room technical function. It is not enough for IT projects to offer the right technical solution – a PwC and Mori survey undertaken in 2003 shows that 85 per cent of projects have the right solution, but 70 per cent fail because of poor implementation. 

11. Information only adds value if it is fit for purpose and is used. The NHS is big and complex so getting information to the right people when they need it will not happen by chance. It requires a structured approach, with clearly identified investment at national and local levels, otherwise there is a real danger that costly IT solutions will not meet stakeholder information needs and support the large scale national change envisaged in Delivering for Health. To date the UK public sector has experienced a number of high profile IT failures, so the challenge for the NHS in Scotland is to ensure that IM&T investment is well managed, represents value for money and delivers the intended benefits. 

The NHS in Scotland does not meet Wanless targets for IM&T investment
12. Effective IM&T is a powerful enabler for change as highlighted in the Wanless Report:17

“Without a major advance in the effective use of information and communications technology, the health service will find it increasingly difficult to deliver the efficient, high-quality service which the public will demand. This is a major priority which will have a crucial impact on the health service over future years.”
 
13. The Wanless Report also makes clear that the NHS has a poor record on investment in IM&T compared with other public services in the UK and with health services in other countries. Average spend on IM&T in the public sector is typically 2-4 per cent of budget. In other industry sectors spend can be seven per cent or more depending on the extent to which IM&T is crucial to the business. The Wanless Report recommended that spending should rise to 3-4 per cent of total UK health spend in order to make progress.
 
14. The White Paper Partnership for Care highlighted the importance of IM&T and the need to invest in this. It states:18

“Staff need to have the tools to do their job. So we are investing heavily, not only in the NHS staff themselves, but also in modernising the infrastructure of NHSScotland and above all in the information systems and communications technology necessary to deliver redesigned healthcare. NHS staff need access to the right information at the right time, if they are to meet patients’ needs. We urgently require an e-health culture to be established, driven by clinical leaders ... Our goal is to deliver an Integrated Care Record jointly managed by patients and professional NHS staff with in-built security of access governed by patient consent.”

15. The SEHD published its strategy to achieve this in 2004, focusing on e-Health.19 There has not been an all-encompassing or fully understood definition of e-Health and the NHS in Scotland appeared to use it to cover those systems that underpin the development of a single record for patients.20 Current documents remain unclear about whether e-Health includes support systems such as finance, human resources and supplies as well as operational elements such as applications management and desktop support. The IM&T strategy needs to ensure that there is appropriate coverage and linkage of support systems such as finance and human resources.   Exhibit 3 provides a summary of current national IM&T programmes and projects in the NHS in Scotland. Appendix 2 provides further information on the current status of these, including budgets and timescales where these are known.

The NHS in Scotland faces real challenges as it seeks to make best use of IM&T

16. Ensuring that people get the information they need to do their jobs and support service change presents enormous challenges for the NHS (Exhibit 4). New systems which provide people with better and more timely information require changes in ways of working for both capturing and using information.
  
17. Failure to meet these challenges leads to information being:

•	unavailable to those who most need it 
•	received too late to be of use 
•	of poor quality 
•	in conflict with that received from alternative sources
•	not relevant.
14	e-Health / IM&T Strategy 2004-2008, NHSS, April 2004.

15	Reports on the NHS overview and staff pay modernisation, Audit Committee, July 2004 and Budget process report to the Finance Committee, Health Committee, Session 2, 2004.
16	For example, A Scottish prescription: Managing the use of medicines in hospitals, Audit Scotland, July 2005 and Overview of the performance of the NHS in Scotland, 2004/05, Audit Scotland, December 2005.
17	Securing our Future Health: taking a long-term view, HM Treasury, April 2002. 
18	Partnership for Care, SEHD, February 2003. 
19	e-Health/IM&T strategy 2004-2008, SEHD, April 2004.
20	Examples include the Community Health Index (CHI) number, Picture Archiving and Communication System (PACS), Emergency Care Summary (ECS), National A&E system, and ePharmacy.

Part 2. Leading the way
Key findings

Delivering for Health signals a more corporate approach in future. This is a significant cultural shift in the way in which IT is managed within the NHS in Scotland and will take time to plan and implement.

At the time of our audit it was not clear who was accountable for directing IM&T strategy development and implementation to ensure that benefits are identified and achieved. The respective roles and responsibilities of the SEHD and NHS NSS still need to be clarified, and the balance between national mandatory requirements (such as standards) and freedom to implement local solutions needs to be agreed. 

There is not an overarching information framework to inform the development of integrated IT solutions in the NHS in Scotland. 

The overall IM&T strategy needs to be revised to reflect the full range of information needs and recent policy initiatives.

The NHS in Scotland needs to build on recent efforts to improve IM&T governance arrangements and the organisational structures which underpin these. 

The SEHD needs to improve the way it funds IM&T programmes in future – for example, funding should be based on a sound business case and it should adopt a ‘stage gate’ approach whereby funds are released on a phased basis. 

The NHS in Scotland does not know exactly how much it spends on IM&T overall. 

Why leadership matters

18. Without strong leadership it will be extremely difficult, if not impossible, to deliver integrated information and technology across Scotland which meets the needs of clinicians, managers and the public and delivers benefits to patients. If shortcomings in leadership are not addressed promptly, there is a real danger that programmes will be unrealistic; NHS boards might refrain from signing up to national solutions; and staff might not fully engage with initiatives. 

Developing a corporate approach will require a cultural shift

19. Those tasked with leading IM&T at a national level need to set a clear vision and goals. They then need to achieve a balance between directing the implementation of those aspects of the IM&T strategy for which they are directly responsible, and influencing implementation at boards and other organisations not directly within their control. 

20. Historically boards have had a significant degree of local autonomy. They have determined whether or not to invest in national or local IM&T solutions depending on their own priorities. As a result the SEHD and NHS NSS have had limited authority and impact in driving the full implementation of national solutions in local NHS boards. The SEHD has used health department letters (HDLs) to seek local action on IM&T development. However, we did not find any formal processes or assurance arrangements for ensuring compliance with these letters and they are not always clearly understood. For example, in April 2005 the SEHD increased delegated limits for capital spending on IM&T by all NHS boards from £1 million to £2 million specifically to provide greater autonomy.22 A further HDL issued in October 2005 required boards to inform the SEHD about any new significant e-Health proposals, defined as those costing more than £100,000.23 The second HDL focuses specifically on board spend on e-Health proposals. Given the lack of a consistent definition of e-Health in the past, there has been the potential for differences in understanding between the SEHD and boards as to whether or not a given procurement falls within the scope of this HDL. 

21. Delivering for Health signals a more corporate approach in future where “... previous freedoms to procure and implement systems locally will be curtailed to ensure that local systems align with the move to Electronic Health Records.”24 This is a significant cultural shift in the way in which IT is managed within the NHS in Scotland and will take time to plan and implement.

22. This is because there are tensions between national and local priorities for IM&T. Boards have been expected to base their IM&T decisions on local business cases and these have been judged against other competing projects and initiatives. This means that boards have opted in or out of national systems depending on their local circumstances. For example:

•	SCI Store is at different stages of adoption in NHS boards with different versions in operation and take-up is elective. Additionally no clear business case, benefits realisation plan or future development plan exist for SCI. The lack of these structured processes increases the risk that the NHS does not get the full benefit from the system and increases the costs of support.
•	Some boards have been allowed to opt out of the national system for A&E where their existing system met national standards. However, there may be an impact on development costs if the SEHD’s reporting requirements change over time.    
•	The potential to achieve cost savings and service improvements in procurement will only be maximised through improved collaborative working across Scotland. BPI aims to support this and, since its inception in 2003, 12 boards have signed up.25 However, it is unclear how well embedded the system is at a local level, or what impact it is having in terms of generating savings. In addition, although NHS Greater Glasgow & Clyde and NHS Fife are engaged in the BPI Programme, they have not yet signed up to the eProcurement Scotl@nd Service. 
23. These examples may reflect the fact that the NHS in Scotland is in transition, moving from the previous approach of local autonomy, which contributed towards the current fragmentation, to one which is more corporate. However, during our fieldwork we did not find evidence of a convergence strategy which would ensure that all boards have plans to comply with the national strategy over time. A paper on convergence has subsequently been considered at the June 2006 meeting of the 
e-Health Strategy Board.

24. We reviewed current IM&T leadership in the NHS in Scotland in terms of:

•	governance and risk management 
•	organisational design
•	strategic planning
•	funding.
Good IM&T governance is essential

25. The principles of good governance help to ensure good management, good performance, good stewardship of public money, good public engagement and ultimately, good outcomes from IM&T programmes and projects.

26. Exhibit 7 shows the elements that need to be in place to ensure that IT solutions support the health service’s objectives. 

The SEHD is reviewing its governance and risk management arrangements

27. The NHS in Scotland recognises that it needs to ensure that its governance arrangements are in line with the elements of good practice outlined in Appendix 3. Responsibility for IM&T within NHS Scotland currently sits with:

•	The SEHD, which is responsible for defining the strategy for IM&T.
•	NHS NSS, which has direct responsibility for the ongoing management of a number of national projects and programme management responsibility for many of the new national IM&T solutions. In addition, NHS NSS provides extensive Scottish health and care statistics to the service.
•	Local boards, which have traditionally been responsible for the delivery of IM&T within their own organisation.
28. At the time of our review it was not clear who was accountable for directing IM&T strategy development and implementation to ensure information needs are met and benefits are identified and achieved. The relationship between the SEHD, NHS NSS and NHS boards still needs to be clarified and the balance between national mandatory requirements (such as standards) and freedom to implement local solutions needs to be agreed. 

29. The SEHD and NHS boards have taken a number of steps to improve governance. For example:

•	ISD has developed a programme office which now provides status reports on the overall e-Health 
programme. The reports provide Red, Amber, Green (RAG) styled progress indicators of status but do not provide any assessment of progress against a benefits realisation plan, risks or inter-programme dependencies. In addition, its scope does not include reporting on other programmes or projects such as the BPI Programme, the Logistics Programme, Shared Services 
or SWISS. 
•	The SEHD commissioned an external consultant to review and develop revised governance arrangements in November 2005. These are being further developed along with implementation plans. These new arrangements incorporate: 
–	an e-Health Strategy Board responsible for the strategic direction of e-Health 
–	an e-Health Delivery Board responsible for the delivery of the e-Health programme
–	a Design Authority responsible for the technical oversight and assurance of the strategic fit of proposed national IM&T solutions.
•	As of February 2006, the e-Health Strategy Board had met once and the other two groups were still to be established. The effectiveness of the groups will depend on their terms of reference, clear roles and responsibilities for group members, and on the right membership. There are documented briefs for the e-Health Strategy Board and steering groups. These identify group members but they do not set out individual members’ responsibilities and accountabilities. 
•	The SEHD has accepted that clarification of its role along with those of NHS NSS and health boards is an important part of the work of the e-Health Strategy Board.
•	In order to ensure that national and local governance arrangements complement each other, the SEHD began a tendering process in May 2006 “to define the governance arrangements and standards required to deliver the e-Health Programme and set out a plan for their delivery.” 
30. The new governance arrangements currently planned reflect good practice but it is too soon for us to assess compliance with these arrangements and their effectiveness. In addition, we remain unclear how the governance of back room support systems, such as human resources and finance, will 
be addressed. 

31. The SEHD also needs to improve existing systems for:

•	Linking funding for initial procurement, implementation, maintenance and future development of IT systems to achieving key milestones or delivering benefits. Existing funding mechanisms do not ensure that the continuation of a programme, project or initiative is dependent on meeting agreed success criteria.
•	Monitoring the progress of IT programmes and projects in the context of the national IM&T strategy. We found little evidence to demonstrate that this was happening, with programme and project boards operating in isolation. A number of key stakeholders commented on this.
•	Managing risks consistently across all projects and programmes and reporting these to programme and project boards. At present, risk management and assurance are carried out at a project level and there is no standardised risk management framework or approach to assurance. The SEHD recognises this and has been working with external consultants to develop a framework which will be introduced as part of the new governance arrangements.
The current organisational structure does not support clear accountability

32. There is no right organisational structure for IM&T. But whatever structure is adopted it is essential that IM&T professionals and others are clear about who is accountable for each of the areas and services shown in Exhibit 8 and how they are delivered to, and by, the NHS in Scotland. 

33. At the time of our review, responsibility for IM&T management was fragmented between the SEHD, NHS NSS and NHS boards, and it was not always clear how functions were shared. This uncertainty made overall accountability unclear and stakeholders were confused about who had overall responsibility. 

34. In particular, we found that:

•	there was a lack of clarity about the roles, responsibilities and accountabilities between those in charge of information and those in charge of IT at NHS NSS and the SEHD, and how they should link 
•	more effective arrangements were needed to fulfil the role of a technical design authority, providing technical oversight and assurance of the strategic fit of proposed national IM&T solutions 
•	there was not a common understanding as to who, or which body, was responsible or accountable for ensuring that the NHS in Scotland was focusing on the right priority areas for IM&T and for reviewing the IM&T programme on an ongoing basis 
•	there were some examples of initiatives to develop practices, methods and tools to support consistent implementation such as the Electronic Clinical Communications Implementation (ECCI). However, no group had lead responsibility for developing and implementing these for all IT systems. Interviews with stakeholders in boards highlighted the Centre for Change and Innovation (CCI) as having a key role to play in supporting the delivery of business change and adoption of best practices. But its remit appeared more aligned to the redesign of front line services rather than supporting the delivery of IM&T programmes, and there was little evidence of CCI involvement in the case studies we reviewed 
•	many stakeholders were of the view that projects tended to be managed in isolation rather than as part of a portfolio. This meant that there was no overarching view of projects to identify what was going well or not so well in relation to other projects and the changing priorities of the NHS. 
35. The SEHD recognised that achieving the major milestones set out in Delivering for Health will require a joint approach between the SEHD and boards and assigned lead responsibility for e-Health to the Director of Policy and Strategy in the SEHD and the Chief Executive of NHS NSS.26 Currently e-Health at the SEHD remains the responsibility of the Director of Primary & Community Care and the Director of Finance leads the Analytical Services Division (ASD) which covers information. In NHS NSS, responsibility for e-Health and information is also split between two interim directors. 

36. Since the completion of our fieldwork the SEHD has announced a new organisational structure for IM&T. This is in line with good practice but it is too early for us to assess the degree of implementation or the effectiveness of the new structure. More work will be required to ensure that the new structure demonstrates a clear connection between information and IT; clarifies the respective responsibilities and accountabilities of the SEHD and NHS NSS; clarifies responsibilities for all IT programmes; and ensures that responsibilities are fully communicated to the wider NHS in Scotland.

Strategic planning for IM&T needs to get better

37. Strategic planning is about ensuring that medium to long-term IM&T plans fully support the health strategy for Scotland. It means being clear about priorities and plans; using the resources available to deliver a defined set of benefits and objectives; and reviewing these on a regular basis to ensure the benefits are realised.

There is not an overarching information framework or strategy to inform the development of integrated IT solutions
38. An IM&T strategy should be driven by a clear information strategy outlining NHS information requirements including outcomes, quality, activity and cost. It should identify how IT systems will help meet these needs. Information needs should be identified in a structured way so that it is clear what information the NHS in Scotland needs; where it should come from; who should provide it; how it should be analysed and quality assured; and how this can be supported by IT.

39. There is not an agreed information framework linked to the health strategy, on which to base strategic plans for improving the availability and quality of information, although there are some initiatives underway which could help address this situation. 

The current IM&T strategy needs to be revised
40. The current National e-Health/ IM&T strategy was published in 2004, and was informed by the health policies in place at that time.27 The strategy identifies areas of the policies that IM&T could support; provides a view of the position at that time; and identifies what more needs to be done. The SEHD is assessing the capability of the current e-Health programme to deliver against Delivering for Health. In doing this it needs to ensure that there is:

•	Clarity about the information needed by the NHS in Scotland in the light of Delivering for Health, including clinical and management information and how IT solutions will address these needs.
•	A basis for developing an implementation plan. This should set out timescales and identify the expected IM&T contribution towards the overall health strategy, including benefits expected such as improved quality, productivity or financial savings. 
•	A basis for local board strategies and implementation plans which complement the national strategy.
•	Compatibility with the Open Scotland Information Age Framework to promote interoperability between IT systems, both within the NHS and between the NHS and its partners, so that data can be shared when it is appropriate to do so.28
41. The SEHD has no key performance indicators (KPIs) to monitor the implementation of the National e-Health/IM&T Strategy and the performance of IT systems. Exhibit 11 provides examples of possible KPIs. These are important in identifying and addressing problems as early as possible. Failure to do this can mean that stakeholders’ commitment is undermined. For example, users have experienced problems with response times for the e-procurement system, and this reduces their willingness to use the system. Instead the SEHD has set a number of target dates that the NHS in Scotland is expected to achieve.

42. During our audit we did not find evidence of an overarching prioritisation process to ensure that investment in individual programmes and projects is driven by corporate strategic priorities. Neither did we find evidence of ongoing evaluation or review of the progress made by current projects against an action plan to deliver the overall National e-Health/IM&T Strategy.

43. The national PACS and A&E systems originated as local projects and have now been adopted nationally. Local procurement would have been expensive and the SEHD decided to procure national solutions in order to gain economies of scale and ensure a standardised system. These have become part of the broader e-Health strategy. They are important developments but there is a risk that without a robust national assessment and decision-making process priorities can get distorted, suboptimal IT solutions may be adopted and future costs may be increased.

44. In March 2006, the SEHD made a commitment to review all systems in light of their contribution to Delivering for Health to determine what to accelerate, continue, pause or stop.29

The SEHD needs to review funding for IM&T

45. Funding should be based on a sound business case which clearly specifies the justification for the investment over the whole lifetime of the project, and the benefits that the investment will deliver. There should be a financial plan that identifies:

•	how funding will be put in place
•	how expenditure will be phased 
•	the points at which expenditure will be authorised
•	the breakdown of costs 
•	the authorisation process for expenditure. 
46. Within leading practice organisations, funding for large programmes is typically based on a gateway approach where both capital and revenue funds (including staff time) are released on a phased basis, depending on the achievement of certain outcomes (Appendix 5 outlines the OGC’s gateway model).This approach is beginning to be used for external suppliers but is not routinely in place for all programmes.

47. The SEHD does not have robust processes for defining whole-life budgets.30 It does identify the full lifetime costs for projects which require a Full Business Case – that is those with capital expenditure of over £2 million – or on smaller projects which involve a procurement. However, this does not happen for projects which are delivered as part of the work of an existing NHS team. This can make it difficult to undertake ongoing cost benefit reviews of the systems and services provided.

48. For new programmes and initiatives in the NHS in Scotland, there is usually a component of national funding. This money is most often used to fund national software purchases and systems development, for example the SCI products (SCI Store, SCI Gateway, SCI Discharge and SCI Outpatients) and the BPI Programme. We found no examples of funding based on a gateway process or staged draw-down of funds; instead national funds are typically released as an annual budget or one-off activity. 

49. Historically, boards have allocated money from their existing budgets to fund local implementation of national projects and the ongoing running of the system. This creates financial challenges for boards, especially those trying to address budget deficits, as the costs of IM&T developments are competing for funding with clinical priorities. More recently there have been examples, such as the A&E project, where the SEHD has provided some central funding for local implementation and for software support. 

50. For the four case studies we reviewed, only one (BPI) had programme reports that provided information on spend against budget (See Appendix 4 for findings from our four case studies). 

51. The NHS in Scotland cannot identify exactly how much it spends in total – capital and revenue – on IT. The SEHD carried out a one-off survey of local spend in 2002/03 but it reports that this was resource-consuming and difficult mainly because of differences in definition of what constituted IM&T spend. Its best estimate of spend by NHS boards is £50 million in 2006/07. It now intends to review the recording of financial data as part of its new governance arrangements.

52. Exhibit 12 provides information from the draft revenue budget for national IT and actual spend for the period from 2002 to 2008. In addition, capital budgets for 2005/06, 2006/07 and 2007/08 are £27 million, £35 million and £40 million respectively. Appendix 2 provides the data available from the SEHD on revenue and capital expenditure for selected IT projects. 

53. Although budgeted and actual expenditure have increased significantly, this still falls short of the 3-4 per cent of the NHS budget recommended by Wanless, which would amount to approximately 
£373 million in 2006/07. Nevertheless, the growth in investment is substantial and there is a need to do more to ensure that it represents value for money and delivers the information that staff need to provide services to patients. 

Recommendations

The SEHD should:

•		Extend proposals for more robust governance and organisational design to cover all IM&T and, once implemented, monitor their effectiveness.
•	Clarify responsibilities and accountabilities for IM&T strategy development and implementation at a national level.
•	Develop and implement a comprehensive IM&T strategy which clearly links IT solutions to information requirements including outcomes, quality, activity and costs.
•	Require NHS boards to develop clear plans to bring any local IM&T solutions, arising from past investments, into line with the national strategy in an acceptable timescale. 
•	Develop business cases and implement funding allocation processes to support a gateway review approach for major IM&T programmes in line with OGC good practice.
•	Identify and monitor total capital and revenue spend on IM&T.  

NHS boards should:

•		Ensure that local governance and organisational design for IM&T align with national arrangements.
•		Clarify responsibilities and accountabilities for IM&T strategy development and implementation at a local level.
•		Develop clear plans to bring any local IM&T solutions, arising from past investments, into line with the national strategy in an acceptable timescale.
•		Develop business cases and implement funding allocation processes to support a gateway review approach for local IM&T programmes in line with OGC good practice.
•		Identify and monitor capital and revenue spend on IM&T. 

22	NHS HDL (2005)16 Delegated Limits.
23	e-Health: Guidance for planning and development pending single record. HDL(2005)46. 
24	http://www.scotland.gov.uk/Publications/2005/11/02102635/26380
25	http://www.eprocurementscotland.com
26	Delivering for Health; Guidance on Implementation, HDL(2006)12, SEHD, 2006.
27 	National eHealth/IM&T Strategy 2004-08, April 2004.
28	OSIAF Open Scotland Information Age Framework, Scottish Executive, 2006.
29	National eHealth Strategy: Review of national project portfolio, SEHD, March 2006.
30   Whole-life budgets mean the revenue and capital expenditure for implementation and running costs throughout the life of the system.
 
Part 3: Involving information users
Key findings

The SEHD and NHS boards recognise the importance of stakeholder engagement, but they need to do more to involve clinicians, managers and policy makers to ensure their information needs are met through IT.

Why stakeholder engagement matters

54. The main stakeholders are information users such as clinicians, managers and policy makers. Stakeholders are those people who are affected by, or can influence, an IM&T project but who are not part of the project team – they are critical to its success. For example, if clinicians are to use new systems to support their work with patients they need to have confidence in the accuracy and timeliness of the information generated. 

55. If stakeholder information needs are not identified and incorporated at the specification stage of an IM&T initiative, it is often expensive and, at best, difficult to meet their information needs. Given the size, complexity and interdependencies of IT systems in the NHS, planning, managing and monitoring stakeholder engagement is crucial (Exhibit 13, overleaf). As the NHS in Scotland moves to a more corporate model for IM&T, led nationally, and as the size and complexity of programmes undertaken increases, this task becomes more difficult. 

There have been problems involving stakeholders 

56. The SEHD and NHS boards recognise the need for stakeholders to be identified, consulted and involved in the development of strategy and in the design and delivery of information systems. 
But the processes for identifying and involving stakeholders lack formality, rigour and impact. 

57. We reviewed the process for engaging stakeholders with the National e-Health/IM&T Strategy 2004-2008. Formal consultation was in the form of an HDL to chief executives but this did not guarantee that all stakeholders had an opportunity to comment. It was also unclear how views were fed into and recognised within the strategy. Neither did we find evidence of an action plan to ensure that stakeholders were signed up to the strategy. 

58. For individual programmes and projects, we found little evidence of formal processes or methods in place (See Appendix 4 for findings from our four case studies). Instead, engagement depends on project managers’ knowledge of the health service and the existence of professional bodies or organisations that could potentially represent one or more stakeholder groups. Stakeholder mapping is important to ensure balanced representation of all groups. Without it there is a risk that IM&T developments do not represent or reflect all relevant information needs within the NHS 
in Scotland.

59. The main formal route for involving stakeholders has been through membership of a programme board or through the user group for existing systems. We did not find evidence of effective communication plans for different groups.

60. There was no evidence of project board members having clear terms of reference for their involvement, so there is a significant risk that individuals sit on the board in an individual rather than a representative capacity. 

The SEHD is taking action to improve stakeholder engagement

61. The SEHD is beginning to do more to engage with stakeholders, for example:

•	Each NHS board is required to have an IM&T lead and a separate clinical IM&T lead. 
•	The Clinical Lead for IM&T at the SEHD is investing significant effort to get key stakeholders (clinical and non-clinical) on board with the A&E programme. The need to do this is becoming pressing as the target dates for systems implementations approach. 
•	The SEHD has commissioned an assessment of the strategic options to take forward the wider primary and community care agenda arising from Delivering for Health. This is essential because of the serious concerns expressed about the General Practice Administration System for Scotland (GPASS). The British Medical Association and the Royal College of GPs raised some of these concerns with the Health Minister in April 2006, and concerns have also been discussed at the Parliament’s Health Committee.31    
•	The Chief Medical Officer (CMO) will have responsibility for engaging with clinicians on the e-Health agenda. But gaps remain for other stakeholders, and work is needed to develop and implement a comprehensive stakeholder plan.
62. It is too early to assess whether these actions will address stakeholder concerns; for example, IM&T leads in NHS boards alone will not guarantee effective stakeholder involvement without an action plan for active engagement.

Recommendation

The NHS in Scotland should:

Take a more formal, structured approach to stakeholder identification, engagement and communication to ensure the successful delivery of IM&T programmes and projects which meet stakeholder information requirements and deliver expected benefits. 

31	Evidence to the Health Committee, Scottish Parliament, 30 May 2006.

  
Part 4. Programme and project management
Key findings

The SEHD needs to ensure that existing good practice in project and programme management is applied consistently throughout the NHS in Scotland. This is essential to alert leaders and senior managers to emerging problems, and inform their IM&T investment decisions about what to start, stop or accelerate to achieve their overall objectives. There is limited evidence that expected benefits are identified, monitored and delivered.

Programme and project management methods are used but not always to best effect

63. Effective programme and project management are essential if the NHS in Scotland is to deliver the ambitions of the e-Health and wider IM&T strategies. Leading practice organisations invest significantly in this area to improve the likelihood of success within large scale, complex technology change work programmes. Effective programme and project management alerts leaders and senior managers to emerging problems; this enables them to intervene early to prevent projects and programmes running over time and budget. 

64. The OGC findings, and experience within the NHS across the UK, highlight the common pitfalls encountered in managing change, with poor risk controls, weak business case rationale and inadequate skills being some of the key findings within programme failures.32 

65. We found examples of good practice in some elements of project management in our case studies.But more needs to be done to ensure that all national and local projects are managed well so that they deliver to time and within budget, and meet user needs. 

66. There is reasonable compliance with generally recognised programme and project management standards according to the evidence from our case studies. PRINCE2, a formal project management method, is often applied but the rigour with which standards are applied varies.33 For example, the quality of content on project status reports varies, and there are variations in the degree to which risks are actively managed. 

67. Our case studies showed that the level of programme and project management skills varies across Scotland. For example, within NHS NSS and NHS 24, the majority of project managers are PRINCE2 trained but this is not generally the case across the NHS in Scotland. Project managers and others indicated that the level of skill in local boards is often low, with very limited understanding of the basics such as the need for, and how to produce, a project initiation document. In addition, programme management reporting – progress against plan, spend against budget, deliverables and outcomes documented with clear accountabilities, assurance and timelines – is not universally practised or well-structured. This means that strategic decision making is likely to be flawed through a lack of accurate information on IM&T performance. 

68. The SEHD requires a business case for all projects requiring a capital investment of £2 million or more to demonstrate the rationale for the investment. A number of the national projects which play a key part in the SEHD’s e-Health strategy do not have a national business case as the capital investment required for these was less than £2 million.34 The lack of a business case for these strategically important projects, each of which has a significant revenue investment, makes it difficult to monitor the extent to which they are delivering the intended benefits and, as they move into ongoing operation, the extent to which they continue to provide their services to NHS Scotland in an efficient and effective manner. A formal project plan is not always established for each project undertaken. As a result projects are not split into key activities and this leads to a lack of accountability for delivery or milestones for progress reporting against objectives. Also, dependencies between projects are not always identified, with limited recognition of local systems which have already been implemented – this increases project risk. 

69. Where business cases are produced, stakeholders indicated that they are not always shared outside of the SEHD or NHS NSS and this means that others are not clear about the rationale for the investment.

70. We did not find a standardised framework for risk management. Accountabilities rest by default with programme managers and in some instances there are no documented controls and procedures for monitoring. A risk assessment is not always undertaken before the start of a project. The four case studies that we reviewed have issues and risk logs, but only two could demonstrate that they were updated on a regular basis and that the necessary action was taken.35

71. The templates for project reporting were generally good, but we found significant variation in the quality and completeness of the information provided in our case studies. The NHS NSS’ project office provides the SEHD with a monthly update of their current IM&T projects and status, together with key issues, but little is actually done with this monitoring information.

72. There is no standard process for undertaking post implementation reviews or for sharing lessons learned across the NHS in Scotland. NHS NSS carries out reviews for projects that it is involved in, but the lessons are not shared. These reviews need to be more rigorous to ensure that programmes deliver the expected benefits to stakeholders. 
   
The SEHD needs to adopt a more rigorous approach to benefits realisation 

73. Major IM&T programmes are expensive so it is vital that decision makers are clear about what benefits the investment can be expected to deliver. They also need to be assured that both at a corporate NHSScotland level and at an NHS board level these expected benefits are actually achieved and have been worth it. For example, many GP users are not convinced that the national investment in GPASS continues to represent value for money. However, current funding arrangements, whereby money is top-sliced by the SEHD, mean that NHS boards are contributing to GPASS even if their GPs exercise their right under the GMS contract to apply for funding to use other systems. A current example of GPs exercising this right is NHS Grampian which has submitted a business case to the SEHD, on behalf of its GP practices, to move away from GPASS.  

74. The OGC stresses the importance of benefits management from the creation of a business case through to benefits realisation.36 

75. Benefits identification, tracking and realisation are not standard components of the NHS in Scotland’s approach to IM&T programmes. For example, programme plans do not specify 
the contribution that their delivery will make to the achievement of the overall strategy for the NHS 
in Scotland.

76. There is a lack of accountability for the realisation of benefits, many of which will be achieved over a period of time once the project or programme has been fully implemented. It is not clear who is responsible for tracking and managing the achievement of benefits during and after implementation, although the financial benefits of some projects should be tracked through the Efficient Government Programme. This is acknowledged by the SEHD and they intend to address this through the introduction of new governance and management arrangements.

Recommendations

The SEHD and NHS NSS should:

•		Extend the role of the national project office to include benefits monitoring and reporting for the national IM&T programmes.
•		Ensure that programme and project boards establish a performance baseline within the start-up phase of future IM&T work programmes in order to provide a basis for measuring future benefits delivered.
The SEHD should:

•		Introduce benefits-led business cases as a mandatory component of programme/project commissioning.
Programme and project boards should:

•		Establish a performance baseline within the start-up phase of future IM&T work programmes in order to provide a basis for measuring future benefits delivered.

32	http://www.ogc.gov.uk/sdtoolkit/seniormanagement/gatewaylessons/index.html
33	PRINCE2 is the designated programme and project management methodology for all large scale projects within the NHS in Scotland.
34	For example, SCI Store, the Emergency Care Summary and the A&E project.
35	The Emergency Care Summary and BPI.
36	http://www.ogc.gov.uk/documents/ManagingBenefitsv101.pdf

If you require this publication in an alternative format and/or language, please contact us to discuss your needs. It is also available on our website: www.audit-scotland.gov.uk

Audit Scotland 
110 George Street 
Edinburgh EH2 4LH 

Telephone 
0845 146 1010
Fax 
0845 146 1009
 
www.audit-scotland.gov.uk 

ISBN 1 905634 38 2
AGS/2006/12