Our Submission to the EPR Inquiry
From Nhs It Info
Submission to the Health Select Committee Inquiry
into
The Electronic Patient Record and its Use
Submitted by
Brian Randell
On behalf of the Group of 23 Senior Academics in Computing and Systems, listed at the end of this document
Executive Summary
This submission addresses the issue: “Current progress on the development of the NHS Care Records Service and the National Data Spine and why delivery of the new systems is up to 2 years behind schedule”. It draws on the Dossier of Concerns regarding NPfIT that we have assembled from a variety of sources, and recently made available to Members of the Select Committee. Despite the difficulty of assessing NPfIT’s plans and progress, caused by the Programme’s size and complexity, the secrecy regarding detailed system specifications, and the atmosphere of fear that prevents many NHS staff from expressing criticisms, our Dossier contains extensive evidence, some but by no means all anecdotal, that supports our assessment that the Programme is in serious danger. The huge range of problems, covering technical matters, methods of procurement, the lack of buy-in from stakeholders, privacy and security questions, delivery delays and spiralling costs, greatly complicate the task of correctly identifying the fundamental causes and most effective remedies. Hence our recommendation that a detailed technical review of the Programme be commissioned, a review that must be open and manifestly independent if public confidence in NPfIT is to be regained.
Introduction
1. We, a group of concerned senior academics in computing and systems, first wrote to the Select Committee last April, saying that we believed that NPfIT was showing many of the symptoms we have seen in major IT systems that had eventually been cancelled, overrun massively in terms of time and anticipated budget, failed to deliver an acceptable service to users or to reach business benefit targets for their organisations. We are pleased that the Committee has now agreed to hold an inquiry into the critical aspect of NPfIT, its support for and use of Electronic Patient Records (EPRs).
2. Since we first wrote we have compiled, and made available in printed form to all Members of the Committee, an extensive Dossier of Concerns. This has been assembled from nearly 600 published and unpublished sources, ranging from scholarly research papers to what might (unwisely) be dismissed as mere media rhetoric. Our submission makes numerous references to this Dossier — for convenience simply by indicating the relevant section number, e.g. “3.8.8”, in the printed version of the Dossier, dated 18 January 2007. (This version can be found online, along with the current further-extended version, via our website at http://nhs-it.info.)
3. As one of the MPs we sent our Dossier to remarked to us, it is very difficult to obtain an adequate picture of NPfIT free of the “entanglements of political axe-grinding, professional jealousies and commercial interests”. An even greater difficulty is the climate of fear in the NHS that has prevented many from expressing publicly the views that they have been willing to share with us in private. Nevertheless, even though much — though by no means all — of the evidence in our Dossier is anecdotal, and from secondary sources, we believe that it provides strong support for our assessment that NPfIT is in serious danger. What is significant about our Dossier is not just its size, but the range of problems noted, covering technical matters, methods of procurement, the lack of buy-in from a range of stakeholders, questions of privacy and security, delays in delivery and spiralling costs.
4. We have also attempted, as outsiders, to assess the technical merits of NPfIT, particularly those of direct relevance to the safety and reliability of the overall system. However many details of NPfIT (even the contractual integrity and availability requirements and specifications) are regarded as commercially highly confidential, making this task virtually impossible. (Such difficult-to-justify secrecy has, we are certain, also contributed to the lack of confidence that many working in the NHS have in the Programme [3.8.8].)
5. Our hope is that the outcome of the Committee’s Inquiry will be the setting up of an open independent constructive technical review, ideally of NPfIT as a whole, but at least of the centrally-provided NHS Care Records Service (NHS CRS), and of the various systems being provided by the Local Service Providers (such as Patient Administrative Systems, Clinical Systems and Departmental Systems), that together support the creation, maintenance and utilisation of EPRs. Dr Granger and his senior colleagues publicly expressed support for such a review following our meeting with them last April. It is we believe the best way of arriving at a disinterested expert assessment of NPfIT and of significantly improving the chances of a successful, timely and well-accepted outcome of this major investment by the NHS. (The current refusal by NHS to contemplate such a review is worryingly reminiscent of management attitudes during London Stock Exchange’s disastrous Taurus project [Drummond 1996].)
The Electronic Patient Record
6. Virtually all the claimed clinical advantages for patients of centralised EPRs (at cluster or national level) could be achieved by replacing paper records with electronic ones at the local (i.e. trust) level [2.6, 3.5.21]. The claimed importance of being able to access a central EPR directly when a patient requires treatment far from home is not supported by evidence [2.7]. Making what could have been local record keeping part of a cluster-level, leave alone an immense national-level, “system-of-systems” introduces system interdependencies that, because of their effect on system complexity, pose risks to system reliability and availability that in our judgement are likely to prove out of all proportion to any potential benefits [3.8.8]. Also the integration of EHR files at cluster, and certainly national, level greatly exacerbates the problem of maintaining patient confidentiality [Javitt 2005, 2.8, 3.5.3, 3.8.25].
7. Electronic records need to be generated as a by-product of relevant medical activity, and at the time of that activity, if they are to be of direct support to health care (for example in preventing possible clinical errors, such as related to drug dosage) [2.1]. If in contrast their generation has (perhaps because of usability or system performance limitations) to be undertaken as a supplementary time-consuming after-the-fact task, especially if it is one of little evident direct benefit to patients or clinicians, there will be much less incentive on the part of staff to undertake detailed record generation or to maintain quality control [Brennan 2005, 7.1.1].
8. Moreover, such are the differing circumstances from hospital to hospital that centrally-imposed standard EPR systems will often prove ineffective, and will not be used as intended, as Professor Eason, for example, has convincingly demonstrated in his study of mental health trusts [Eason 2007]. The clear implication is that it would be better to let local experts decide how best to satisfy local needs and circumstances, to identify minimum standards needed for interlinking local systems, and to defer such interlinking until after local systems have been successfully implemented and gained general support.
The causes of delayed delivery of the new systems
9. Many studies have reported significant time delays, cost overruns and, all too frequently, complete failures of projects that involve extensive software development or customisation. For example, a 1995 study [Jones 1995] of 164 software projects concluded that over 24% of projects were cancelled and that two-thirds experienced significant cost and time overruns. A 2001 survey [Taylor 2001] reported that of more than 500 development projects, only three met the survey’s criteria for success. A 2002 survey of 13,552 IT projects [Standish Group 2002] reported that only 34% were completed on time and to budget and that 51% though completed and operational were over-budget, over the time estimate, and offering fewer features and functions than originally specified. 10. The ever-growing levels of ambition on the part of system builders and system commissioners are such that the situation is not improving: in 2007 a US National Institute of Standards and Technology report stated that: “By most estimates, over half of all large application development projects . . . end in failure — after all the time and money is spent, the product still cannot be used operationally”.
11. Hence it is clear that the utmost care has to be taken to (i) avoid undue ambition, (ii) make sure that the most circumspect software acquisition processes are employed, (iii) minimise undue dependence on the continuous correct functioning of the complete system, (iv) evolve towards the intended overall system via a sequence of practical and cost-effective intermediate systems, and (v) avoid at each stage the trap of specifications that are vague, changing or in conflict [Curtis 1988, 3.8.25].
12. Many healthcare systems are necessarily large and complex, but the NHS is huge and organisationally highly intricate [Beynon-Davies 1994; Wyatt 1995]. A fully-integrated NHS-wide healthcare system is vastly larger and more complex than any previous healthcare system [Javitt 2004]. Indeed it is admitted to be the world’s largest civil IT project ever. Thus NPfIT is by definition an undertaking that is inherently “at risk”. Moreover, given the crucial and pervasive role played by EPRs, we believe much of this risk arises from the complex systems supporting cluster-level and national-level EPRs.
13. Adverse outcomes from large IT projects rarely can be linked to a single cause (Lyytinen and Hirschheim 1987). Indeed analyses have identified many different causes. Some we list here, with representative references to relevant sections of our Dossier, as a contribution to identifying problematic aspects of NPfIT.
- cryptic or concealed agendas [3.3.2, 3.8.31]
- not involving users or not incorporating organisational needs [2.1, 2.3-4, 3.4.1-3, 3.4.9-10, 3.4.12, 3.4.16-17]
- treating a project as an IT project rather than as a business change project [3.8.5, 3.8.8-9, 4.2]
- ill-defined, unrealistic or conflicting objectives [2.1, 3.1.18, 3.7.33, 3.8.29]
- suppression or mismanagement of risks and uncertainties [2.8, 3.1.21, 5.3.7]
- ineffective project or resource planning [2.2, 3.1.10-12, 3.1.14, 3.4.6, 3.7.8]
- planning for “big-bang” instead of evolutionary delivery and failure to plan for longer-term evolution [2.2, 3.8.13-14, 3.8.28]
- fixing objectives, timeframes or costs when excessive uncertainty still exists [1.2.5, 3.1.11, 3.7.28]
- ignoring costs of business change [2.2, 3.8.1, 3.8.8]
- insufficient political support [2.3, 3.4.18-19]
- over-dependency on, and/or failure to control, suppliers [3.1.14, 3.1.26, 3.3.10-13, 3.3.20, 3.7.27]
- excessive project size and complexity [3.1.12, 3.2.11, 3.3.17, 3.7.32, 3.8.15]
- unproven technology or approach, especially exceeding the limits of proven performance [3.1.25, 3.1.30]
- inadequate provision for data transfer and quality [2.6, 3.3.14, 3.8.9]
- lack of provision for information governance [3.8.26, 3.8.47-48]
- failure to identify or achieve relevant standards [2.6, 3.7.25, 3.8.26]
- failure to plan for necessary levels of safety, security or recovery [2.8, 3.3.19, 3.5.1-47, 3.6.1-23]
- lack of project or technical skills [1.6.4, 2.2, 3.1.14, 3.7.32, 3.8.21]
- undefined exit conditions [1.2.5]
- inflexibility, inappropriate aggression and machismo [3.3.13, 3.5.34-35, 3.8.34]
- loss of political support [1.6.5, 3.4.11, 3.8.33]
- resources not properly allocated [3.3.6-7, 3.8.44]
- failure to create effective partnership with contractors [3.2.2, 3.2.9, 3.8.22, 3.8.46]
- unreliable progress reporting [3.1.14, 3.3.9, 3.8.16]
- unrealistic timeframes or budgets [1.2.5, 3.1.11, 3.7.30, 3.8.21, 3.8.40]
- inattention to quality [3.1.14, 3.3.14]
- barriers to communication [3.8.17, 3.8.31]
- fear of failure [3.6.8, 3.8.14]
14. We find it quite remarkable, and extremely worrying, that our Dossier shows that all of the above lengthy list of generic system problems would appear to exist in NPfIT.
15. Our Dossier also provides extensive evidence of a number of further problems that are specific to, or particularly challenging in, NPfIT including:
- inadequate clinical engagement during system requirements analysis and specification [2.3-4, 3.4.3, 3.4.9, 3.4.16-17, 3.8.22, 5.3.9]
- replacement of existing successful well-trusted systems by standard systems that are perceived to be inferior [3.6.8, 3.8.9, 3.8.39]
- employing Patient Administration Systems that were designed for the very different US healthcare market [3.8.10]
- inadequate response to the medical profession’s concerns regarding issues of patient confidentiality [3.4.5, 3.5.1, 3.5.11, 3.5.30, 3.5.34]
- excessive reliance on IT consultants and suppliers with little knowledge of UK healthcare and the NHS [2.4, 3.8.7]
- implementing a complex centralised system in a situation in which the NHS is constantly faced by changes in organisation, medical practice and even the law [2.6, 3.8.23, 9.3]
- identifying and managing the changes in NHS working practices needed to complement, shape, and exploit proposed new IT facilities [3.8.5, 3.8.8, 3.8.9]
17. It will, we believe, take such a review to pinpoint the most important causes of the present delayed delivery of the Programme, in particular those aspects related to EHRs. More importantly, such a review is needed to determine whether there are, as we suspect, strong reasons to assume that the Programme will actually fail, not just continue to over-run its schedule and budget, unless appropriate remedial action is identified and undertaken urgently.
Concluding remarks
18. A well-known aphorism in the IT industry is: “A complex system that works has evolved from a simple system that worked” [Gall, 1975]. Another hard-won insight is that the most successful highly-critical large IT systems, such as the worldwide VISA payments system [Stearns 2006], achieved their success through ruthless control of their complexity, and minimisation of the level of dependence that needed to be placed on them, as well as through high levels of hardware/software reliability. A further crucial insight concerns the criticality of employing evolutionary procurement methods, and socio-technical expertise, in order to determine precise specifications that meet the various stakeholders’ requirements [2.2]. A detailed constructive review of NPfIT in the light of these insights could, we argue, greatly increase the likelihood of the project’s eventual success. 19. The NHS has many good working systems, and NPfIT is planning and delivering others, but our and others’ research and experience suggests that NPfIT’s problems, especially those centred on EHRs, are daunting; our expert opinion therefore is that there is a high (but probably avoidable) risk that the Programme will fail. Hence our recommendation that an open independent technical review is an essential first step toward managing that risk in a professional manner. (In making this recommendation we are not seeking to review NPfIT ourselves – being entirely independent of NPfIT, we are simply acting out of strong professional concern and in the public interest.)
References
1. Beynon-Davies, P. Information management in the British National Health Service. Int. J. Information Management (1994;14): pp.84-94.
2. Brennan, S. The NHS IT Project. Radcliffe Publishing Ltd (2005).
3. Curtis, B, et al. A field study of the software design process for large systems. Communications of the ACM 31,11 (Nov 1988) pp. 1268-1287.
4. Drummond, P.H. Escalation in Decision-Making: The Tragedy of Taurus. Oxford University Press (1996).
5. Eason, K. Local socio-technical system development in the NHS National Programme for Information Technology (To appear in the J. Information Technology).
6. Gall, J. Systemantics: how systems work and especially how they fail. New York Times Book Co. 1975).
7. Javitt, J. C. How to succeed in health information technology. Health Affairs (25 May 2004) pp.321-324.
8. Jones, C. Patterns of large software systems: failure and success. IEEE Computer (March 1995) pp.86-87.
9. Lyytinen, K., Hirschheim R. Information systems failures. Oxford Surveys in Information Technology (1987;4) pp.257-309.
10. Standish Group, The CHAOS Report (2002).
11. Stearns, D. In plastic we trust: dependability and the Visa payment system. DIRC Conference, Newcastle (April 2006). http://www.sociology.ed.ac.uk/finance/Papers/StearnsDIRC06.pdf
12. Taylor, A. IT projects sink or swim. BCS Annual Review (2001) pp.61 - 64.
13. US National Institute of Standards and Technology. Advanced technology program on component-based software (2007). http://www.atp.nist.gov/focus/cbs.htm.
14. Wyatt, J. C. Hospital information management. BMJ (1995;311) pp.175-178.
Brian Randell was with IBM Research in the States 1964-69, before being
appointed to a Chair at Newcastle University. He has published nearly two
hundred technical papers and reports, and is co-author or editor of seven
books. He is now Emeritus Professor of Computing Science, and Senior Research
Investigator, at Newcastle University. He received the 2002 IEEE Emanuel R.
Piore Award for his research on system dependability.
SUBMISSION FROM:
Ross Anderson
Professor of Security Engineering
Cambridge University
James Backhouse
Director, Information System Integrity Group
London School of Economics
David Bustard
Professor and Head of Computing and Information Engineering
University of Ulster
Ewart Carson
Professor of Systems Science
Centre for Health Informatics
City University
Patrik O’Brian Holt
Professor
School of Computing
The Robert Gordon University
Roland Ibbett
Professor
School of Informatics
University of Edinburgh
Ray Ison
Professor of Systems
The Open University
Achim Jung
Professor
School of Computer Science
University of Birmingham
Frank Land
Emeritus Professor
Information Systems Department
London School of Economics
Bev Littlewood
Professor of Software Engineering
City University
John A McDermid
Professor of Software Engineering
University of York
Julian Newman
Professor of Computing
Glasgow Caledonian University
Brian Randell
Professor
School of Computing Science
University of Newcastle
Uday Reddy
Professor
School of Computer Science
University of Birmingham
Peter Ryan
Professor of Computing Science
University of Newcastle
Geoffrey Sampson
Professor
Department of Informatics
University of Sussex
Martin Shepperd
Professor of Software Technologies
Brunel University
Michael Smith
Visiting Professor
Department of Computer Science
University College London
Tony Solomonides
Reader in Computer Science and Medical Informatics
University of the West of England
Ian Sommerville
Professor
Computing Department
Lancaster University
Harold Thimbleby
Professor of Computer Science
Swansea University
Martyn Thomas
Visiting Professor of Software Engineering
Computing Laboratory
Oxford University
Colin Tully
Professor of Software Practice
School of Computing Science
Middlesex University
