System Reliability and Performance
From Nhs It Info
Revision as of 08:29, 10 October 2006
Major incidents hit NHS national systems
Computer Weekly, 19 Sep 2006
"More than 110 "major incidents" have hit hospitals across England in the past four months, after parts of the health service went live with systems supplied under the £12.4bn National Programme for IT (NPfIT) in the NHS. Many of the incidents, which have been reported by Connecting for Health, the body that oversees the NPfIT, involve the failure of x-ray retrieval hardware and software, known as Pacs (picture archiving and communications systems) which allow clinicians to view digitised x-rays on screen. . . The major incidents also involve hospital patient administration systems, which hold patient details such as appointments and planned treatments. The specifications for services to be supplied under the NPfIT built up an expectation among NHS staff and clinicians that they would receive sub-second response times, and that equipment would be available to them 99.99% of the time. But the list of major incidents seen by Computer Weekly shows that in some cases NHS staff and clinicians have lost access to their main hospital systems. More than 20 major incidents have affected multiple NHS sites. This raises questions about whether the risks of failure after go-live have been adequately assessed, and whether any independent regulator has an overview of the riskiest implementations across England. . . Some of the listed incidents were fixed quickly, though others lasted much longer. . ."
NPfIT systems failing repeatedly
e-Health Insider, 20 Sep 2006
http://www.e-health-insider.com/news/item.cfm?ID=2144
“More than 110 major incident failures have been reported by hospitals and GPs over the past four months relating to systems provided by the NHS National Programme for IT. The problems, which have affected dozens of hospitals across England, were serious enough to be logged by NHS managers as ‘major incidents’. The issues were revealed by an anonymous NHS IT director speaking to Computer Weekly. The IT director told the magazine: "Some NHS trusts that have implemented Connecting for Health [centrally-bought] solutions are struggling to cope with poor system performance and service availability issues. "The local service provider is working flat out to resolve the issues. However, a great deal of damage has been done in terms of deteriorating end-user confidence and satisfaction with respect to the systems." E-Health Insider understands that the 110 serious incidents reported by Computer Weekly may actually understate the true number of problems. Industry sources say that some problems are routinely not reported or recorded or classified as less serious. For instance, the July data centre failure that affected 80 trusts is understood to have been counted as a single major incident. EHI has also learned that a 9 September failure that resulted in the iSoft system delivered by Computer Sciences Corporation to Morecambe Bay Hospital NHS Trust becoming unavailable to all staff was only treated as an 'amber' incident, rather than a 'red' major incident. The contractual specifications for services to be supplied under the NPfIT say that staff and clinicians will receive sub-second response times, with 99.99% availability. But in many cases staff have found systems can either be extremely slow, impossible to access or unavailable to them for hours or even days. . . While the early problems will hopefully just prove teething problems, they raise the spectre that staff will not be able to fully rely on CfH systems and will still need to maintain old systems and paper records. The programme has yet to begin widespread delivery of clinical rather than administrative systems. . .”
NHS suppliers face review of disaster plans
Computer Weekly 15 Aug 2006
“Connecting for Health (CfH), which runs the National Programme for IT in the NHS, has ordered a review of disaster recovery arrangements for all five of its local service providers following failures at a datacentre run by CSC earlier this month. The outages left hospital trusts in the North West and West Midlands without access to patient administration systems for up to five days. CfH contracts with local service providers specify that storage area networks at the heart of disaster recovery provisions must have no single point of failure, 99.9% availability and zero data loss. “The disaster recovery restored time within contracts depends upon the services affected. This is currently between two and 72 hours. However, by January 2007, all services must be restored within two to 12 hours,” said a CfH spokesman.”
E-mail reveals outage disrupted patient care
e-Health Insider, 7 Aug 2006
http://www.e-health-insider.com/news/item.cfm?ID=2054
“One week on from the computer failure that left 80 trusts in the North-west and West Midlands without access to their IT systems the extent of the disruption to patient care of the biggest ever NHS IT failure is coming to light. Despite claims to the contrary by NHS Connecting for Health, E-Health Insider has received documentation showing the failure disrupted patient care at Birmingham Children’s Hospital (BCH) NHS Trust - one of eight acute trusts that lost access to patient data last week. As a result BCH has begun a review of its contingency arrangements. . . An internal e-mail from Richard Beekan, the trust’s director of operations, is explicit about the impact the loss of the Lorenzo patient administration system had. Once the trust lost access to the patient administration system (PAS) it had to revert to paper based “business continuity systems. This system was introduced expecting the system only ever to be unavailable for a maximum of 12 hours and therefore during the last three days we have experienced issues we had not planned for. In particular the absence of our case note tracking system and an ability to know where notes were had an impact in both out patients and inpatient areas.” Last week NHS Connecting for Health (CfH), the agency responsible for the NHS IT modernisation project, publicly stated in bulletins that the failure at the CSC data centres had no impact on patient care. On 2 August, CfH said: “To date no impact on the delivery of patient care has been reported.”
NHS computer system 'won't work'
The Observer 6 Aug, 2006
http://observer.guardian.co.uk/politics/story/0,,1838470,00.html
Leaked analysis says hospitals would be better off without national upgrade. The project to overhaul the NHS's computer systems, costing millions, is so beset by problems that hospitals would be better off if they had never tried to implement it, according to a confidential document apparently sent by one of the scheme's most senior executives. A 12-page analysis detailing why the project will never work was sent anonymously to an MP on the Public Accounts Committee from the computer of David Kwo who, until last year, was in charge of implementing the Connecting for Health system across London. . . Kwo did not return emails or telephone calls from The Observer, but the Microsoft Word document reveals that it was written on his computer. What is irrefutable is that the devastating analysis of the flawed computer system - which is two years behind schedule - could have been written by only a handful of senior NHS IT experts who have worked on the project. 'The conclusion here is that the NHS would most likely have been better off without the national programme, in terms of what is likely to be delivered and when,' states the document, sent to Conservative MP Richard Bacon and obtained by The Observer. 'The national programme has not advanced the NHS IT implementation trajectory at all; in fact, it has put it back from where it was going.' As the problems have increased, GPs' surgeries have opted to implement their own systems, something which the document observes is 'fragmenting the national programme further'. Many hospitals are 'being forced to deliver outdated legacy systems, which the programme was established to replace.
Experts try to fix NHS IT failure
BBC News 1 Aug 2006
http://news.bbc.co.uk/1/hi/health/5233604.stm
“Technicians are trying to solve a computer failure that has prevented 80 NHS trusts gaining access to patients' records and admissions since Sunday. Eight major hospitals and more than 70 primary care trusts in north-west England and the West Midlands were hit. . . The problem affects trusts in Birmingham and the Black Country, Cheshire and Merseyside, Cumbria and Lancashire, Greater Manchester, Shropshire and Staffordshire and the southern part of the West Midlands. Computer company CSC, which runs the system, said experts were working around the clock to resolve the situation. A spokesman for NHS Connecting for Health, which oversees the multi-billion pound NHS IT service, said that no data had been lost, and that the incident was caused by "storage area network equipment failure".”
Fears over faults in NHS patient records system
The Observer 25 Jun 2006
http://observer.guardian.co.uk/uk_news/story/0,,1805437,00.html
"The multi-billion pound computer system built to run NHS patient records is experiencing so many problems that there are concerns people could be put at 'clinical risk', with missed appointments and lost records meaning that some hospitals have pulled out of the scheme in despair. Confidential documents and emails obtained by The Observer reveal the scheme's progress is plagued by technical problems that threaten lengthy delays for patients needing to see specialists. . . Industry sources familiar with the project told The Observer that the problems have seen many hospitals or trusts postpone the system's implementation. Just 12 of England's 176 major hospitals have implemented even the most basic part of the new system which electronically books patient appointments with specialist consultants - despite the fact 104 had agreed to have it operating by April. Furthermore, not one NHS trust or hospital in England has implemented the second phase of the system, which will allow doctors to order clinical services such as blood tests or X-rays electronically - contrary to the Department of Health's planned timetable.
Paper working after disaster 'not acceptable'
e-Health Insider, 1 Feb 2006
http://www.e-health-insider.com/news/item.cfm?ID=1678
". . . some acute trusts in Accenture's regions found their patient administration systems (PAS) were not working for a week and had to go back to working on paper."
A spineless performance
The Guardian, 12 Jan 2006
http://society.guardian.co.uk/e-public/story/0,,1684068,00.html
"The system at fault was not the booking software as such, but in the underlying digital "spine" supposed to connect all parts of the NHS in England. Officials had previously boasted that the spine would be available 99.8% of the time, with recovery within 30 minutes of any crash. . . The trouble began on December 18 with the installation of a major upgrade of the spine software. . . The new software reacted badly with one of the many different systems used by GPs to manage their practices, and generated spurious messages that overwhelmed networks and servers. This rogue behaviour masked other incompatibilities between the new demographics service and the "choose and book" software. "We were into Christmas before we were able to start diagnosing," said one of the team who worked over the holiday to resolve it."
Patient data errors created by iSoft's iPM system
e-Health Insider, 9 Jan 2006
http://www.e-health-insider.com/news/item.cfm?ID=1632
"A flaw has been identified in the iSoft iPM patient administration system being provided as the standard solution to NHS trusts in the North West and West Midlands that can corrupt patient data creating suspected clinical risks to patients"
NHS User Survey: Appendices 1-6
TFPL Ltd. for NHS NHS Connecting for Health, 17 Jun 2005
http://www.library.nhs.uk/nlhdocs/Appendices_1-6.pdf
"Not surprisingly the professional population canvassed are comfortable using e-resources though not everyone was confident that they used them well. House officers experience frustration with changing Athens passwords as they moved locations. Manager's views of Athens were mixed – some had no issues, others experienced technical unreliability. Firewalls present another issue – managers get over this by using some resources from home. . . Access to Athens needs to be more reliable and easier to use. Athens takes too long to use and access is not technically reliable enough."
