NHS West Midlands Workforce Deanery Briefing Note
From Mmc
Modernising Medical Careers – Medical Training and Application Service (MTAS)
A briefing note
The Workforce Deanery fully supports Modernising Medical Careers and the philosophy behind the MTAS system and welcomes the national review process to get the system to work effectively for the good of junior doctors, SHA staff, the Service and patients.
Modernising Medical Careers & PMETB
• Modernising Medical Careers is an overdue reform of medical education. It has its origins in Calman’s reforms of specialist training (1995) and to Donaldson’s later review of SHO training.
• MMC’s introduction is coupled with the establishment of the new independent regulator for postgraduate medical training - Postgraduate Medical Education& Training Board (PMETB). PMETB’s role is to set the standards for training programmes.
• Both MMC and PMETB are perceived by the medical royal colleges as being agents of the DH.
• Both MMC and PMETB have challenged the central role that the colleges had in postgraduate medical training. The colleges are also threatened by the increasing role of deaneries in the delivery and QA of training programmes.
• The BMA has opposed the shortened training programmes and selection system.
• The overseas doctors issue has polarised opinion – on one hand the feeling is that we need to protect the increasing number of UK graduates, on the other a desire to support the many thousands of overseas doctors who have come to this country – far in excess of the programmes that were available or will be available in the future – the eligibility of overseas doctors is a complex issue and the subject of recent judicial reviews and court action.
• But MMC is live has already successfully delivered new 2-year Foundation Programmes across the UK – specialty training begins in August 2007.
• MMC/PMETB together will raise standards and help produce the types of specialists and GPs that we need for the service and more efficiently.
Entry points to training programmes and the number of programmes
• The number of entry points to GP and specialty training programmes available this August in WM and across the UK is the largest ever.
• We have not reduced any specialty posts in any hospital in WM because we have promised to work with CEOs to ensure the service is not compromised.
• The next problem is how to reduce the number of surgery and anaesthesia SHO posts while not adversely affecting the service and move their funding to create training programmes in specialties that the service needs e.g GP or acute medicine (In WM this means reducing surgery in year one from 120+ to 36 (the same in year 2 and 3).
• We are planning for the changes to begin in 2008 and have set up a joint delivery board with our CEOs in the West Midlands, chaired by Peter Blythin
Selection & interviews
• The old system for recruitment required often hundreds of applications to individual hospitals and deaneries for SHO and GP / specialty training posts – it was highly inefficient for the candidates and for the service.
• GP selection has been under development since 2000 – it is centralised and run through an office in the WM deanery – it’s highly successful and has had no problems so far (though there is concern re functionality of the MTAS computer).
• There have been significant problems with the MTAS computer system – see below.
• There are problems with the short listing system – the design of the questions did not discriminate enough between candidates – this is one of the major concerns of our surgeons.
• We have had more applicants than we expected (12,700 in the WM – we planned for 8000) .
• Increasing numbers of doctors from the Europe and overseas have applied for our training programmes - the problem is how to address the competing demands of the increasing numbers of UK graduates with the demands of overseas doctors – there will be unemployment of doctors because the number of programmes available across the UK (19000) is less that applicants (33000) – it was always like this because of the large numbers of EC and overseas doctors trying to get into the UK – this should have been sorted by the DH / Home Office but there continues to be problems with the eligibility rules for overseas doctors / HSMPS – fraught with legal issues.
• Despite the problems with short listing and the IT system, the feedback from our interviewers has been excellent. The interviews processes were designed with our consultants’ input – there were many excellent candidates – the interviews continue at the deanery.
• The DH review will add more pressure on the staff – we anticipate the need to bring in some temporary / transitional staff and are preparing a project plan.
• Unlike most deaneries, our additional costs are very low because of the use of the research park accommodation – we did not need to book expensive hotels or football stadia. We did not appear in the Sunday Telegraph hit list!
Applicants and Interviews in the West Midlands – the story so far
• 12,758 applicants to the West Midlands.
• 1568 WM candidates and 115 Defence candidates have been interviewed so far • 3387 WM candidates have been called to interview (156 Defence).
• 10 our General Surgeons refused to interview Surgery ST3 candidates on the morning of their interview.
• They voted and took action – one military surgeon and one administrator from UHB abstained.
• Over 81 trainees were affected – all candidates were sent home after being spoken to by those consultant surgeons - some had travelled from overseas (one from Australia, one from New Zealand).
• The surgeons issued a press statement and appeared on television. Note that the emails on the Drs Net UK and Telegraph discussion for a have all supported their stand. Many of our consultants have questioned their action particularly after seen how well the interviews have gone.
• On no occasion have the surgeons criticised the deanery except for stating that due to the MTAS problems we were not able to long-list before short listing for the interviews (they are correct for surgery). This resulted in a few ineligible candidates appearing for interview. Their anger is directed at MTAS; the computer problems and the short listing forms which they feel failed to discriminate between candidates at the ST3 level.
• I would agree with many of their complaints but not their action which was wrong and many feel their actions are unprofessional.
• The only interviews that we have failed to complete so far in the West Midlands are Surgery ST3 (our cardiothoracic surgery STC has postponed their interviews but have not refused).
• We are working with them to rearrange the interviews for the ST3 cohort that was affected by their walk out. Some of the rebel surgeons did attend to interview for surgery at ST2 level last week.
• There have also been a number of consultants who have raised similar concerns. Some have written to the press and the SoS. A few individuals refused to interview in anaesthesia and other specialties but the vast majority continued with their interviews.
• Interviews are going well and according to time table
Implications of the Review
There are major implications for staffing, both medical and administrative. For junior doctors : ST1 - will need to review all non short listed applicants. ST2 – face to face interviews with a trained medical advisor for all applicants not short-listed. ST3/4 – all first and second choice for the West Midlands guaranteed an interview. This will mean circa 10,000 reviews for the West Midlands.
NB See appendix A for more detail
Steve Field March 18th March 2006
Appendix A
Below is the text of what we submitted to the DH Review and more detail about the MTAS problems
NHS West Midlands Workforce Deanery
Modernising Medical Careers – MTAS
Dear Keith,
I promised to send you an update and some thoughts re the process to date for use at the Review Group.
Following the walk out of 12 of our General Surgical colleagues, we have spent a lot of time reflecting on our own deanery processes and the MTAS system generally. I have met with the surgeons concerned and debated their concerns. While I disapprove of their action, they did raise similar concerns to those raised by deanery staff.
As you know, I presented our concerns at the last meeting of the Modernising Medical Careers Programme Board and would have also shared concerns at the UK Modernising Medical Careers Strategy Group if it had not been abandoned due to the fire / bomb scare. I have also listened to the concerns of the BMA and others when I chaired the UK Modernising Medical Careers Advisory Board, but I did not share my thoughts openly at that meeting.
The workforce deanery fully supports Modernising Medical Careers and the philosophy behind the MTAS system and wishes to be part of the review process to get the system to work effectively for the good of our junior doctors, our staff, the service and our patients.
Update
The only interviews that we have failed to complete so far are Surgery ST3 and one person who was short listed for academic rheumatology has yet to be interviewed. You will be well aware that the surgical problem has featured across the national press this week. I know that the group have forwarded comments and concerns to the review group. We are planning to interview again for Surgery ST3 in 2 weeks pending news re whether the President of the RCS has gained agreement (or not) for all those with MRCS to be interviewed – I have sought clarification from the Modernising Medical Careers Team.
One academic group selecting Walport doctors for rheumatology have refused to interview because someone they feel should have been short listed (having looked at the academic forms) didn’t make it through the first stage. I am dealing with that problem.
There have also been a number of consultants in other specialties who have raised similar concerns but have continued with their interviews. The cardiothoracic surgery STC has postponed their interviews and awaits a national announcement from your review group. An e-mail has been distributed to all STC Chairs within the West Midlands by three STC Chairs seeking support for an immediate suspension of round one. I have spoken to the authors, one of which was one of the surgeons. There has been a flurry of phone calls and emails from consultants that I have dealt with personally over the last 5 days – the concerns generally reflect those of our surgical colleagues. Most interview panels and STCs have proceeded but we have lost a handful of consultants who have refused to interview.
Better news
We are beginning to get feedback from many interview panels that despite their concerns, they are able to select some excellent candidates from those interviewed. The mood is lifting in the consultants who have seen the interview process in action. There is nothing but praise for the deanery staff.
MTAS
As a workforce deanery we have major concerns about the MTAS process that we have been involved in during round 1.
Having been involved in developing the GP selection system and having run the original MDAP process for foundation training for three years up to 2006, we are well aware of the advantages and the problems of electronic systems and are up to date on HR selection issues.
The timetable for delivery was tight but became impossible because of the delayed functionality of the MTAS system and its many glitches.
My staff, like those of other deaneries, worked very late each night and over the weekends in order to try and meet what was an impossible deadline. We had 12,700 approx applications – we had expected about 8000. Luckily we had fantastic support from the multiprofessional deanery admin and specialist staff.
We remain very concerned that the change to the London UofA created an apparent 10 day delay in functionality which produced further problems down stream. The closing date for short listing should have been delayed further and not announced just before the weekend – this together with added pressure from some DH staff to deliver caused undue pressure, forced staff to work 12 hour days over the weekend and put the deanery at increasing risk of error. EWTD!! We could not practice what we preach to our junior doctors. I cannot and will not expect my staff to work under such conditions for round 2.
Some suggestions
While we have lessons to learn re our internal deanery systems, I will address the national issues below.
We must all work hard to regain the confidence of our juniors and seniors. A unified communications strategy is essential. Any changes must be communicated as soon as possible. A regular flow of information is necessary to keep our consultants and juniors on board. Deans have an important role to play in this. Information on applications per specialty, competition ratios, numbers of interviews and success rates must be made available quickly; regular updates are essential. Our CEOs must be kept informed so that they can make plans for their consultants to be released from service in good time.
The access to the system for overseas doctors must be clarified as a matter of urgency.
We support the national review of MTAS round one and the need to learn lessons before embarking on a new round two.
We have debated the situation at length and believe that we should proceed with the round one interviews but we must make changes for round two. We do not believe that interviewing all applicants for round one is feasible. We do not believe that it would be supported by the service. If that decision is made then David Nicholson would need to add his weight to the decision and work with SHA CEOs to ensure that consultants were freed up for the deaneries.
We believe that the MTAS system is the way forward. But, it is essential that the MTAS electronic system is refined and the bugs / glitches sorted out. The E system for long listing and short listing must be reviewed and streamlined. The short listing criteria need reviewing e.g. more marks for academic excellence, more discriminatory questions and better instructions to help assessors give marks is essential.
We believe that the GP system of a factual MCQ followed by multi station selection is the way forward for all specialties. The GP system was developed over a 6 year period and works well. We acknowledge, however, that it is too late to develop and validate MCQ tests for the specialties. We must concentrate, therefore, on short listing and the interview process.
The new timetable must be realistic and allow time to change the short listing forms and develop the interview process in partnership with deans, the royal colleges and deans. The timetable must allow adequate preparation time and most importantly, rest for deanery staff – this is a safety issue not just for the staff but also for the candidates, in order to reduce the potential for error. All deans want to provide an excellent service; therefore, deans be supported and must be involved in reviewing progress throughout and must have a veto on delivery dates free from any external pressure.
I trust these comments and suggestions will be helpful to the review group.
Best Wishes
Professor Steve Field
Head of Workforce and Regional Postgraduate Dean NHS West Midlands Workforce Deanery
8th March 2007
Some detailed MTAS issues from the West Midlands
I asked my deputy to collate the problems encountered with MTAS 2 weeks ago (we are now collating the issues in more detail but I add to this paper Alan’s original complaints for information – more will be available soon)
As you know, I presented these concerns to the Modernising Medical Careers Programme Board – these are his words:
Long listing • We were led to believe that the long-listing flags would highlight specific areas of the application form requiring attention so that staff could investigate that area. In MTAS when a flag shows the staff have to go through all 15 pages of the application entry criteria looking for the relevant area. This adds a considerable time to the process which on average is approximately 15 – 20 minutes each application. We were aware that the vast majority of the applications would be submitted in the last couple of days (Friday 2nd, Sat 3rd and Sun 4th Feb) thus making it impossible to long-list before short-listing commenced. Attempts to access MTAS in order to long-list were very slow during the period leading up to the closing date. The long-listing/shot-listing guidance was also issued very late.
• The first opportunity our staff had of seeing the Entry Criteria screens was on 22nd Jan – the day applications opened to candidates.
• Due to the time that it took to undertake long-listing most candidates will be allowed to progress though to short-listing. (This is the only complaint that our surgeons appear to have with the West Midlands part of the process)
Short-listing
• Staff started the process of identifying applications for each of the specialties/levels first thing on Monday 5th Feb. A decision had already been made that the West Midlands would not use the electronic short-listing process as we had concerns that this part of MTAS would not be ready. We gained this function on the Friday.
• In order to be in a position to service the pre-organised short-listing panels a decision was made to start printing off applications, which at that stage did not include the applicants reference number. They then had the laborious task of double checking the screens and writing the reference on the top of the applications. At approximately 4.30pm on Monday 5th Feb a reference number started to appear on applications although the specialty and level was still not present.
• The printing of a batch list summarising the applications was difficult as applications were sometimes missing from the print.
• It was not possible to count the number of applications either for each specialty, level or total number of applications received for the whole Unit of application. • Short list forms – we had several cases of data loss when we tried to use the computer – but either way the white-space questions were not written with an understanding that most of the doctors would look so similar – i.e. 5 years of medical school and 2 – 3 years of foundation / SHO and the academic questions could have offered marks for Masters degrees etc. Someone who was coached could gain lots of marks and be a poorer candidate who was honest and factual. Plagiarism software was absent.
Post Upload
• All posts had to be individually added to MTAS as there was no bulk upload facility. This resulted in double data entry from our own spreadsheets and resulted in approximately 10 days additional data entry each from 3 people. This was available for Foundation selection.
• The computer was extremely slow on a number of occasions which often lasted some hours. Scoring
• The National Short-listing Scoring Indicators cannot be used by short-listing panel members as they do not have an area for the applicant number to be added or for the panel member to add comments and their signature. A whole series of additional score-sheets have had to be produced for each specialty and level. Applicant “Choose & Book” • Functionality was very late arriving – this seems to be a general problem with Methods. Training
• There has been no national training on MTAS looking specifically at the ST module – this is due to the modules not being available until the day of launch. Training had to be done in a piece-meal fashion with staff as and when elements of MTAS become available. Bugs on MTAS
• There are numerous bugs identified such as:
- GP questions appearing on the CMT long-listing sections
- Application “sign off” by applicants not being mandatory
- Employment history start dates not being mandatory – making it impossible to gauge the length of an applicants experience.
- The Plagiarism Finder did not work
- Countless problems with data loss – we lost over 1300 applications on the day before the closing date and had many separate episodes of data loss – candidates also appeared on the screen unannounced during the short listing period! – As a consequence, the staff have no confidence in the system.
