A programme of work evaluating the University Clinical Aptitude test's (UCAT) properties

The UCAT is a selection assessment required for entry to undergraduate medicine to the majority of UK medical schools. This programme of quantitative work seeks to generate evidence regarding the properties of the UCAT and how scores from the assessment relate to outcomes of interest. These include performance at knowledge based exams and clinical simulations, as well as any professionalism issues. This work is providing an evidence base for medical selection. This is vital to ensure that selection assessments are used in a way which is fair and helps choose those most suited to a medical career, whilst avoiding negative impacts on the recruitment of underrepresented groups to the profession (i.e. widening access). In order to gain this evidence we apply psychometric and epidemiological statistical methods to analyse national cohort data, often from the UK Medical Education Database.

Our previous work has focussed on whether scores from the UCAT predict undergraduate outcomes, such as examination scores, and how UCAT scores relate to other metrics used in selection, such as A-level performance, as well as sociodemographic characteristics.

Our current work includes analysing whether UCAT scores can predict performance later into postgraduate training, and also what role the UCAT situational judgement test should optimally play in medical selection.

We are also undertaking research into the four extended versions of the UCAT, which exist for those entitled to reasonable adjustments. Around 6% of applicants sit an extended version of the UCAT. However, relatively little is known about those candidates who sit an extended version of the UCAT, or how scores may be related to demographic factors, such as gender or age. This work will thus be vital to informing developments of extended versions of the UCAT, will have implications for eligibility criteria and outreach, and will inform future research into UCATSEN.

We use national data to build statistical models which evaluate the relationship between those factors we are interested in (such as scores on the UCAT) and outcomes of interest (such as performance in postgraduate training.

Predictive validity of the UKCAT for medical undergraduate performance: a national prospective cohort study. Tiffin, P. A., Mwandigha, L. M., Paton, L. W., Hesselgreaves, H., McLachlan, J. C., Finn, G. M. and Kasim, A. S. BMC Medicine, 2016:14;1-19

Predictors of Fitness to Practise Declarations in UK Medical Undergraduates. Paton, L. W., Tiffin, P. A., Smith, D., Dowell, J. S. and Mwandigha, L. M. BMC Medical Education, 2018:18.

What is the effect of secondary (high) schooling on subsequent medical school performance? A national, UK-based, cohort study. Mwandigha, L. M., Tiffin, P. A., Paton, L. W., Kasim, A. S and Boehnke, J. R. BMJ Open, 2018:8;e020291

The ability of ‘non-cognitive’ traits to predict undergraduate performance in medical schools: a national linkage study, Finn, G. M., Mwandigha, L. M., Paton, L. W and Tiffin, P. A. BMC Medical Education 2018:18

Artificial or intelligent? Machine learning and medical selection: possibilities and risks. Tiffin, P.A and Paton, L.W. MedEdPublish, 2018:7;4.

Does ‘online confidence’ predict application success and later academic performance in medical school? A UK-based national cohort study. Tiffin, P.A and Paton, L.W. BMJ Open, 2019:9;e034437

Can achievement at medical admissions tests predict future performance in postgraduate clinical assessments? A UK-based national cohort study Paton, L. W., McManus, I. C., Cheung, K. Y. F., Smith, D. T. & Tiffin, P. A. BMJ Open, 2022;12:e056129.