Gingerich, A, Schokking, E and Yeates, P ORCID: 0000-0001-6316-4051 (2018) Comparatively salient: examining the influence of preceding performances on assessors' focus and interpretations in written assessment comments. Advances In Health Sciences Education. ISSN 1573-1677

[img] Text
Comparatively salient ACCEPTED VERSION Jul 5.docx - Accepted Version
Restricted to Repository staff only until 6 July 2019.
Available under License Creative Commons Attribution Non-commercial.

Download (57kB)

Abstract

Recent literature places more emphasis on assessment comments rather than relying solely on scores. Both are variable, however, emanating from assessment judgements. One established source of variability is "contrast effects": scores are shifted away from the depicted level of competence in a preceding encounter. The shift could arise from an effect on the range-frequency of assessors' internal scales or the salience of performance aspects within assessment judgments. As these suggest different potential interventions, we investigated assessors' cognition by using the insight provided by "clusters of consensus" to determine whether any change in the salience of performance aspects was induced by contrast effects. A dataset from a previous experiment contained scores and comments for 3 encounters: 2 with significant contrast effects and 1 without. Clusters of consensus were identified using F-sort and latent partition analysis both when contrast effects were significant and non-significant. The proportion of assessors making similar comments only significantly differed when contrast effects were significant with assessors more frequently commenting on aspects that were dissimilar with the standard of competence demonstrated in the preceding performance. Rather than simply influencing range-frequency of assessors' scales, preceding performances may affect salience of performance aspects through comparative distinctiveness: when juxtaposed with the context some aspects are more distinct and selectively draw attention. Research is needed to determine whether changes in salience indicate biased or improved assessment information. The potential should be explored to augment existing benchmarking procedures in assessor training by cueing assessors' attention through observation of reference performances immediately prior to assessment.

Item Type: Article
Additional Information: This is the accepted author manuscript (AAM). The final published version (version of record) is available online via Springer at http://doi.org/10.1007/s10459-018-9841-2 - please refer to any applicable terms of use of the publisher.
Uncontrolled Keywords: assessment of clinical performance, assessor cognition, comparative distinctiveness, contrast effects, mini-CEX, rater bias, rater cognition, rater error, rater-based assessment, salience
Subjects: R Medicine > R Medicine (General) > R735 Medical education. Medical schools. Research
Divisions: Faculty of Medicine and Health Sciences > School of Medicine
Related URLs:
Depositing User: Symplectic
Date Deposited: 27 Jul 2018 10:55
Last Modified: 27 Jul 2018 11:02
URI: http://eprints.keele.ac.uk/id/eprint/5138

Actions (login required)

View Item View Item