A study conducted by researchers from Cambridge Assessment has compared the marking of GCSE English Literature essays on-screen and on paper. The aim was to explore how the change in mode might affect a range of aspects of marking.
A fundamental question to be addressed was how far the change might impact on the reliability of marking and the design of the study enabled this question to be investigated empirically. Other issues investigated included: whether markers considered different features of the essays when marking in different modes; how the cognitive load was affected when digital images were marked on-screen; differences in navigation, annotation and speed of marking between modes.

A mixed method design involved marks being statistically compared across both modes with an independent reference mark for each essay. Kelly’s Repertory Grid technique was used to elicit constructs to which the markers attended in each mode. The marking reliability analyses were then used to infer any potential relationship between construct recognition and mode. Examiners’ cognitive load was measured during task completion using the National Aeronautics and Space Administration Task Load Index (Hart and Staveland, 1988). Navigation of texts and annotation practices were coded, observed and compared across a sample of scripts and across both modes. The quantitative and qualitative mixed method approach resulted in a rich data source through which the impact of the change in marking mode could be probed.

Knowledge of how the introduction of new technology into marking practices may affect marking processes is important, especially in the context of essay marking where more creative writing tasks may be assessed. The results from this study will be presented and the implications of the findings for marking longer texts on-screen will be discussed.