Task Characteristics and Calibration

Abstract

This chapter outlines the procedures for calibrating and establishing the properties of the collaborative problem solving tasks in the ATC21STM project (The acronym ATC21STM has been globally trademarked. For purposes of simplicity the acronym is presented throughout the chapter as ATC21S.). The chapter deals with the interpretation of these tasks and provides an outline of how they were used, discussing the data they yielded, the interpretation of the CPS construct and the calculation of the student skill-levels measured. Using item response theory, the tasks were calibrated separately and jointly. One and two parameter item response models were used to explore the data and to determine dimensionality. The data were analysed on one, two and five dimensions, corresponding with the theoretical components of the collaborative problem solving construct. Tasks were calibrated in sets of three and these sets were used to determine that there were no significant differences between countries in the difficulty of the items. Difference in mean latent ability of Student A and Student B was also analysed, and it was concluded that there was no advantage or disadvantage to students adopting either role. The task calibrations were used to determine the hierarchy of the indicators, and describe student competency levels as measured by the tasks. Skills progressions were created for one, two and five possible dimensions as interpretations of the collaborative problem solving continuum. In this chapter we describe the methods used to develop the progressions from novice to expert, which provide a framework for teachers to use in interpreting their observations of student behaviour regarding collaborative problem solving.