Evaluation of teachers' orchestration tools usage in robotic classrooms

Abstract

Teachers' self-efficacy in managing classrooms is an important consideration when it comes to bringing educational robots to classrooms. Orchestration tools support teachers by providing awareness indicators of students' progress as well as levers to control the flow of the lesson. We designed and evaluated the impact of an orchestration tool for a series of robot-based learning activities to teach a basic concept in mathematics to children, aged 7-10. Six teachers in primary schools across Switzerland used the orchestration tool to manage the activities in six sessions involving a total of ninety-one students. We observed teachers' usage of the orchestration tool during the sessions and interviewed them after the sessions about the usefulness of these functionalities. Our findings show that even though teachers used the tool in different ways from each other, in general, it supported them in their classroom orchestration practices, mainly to manage the activity sequence and get aware of the robot technical failures and, to a lesser degree, get aware of students' progress for the purpose of activity transitions and enriching class discussions. We discuss the theoretical implications of these results, relating our findings to the literature on classroom orchestration tool design, especially highlighting the importance of educational level and the type of learning technology as contextual factors affecting teachers' usage of orchestration tools. We also provide implications for designing orchestration tools, focusing on the necessity of providing different types of awareness indicators and multiple options for activity management to fulfil the variety of teachers' orchestration needs.