Robot-Assisted Shoebox-tasks

Shoebox-tasks are learning activities used in education for persons with Autism Spectrum Disorder (ASD). Their clear visual structure makes them suitable for vision-based machine learning. Our research focuses on automatically analysing these tasks so a robot can solve them autonomously. We convert camera data into symbolic task descriptions (PDDL), action plans, and robot actions.

There are two main use cases: the robot acts as a teacher, guiding the student during task solving, or the student guides the robot and teaches him how to solve the task. For the second case, we develop a visual programming interface.

Inspired by visual schedules used in educational approaches such as TEACCH®, where they support students by visualizing daily activities, we use analogous visual schedules to represent algorithms for the robot.

Our goal is to motivate students with ASD to engage with robotics through a structured and supportive environment. We hope to provide new opportunities for learning, while also opening potential pathways toward future interests and studies.

Lada Kudláčková
Charles University, Faculty of Mathematics and Physics