Enabling robots to make use of the structure of human actions - a user study employing Acoustic Packaging

Human learning strongly depends on the ability to structure the actions of teachers in order to identify relevant parts. We propose that this is also true for learning in robots. Therefore, we apply a method for multimodal action segmentation called Acoustic Packaging to a corpus of pairs of users teaching object names to a robot. Going beyond previous use cases, we analyze how the structure of human actions changes if the robot is learning quickly or slowly. Our results reveal differences between action structuring in the conditions such as longer utterances and more motion when the robot learns slowly. We also evaluate how the partners in the pair influence each other's action structuring. The results show a strong correlation between the participants in the pairs, even more so in the trials where the robot is learning slowly. We conclude that the action structuring based on Acoustic Packaging allows robots to differentiate how well the interaction with multiple users is going and is, thus, a vehicle for feedback generation.