Mina Huh, an undergraduate student in KAIST School of Computing, received the Grand Prix at the 3rd presentation hall of the 2020 Winter/Spring Undergraduate Research Participation (URP) Program Workshop held online on Thursday, August 27th. (advisor: Prof. Juho Kim, TA: Ph.D. Candidate Minsuk Chang).

URP program is a project supported by the Ministry of Education and the Korea Science and Creativity Foundation to enhance the creative research capabilities of undergraduate students and provide opportunities for self-directed research.

Title:
Data Modeling Techniques for Supporting Conversational Interaction in Video Tutorials

Abstract:
When watching video tutorials, viewers often need to pause, rewind, or skip to clarify misunderstandings, and to find missing or anticipated information. The target scene can be referenced in terms of their content, or by their temporal location. Voice interfaces are theoretically useful for controlling video tutorials, especially when the hands are tied by following physical tasks. However, they only support remote-control like commands such as “skip 30 seconds” that are based on temporal references. As a result, viewers must remember a sequence of events, their temporal locations and maintain a mapping between them while following the tutorial. We present RubySlippers, a multimodal system that allows users to navigate video tutorials by conversational description of the content, while keeping the time-driven navigation. The system runs on an NLP pipeline which compares the semantic composition of the user’s utterance and that of the video content to populate navigation target candidates. Our evaluation with 12 participants shows that users are able to navigate quicker with lower mental demand with RubySlippers than existing time-driven voice interfaces.

Congratulations on the award.