Dialogue and Multimodal Interaction
A list of potential topics for PhD students in the area of Dialogue and Multimodal Interaction.
Robot Learning via Trial and Error and an Extended Conversation with an Expert
Supervisor: Subramanian Ramamoorthy
A field of robotics known as Learning from Demonstration teaches robots new skills through a mix of trial and error and physical enactment or manipulation by a human expert. There is some preliminary work that enhances this evidence with linguistic utterances, but their underlying messages are rudimentary (e.g., "no"), or pertain to just the current situation (e.g., "go left"). This project will investigate how using current semantic parsing and symbol grounding can enhance the task of learning optimal policies, when the expert's utterances include quantification and abstraction (e.g., "when putting fruit in a bowl, always grasp it softly and lower it slowly").
Next-Generation Multimodal Conversational Task Assistance
Supervisor: Jeff Dalton
The aim of this project is to build on next-generation conversational assistants that help people perform real-world tasks. It builds on multimodal generative language models for automatically understanding tasks and using them to perform personalized task adaptation based on factors like expertise, preferences, and constraints. It develops new methods for knowledge-grounded generation of information for a dynamic environment. It studies the interplay between Human and AI systems to not just perform tasks, but also to teach and entertain in the process. It builds on the open-source Open Assistant Toolkit deployed to millions of users in the Amazon Alexa Prize to study real-world use cases.
Computational Models of Privacy
Supervisor: Nadin Kökciyan
Goal: To develop a personal privacy assistant that will work with its user to handle multi-party privacy in online systems
Online systems, such as social networks, may violate the privacy needs of the users who have little control of their own data. Privacy is one of the most important ethical values to preserve, as highlighted by its protection in law, such as under GDPR. This project aims to develop a personalised privacy assistant that can: (i) analyse content (e.g.; text, image) by using neurosymbolic approaches to ensure its user's privacy is preserved before sharing data, (ii) enable multi-party privacy when the content is revealing sensitive information about multiple users (e.g. a picture with friends), (iii) conduct dialogues in natural language to collaborate with its user to make ethical privacy decisions.