IPAB Workshop-25/04/2024

Speaker: Stefano Albrecht

Title: From Deep Reinforcement Learning to LLM-based Agents: Perspectives on Current Research

Abstract: Since the recent successes of large language models (LLMs), we are beginning to see a shift of attention from deep reinforcement learning to LLM-based agents. While deep RL policies are typically learned from scratch to maximise some defined return objective, LLM-agents use an existing LLM at their core and focus on clever prompt engineering and downstream specialisation of the LLM via supervised and reinforcement learning techniques. In this talk, I will first provide a broad overview of my group’s research in deep RL, which focuses among other topics on developing sample-efficient and robust RL algorithms for both single- and multi-agent control tasks, including industry applications in autonomous driving and multi-robot warehouses. I will then present our recent research into LLM-agents, where we propose an approach for household robotics that takes into account user preferences to achieve more robust and effective planning. I will conclude with some personal observations about the state of LLM-agent research: (a) many papers in this field follow essentially the same recipe by focussing on prompt engineering and downstream specialisation; (b) this recipe makes their scientific claims brittle as they depend crucially on the specific LMM engine, and (c) LLMs are not natively designed to maximise objectives for optimal control and decision making. Based on these observations, I believe some fruitful research avenues can be identified.