ANC Workshop - Asa Cooper Stickland and Paul Micaelli

Chair: Angus Chadwick

Speaker: Asa Cooper Stickland

Title: Robustness and Calibration on Out-of-domain Data with Ensembles and Language Models.

Abstract:

Modern deep neural networks can produce badly calibrated predictions, especially when train and test distributions are mismatched. In a recent ICML workshop paper 'Diverse Ensembles Improve Calibration', we propose a simple technique to improve calibration, using a different data augmentation for each ensemble member. These techniques improve calibration and accuracy over strong baselines on the CIFAR10 and CIFAR100 vision benchmarks, and out-of-domain data from their corrupted versions. I'll also present some early results on calibration for large language models like BERT on out-of-domain data.

Bio: Asa is a 2nd year PhD student with Iain Murray, working on transfer learning and robustness in NLP. He's previously worked on adapting large language models like BERT to new tasks, and completed an internship at Facebook AI concentrating on adapting multilingual pre-trained models to machine translation. More recently, Asa has been interested in calibration and robustness to domain shifts.

Speaker:  Paul Micaelli

Title: Meta-learning for problems requiring many gradient steps.

Abstract:

Meta-learning, or learning to learn, has seen a resurgence of interest in the last 3 years. Unfortunately, memory scaling and gradient degradation issues have limited meta-learning methods (e.g. MAML) to simple inner problems with small horizons. In this talk, I will cover two approaches that enable meta-learning over many steps:  forward-mode differentiation and implicit differentiation. These methods have the potential to make meta-learning a ubiquitous part of any machine learning pipeline.

Bio:

Paul is a PhD student supervised by Amos Storkey. His research first considered zero-shot knowledge transfer of large neural networks, but is now mainly dealing with meta-learning. In particular, Paul is interested in scaling meta-learning tools to real-world settings where datasets and models are large.