8th March 2021 - 5pm - Matt Gardner: Seminar

Title: Contrastive evaluation and learning in NLP

Abstract:

Any dataset created by humans will almost unavoidably have spurious correlations between inputs and outputs.  This means that when we collect data and split it into train and test sets, models that maximize the likelihood of the data will tend to find these spurious correlations, and they will use them to perform better than they should at test time.  I will show that this problem is pervasive in natural language processing, extending even to traditional NLP tasks such as dependency parsing, and I will briefly demonstrate one method to partially solve this problem in our evaluations, by generalizing the long-standing notion of a "minimal pair".  Solving the problem during training is more challenging.  As a start, I will present work that leverages consistency on related examples during training to improve compositional reasoning in neural module networks.  This is admittedly a very narrow solution to the problem; if time permits, I will also discuss some ongoing work that generalizes these ideas.

Bio

Matt is a senior research scientist at the Allen Institute for AI on the AllenNLP team. His research focuses primarily on getting computers to read and answer questions, dealing both with open domain reading comprehension and with understanding question semantics in terms of some formal grounding (semantic parsing). He is particularly interested in cases where these two problems intersect, doing some kind of reasoning over open domain text. He is the original architect of the AllenNLP toolkit, and the instigator of the NLP Highlights podcast.

Add to your calendar

 vCal  iCal