11 November 2019 - Chris Dyer: Seminar

TITLE:   Unsupervised Linguistic Structure Discovery with Deep Learning

ABSTRACT: 

The learning mechanisms that enable children to robustly acquire knowledge of latent linguistic structures (words, phrases, syntactic relationships) from unstructured examples of language observed in their environments have been the subject of intense study for over half a century. Despite this scientific effort—not to mention the reliable success that children learning their first language exhibit—we still have no precise model of this learning process. In this talk I report on my attempts to use deep neural networks to recover these structures without supervision, focusing on two general approaches: one that identifies correlates of these structures in non-structured models such as LSTMs ("structure as analytical artifact") and another that reifies them as actually existing components in structure-based models ("structure exists"). I provide empirical evidence that explicitly structured models result in more accurate generalization and better (latent) structure discovery, as well as theoretical arguments that searching for correlates of structure in deterministic neural networks such as LSTMs cannot account for all of the behavior that latent structure explains in conventional linguistic analysis. Additionally, while deep architectures often provide substantially better estimates of held out data distributions compared to classical statistical models, they also tend to recover consistent and accurate latent structures less reliably.

BIOGRAPHY: 

Chris Dyer is a senior staff scientist with DeepMind Technologies and also maintains a Consulting Professor appointment in the School of Computer Science at Carnegie Mellon University. He is a 2016 recipient of the Presidential Early Career Award for Scientists and Engineers (PECASE), and has won best paper awards at ACL, EMNLP, and NAACL. He lives in London and plays the cello.

 vCal  iCal