26 April 2019 - Willem Zuidema: Seminar
Understanding neural grammars to understand the brain
Deep neural networks have in the last 5 years quickly come to dominate Natural Language Processing. For understanding how humans process and learn language, however, these models are currently less useful, given (i) the lack of understanding of how they actually solve the task, (ii) the lack of a mapping from components of the model to components of the human brain and mind; and (iii) the massive amounts of data needed to train them. In my talk, I discuss progress on the first two of these challenges. First, we open up the ‘black boxes' of DNNs using a methodology we call ‘diagnostic classification', that involves formulating and testing symbolic hypotheses on their inner workings. Using this methodology, we can determine how, e.g., LSTMs process hierarchical structure and represent logical inference relations between sentences in artificial tasks. Second, we study the mapping from components of the LSTMs to brain activation, by training a separate machine learning model to predict brain activation, as measured using fMRI, from the internal variables (hidden layers and gate values) of the LSTM. Using this approach, and applying it to the fMRI dataset of Wehbe et al. 2014, we can predict, with better accuracy than existing baselines in the literature, the brain activation associated with words when participants are reading a chapter from Harry Potter. Moreover, we can track which parts of the brain show activity that is best or least predictable from components of the LSTM. Integrating the work on interpretability and brain prediction holds the promise of providing a novel way of linking brain activation to linguistic function, and might indirectly, I will argue, help reducing the amount of data DNNs need.
Willem Zuidema is reader in computational linguistics and cognitive science at the Institute for Logic, Language and Computation of the University of Amsterdam. He obtained his PhD (2005) at Edinburgh, did postdocs at Leiden and Amsterdam. He is recipient of the VENI-fellowship, a NIAS-Lorentz fellowship, and currently is research fellow in the national Language in Interaction project. His research focuses on interpretability and cognitive relevance of modern natural language processing models.