ANC Workshop - 21/01/2020

Chair

Michael Gutmann

Speaker

Cian Eastwood

Title

Online adaptation of neural networks in the face of covariate shift

Abstract 

In the real world, environments are generally non-stationary and the conditions under which a model was developed will often differ from those in which it will be used. Unfortunately, most learning algorithms for neural networks work by ignoring these environmental "shifts" or condition "mismatches", blindly (and usually incorrectly) assuming that the data distribution does not change. Assuming that the data distribution does not change makes these algorithms ill-prepared to react when it does. Lacking mechanisms to determine what has changed, they are unable to determine how learnt knowledge should be updated, i.e. which parameters should be updated, and which should not. In this talk, I will present an approach to determining which parameters in the network should update, and which should not, so as to facilitate fast online adaptation to new data distributions. The proposed method determines the "plasticity" of a given unit?s input parameters (i.e. whether or not they can be updated) as a function of

i) recent activation history and ii) modulatory signals from neighbouring units. Our aim is to show that these history-dependent, locally-coordinated updates allow a small number of parameters to "take responsibility" for an adaptation. Furthermore, we aim to show that these sparser updates better-preserve learnt structure, and as a result, achieve faster adaptation by transferring past knowledge.

Note: I will present our preliminary results and planned future work towards these aims.

 

Speaker

Sohan Seth

Title

Archetypal Distributions

Abstract

Archetypal analysis is an unsupervised learning tool commonly used in exploratory data analysis dimensionality reduction interpretation and visualization. We extend this idea to find archetypal distributions given a set of probability distributions. This is useful for example when we report the uncertainty in a measurement alongside the measured value. We propose a principled approach to tackle this situation using partial membership model. We discuss the connection between the proposed approach and existing extensions of archetypal analysis namely probabilistic archetypal analysis kernel archetypal analysis interval archetypal analysis and statistical archetypal analysis and apply this approach to both synthetic and real data to investigate its properties and effectiveness.

 

Jan 21 2020 -

ANC Workshop - 21/01/2020

Sohan Seth, Cian Eastwood

G.03, IF