ANC Workshop - Chris Williams

Tuesday, 8th November 2022

On Suspicious Coincidences and Pointwise Mutual Information

Abstract: 

Barlow (1985) hypothesized that the co-occurrence of two events A and B is 'suspicious' if P (A, B) >> P (A)P (B). We first review classical measures of association for 2 ?? 2 contingency tables, including Yule's Y (Yule, 1912), which depends only on the odds ratio lambda, and is independent of the marginal probabilities of the table. We then discuss the mutual information (MI) and pointwise mutual information (PMI), which depend on the ratio P (A, B)/P (A)P (B), as measures of association. We show that, once the effect of the marginals is removed, MI and PMI behave similarly to Y as functions of lambda. The pointwise mutual information is used extensively in some research communities for flagging suspicious coincidences. We discuss the pros and cons of using it in this way, bearing in mind the sensitivity of the PMI to the marginals, with increased scores for sparser events.

Inference and Learning for Generative Capsule Models

Abstract: 

Capsule networks (see e.g. Hinton et al., 2018) aim to encode knowledge of and reason about the relationship between an object and its parts. In this paper we specify a generative model for such data, and derive a variational algorithm for inferring the transformation of each model object in a scene, and the assignments of observed parts to the objects. We derive a learning algorithm for the object models, based on variational expectation maximization (Jordan et al., 1999). We also study an alternative inference algorithm based on the RANSAC method of Fischler and Bolles (1981). We apply these inference methods to (i) data generated from multiple geometric objects like squares and triangles ("constellations"), and (ii) data from a parts-based model of faces. Recent work by Kosiorek et al. (2019) has used amortized inference via stacked capsule autoencoders (SCAEs) to tackle this problem???our results show that we significantly outperform them where we can make comparisons (on the constellations data).

Joint work with Alfredo Nazabal and Nikolaos Tsagkas.

Event type: Workshop

Date: Tuesday, 8th November 2022

Time: 11:00

Location: G.03

Speaker(s): Chris Williams

Chair/Host: Andrea Weisse