Governance Node Seminar - Svetlin Penkov & Daniel Angelov, Efemarai

The industrial machine learning operations pipeline needs to meet demands that go beyond the standard (1) experiment management; (2) model development and training; (3) deployment; (4) monitoring. The deployment of safety critical applications has shown that the current evaluation strategies lack the abilities to correctly evaluate the robustness of the models and their expected behaviour with their deployment in the wild. Drawing parallels from the software engineering industry, electronic circuit board design to aircraft safety, the need for an independent quality assurance step is long overdue. We take inspiration from the latest in property based testing for software and translate it to the world of machine learning. During this talk we’ll walk through the Efemarai Continuum platform and how it aids with measuring the robustness of the machine learning models within an operational domain. We’ll highlight results from our public tests and show what industrial applications may expect as an outcome of using our platform.