IPAB Seminar - 25/02/2021

Speaker:  Sebastian Starke

Title: Neural Animation Layering for Synthesizing Martial Arts Movements

Abstract: Interactively synthesizing novel combinations and variations of character movements from different motion skills is a key problem in computer animation. In this research, we propose a deep learning framework to produce a large variety of martial arts movements in a controllable manner from unstructured motion capture data. Our method imitates animation layering using neural networks with the aim to overcome the typical challenges when mixing, blending and editing movements from unaligned motion sources. The framework can synthesize novel movements from given reference motions and simple user controls, and generate unseen sequences of locomotion, punching, kicking, avoiding and combinations thereof, but also reconstruct signature motions of different fighters, as well as close-character interactions such as clinching and carrying. We adopt a modular framework that is composed of the motion generator, that is trained in a task-agnostic manner to map novel mixed/edited trajectories to natural full-body motions, and a set of task-specific control modules that produce such trajectories from the user inputs. Our framework provides a transparent control interface for animators that allows modifying or combining movements after network training, greatly reduces network iteration time when working with large-scale datasets and enables iterative adding of different motion tasks and behaviours. Our system can be used for offline and online motion generation alike and is relevant for real-time applications such as computer games.