Innate and Baseline Representations - ICLR 2020 submission

In this position paper, we attempt to distil lessons from recent research in AI. These insights are combined with findings from a number of disciplines (Animal Cognition, Neuroscience, Comparative Psychology) and point towards a need for a paradigm shift for progress towards an AI that is able to learn and reason. We use a representation learning lens to review work across adversarial attacks, curriculum learning, meta learning, multi-task learning, self-supervised learning and foundational research on generalization. We find evidence supporting a need to (1) focus on crafting baseline representations, (2) treat some of these representations as innate, (3) find ways to combine learning and innate representations. Finally, we pose the question of how to build this hierarchy of representations and how to embed reasoning, indicating Analogy Reasoning as a promising approach.