AI3 Seminar - Speaker: Meir Feder, Professor and Jokel Chair in Information Theory, School of Electrical and Computer Engineering, Tel Aviv University

Abstract

Information theory offers a unified view of learning as universal prediction under log loss, measured through regret bounds. Unlike classical approaches with uniform regret over small model classes, this framework provides non-uniform, model-dependent bounds using a new notion of architecture-based model complexity. This complexity depends on the volume of models near a target model in informational distance, approximated through the Fisher Information Matrix or spectral properties of the expected Hessian.