Information Bottleneck (and Unsupervised Learning)
Date:
This short presentation introduces the method of information bottleneck by describing its formulation and by illustrating its application in analyzing the behaviour of deep neural networks. The presentation ends discussing the problem of using a similar information-theoretic method to study the behaviour of unsupervised learning algorithms, focusing in particular on the analysis of the sparse filtering algorithm.