Deep Neural Networks

Recently, while contemplating about a machine learning problem in my HIS project I learned about the following beautiful paper Understanding Deep Convolutional Networks by Stéphane Mallat. It describes some mathematical ideas on how to understand the power behind deep networks. It contains beautiful math stuff combining analysis and wavelets, stochastic, statistics, geometry and group actions, diffeomorphism etc. and connections to statistical physics. There is also a nice talk from Stéphane available on youtube. Seems that we are still far from understanding what is going on but at least there is a glimpse. And the math is terribly interesting.