- Recurrent-Neural Networks [X]
- Graves, Mohamed and Hinton. Speech recognition with Deep Recurrent Neural Networks. [http://arxiv.org/abs/1303.5778]
- Unsupervised + Supervised [JW]
- Erhan, Courville, Bengio, Vincent. Why does unsupervised pre-training help deep learning?. AISTATS 2010 [PDF]
Monday, September 22, 2014
Recurrent Neural Networks + AISTATS 2010
Subscribe to:
Post Comments (Atom)
How exactly is unsupervised pre-training done. I am unable to understand how a unsupervised pre train could possibly help a different supervision task.
ReplyDeleteFrom what I deduce from the paper is that the pre training is done only for the first few layers. So my question is what is exactly is the structure of the unsupervised network used in pre training. Is it a sparse auto encoder; that would make sense given that the first few layers anyway encode low level features.
Hi,
ReplyDeleteCould someone guide me to other regularization methods used in deep networks,