Differences

This shows you the differences between two versions of the page.

Link to this comparison view

implicit_ensemble [2016/11/23 20:00]
127.0.0.1 external edit
implicit_ensemble [2018/03/15 11:10] (current)
admin
Line 57: Line 57:
  
 We propose zoneout, a novel method for regularizing RNNs. At each timestep, zoneout stochastically forces some hidden units to maintain their previous values. Like dropout, zoneout uses random noise to train a pseudo-ensemble,​ improving generalization. ​ We propose zoneout, a novel method for regularizing RNNs. At each timestep, zoneout stochastically forces some hidden units to maintain their previous values. Like dropout, zoneout uses random noise to train a pseudo-ensemble,​ improving generalization. ​
 +
 +https://​arxiv.org/​abs/​1703.02065v4 On the Expressive Power of Overlapping Architectures of Deep Learning
 +
 +Our analysis shows that having overlapping local receptive fields, and more broadly denser connectivity,​ results in an exponential increase in the expressive capacity of neural networks. Moreover, while denser connectivity can increase the expressive capacity, we show that the most common types of modern architectures already exhibit exponential increase in expressivity,​ without relying on fully-connected layers.
 +