Differences

This shows you the differences between two versions of the page.

Link to this comparison view

differentiable_layers [2017/03/03 19:30]
127.0.0.1 external edit
differentiable_layers [2017/10/25 10:26] (current)
admin
Line 79: Line 79:
  
 We develop a framework for combining differentiable programming languages with neural networks. Using this framework we create end-to-end trainable systems that learn to write interpretable algorithms with perceptual components. We explore the benefits of inductive biases for strong generalization and modularity that come from the program-like structure of our models. In particular, modularity allows us to learn a library of (neural) functions which grows and improves as more tasks are solved. Empirically,​ we show that this leads to lifelong learning systems that transfer knowledge to new tasks more effectively than baselines. We develop a framework for combining differentiable programming languages with neural networks. Using this framework we create end-to-end trainable systems that learn to write interpretable algorithms with perceptual components. We explore the benefits of inductive biases for strong generalization and modularity that come from the program-like structure of our models. In particular, modularity allows us to learn a library of (neural) functions which grows and improves as more tasks are solved. Empirically,​ we show that this leads to lifelong learning systems that transfer knowledge to new tasks more effectively than baselines.
 +
 +https://​arxiv.org/​abs/​1710.08717 Auto-Differentiating Linear Algebra
 +
 +However, it is currently not easy to implement many basic machine learning primitives in these systems (such as Gaussian processes, least squares estimation, principal components analysis, Kalman smoothing), mainly because they lack efficient support of linear algebra primitives as differentiable operators. We detail how a number of matrix decompositions (Cholesky, LQ, symmetric eigen) can be implemented as differentiable operators.