Gaussian Pre-Activations in Neural Networks: Myth or Reality?
Finnish Center for Artificial Intelligence FCAI via YouTube
Overview
Syllabus
Introduction
Scaling
Framework
Naive heuristic
Outline
Edge of Cows
Recurrence Equation
Gaussian PreActivations
The Edge of Chaos
Experiments
Gaussian regulations
Assumption of edge of chaos
Preservation of variance
Solution
Summary
Constraints
Density
activation functions
numerical approximations
training experiments
training losses
conclusion
future work
questions
data patterns
impossibility results
Cons
Training Loss
Taught by
Finnish Center for Artificial Intelligence FCAI