Explore a 30-minute conference talk on Flux, an innovative machine learning library for the Julia programming language. Discover how Flux combines ease of use with state-of-the-art performance through Julia's advanced features and modern compiler technology. Learn about various applications of Flux, including image recognition, speech recognition with CUDA, and reinforcement learning. Delve into the library's internals, its integration with ONNX, and its ability to export models to browsers using FluxJS.jl. Gain insights into the future plans for Flux, focusing on automatic differentiation as a compiler problem. Understand the development of Zygote.jl, a novel approach to automatic differentiation that improves upon traditional methods. Witness demonstrations, benchmarks, and discussions on custom gradients, error messages, and the flexibility of Julia's compiler. Engage with a Q&A session covering topics such as Zygote.jl's capabilities, limitations, and its relationship with other Julia packages.
Overview
Syllabus
Welcome!.
Quick overview of Flux.jl.
Quick glimpse on Flux.jl internals.
Summer of Code with Flux.jl.
Reading ONNX files with ONNX.jl.
Images recognition models.
Speech recognition model with CUDA.
Demo of speech recognition.
Reinforcement learning and AlphaGo.jl.
Exporting Flux.jl to the browser with FluxJS.jl.
Cheers for people from Summer of Code.
Plans for the future: ML for us is a compiler problem.
Our main compiler problem: automatic differentiation (AD).
AD normally needs expression trees.
Can we do better? Our answer: Zygote.jl.
Taking derivative and Julia IR (Internal Representation).
Benchmarking more complex examples.
Speed, but at what cost?.
Defining custom gradients.
Convenient error messages.
We have fully dynamics AD.
In Julia, we can just hack compilers with different tricks when we need it.
Demo of simple derivative.
Question: what is that arrow?.
Q&A: can you differential different functions that number to number (scalar to scalar)?.
Comment: removing of the stack in some cases.
Q&A: what are the Zygote.jl limitations right now?.
Q&A: what is relation of Zygote.jl and Casset.jl?.
Q&A: can we use Zygote.jl to differentiate function from one parameter to many parameters?.
Annulment of Flux.jl on hackathon.
Taught by
The Julia Programming Language
