Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

YouTube

Deep Dive into the Transformer Encoder Architecture

CodeEmporium via YouTube

Overview

Dive deep into the transformer encoder architecture in this 21-minute video tutorial. Explore the intricacies of initial embeddings, positional encodings, and the encoder layer structure. Learn about query, key, and value vectors, self-attention matrix construction, and the importance of scaling and softmax. Understand the combination of attention heads, residual connections, layer normalization, and the role of linear layers, ReLU, and dropout. Conclude with insights on final word embeddings and a sneak peek at the code implementation.

Syllabus

Introduction
Encoder Overview
Blowing up the encoder
Create Initial Embeddings
Positional Encodings
The Encoder Layer Begins
Query, Key, Value Vectors
Constructing Self Attention Matrix
Why scaling and Softmax?
Combining Attention heads
Residual Connections Skip Connections
Layer Normalization
Why Linear Layers, ReLU, Dropout
Complete the Encoder Layer
Final Word Embeddings
Sneak Peak of Code

Taught by

CodeEmporium

Reviews

Start your review of Deep Dive into the Transformer Encoder Architecture

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.