Information Theory Courses and Certifications

Introduction to Complexity

145 reviews

In this course you'll learn about the tools used by scientists to understand complex systems. The topics you'll learn about include dynamics, chaos, fractals, information theory, self-organization, agent-based modeling, and networks.

• Complexity Explorer
• 10 weeks, 3-6 hours a week
• On-Demand
• Free Online Course

• Best of All Time

Information Theory of Deep Learning - Naftali Tishby

Explores the connection between deep learning and information theory, offering new insights into neural network design, generalization, and optimization through an information bottleneck framework.

• YouTube
• 1 hour 25 minutes
• On-Demand
• Free Video

Data Compression I - Course Introduction and Lossless Compression Basics - Lecture 1

Introduction to lossless data compression fundamentals, covering key concepts and techniques for efficient information storage and transmission.

• YouTube
• 1 hour 3 minutes
• On-Demand
• Free Video

Data Compression I - Lecture 2: Prefix Free Codes

Explore prefix-free codes in data compression, covering theory and applications. Gain insights from Stanford experts on efficient information encoding techniques.

• YouTube
• 1 hour 17 minutes
• On-Demand
• Free Video

Data Compression I - Lecture 5: Asymptotic Equipartition Property

Explore the Asymptotic Equipartition Property in data compression, a fundamental concept for understanding information theory and efficient coding techniques.

• YouTube
• 1 hour 20 minutes
• On-Demand
• Free Video

Context-based Arithmetic Coding and LLM Compression - Stanford EE274 Lecture 9

Explore context-based arithmetic coding and compression techniques for large language models in this advanced lecture on data compression theory and applications.

• YouTube
• 1 hour 20 minutes
• On-Demand
• Free Video

Data Compression I - Lecture 4: Huffman Codes

Explore Huffman coding, a fundamental data compression technique, in this Stanford lecture. Learn its principles, implementation, and applications in information theory.

• YouTube
• 1 hour 15 minutes
• On-Demand
• Free Video

Data Compression I - Lecture 3: Kraft Inequality, Entropy, and Introduction to SCL

Explore Kraft Inequality, entropy, and Source Coding Limit in data compression. Gain insights into fundamental concepts and their applications in information theory and coding.

• YouTube
• 1 hour 17 minutes
• On-Demand
• Free Video

Stanford EE274: Data Compression - Beyond IID Distributions: Conditional Entropy - Lecture 8

Explore conditional entropy and advanced data compression techniques beyond IID distributions in this Stanford lecture, enhancing understanding of complex information theory concepts.

• YouTube
• 1 hour 16 minutes
• On-Demand
• Free Video

LZ and Universal Compression - Lecture 10

Explore Lempel-Ziv algorithms and universal compression techniques in this Stanford lecture, delving into advanced data compression theory and applications.

• YouTube
• 1 hour 25 minutes
• On-Demand
• Free Video

Arithmetic Coding - Stanford EE274 Data Compression I Lecture 6

Explore arithmetic coding in data compression, covering theory and applications. Learn from Stanford experts about this advanced technique for efficient information encoding.

• YouTube
• 1 hour 18 minutes
• On-Demand
• Free Video

Asymmetric Numeral Systems (ANS) - Data Compression Lecture 7

Explore Asymmetric Numeral Systems (ANS), an advanced data compression technique, in this Stanford lecture. Gain insights into its principles, applications, and advantages in modern compression algorithms.

• YouTube
• 1 hour 20 minutes
• On-Demand
• Free Video

On the Foundations of Info-Metrics

Explore the science of info-metrics for modeling and inferring from limited data, with insights on reasoning under uncertainty and insufficient information.

• YouTube
• 58 minutes
• On-Demand
• Free Video

The Information Edge: Creation in Life, the Economy, and the Universe - Lecture 1

Explore the Information Edge principle and its role in creation across the universe, from cosmic structures to economic systems, with MIT professor Seth Lloyd.

• YouTube
• 1 hour 30 minutes
• On-Demand
• Free Video

The Information Edge: Collapse and Rebirth in Complex Systems - Lecture 2

Seth Lloyd explores the Information Edge's role in cosmic destruction and rebirth, from black holes to mass extinctions, and how complex systems regain balance after collapse.

• YouTube
• 1 hour 27 minutes
• On-Demand
• Free Video