Black Friday 2024: 50 Ways to Save on Online Learning

Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

NPTEL

Introduction to Large Language Models (LLMs)

NPTEL via Swayam

Go to class Write review

Overview

Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Grab it
ABOUT THE COURSE:This course introduces the fundamental concepts underlying Large Language Models (LLMs). It starts with an introduction to the various problems in NLP, and discusses how to approach the problem of language modeling using deep learning. It describes the architectural intricacies of Transformers and the pre-training objectives of the differen Transformer-based models. It also discusses the recent advances in LLM research, including LLM alignment, prompting, parameter-efficient adaptation, hallucination, bias and ethical considerations. This course prepares a student to comprehend, critique and approach various research problems on LLMs.INTENDED AUDIENCE: UG and PG students in CSE, EE, ECE, IT, Maths, etc.PREREQUISITES: Mandatory: Machine Learning, Python Programming Optional: Deep LearningINDUSTRY SUPPORT: All those industries whose work involves machine learning, such as Google, Microsoft, Adobe, IBM, Accenture, Adobe, JP Morgan, Wipro, Flipkart, Amazon, etc

Syllabus

Week 1
  1. Introduction to NLP – Part 1 (NLP pipeline, applications of NLP)
  2. Introduction to NLP – Part 2 (distributional semantics)
Week 2
  1. Introduction to Deep Learning (Perceptron, ANN, backpropagation, CNN)
  2. Word vectors (Word2Vec, GloVe, fastText)
Week 3
  1. Introduction to Statistical Language Models (N-gram LM, Perplexity, Smoothing)
  2. Language Models with CNN and RNN
Week 4
  1. Introduction to PyTorch
  2. Implementation of RNNs and LSTMs using PyTorch
Week 5
  1. Sequence-to-sequence models, Beam search
  2. Attention and self-attention
Week 6
  1. Introduction to Transformers
  2. Positional Embedding, various tokenization strategies
  3. Implementation of Transformers using PyTorch
Week 7
  1. Transfer Learning: ELMo, BERT (Encoder-only Model)
  2. Transfer Learning: GPT (Decoder-only Model), T5 (Encoder-decoder model)
Week 8
  1. Prompting (hard and soft) and Instruction fine-tuning (FLAN)
  2. Advanced prompting (Chain of Thoughts, Graph of Thoughts, Prompt Chaining, etc.)
  3. Introduction to HuggingFace Library
Week 9
  1. Alignment with Human Feedback: RLHF, RLAIF
  2. Parameter-efficient adaptation (Prompt tuning, Prefix tuning, LoRA)
Week 10
  1. Knowledge graphs (KGs)
    a. Representation, completion
    b. Tasks: Alignment and isomorphism
    c. Distinction between graph neural networks and neural KG inference
Week 11
  1. Open-book question answering: The case for retrieving from structured and unstructured sources;retrieval-augmented inference and generation
  2. Retrieval augmentation techniques
    a. Key-value memory networks in QA for simple paths in KGs
    b. Early HotPotQA solvers, pointer networks, reading comprehension
    c. REALM, RAG, FiD, Unlimiformer
    d. KGQA (e.g., EmbedKGQA, GrailQA)
Week 12
  1. Overview of recently popular models such as GPT4, Llama 3, Claude 3,Mistral, and Gemini
  2. Ethical NLP – Bias and Toxicity
  3. Conclusion

Taught by

Prof. Tanmoy Chakraborty, Prof. Soumen Chakraborti

Reviews

Start your review of Introduction to Large Language Models (LLMs)

Go to class

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Sign up for free
Someone learning on their laptop while sitting on the floor.