ABOUT THE COURSE:Natural Language Processing finds its applications nearly everywhere -- be it Machine Translation,Question answering, Text Summarization, Dialogs, etc. In the last decade, Deep Learning basedmethods have given very good performance across a variety of NLP tasks, and have become a default choice for NLP problems. This course aims to give a thorough understanding of various deep learning architectures along with their specific use-cases in NLP.The course will also introduce the fundamental ideas behind training as well as fine-tuning/prompting the Large Language Models, which include in-context-learning, Parameter-efficient-fine-tuning,Reinforcement Learning through Human Feedback (RLHF). The course will also offer hands-on tutorials to help students master this subject.INTENDED AUDIENCE: Advanced UG and PG studentsPREREQUISITES: Participants should have done a course on MachineLearning, and should know basics of PythonProgrammingINDUSTRY SUPPORT: Google,Microsoft, Amazon, Flipkart, Adobe
Overview
Syllabus
Week 1:
Week 3:Word Representations
Week 5:Attention Mechanism
Week 6:Self-supervised learning (SSL), Pretraining
Week 7:
Week 8:Instruction Fine-tuning, FLAN-T5, Reinforcement Learningthrough Human Feedback (RLHF)
Week 9:In-context learning, chain-of-thought prompting. ScalingLaws. Various Large Language Models and unique architectural differences
Week 10:Parameter Efficient Fine-tuning (PEFT) - LoRA, QLoRA
Week 11:Handling Long Context, Retrieval Augmented Generation(RAG)
Week 12:Analysis and Interpretability, ethical considerations
- Introduction to NLP: What is Natural LanguageProcessing? A brief primer on word and sentence level tasks and n-gram language Model.
- Shallow and Deep Neural Networks
- Representation Learning
Week 3:Word Representations
- Word2Vec
- Glove
- fastText,
- Multilingualrepresentations with emphasis on Indian Languages
- RNN LMs
- GRUs, LSTMs, Bi-LSTMs
- LSTMs for Sequence Labeling
- LSTMs for Sequence to Sequence
Week 5:Attention Mechanism
- Sequence to Sequence with Attention
- Transformers: Attention is all you need
Week 6:Self-supervised learning (SSL), Pretraining
- Designing SSL objectives
- Pretrained Bi-LSTMs: ELMO
- Pretrained Transformers: BERT, GPT, T5, BART
Week 7:
- Applications: Question Answering, Dialog Modeling, TextSummarization
- Multilingual extension with application to Indian languages
Week 8:Instruction Fine-tuning, FLAN-T5, Reinforcement Learningthrough Human Feedback (RLHF)
Week 9:In-context learning, chain-of-thought prompting. ScalingLaws. Various Large Language Models and unique architectural differences
Week 10:Parameter Efficient Fine-tuning (PEFT) - LoRA, QLoRA
Week 11:Handling Long Context, Retrieval Augmented Generation(RAG)
Week 12:Analysis and Interpretability, ethical considerations
Taught by
Prof. Pawan Goyal
