Theory and Practice of Large Language Models (Master S2 Class)

Basic information

Description

This lecture delves into the foundational theories and practical applications of advanced NLP techniques. Students will explore the architecture and training methods of (large) language models such as GPT and BERT. Through a combination of lectures, hands-on projects, and case studies, learners will understand how to implement, fine-tune, and deploy these models. Key topics include model scalability, ethical considerations, scientific questions raised by LLMs, analysis of neural representations, prompting, and retrieval-augmented generation (RAG). The course aims to equip students with both the theoretical knowledge and practical skills required to leverage large language models in various domains. By the end of the course, participants will be adept at handling complex NLP tasks and contributing to cutting-edge AI research.

Prerequisites

A very good knowledge of python is required: the course includes many practical exercises using the main NLP (Hugging Face 🤗 & spacy), deep learning (pytorch) and data science libraries (seaborn, pandas & matplotlib).
Knowledge of the fundamentals of statistical learning (train/test separation, linear classifier, loss function, etc.) is also required.

Learning outcomes

On successful completion of this course, students should:

  • be able to explain the inner workings of models such as ChatGPT;
  • know the main methods for developing “modern” NLP systems: fine-tuning & prompting;
  • understand the scientific issues raised by LLMs;
  • be able to describe the capabilities and limitations (particularly ethical and social) of LLMs.