Natural Language Processing (NLP)

Have you ever wondered how systems like ChatGPT, which can generate human-like text, are built? Are you intrigued by the idea of creating a system that can process, understand, or generate text automatically? Are you interested in building applications that can translate between languages, answer questions, or recognise named entities in text? If so, this course is designed for you.

This course provides an introduction to the fundamentals of Natural Language Processing (NLP), which involves computational models of language and their applications to text. As language is the core of human intelligence, NLP holds a pivotal role in Artificial Intelligence research and development.

We will integrate machine learning (ML), including its fundamental formalisms and algorithms, with a robust hands-on experience. This means you will gain practical skills in implementing these methods for real-world NLP problems.

The course utilises interactive lecture materials constructed with Jupyter notebooks. Course materials from last year are publicly available here. The course will closely follow the structure of the previous year's iteration. If you're unsure about the course prerequisites or content, please review these materials.

The course covers the following topics:

• NLP tasks: tokenisation, text classification, language modelling, named entity recognition, part-of-speech tagging, parsing, information extraction, machine translation, question answering

• Methods: log-linear models, structured prediction, conditional random fields, beam search, and neural network models such as transformers (including representation learning, pre-training, transfer learning and interpretability methods)

• Implementations: relationship between NLP tasks, efficient implementations, and the use of modern NLP libraries such as Hugging Face's Transformers

Throughout the course, we will also explore the themes of discriminative and generative learning and various ways of obtaining supervision for training statistical NLP models. An important aspect of our discussions will be the application of these techniques in multilingual settings, understanding how NLP can be adapted and applied to a variety of languages beyond English.