aivancity NLP

Main repository for the 2024-2025 Natural Language Processing class at aivancity by Paul Lerner (both semesters)

Classes

• :download:`Introduction to NLP and Distributional Semantics <./docs/_static/NLP_1_intro_sem.pdf>`
• :download:`LLM Architectures: Attention Mechanism and Transformers <./docs/_static/NLP_2_transformers.pdf>`
• :download:`Large Language Models from Shannon to ChatGPT <./docs/_static/NLP_3_LLM.pdf>`
• :download:`Benchmarking / Ethical, social, and environmental issues <./docs/_static/NLP_4_evaluation_ethics.pdf>`

Practical Works

• Practical Work 1: Distributional Semantics/Skipgram/word2vec https://colab.research.google.com/github/PaulLerner/aivancity_nlp/blob/main/pw1_embedding.ipynb
• Practical Work 2: Transformers https://colab.research.google.com/github/PaulLerner/aivancity_nlp/blob/main/pw2_transformers.ipynb
• Practical Work 3: Large Language Models https://colab.research.google.com/github/PaulLerner/aivancity_nlp/blob/main/pw3_llm.ipynb
• Practical Work 4: Information Extraction https://colab.research.google.com/github/PaulLerner/aivancity_nlp/blob/main/pw4_eval_ie.ipynb

Contributing

Add Google Colab badges to PWs with https://openincolab.com/

Build docs using sphinx-build -b html . docs

Acknowledgements

This class directly builds upon:

• Jurafsky, D., & Martin, J. H. (2024). Speech and Language Processing : An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition with Language Models (3rd éd.). https://web.stanford.edu/~jurafsky/slp3/ed3bookaug20_2024.pdf
• Eisenstein, J. (2019). Natural Language Processing. 587. https://nlp.cs.princeton.edu/cos484-sp21/readings/eisenstein-nlp-notes.pdf
• Yejin Choi. (Winter 2024). CSE 447/517: Natural Language Processing (University of Washington Paul G. Allen School of Computer Science & Engineering)
• Noah Smith. (Winter 2023). CSE 447/517: Natural Language Processing (University of Washington Paul G. Allen School of Computer Science & Engineering)
• Benoît Sagot. (2023-2024). Apprendre les langues aux machines (Collège de France)
• Chris Manning. (Spring 2024). Stanford CS224N: Natural Language Processing with Deep Learning
• Classes where I was/am Teacher Assistant:
  • Christopher Kermorvant. Machine Learning for Natural Language Processing (ENSAE)
  • François Landes and Kim Gerdes. Introduction to Machine Learning and NLP (Paris-Saclay)

Also inspired by:

• My PhD thesis: Répondre aux questions visuelles à propos d’entités nommées (2023)
• Noah Smith (2023): Introduction to Sequence Models (LxMLS)
• Kyunghyun Cho: Transformers and Large Pretrained Models (LxMLS 2023), Neural Machine Translation (ALPS 2021)
• My former PhD advisors Olivier Ferret and Camille Guinaudeau and postdoc advisor François Yvon
• My former colleagues at LISN