Skip to content

Commit

Permalink
Updated chapter 01
Browse files Browse the repository at this point in the history
  • Loading branch information
MikeySaw committed Apr 23, 2024
1 parent 776c921 commit e5f528f
Show file tree
Hide file tree
Showing 6 changed files with 17 additions and 10 deletions.
2 changes: 1 addition & 1 deletion content/chapters/01_introduction/01_01_course_intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "Chapter 01.01: Introduction and Course Outline"
weight: 1001

---
This chapter introduces the people responsible for the course aims to answer all open question and should give an impression of the expected workload.
This chapter introduces the people responsible for the course, aims to answer all open question and should give an impression of the expected workload.

<!--more-->

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "Chapter 01.02: Learning Paradigms"
weight: 1002

---
This chapter introduces different learning paradigms, such as embeddings, prompting and pre-training & finetuning a model
This chapter introduces different learning paradigms, such as embeddings, prompting and pre-training and finetuning a model. These are all very important concepts in the context of Deep Learning for NLP. The purpose of this chapter is to give an overview over these concepts.

<!--more-->

Expand Down
2 changes: 1 addition & 1 deletion content/chapters/01_introduction/01_03_tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "Chapter 01.03: NLP tasks"
weight: 1003

---
This chapter discusses various NLP tasks of different types (low- vs. high-level) and purely Linguistic task vs. more general classification task.
Here you will learn about the distinctions between low-level and high-level tasks, as well as the differences between linguistic tasks and broader classification tasks.

<!--more-->

Expand Down
5 changes: 4 additions & 1 deletion content/chapters/01_introduction/01_04_nplm.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,14 @@ title: "Chapter 01.04: Neural Probabalistic Language Model"
weight: 1004

---
This chapter introduces the Neural Probabalistic Language Model as a first approach to use Deep Neural Networks for language modelling.
In this chapter, take your first steps into the world of Deep Neural Networks for language modeling with the Neural Probabilistic Language Model (NPLM) [1]. Explore how NPLM represents a foundational approach in utilizing deep learning techniques for understanding and generating natural language. Uncover the principles behind NPLM and its significance in paving the way for more sophisticated language models.

<!--more-->

### Lecture slides

{{< pdfjs file="https://github.com/slds-lmu/lecture_dl4nlp/blob/main/slides/chapter01-basics/slides-14-nplm.pdf" >}}

### References

- [1] [Bengio et al., 2003](https://www.jmlr.org/papers/volume3/bengio03a/bengio03a.pdf)
10 changes: 9 additions & 1 deletion content/chapters/01_introduction/01_05_embeddings.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,19 @@ title: "Chapter 01.05: Word Embeddings"
weight: 1005

---
This chapter introduces Word Embeddings and discusses different methods of creating them, such as Word2Vec and CBOW.
Embeddings allow us to represent tokens with a vector representation, enabling computers to efficiently understand the meaning and context of different words. In this chapter we will first explain the general concept of embeddings and then introduce two popular approaches, namely Word2Vec [1] and FastText [2]. Word2vec is a neural network model that learns distributed representations of words in a continuous vector space based on their contextual usage in a corpus of text. Word2vec achieves this by training on a large dataset to predict the surrounding words given a target word, embedding each word as a dense vector where similar words are closer together in the vector space. FastText is an extension of Word2vec that represents words as bags of character n-grams, enabling it to capture morphological information alongside semantics. It accomplishes this by breaking down each word into character n-grams, creating embeddings for these subword units, and then summing or averaging these embeddings to obtain the representation for the whole word.

<!--more-->

### Lecture slides

{{< pdfjs file="https://github.com/slds-lmu/lecture_dl4nlp/blob/main/slides/chapter01-basics/slides-15-embeddings.pdf" >}}

### References

- [1] [Mikolov et al., 2013](https://arxiv.org/abs/1301.3781)
- [2] [Bojanowski et al., 2016](https://arxiv.org/abs/1607.04606)

### Additional Resources

- [Word2vec Blogpost](https://israelg99.github.io/2017-03-23-Word2Vec-Explained/)
6 changes: 1 addition & 5 deletions content/chapters/01_introduction/_index.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,6 @@
---
title: "Chapter 1: Introduction to the course"
---
This chapter gives a quick introduction to the basic concepts of Deep Learning for NLP, such as learning paradigms, tasks, neural probalistic language model and embeddings. A comprehensive introduction is given in [1].
In this chapter, you'll dive into the fundamental principles of Deep Learning for Natural Language Processing (NLP). Explore key concepts including learning paradigms, various tasks within NLP, the neural probabilistic language model, and the significance of embeddings.

<!--more-->

### References

- [1] [Goodfellow et al., 2016](https://www.deeplearningbook.org/)

0 comments on commit e5f528f

Please sign in to comment.