-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Add] chapter 8 decoding and rename other chapters
- Loading branch information
Showing
15 changed files
with
93 additions
and
49 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
--- | ||
title: "Chapter 08.01: What is Decoding?" | ||
weight: 8001 | ||
--- | ||
|
||
|
||
|
||
<!--more--> | ||
|
||
### Lecture Slides | ||
|
||
{{< pdfjs file="https://github.com/slds-lmu/lecture_dl4nlp/blob/main/slides/chapter12-decoding/slides-121-intro.pdf" >}} | ||
|
||
### References | ||
|
||
- [1] [Radford et al., 2018](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
--- | ||
title: "Chapter 08.02: Greedy & Beam Search" | ||
weight: 8002 | ||
--- | ||
|
||
|
||
|
||
<!--more--> | ||
|
||
### Lecture Slides | ||
|
||
{{< pdfjs file="https://github.com/slds-lmu/lecture_dl4nlp/blob/main/slides/chapter12-decoding/slides-122-determ.pdf" >}} | ||
|
||
### References | ||
|
||
- [1] [Radford et al., 2018](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
--- | ||
title: "Chapter 08.03: Stochastic Decoding & CS/CD" | ||
weight: 8003 | ||
--- | ||
|
||
|
||
|
||
<!--more--> | ||
|
||
### Lecture Slides | ||
|
||
{{< pdfjs file="https://github.com/slds-lmu/lecture_dl4nlp/blob/main/slides/chapter12-decoding/slides-123-sampling.pdf" >}} | ||
|
||
### References | ||
|
||
- [1] [Radford et al., 2018](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
--- | ||
title: "Chapter 08.04: Decoding Hyperparameters & Practical considerations" | ||
weight: 8004 | ||
--- | ||
|
||
|
||
|
||
<!--more--> | ||
|
||
### Lecture Slides | ||
|
||
{{< pdfjs file="https://github.com/slds-lmu/lecture_dl4nlp/blob/main/slides/chapter12-decoding/slides-124-hyper-param.pdf" >}} | ||
|
||
### References | ||
|
||
- [1] [Radford et al., 2018](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
--- | ||
title: "Chapter 08.05: Decoding Hyperparameters & Practical considerations" | ||
weight: 8005 | ||
--- | ||
|
||
|
||
|
||
<!--more--> | ||
|
||
### Lecture Slides | ||
|
||
{{< pdfjs file="https://github.com/slds-lmu/lecture_dl4nlp/blob/main/slides/chapter12-decoding/slides-125-eval_metrics.pdf" >}} | ||
|
||
### References | ||
|
||
- [1] [Radford et al., 2018](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
--- | ||
title: "Chapter 8: Decoding Strategies" | ||
--- | ||
|
||
This chapter is about various decoding strategies. You will learn about deterministic methods (greedy search, beam search, contrastive search, contrastive decoding) and stochastic methods (top-k, top-p, sampling with temperature). This chapter also covers evaluation metrics for open ended text generation. |
4 changes: 2 additions & 2 deletions
4
...apters/08_llm/08_01_instruction_tuning.md → ...apters/09_llm/09_01_instruction_tuning.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 changes: 2 additions & 2 deletions
4
content/chapters/08_llm/08_02_cot.md → content/chapters/09_llm/09_02_cot.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 changes: 2 additions & 2 deletions
4
content/chapters/08_llm/08_03_emerging.md → content/chapters/09_llm/09_03_emerging.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 1 addition & 1 deletion
2
content/chapters/08_llm/_index.md → content/chapters/09_llm/_index.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,5 @@ | ||
--- | ||
title: "Chapter 8: Large Language Models (LLMs)" | ||
title: "Chapter 9: Large Language Models (LLMs)" | ||
--- | ||
|
||
In this chapter we cover LLM concepts, such as Instruction Fine-Tuning, Chain-of-Thought prompting and discuss the possbility of emerging abilities of LLMs. |
12 changes: 0 additions & 12 deletions
12
content/chapters/10_multilingual/10_01_why_multilinguality.md
This file was deleted.
Oops, something went wrong.
12 changes: 0 additions & 12 deletions
12
content/chapters/10_multilingual/10_02_cross_lingual_word_embeddings.md
This file was deleted.
Oops, something went wrong.
12 changes: 0 additions & 12 deletions
12
content/chapters/10_multilingual/10_03_multi_lingual_transformers.md
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
2 changes: 1 addition & 1 deletion
2
content/chapters/09_rlhf/_index.md → content/chapters/10_rlhf/_index.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters