Skip to content

Commit

Permalink
Added Analysis Section to README
Browse files Browse the repository at this point in the history
  • Loading branch information
NikhilGupta1997 authored Mar 20, 2019
1 parent c46e729 commit 153d26a
Showing 1 changed file with 31 additions and 0 deletions.
31 changes: 31 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,34 @@ To obtain metric scores on the best model run `main.py` with *--train=False* or
```console
❱❱❱ python main.py --task_id 1
```

# Analysis

## Trainable Paramters
The following numbers are reported using *embedding_size 256*, *batch_size 64*, *hops 6*

| | Task 1 | Task 2 | Task 3 | Task 4 | Task 5 | CamRest | SMD |
| ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ |
| **BossNet** | 1204993 | 1198849 | 1478913 | 1715457 | 1943809 | 1968129 | 2187265 |
| **Mem2Seq** | 776030 | 780127 | 3426789 | 4508397 | 5049201 | 5274536 | 6880560 |
| **Seq2Seq + Copy** | 6379859 | 6378321 | 6870481 | 7083494 | 7167315 | 7223452 | 7504137 |
| **Seq2Seq** | 6905172 | 6903634 | 7395794 | 7608807 | 7692628 | 7748765 | 8029450 |

## Running Times
The following numbers are reported using *embedding_size 128*, *batch_size 64*, *hops 3*
Times are reported as ` sec. per train epoch (avg. no. of epochs till convergence) **total train time** `

| | Task 1 | Task 2 | Task 3 | Task 4 | Task 5 | CamRest | SMD |
| ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ |
| **BossNet** | 38.1 (15) **571.5** | 65.2 (10) **652.0** | 142.4 (25) **3560** | 16.9 (2) **33.8** | 231.3 (6) **1387.8** | 113.5 (6) **681** | 1252 (10) **12520** |
| **Mem2Seq** | 10 (100) **1000** | 32 (30) **960** | 51 (90) **4590** | 4 (10) **40** | 136 (60) **8160** | 22 (40) **880** | 81 (40) **3240** |

## Hyperparameters

| | Task 1 | Task 2 | Task 3 | Task 4 | Task 5 | CamRest | SMD |
| ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ |
| Learning Rate | 0.001 | 0.001 | 0.005 | 0.001 | 0.0005 | 0.0005 | 0.0005 |
| Hops | 1 | 1 | 3 | 1 | 3 | 6 | 3 |
| Embedding Size | 128 | 128 | 128 | 128 | 256 | 256 | 256 |
| Disentangle Loss Weight | 1.0 | 1.0 | 1.5 | 1.0 | 1.0 | 1.0 | 1.0 |
| DLD | 0.2 | 0.2 | 0.2 | 0.2 | 0.2 | 0.2 | 0.1 |

0 comments on commit 153d26a

Please sign in to comment.