GAN integration #12

edorado93 · 2018-07-04T02:29:22Z

Steps to run the code:-

cd into your Writing-editing-Network folder
git fetch origin && git reset --hard origin/pr/12
Activate the conda writing-editing-network virtual environment.
python Writing-editing\ network/main.py --cuda --mode 0 --conf random

The sizes of log probabilities, discriminator input and the sequence length are being printed. So the generator is providing the correct input.

A successful run would give Generator Trained successfully

edorado93 · 2018-07-09T05:33:31Z

Writing-editing network/main.py

@@ -183,7 +192,8 @@ def train_generator(input_variable, input_lengths, target_variable, topics, mode
        # this is not the eval mode.
        if not is_eval:
            """ Call Discriminator, Critic and get the ReINFORCE Loss Term"""
-            reinforce_loss = None
+            est_values = critic_model(input)


What is the input expected here ?

edorado93 · 2018-07-09T05:34:40Z

Writing-editing network/main.py

@@ -183,7 +192,8 @@ def train_generator(input_variable, input_lengths, target_variable, topics, mode
        # this is not the eval mode.
        if not is_eval:
            """ Call Discriminator, Critic and get the ReINFORCE Loss Term"""
-            reinforce_loss = None
+            est_values = critic_model(input)
+            reinforce_loss = reinforce(gen_log, dis_out, est_values, seq_length, CommonConfig())


gen_log is I think the generator's log probabilities. What is dis_out? If it is the output of the discriminator, then I cannot see a call to the discriminator here that would get the output. Please add that.

shruthi0898 · 2018-07-09T05:53:23Z

Yes, it is the discriminator s output, can you just add the call and try. I am out now. I'll go back and work on anything required

…

On Sun, Jul 8, 2018, 10:34 PM Sachin Malhotra ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In Writing-editing network/main.py <#12 (comment)> : > @@ -183,7 +192,8 @@ def train_generator(input_variable, input_lengths, target_variable, topics, mode # this is not the eval mode. if not is_eval: """ Call Discriminator, Critic and get the ReINFORCE Loss Term""" - reinforce_loss = None + est_values = critic_model(input) + reinforce_loss = reinforce(gen_log, dis_out, est_values, seq_length, CommonConfig()) gen_log is I think the generator's log probabilities. What is dis_out? If it is the output of the discriminator, then I cannot see a call to the discriminator here that would get the output. Please add that. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#12 (review)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AQgbpnBLCs-slXNKwflKv3Fiq0EbtM4lks5uEutwgaJpZM4VB3jH> .

…onfig arguments and run

edorado93 · 2018-07-09T18:37:35Z

@shruthi0898
That's the error I see. Mostly it is because of the wrong usage of batch size. I can see in the critic code that you have hardcoded the value 2 in certain dimensions. It should be batch_size there. Look into that and try running. A successful run would give you Generator Trained successfully for now and the model would exit. Once that is done we can test the discriminator's training.

shruthi0898 · 2018-07-09T18:41:21Z

Okay. I am on my way to the lab. l'll change that into batch_size and push again.

…

On Mon, Jul 9, 2018, 11:37 AM Sachin Malhotra ***@***.***> wrote: [image: screen shot 2018-07-09 at 11 34 48 am] <https://user-images.githubusercontent.com/11422365/42469327-3babb3ea-836c-11e8-9de8-e157437907bc.png> That's the error I see. Mostly it is because of the wrong usage of batch size. I can see in the critic code that you have hardcoded the value 2 in certain dimensions. It should be batch_size there. Look into that and try running. A successful run would give you Generator Trained successfully for now and the model would exit. Once that is done we can test the discriminator's training. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#12 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AQgbpp_eHoso2KVYd4F8nSBsQUsW4s3zks5uE6LwgaJpZM4VB3jH> .

edorado93 · 2018-07-09T18:55:17Z

Writing-editing network/seq2seq/discriminator.py

+        output = output.squeeze(2)
+        return output, hidden
+
+def decoder_train(input_tensor, encoder_hidden, decoder, seq_length):


Combine this with the DecoderRNN's forward function. These shouldn't be separate ideally. Also, try and get rid of the explicit for loop.

@shruthi0898 , the decoder_train function shouldn't be a separate one. You should just call the DecoderRNN's forward function.

edorado93 · 2018-07-09T18:55:28Z

Writing-editing network/seq2seq/discriminator.py

+        return output, hidden
+
+def decoder_train(input_tensor, encoder_hidden, decoder, seq_length):
+    decoder_input = torch.zeros((2,1), dtype=torch.long)


What is the 2 here ?

edorado93 · 2018-07-09T18:56:00Z

Writing-editing network/seq2seq/discriminator.py

+        self.hidden2tag = nn.Linear(hidden_dim, 1)
+        self.hidden = self.init_hidden()
+
+    def init_hidden(self):


All these 2s seem to be hardcodings. Take batch size as input and replace that in the code.

edorado93 · 2018-07-09T18:56:26Z

Writing-editing network/seq2seq/discriminator.py

+        self.batch_size = batch_size
+
+    def forward(self, input, hidden):
+        output = self.embeddingF(input).view(self.batch_size, 1, -1)


There's no variable as embeddingF in your code.

shruthi0898 · 2018-07-09T18:58:06Z

Hey. Didn't understand. Which one should I try and combine? And can you tell me when you are free to call?

…

On Mon, Jul 9, 2018, 11:55 AM Sachin Malhotra ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In Writing-editing network/seq2seq/discriminator.py <#12 (comment)> : > + self.embedding_size = embedding_size + + self.embedding = nn.Embedding(vocab_size, embedding_size) + self.lstm = nn.LSTM(embedding_size, hidden_size, batch_first=True) + self.out = nn.Linear(hidden_size, output_size) + self.batch_size = batch_size + + def forward(self, input, hidden): + output = self.embeddingF(input).view(self.batch_size, 1, -1) + output = F.relu(output) + output, hidden = self.lstm(output, hidden) + output = self.out((output.view(self.batch_size, 1, -1))) + output = output.squeeze(2) + return output, hidden + +def decoder_train(input_tensor, encoder_hidden, decoder, seq_length): Combine this with the DecoderRNN's forward function. These shouldn't be separate ideally. Also, try and get rid of the explicit for loop. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#12 (review)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AQgbpipATKip4TMdrc1jVXsiIyYLcspFks5uE6cWgaJpZM4VB3jH> .

1. Optimizer takes in generator's parameters. We need a single optimizer for both the discriminator and the generator. 2. Reinforce returns a tuple 3. Critic returns a tuple 4. Hidden and cell states should be reinitialized on every run of critic, encoder and decoder and not just once during object initialization. 5. Model not working on CUDA, working fine on CPU. Hidden states need to be .cuda() as well. That was an issue. Fixed now.

edorado93 · 2018-07-12T21:46:46Z

@shruthi0898 , please look into the latest commit. There were some bugs that I fixed in your code. I have mentioned the description in the commit itself. Please verify those changes.

edorado93 · 2018-07-14T06:46:57Z

@shruthi0898 ,
The generator is training successfully. As for the discriminator, the dis_out and dis_sig are of wrong dimensions. Right now, they are of dimensionality (20,633) where 633 is the sequence length. I see that the discriminator makes a prediction for every word. But we need a prediction for the entire sentence. Please look into this.

Also, I have pushed the data as a separate commit in this PR. Pulling the latest changes should give you the dataset.

edorado93 · 2018-07-14T06:49:40Z

Writing-editing network/main.py

+def train_discriminator(input_variable, target_variable, is_eval=False):
+    sequence_length = input_variable.shape[1]
+    '''add other return values'''
+    dis_out, dis_sig = discrim_model(input_variable, sequence_length, config.batch_size)


Wrong dimensions are coming here. Kindly check. The dimensions of dis_out and target_variable should match. The target variable has a dimensionality of (20, 1)

edorado93 · 2018-07-27T18:00:24Z

@shruthi0898
I think your latest changes broke the Generator's training. Getting the following error now. Please look into this and fix it. On a successful run, you will get the losses from the Generator and the Discriminator.

…check

edorado93 and others added 4 commits July 3, 2018 18:03

Initial GAN changes

9f6b8a6

Finished changes

93ea85a

Bug fixes

f640126

Discriminator Changes

2d26606

edorado93 force-pushed the GANs branch from ac507c3 to 2d26606 Compare July 9, 2018 05:24

edorado93 commented Jul 9, 2018

View reviewed changes

Added configs, moved file to different folder.

a699fe8

shruthi0898 and others added 2 commits July 9, 2018 22:26

Added Discriminator call, added comments for your reference, change c…

5467641

…onfig arguments and run

Completed missing portions.

862e181

Made models cuda() and added completion logs

0dead58

edorado93 force-pushed the GANs branch from 4777ac1 to 0dead58 Compare July 9, 2018 18:54

edorado93 commented Jul 9, 2018

View reviewed changes

edorado93 force-pushed the GANs branch from 72c0e55 to 4f68a84 Compare July 9, 2018 19:06

Added random config

9236f7f

edorado93 force-pushed the GANs branch from 4f68a84 to 9236f7f Compare July 9, 2018 19:09

shruthi0898 and others added 2 commits July 12, 2018 00:38

Removed Typos and hardcoded values

1314fc9

edorado93 added 3 commits July 13, 2018 23:15

Fixed cuda issues. Generator is training successfully

b1ce70d

Added dataset

1885fe6

Discriminator changes added

5727622

edorado93 commented Jul 14, 2018

View reviewed changes

shruthi0898 and others added 3 commits July 20, 2018 21:59

Changed dis_out to be batch*1

8dfc6e3

Annotated dataset

6e3d6ea

dis_loss shape

187c90d

edorado93 and others added 6 commits July 27, 2018 11:00

Removed unnecessary print. Added one milestone print statement

a068a65

Changed Discriminator output, added critic optimizer, yet to run and …

2bdbf79

…check

Runnable state - Integration

62c9903

RUNNING GANS being tuned

39c494b

GAN changes - args and ignores

36d809e

tried bleu

947b396

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GAN integration #12

GAN integration #12

edorado93 commented Jul 4, 2018 •

edited

Loading

edorado93 Jul 9, 2018

edorado93 Jul 9, 2018

shruthi0898 commented Jul 9, 2018 via email

edorado93 commented Jul 9, 2018 •

edited

Loading

shruthi0898 commented Jul 9, 2018 via email •

edited by edorado93

Loading

edorado93 Jul 9, 2018

edorado93 Jul 9, 2018

edorado93 Jul 9, 2018

edorado93 Jul 9, 2018

edorado93 Jul 9, 2018

shruthi0898 commented Jul 9, 2018 via email

edorado93 commented Jul 12, 2018

edorado93 commented Jul 14, 2018

edorado93 Jul 14, 2018

edorado93 commented Jul 27, 2018

GAN integration #12

Are you sure you want to change the base?

GAN integration #12

Conversation

edorado93 commented Jul 4, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shruthi0898 commented Jul 9, 2018 via email

edorado93 commented Jul 9, 2018 • edited Loading

shruthi0898 commented Jul 9, 2018 via email • edited by edorado93 Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shruthi0898 commented Jul 9, 2018 via email

edorado93 commented Jul 12, 2018

edorado93 commented Jul 14, 2018

Choose a reason for hiding this comment

edorado93 commented Jul 27, 2018

edorado93 commented Jul 4, 2018 •

edited

Loading

edorado93 commented Jul 9, 2018 •

edited

Loading

shruthi0898 commented Jul 9, 2018 via email •

edited by edorado93

Loading