Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to continue to train? #4

Open
GulerEnes opened this issue May 4, 2022 · 3 comments
Open

How to continue to train? #4

GulerEnes opened this issue May 4, 2022 · 3 comments
Labels
enhancement New feature or request

Comments

@GulerEnes
Copy link

Hi, I am trying to use your codes on Google Colab. But sometimes Colab suddenly closes itself. Is there a way to continue training?

I didn't see any flag in main.py about this.

@Akhilesh64
Copy link
Owner

Hi @GulerEnes this is an issue with colab. It shuts down if no activity is detected for sometime. There is no problem with the code.

@GulerEnes
Copy link
Author

Hi @GulerEnes this is an issue with colab. It shuts down if no activity is detected for sometime. There is no problem with the code.

Yes, I know the codes are working well. Each 10 epoch a new model.h5 file saving. I just want to continue the training from the last saved model.h5 for any reason. How can I do this?

@Akhilesh64
Copy link
Owner

The scripts don't yet support resuming from checkpoint but I'll try to implement this if I get time or you can do these changes yourselves because this seems pretty straightforward.

@Akhilesh64 Akhilesh64 added the enhancement New feature or request label May 10, 2022
@Akhilesh64 Akhilesh64 reopened this May 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants