Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Look into systematizing the noctua-models migration SOP into a pipeline #289

Open
kltm opened this issue May 27, 2022 · 6 comments
Open

Comments

@kltm
Copy link
Member

kltm commented May 27, 2022

Currently, I'm cutting and pasting from the SOP doc. This is error-prone and an all around not-so-great idea. Take a look at getting the meat of the noctua-models-migration process encoded in a pipeline file:

  1. So I don't have to run in locally and transfer large files in and out
  2. Get access to a larger machine == goes faster
  3. Anybody can run it
  4. (Opens the door for easily testing changes before we apply them)
@kltm kltm changed the title Look into systematizing the Look into systematizing the noctua-models migration SOP into a pipeline May 27, 2022
kltm added a commit that referenced this issue May 27, 2022
@kltm
Copy link
Member Author

kltm commented May 27, 2022

Oh yeah--the i/o issue on our RAIDed servers...

@kltm
Copy link
Member Author

kltm commented Jun 3, 2022

Still poking at it and the GH commands seem to gum up a little locally. Trying to reset the state into the known: all other work gone, upstream master at HEAD.

@kltm
Copy link
Member Author

kltm commented Jun 4, 2022

Okay, some progress.
We probably want to experiment with this with on a branch so we can semi-manually complete the process and see how it works.
At an hour, that takes a while; but it is mechanical, so that hopefully makes up for it taking out the guesswork.

@kltm
Copy link
Member Author

kltm commented Jun 24, 2022

@vanaukenk Moving forward, I believe I've found a way of generating the potential diffs for a migration within a pipeline. I've set it up to drop the reports here: http://skyhook.berkeleybop.org/noctua-models-migrations/reports/
Assuming we are able to run it end-to-end, we should be able to a test run in about an hour with the push of a button.
geneontology/noctua#782

@kltm
Copy link
Member Author

kltm commented Jun 29, 2022

Wondering if I can just commit straight from the repo after running this? Might be the best of both worlds...

@kltm
Copy link
Member Author

kltm commented Jul 14, 2022

As an intermediate step, I could pause the pipeline and copy the resulting models out. That gets us a little closer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

1 participant