GOC / GOA Joint Pipeline

[kltm]

Project link

https://github.com/orgs/geneontology/projects/159

Project description

As part of the discussion from the UNC GO meeting, we decided to go ahead with a joint GO/GOA pipeline. The intended division of labor would be:

GO Central produces:

• Ontology
• User, groups, dbxrefs, and other metadata
• Curation tool resources
  • PAINT annotations
  • Noctua-derived annotations (standard and GO-CAM)
  • "Automated upstreams" (MGI)
• Derived products to drive GO Central interfaces and downloads
  • Solr index, Blazegraph, GO API and required files, etc.

GO Central consumes from GOA:

• GAFs and GPAD/GPI(?) files from GOA
• GOA annotation error reports

GOA produces:

• GAFs and GPAD/GPI(?) files from GOA
• GOA annotation error reports

GOA consumes from GOC:

• Ontology
• Metadata
• Noctua GPADs
• PAINT
• "Automated upstreams" (MGI)

This joint pipeline would solve several ongoing issues that GO faces wrt the pipeline

• speed and stability of releases
• detailed error reporting
• ontology synchronization
• consistence of data products for the GO community
• infrastructure expense (leveraging EBI compute)

PI

Chris

Product owner (PO)

Pascale

Technical lead (TL)

Seth/Alex

Other personnel (OP)

Dustin (at need)

Technical specs

Rolling technical discussions w/Alex and Pascale:
https://docs.google.com/document/d/1Jxl2WaOCpxmiuPrTZZB3LFs4lYzXQtjDxqAjvuidxRE/edit
Explanatory slides: (IN PROGRESS)

Other comments

N/A

[kltm]

Tagging @pgaudet, noting that this not "officially" exists.

[kltm]

Items to be added to project at need.

We still need to work out a lot of technical details (e.g. file granularity, downloads, timing and freezes wrt ontology and curation tools for release processing, etc.). Early days.

[kltm]

Current Pascale working doc: https://docs.google.com/document/d/1klJX7do0JGaWkFY3fcGsXBXGPPG9a42y_4YM7arCPK4/edit?pli=1