Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ML warehouse JSON output capability to TreePublisher #363

Open
3 tasks done
kjsanger opened this issue Apr 5, 2022 · 0 comments
Open
3 tasks done

Add ML warehouse JSON output capability to TreePublisher #363

kjsanger opened this issue Apr 5, 2022 · 0 comments
Assignees

Comments

@kjsanger
Copy link
Member

kjsanger commented Apr 5, 2022

The main Illumina pipeline iRODS publisher can produce a JSON dump containing metadata about the files it has deposited in iRODS.

Calback constructing the JSON here:

my $mlwh_json_cb = sub {

We would like a similar capability for the the tree publisher https://github.com/wtsi-npg/npg_irods/blob/devel/lib/WTSI/NPG/HTS/TreePublisher.pm and to have that exposed through its CLI https://github.com/wtsi-npg/npg_irods/blob/devel/bin/npg_publish_tree.pl

  • Propose a version of the JSON metadata generalised beyond the core Illumina pipeline

We will allow the metadata to describe just a collection path, being the root of the uploaded directory hierarchy,
without including paths of all the consitituent data objects.

  • Propose a route for the metadata to reach the ML warehouse
  • Add to the TreePublisher the option to create JSON file dump
  • Expose the new option on the CLI
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants