TG2-TO DO List for standard submission

[Tasilee]

A list of things for @ArthurChapman, @Tasilee, @tucotuco, @chicoreus and @ymgan to complete the BDQ Core standard submission

Introduction Document and Landing Pages

• (Done) Replicate target directory structure for the standard within _review @tucotuco
• (Done) Files under _review directory that require some manual content @chicoreus (content parsed out of supplemental into users guides and other documents).
• (Done) Diagrams showing a generic approach to using Validations, Measures, Amendments, Validations, Measures @ymgan
• (Done) Place diagram svg files within _review directory. in _review/docs/intro @chicoreus
• (Done) Modfications to target directory structure to include derivative files for review @chicoreus See: _review/README for current state of files and directories
• (Done) Identify files that are currently maintained outside of _review and copied into it. @chicoreus see: https://github.com/tdwg/bdq/blob/master/tg2/_make_review/copy_files.sh
• Pare down Introduction document to minimum normative and non-normative needed @tucotuco (done through section 1)

Supplementary Document

• (Done) Edit Supplementary Document @Tasilee
• (Done) Put supplementary information into supplementary document @Tasilee
• (Done) Move normative information from supplementary document to appropriate other documents, limit content of supplementary to just supplementary. @chicoreus
• (Done) Format supplementary document @ArthurChapman
• (Done) Copy the bdqTag vocabulary into a markdown table in the supplementary document and remove from vocabulary/glossary_terms.csv @chicoreus
• (Done) Copy the bdqTestField vocabulary into a markdown table in the supplementary document and remove from vocabulary/glossary_terms.csv. @chicoreus
• (Done) Add history of the development of the tests to Section 3.
• (Done) Merge the three duplicative sets of Test Field descriptions in the supplementary documentation in to one.
• Add Competency Questions for bdqffdq and bdqcore to the supplementary documentation. @chicoreus (in progress)
• Review Supplement Document (in progress) @ArthurChapman

Vocabularies and termlist documents

• (Done) Flesh out Vocabulary landing page @ArthurChapman, @tucotuco
• (Done) Review which terms go where @ArthurChapman @chicoreus
• (Done) Review bdq and bdqdim term definitions @ArthurChapman
• Update Ontology @chicoreus @ArthurChapman
  • (Done) Review bdq/bdqffdq vocabulary definitions @ArthurChapman @chicoreus
  • (Done) Move ontology into tdw/bdq respository @chicoreus
  • (Done) Update bdqffdq ontology skos:definitions and skos:comments @chicoreus
  • (Done) Give each bdqffdq term a skos:prefLabel, skos:definition, rdfs:label, and rdfs:comment @chicoreus
  • (Done) Create csv export of ontology @chicoreus (note, needs manual edit with each export from owl).
  • Integrate First draft Python code to produce human readable display of ontology into _review/build/ @chicoreus
• Provide rdf distributions of Vocabulary Term List documents following SDS section 2.2.4. (owl document for bdqffdq)
  • (Done) rdf artifacts for Vocabulary Terms lists of bdq, bdqdim, bdqenh, bdqcrit @chicoreus
  • (Done) Update rdf artifact for bdqcore generated with kurator-ffdq @chicoreus
    • (Done) Multiple tests in each Policy @chicoreus,
    • (Done) Class names matched to bdqffdq. @chicoreus
    • (Done) Add Criterion, Enhancement, DataQualityDimension instances from bdqcrit, bdqenh, bdqdim vocabularies @chicoreus
    • (Done) Stable identifiers for methods, specifications, information elements in RDF generated from CSV @chicoreus (tg2/core/information_element_guids.csv and tg2/core/TG2_tests_additional_guids.csv used by kurator-ffdq with -guidFile and -ieGuidFile options)
      • (Done) Provide for reuse of uuids between builds of RDF artifact from configuration files @chicoreus
      • (Done) stable identifier file for core test methods/specifications/profiles @chicoreus
      • (Done) stable identifier file for information elements @chicoreus
      • (Done) add stable identifiers for methods/specifications/profiles for multirecord measures @chicoreus
• Build Vocabulary term list files in _review/docs/list/{term}/index.md files @chicoreus
  • (Done) Build script for term list files modified to run with local configuration files for draft standard @chicoreus
  • (Done) Configure and run for bdq, bdqdim, bdqenh, bdqcrit vocabularies. @chicoreus
  • Update Vocabulary Landing page
  • links to point to new location for final term list documents rather than termlist header documents, when the former are ready @tucotuco
• Update Dimensions Vocabulary term-list page to reflect bdqffdq:DataQualityDimension @ArthurChapman ?Done?, page has moved, edit the template in _review/build/templates/list/bdqdim/ (reference was to a Dimensions Vocabulary Landing page, but there is no such file, this must be a reference to the term list page in list/bdqdim (noted as done below)).
• Review human readable term-list files in
  - (Done) https://github.com/tdwg/bdq/blob/master/tg2/_review/docs/list/bdq/index.md @ArthurChapman
  - (Done) https://github.com/tdwg/bdq/blob/master/tg2/_review/docs/list/bdqdim/index.md @ArthurChapman
  - (Done) https://github.com/tdwg/bdq/blob/master/tg2/_review/docs/list/bdqcrit/index.md @ArthurChapman
  - (Done) https://github.com/tdwg/bdq/blob/master/tg2/_review/docs/list/bdqenh/index.md @ArthurChapman
  - (Done) Checked for consistency of headings between all four documents. Header files updated @ArthurChapman @chicoreus
• (Done) Document and close Issue BDQ Core - VOCABULARY of terms #152 as this vocabulary has been split into separate files and added to _review @ArthurChapman
• (Done) Add vocabularies and pages for terms to apply to Criterion and Enancement fundamental concepts @chicoreus
• (Done) Hold discussion with Cecilie Svenningsen (GBIF) on Vocabularies @ArthurChapman
• (Done) Provide GBIF with list of future Vocabularies needed for Data Quality @ArthurChapman
• Review bdq term list header document @Tasilee

Quick Reference Guide

• (Done) Create Preface for the Quick Reference Guide @Tasilee
• (Done) Decide what descriptors that will be in the Quick Reference Guide @Tasilee
• (Done) Build bdqcore Quick Reference Guide @Tasilee
• (Done) Create draft QRG Footer (Citation based on Darwin Core QRG) @Tasilee
• (Done) Python script to produce [quick reference guide] @chicoreus (https://github.com/tdwg/bdq/blob/master/tg2/_review/docs/terms/bdqcore/index.md) for combined singlerecord and multirecord bdqcore tests @chicoreus .
• (Done) Incorporate this script with header/footer assembly @tucotuco @chicoreus

Users Guide

• (Done) Complete template in _review/build/template/guide/bdqffdq/ @chicoreus
• Review content to confirm no missing assumptions in understanding and clarity of text for consumers.
• **(Done)**Generate TOC (sections for populating) @chicoreus
• allocate sections to team members (https://github.com/tdwg/bdq/blob/master/tg2/_review/docs/users/index.md) @chicoreus
• **(Done)**Script to build and assemble materials into _review/docs/guides directory @chicoreus

Implementation Guide

• (Done) Complete template in _review/build/template/guide/bdqffdq/ @chicoreus
• (Done) Generate TOC (sections for populating) @chicoreus
• allocate sections to team members (https://github.com/tdwg/bdq/blob/master/tg2/_review/docs/implementers/index.md)
• (Done) Script to build and assemble materials into _review/docs/guides directory @chicoreus
• Add note on stub code generation by kurator-ffdq @chicoreus

bdqffdq Guide

• (Done) Complete template in _review/build/template/guide/bdqffdq/ @chicoreus
• (Done) Create visualizations to explain ontology, add to bdqffdq guide page. @chicoreus (files are in _review/build/template/guide/bdqffdq/)
• Edit the above ontology visualizations @ArthurChapman (in progress)
• (Done) Script to build and assemble materials into _review/docs/guides directory @chicoreus

Standards Maintenance Document

• (Done) Stub for Standard Maintenance document
• (Done) Add a section on Test Data @Tasilee
• Add a section on supporting code (bdq_issue_to_csv, bdqtestrunner, kurator-ffdq) @chicoreus

Tests

• (Done) Review/edit (English) Description in the light of current Expected Responses (CORE) @ArthurChapman
• (Done) Add skos:prefLabel for multi-record measures in bdq_issue_to_csv @chicoreus
• Update and add links to source code for implementations in test descriptors for filtered push libraries @chicoreus
• Freeze editing of markdown table representations of tests (decision by all)
  • Final export from markdown tables with bdq_issue_to_csv @chicoreus
  • Then: Update rdf artifact for bdqcore generated with kurator-ffdq @chicoreus
  • (Done) Replace all instances of EMPTY and NOT EMPTY, etc. in the test specifications with bdq:Empty @Tasilee @ArthurChapman
• (Done) Build script that copies latest validation data files (test data and non-printing characters) into _review directory. @chicoreus See _make_review/copy_files.sh
• (Done) Add implementation for TG2-ISSUE_COORDINATES_CENTEROFCOUNTRY #287 run against validation data. @chicoreus
• (Done) Check generated Examples in Test Data match those in the Tests and align accordingly @Tasilee
• (Done) Check and align inconsistencies in phrasing in the Specification of "in bdq:sourceAuthority" vs "in the bdq:sourceAuthority" @Tasilee
• (Done) Check consistency of use of bdq:sourceAuthority and ISO references in all tests and align @Tasilee
• Complete review of FAIL cases in validation data, update test specifications, validation data, or implementation as needed (also under Test Data below) @chicoreus @Tasilee @ArthurChapman

References

• (Done) Prepare intro for References @Tasilee
  • (Done) Format RFC 2119 reference and include in reference list @Tasilee
• (Done) @Tasilee to send @ArthurChapman a list of References that need correction
• (Done) Correct References in the Test Issues @ArthurChapman, @Tasilee
• (Done) Prepare a combined list of References @Tasilee

Test Data

• (Done) Add new column for 'Date Last Updated' and populate with current date @Tasilee
• (Done) Rationalize (align for consistency) of Response.comments and align with Test Examples (e.g., tense, phrasing) @Tasilee
• Complete review of FAIL cases in validation data, update test specifications, validation data, or implementation as needed @chicoreus @Tasilee @ArthurChapman
  • Resolve 3 apparent duplicate rows @Tasilee Note: All filtered push qc implementations passing all validation rows except the 3 that appear as duplicates, and the annotation test for which there is no implementation.

General

• (Done) Identify potential Review Managers
• Identify potential Maintenance Managers
• (Done) Prepare and send off Report to Executive (All)
• (Done) Make verbal report and summary to TDWG Business Meeting at TDWG2024 @ArthurChapman
• Add configuration file for acknowledgements? Seems a good idea that works with authors configuration file. In progress @chicoreus @ArthurChapman @Tasilee
• (Done) Prepare diagrams for each of the test types (NAME, SPACE, TIME, OTHER) showing linkages between the tests @ArthurChapman
  • produce RDF triples from the diagrams using showing linkages between the tests using improvedBy, targetedIssue, targetedMeasure, targetedValidation terms (In progress) @ArthurChapman ??
  • Write sparql and use these as competency questions in supplement @chicoreus
• Final (editorial) review of all documents :) @tucotuco

[chicoreus]

Fixed the nested list formating in the list of actions, see: https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#nested-lists