Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify schema #43

Merged
merged 4 commits into from
Jul 26, 2023
Merged

Simplify schema #43

merged 4 commits into from
Jul 26, 2023

Conversation

richfitz
Copy link
Member

@richfitz richfitz commented Jul 20, 2023

In the end, very small changes to the schema ahead of moving it into its own repo, I think this is probably reasonable. The biggest changes:

  • drop session from top-level schema, move into the custom orderly one
  • drop packages from the orderly one, we're not even populating this
  • location is a compulsory entry in the config schema (otherwise we don't know what local refers to)
  • drop scripts from the top-level schema as it was always orderly.R and can just go in the custom schema now if people need it

Other changes:

  • Tidy up filenames (locationId to location-id, id to packet-id)
  • Log schema should be extensible
  • time property must include start and end buy may include other things, so long as they are also numbers
  • the args field was left out of the location configuration
  • no longer add a hash field to the deserialised metadata contents (I'd forgotten about this and got confused when spotting it)

I did wonder about:

  • dropping size from the file path/hash/size object, but we use that in outpack_server and the alternative there feels worse (having to hit the store to check the size). At the same time it is strictly redundant as it is implied (though unknowable) from the hash, and makes me wonder how/if we'd cope with an inconsistency there. It's also somewhat ill defined for text files if the encoding is liable to change.
  • in the time element, are start and end really special? they probably are.
  • scripts: this one feels not that interesting, but at the same time once I dropped it it felt wrong too. So I put it back, but this is one that I think is worth potentially being ruthless about
  • session: I think this should move inside the orderly metadata part, and might just do that. Thoughts?

@richfitz richfitz marked this pull request as ready for review July 20, 2023 10:27
@richfitz richfitz requested a review from r-ash July 20, 2023 10:28
@codecov
Copy link

codecov bot commented Jul 25, 2023

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (76bf6e0) 100.00% compared to head (8176bcd) 100.00%.

❗ Current head 8176bcd differs from pull request most recent head 3ac87a1. Consider uploading reports for the commit 3ac87a1 to get more accurate results

Additional details and impacted files
@@            Coverage Diff            @@
##              main       #43   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           35        35           
  Lines         2769      2757   -12     
=========================================
- Hits          2769      2757   -12     
Files Changed Coverage Δ
R/outpack_metadata.R 100.00% <ø> (ø)
R/outpack_packet.R 100.00% <ø> (ø)
R/outpack_tools.R 100.00% <ø> (ø)
R/run.R 100.00% <100.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@r-ash r-ash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chatted through this, looks good to me!

@r-ash r-ash merged commit f05e8c9 into main Jul 26, 2023
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants