Commits on May 24, 2024

1. build: Add --exclude-from-upload and --exclude-from-download options …

   Excluding files from upload is very handy for ad-hoc skipping of large
   ancillary files or previous output files in the build directory that the
   user wants to ignore for the remote build on AWS Batch (i.e. to start it
   "fresh").  Existing workarounds for the lack of an exclusion mechanism
   include git worktrees to obtain a clean state and moving files or
   directories temporarily out of the build directory.
   
   A future improvement would be adding support to also specify these
   patterns via a file, which can then be checked in to a build repo and
   shared by all users.
   
   As a broader improvement, I'd like to design away this issue by
   separating out the workflow-as-program (rules, code, etc.) from the
   workflow-as-state (config, inputs, outputs, etc.).  This comes out of
   several other goals, but would apply to this "excluded files" need too.
   For example, instead of relying on the implicit state combined with the
   code like now in a single build directory, we'd instead explicitly pass
   an empty starting state separate from the workflow program.
   
   This change includes a small bug fix for --download to allow _only_
   negated patterns.
   
   Resolves: <#219>

   

   tsibley committed May 24, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for 6d465f0
   • Browse repository at this point

   Copy the full SHA

   6d465f0 View commit details

   Browse the repository at this point in the history