need --purge option to remove and compress output for long-term storage

[RickKessler]

No description provided.

[djbrout]

1_SIM: Recursively Tar Folders
2_LCFIT:
- allow pipin to continue after submit_batch_jobs.sh --purge
- recursive tar option for archiving
3_CLASS:
- reduce log output (need to ask people who implemented classifiers to help)
- recursive tar option for archiving
4_AGG:
- gzip the .csv and .key files immediately
- recursive tar option for archiving
5_MERGE:
- recursive tar option for archiving
6_BIASCOR:
- allow pipin to continue after submit_batch_jobs.sh --purge
- recursive tar option for archiving
7_CREATE_COV:
- allow for gzipping immediately of covariance and correlation matrices. may have to implement this for cosmomc covariance matrices.
- recursive tar option for archiving
8_COSMOMC:
- remove chains log files except last one
- catenate chains and gzip just like its done in 9_ANALYZE (but dont get rid of the original walkers until arxiving, but can gzip the walkers)
- arxiving: remove walkers and recursive tar
9_ANALYSE
- dont recopy the chains here, just make symlink
- remove parenthesis and blank spaces in filenames
- recursive tar option for archiving (maybe not needed, but cant hurt?)

[djbrout]

cfg.yml - should have a max_output (to prevent someone accidentally generating terabytes of data)

[djbrout]

cfg.yml should have parallel outputs for SCRATCH and PROJECT. each of these directories should have an ALLOWED.TXT for SCRATCH it will be * but for PROJECT it will be PANTHEON or something that only permits certain outputs to be generated to limit the project area from being overrun.

[OmegaLambda1998]

More suggestions from @RickKessler (#111)

under 3_CLAS/XYZ

compress output.log
compress predictions.csv
remove /dump directory unless user requests it (there is a huge .pickle file that is likely not used)

In stage 7_CREATE_COV, suppress /cosmomc output unless specifically requested.