…nto ual-clean

- cleaned run.py and settings.yaml
- incorporated latest commits in LBNL-UCB-STI ual-clean-v2 branch
Notes: atlas postprocessor functions disabled for now (due to an unresolved io bug)

ual-clean-v2 settings IWAIT XWAIT caused key errors; changed them back to ual-clean settings

…nto ual-clean

� Conflicts:
�	pilates/activitysim/preprocessor.py
�	settings.yaml
�	upload_and_clean.sh

custom_mpo and model_data_*.h5 have slightly different data structure. Modified preprocessor to resolve this problem

run_atlas upgraded to run_atlas_auto in run.py; atlas container fix (now v1.0.5); minor naming changes in postprocessor for clarity

accessibility_2015.RData and modeaccessibility.csv are atlas inputs used for any year <=2014. They are small files, so added into git too for easy fetch

# Conflicts:
#	pilates/activitysim/preprocessor.py

This set of code has been tested for sfbay 2010-2012 usim-atlas-asim run; atlas version 1.0.5; input data included (RData and csv)

pull latest commits from lbnl repo

atlas sometimes encounters "error reading from connection", which is related to parallel computing and can be fixed by a simple re-submission. Run_atlas_auto function will handle this re-submission when needed. I added "try except" statement to make sure it will not get broken when such errors happen.

Ridehail skims

# Conflicts:
#	pilates/beam/postprocessor.py
#	run.py

…om/LBNL-UCB-STI/PILATES into improve-output-directory-structure

…om/LBNL-UCB-STI/PILATES into improve-output-directory-structure

…om/LBNL-UCB-STI/PILATES into improve-output-directory-structure

…om/LBNL-UCB-STI/PILATES into improve-output-directory-structure

…ucture

Improve output directory structure

# Conflicts:
#	run.py
#	settings.yaml

# Conflicts:
#	settings.yaml

integrate ATLAS into PILATES

since atlas docker is updated to v1.0.6, the "vintage_category" to "model_year" translation in atlas postprocessor can be removed now

keep track of LBNL-UCB-STI master + atlas postprocessor minor fix

atlas output "model_year" -> "modelyear"

major changes:
1. create first skims at warm start atlas stage instead of warm start asim
2. add accessibility calculation in atlas preprocessor, and add atlas_beamac in settings.yaml to turn it on/off
3. solve potential parallel errors in atlas by adding an auto-resubmission when run fails due to parallel errors
4. add vehicleTypeId column in atlas output for BEAM team to read as input

correct a minor error in atlas preprocessor

v1.0.7: atlas_beamac will be parsed to atlas docker, if 0, atlas will read from "accessibility_2015.RData", otherwise atlas preprocessor will calc accessibility itself.

an atlas input needed for calculating accessibility in preprocessing stage (when atlas_beamac > 0)

atlas group updates

function not changed. Minor simplifications: 1) two-line commands simplified to one line "ODmatrix = ODmatrix<=30"; 2) get zone_count directly from ODmatrix size without opening skims.omx again

Atlas group update

raise warning (in run.py parse_args_and_settings) if atlas_beamac = 1 for non-sfbay-taz runs. By default atlas_beamac should be 0. When atlas_beamac =1, accessibility is calculated internally and it relies on a "taz_to_tract_sfb.csv" file, which is externally generated by ATLAS team.

add atlas_beamac warning

Configs outside docker

copy additional files