Skip to content

Latest commit

 

History

History
114 lines (90 loc) · 4.38 KB

README.md

File metadata and controls

114 lines (90 loc) · 4.38 KB
Raw

The code for the successor version, Symnet 3.0, published in UAI 2023, is available at SymNet 3.0

SymNet 2.0

Repo contains the code for UAI 2022 paper "SymNet 2.0: Effectively handling Non-Fluents and Actions in Generalized Neural Policies for RDDL Relational MDPs".

Libraries required:

  1. python3
  2. tensorflow=2.0
  3. unittest
  4. multiprocessing
  5. threading
  6. shutil
  7. better_exceptions
  8. pickle
  9. networkx
  10. scipy

Note that we also use Spektral, however, there is no need to install it as we have incorporated that in the repo.

How to run an experiment:

  1. Generate trajectories using PROST as

    1. Follow the instructions mentioned at https://github.com/prost-planner/prost. to setup PROST 2014.
    2. Run PROST using below command and save the logs generated by PROST to a file $RESULT_FILE.
    ./prost.py $INSTANCE "[Prost -s 1 -se [IPC2014]]" > $RESULT_FILE

  2. Generate a dataset from a raw trajectory:

python dataset_builder.py --domain DOMAIN --start_instance START_INSTANCE --num_instances NUM_INSTANCES --prost_log PROST_LOG --save_folder SAVE_FOLDER

For example, assume that PROST_LOG contains files 1.result, 2.result...10.result of domain academic_advising. To generate the dataset for all these files, run

python dataset_builder.py --domain academic_advising --start_instance 1 --num_instances 10 --prost_log PROST_LOG --save_folder SAVE_FOLDER
  1. Set hyperparameters of the experiment as explained at the end of this readme.
  2. Run these commands from the main folder once:
for i in {1..10}
do
cp -r ./rddl/lib ./gym/envs/rddl/rddl
cp ./gym/envs/rddl/rddl/lib/clibxx.so ./gym/envs/rddl/rddl/lib/clibxx$i.so
cp ./rddl/lib/clibxx.so ./rddl/lib/clibxx$i.so
done
  1. To train SymNet2.0 on a domain run below command.
python ./multi_train/deep_plan/train_supervised.py --domain DOMAIN --model_dir MODEL_DIR --train_instances TRAIN_INSTANCES --val_instances VAL_INSTANCES

Example, to train on academic_advising instances 1,2,3, and validate on instance 4, run:

python ./multi_train/deep_plan/train_supervised.py --domain academic_advising --model_dir MODEL_DIR --train_instances "1,2,3" --val_instance "4"
  1. To validate all models:
python ./multi_train/deep_plan/validate.py --domain DOMAIN --trained_model_path PATH --num_threads NUM_THREADS --val_instances VAL_INSTANCES

Example, to validate on instance 4 of academic_advising using 6 threads,

python ./multi_train/deep_plan/validate.py --domain academic_advising --trained_model_path PATH --num_threads 6 --val_instances "4"
  1. To test the best model:
python ./multi_train/deep_plan/test.py --domain DOMAIN --trained_model_path PATH --num_threads NUM_THREADS --test_instances TEST_INSTANCES

Example, to test on instance 1,2,3,4,5 of academic_advising using 2 threads,

python ./multi_train/deep_plan/test.py --domain academic_advising --trained_model_path PATH --num_threads 2 --val_instances "1,2,3,4,5"

Hyper Parameters explained:

You can set the general hyperparameters in two files: For my_config:

  1. domains: academic_advising, crossing_traffic, game_of_life, navigation, skill_teaching, sysadmin, tamarisk, traffic, wildfire
  2. num_validation_episodes: Number of episodes to validate on
  3. num_testing_episodes: Number of episodes to test on
  4. dataset_folder: Folder storing the datasets generated using scripts mentioned above
  5. batch_size: Batch size for supervised training
  6. epochs: Number of epochs to train for
  7. ckpt_freq: Frequency of saving checkpoint
  8. lr: Learning rate


Code Organization:

  1. ./gym contains the code for setting up the RDDL domains and instances in ATARI gym based framework (used for generating the trajectories during validation and testing).
  2. ./rddl contains files (RDDL, parsed files and dbbn files) for various domains and instances used.
  3. ./multi_train/deep_plan contains the main code.

For the predecessor version refer to SymNet 1.0