SymNet3.0

Code for SymNet3.0: Exploiting Long-Range Influences in Learning Generalized Neural Policies for Relational MDPs, UAI 2023(link)

Please download datasets and instances from here into the main folder and unzip it.

Libraries required

1. python3
2. tensorflow=2.0
3. unittest
4. multiprocessing
5. threading
6. shutil
7. better_exceptions
8. pickle
9. networkx
10. scipy
11. tqdm
12. pandas
13. numpy

Running an experiment

To run an experiment, set parameters in my_config.py. The important ones are the 'domain', 'mode', 'model_dir' and 'setting'. To train a model, just run:

python train.py

This file both trains and validates models. The best model is stored in the checkpoints/ subfolder of the model folder. The meta_logging.csv file shows reward progress across training.

To run inference on the best model, just run:

python test.py

This will generate a file results.csv in the model folder which will contain rewards obtained by the best model.

Important parameters explained

Parameters can be set in two files in multi_train/deep_plan:

1. my_config.py
2. symnet3_config.py

The most important parameters are:

1. mode: This sets the training setting and architecture. Use "no_dist" for SymNet2.0, "kl" for SymNet3.0+KL, "no_kl" for SymNet3.0-KL and "kl_decay" for SymNet3+KL_{decay}
2. setting: This sets the instances to be used for training. "lr" uses the Long Range instances and "ippc" uses the IPPC instances. The instances used for training are given in train.py.

Code Organization

1. data/ contains trajectory data generated by PROST and the generators used for instance generation.
2. gym/ contains code for initializing and simulating RDDL environments.
3. rddl/ contains all the instance files along with their DBN .dot files and parsed files.
4. utils/ contains code for extracting. instance graph and generating other utilities for all the environments.
5. spektral/ contains code used for GATs and the Influence Layer.
6. multi_train/ contains code for training and evaluation.
7. multi_train/deep_plan/networks contains code for the architecture.

Feel free to contact the authors if you run into issues.