Confused about the store #404
Replies: 2 comments 4 replies
-
A # put_classifiers_into_store.py
from hydra_zen import store
sklearn_store = store(group="classifier")
# add classifier configs to store
sklearn_store(...) # put_datasets_into_store.py
from hydra_zen import store
sklearn_dataset = store(group="dataset")
# add dataset configs to store
sklearn_dataset (...) # train_models.py
# Need to load the files to add configs to store.
#
# Importing each module runs the code in said module. I.e.
# it adds the dataset/classifier configs to the store.
import put_datasets_into_store
import put_classifiers_into_store
from hydra_zen import store, make_config, zen
store(
make_config(
hydra_defaults=["_self_", {"dataset": "moons"}, {"classifier": "knn"}],
dataset=None,
classifier=None,
),
name="config",
)
def task(dataset, classifier):
# training task
...
if __name__ == "__main__":
# store is only in-memory. You must add the configs to the
# Hydra's ConfigStore so Hydra "sees" them when you configure and launch your job
store.add_to_hydra_store()
zen(task).hydra_main(config_path=None, config_name="config") If you do dataset:
_target_: sklearn.datasets._samples_generator.make_moons
_partial_: true
noise: 0.3
random_state: 0
classifier:
_target_: sklearn.neighbors._classification.KNeighborsClassifier
n_neighbors: 3
weights: uniform
algorithm: auto
leaf_size: 30
p: 2
metric: minkowski
metric_params: null
n_jobs: null |
Beta Was this translation helpful? Give feedback.
-
Hi @francisco-camargo thanks for the feedback. As a temporary band aid, I added a little bit more inline explanation in the how-to: That being said, it sounds like our docs need to do a much better job giving the reader more hands-on experience with and intuition for We really appreciate you taking the time to share your experience with us. |
Beta Was this translation helpful? Give feedback.
-
Hey all,
Been staring and playing with the scikit-learn How-To. It help me quite a bit in understanding some basics about how to structure a project. However, it also brought into focus something that I'm realizing I have found confusing throughout the Tutorials and HowTo's that I've read: the store. Having read through the documentation, I still just don't have a mental model of how to think about the store, and I therefore don't really know how to use it.
store(make_config(
is used kind of makes sense if the scope is just to set some default behavior. I suspect the syntax would be made clear if I reread the docs againif __name__ == "__main__":
is where the wheels fall off for me. Several functions, methods, and options are used and I don't know where to begin to understand what's happening there, nor do I have any real idea of the scope of what each line of code is trying to accomplishIn trying to play with the How-To code, I was thinking of trying to see if I could separate out the scikit-learn version of
my_app.py
into several bite-sized scripts. However this is how I realized I was not understanding the store. For example, do objects added to the store get written to disk somewhere such that I could call them later? Such that I could go to the command line and do something likeIn other words, can I run each piece of the code separately? Or is it the case that I must have a single function that kicks-off putting objects into the store as well as running the
task
function?I hope this makes some sense! Thanks!
Beta Was this translation helpful? Give feedback.
All reactions