Hyperparameter optimization with Optuna #24
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1. Running with a .yaml config file will lead to usual execution. 2. Running with the --optimized flag will tune hyperparameters with optuna. No input file needed.
1. objective.py defines the objective function for tuning. Contains the fastvpinns object returning metric for tuning. 2. optuna_tuner.py manages the hyperparameter tuning process.
1. Accept number of trials and number of training iteration for each trial as an argument.
1. Accept an is_optimized argument, True if hyperparameter optimization with Optuna is being used. 2. If is_optimized is True, geometry module doesn't print out the test mesh and VTK file for each trial. 3. Backward compatibility - default value of is_optimized is False, existing code with config file should work as is.
1. Creates an SQLite database if it doesn't exist. Can be used for stalled runs or parallel implementation. 2. Lists available number of GPUs and divides jobs.
thivinanandh
approved these changes
Sep 20, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implement Hyperparameter Optimization using Optuna
Description
This pull request introduces hyperparameter optimization capabilities to FastVPINNs using Optuna. The changes allow for efficient, parallelized optimization across multiple GPUs.
New Files
fastvpinns/hyperparameter_tuning/optuna_tuner.pyfastvpinns/hyperparameter_tuning/objective.pyKey Changes
main_poisson2d.pyto support both YAML config and optimized hyperparametersUsage
To use the new hyperparameter optimization feature:
Run with YAML config file:
python main_poisson2d.py input.yamlRun with hyperparameter optimization:
python main_poisson2d.py --optimized --n-trials 200 --n-epochs 50000The
--optimizedflag triggers the hyperparameter optimization process. The--n-trialsargument specifies the number of optimization trials to run. The--n-epochsargument specifies the number of training iterations for each trial.Dependencies
GPU Utilization
The optimization process automatically detects available GPUs and distributes trials across them for efficient parallel execution.
Example
On a system with 2 GPUs, running:
python main_poisson2d.py --optimized --n-trials 200will execute 200 optimization trials, automatically distributed across both GPUs.
Notes
objective.pyfilefastvpinns_optuna.dbfor persistenceTesting
Future Work