Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how to do hyper-parameter search #5

Open
ruilinchen opened this issue Oct 19, 2020 · 1 comment
Open

how to do hyper-parameter search #5

ruilinchen opened this issue Oct 19, 2020 · 1 comment

Comments

@ruilinchen
Copy link

Thank you for making all the code available!
I'm testing the RegressionTsetlinMachine module with my own data, and I am just wondering if there is any guideline concerning hyper-parameter search.

My features can be converted into series of 20,000 bits and my output is continuous and has about 2,000 observations.
It took me about 3min to run an experiment on my local machine when I set (number_of_clauses=1000, T=5000, s=2.75), so I cannot afford to run too many experiments to optimize these values, but there just seems to be so many different combinations that I felt a little overwhelmed.
.
Therefore, I would really appreciate it if the team can provide some insights on how I might optimize my hyper-parameter search space for (number_of_clauses, T, s) given the data that I have.

@olegranmo
Copy link
Member

olegranmo commented Feb 8, 2021

Hi @ruilinchen! Thanks for your comment. One trick that may help is that when you find a ratio between number_of_clauses and T that works well, you can keep that ratio fixed and adjust both parameters together, e.g., doubling/halving both. When it comes to the s-parameter, values in the range 2.0 - 20 often work well. To avoid adjusting s, you can try out multi-granular clauses by e.g setting s=2.0 and s_range=20.0. Then the clauses will have different s-values, from 2.0 to 20.0. This will bring performance close to the best s-value. However, to maximize performance, finding the best s-value works best. Hope these guidelines help finding good hyper parameters!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants