Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pseudo-experiments notebook is giving nans #63

Closed
matthewfeickert opened this issue Aug 19, 2023 · 1 comment · Fixed by #64
Closed

Pseudo-experiments notebook is giving nans #63

matthewfeickert opened this issue Aug 19, 2023 · 1 comment · Fixed by #64
Assignees
Labels
bug Something isn't working

Comments

@matthewfeickert
Copy link
Member

Following PR #61, in the Playing with Toys notebook, in the Hypothesis Testing (low stats) section using the toybased calculator is yielding nans in the calculation:

CLs_obs, CLs_exp = pyhf.infer.hypotest(
    1.0,  # null hypothesis
    [5.0, 7.0] + model.config.auxdata,
    model,
    test_stat="qtilde",
    return_expected_set=True,
    calctype="toybased",
    ntoys=1000,
    track_progress=False,
)
print(f"      Observed CLs: {CLs_obs:.4f}")
for expected_value, n_sigma in zip(CLs_exp, np.arange(-2, 3)):
    print(f"Expected CLs({n_sigma:2d} σ): {expected_value:.4f}")
/usr/local/venv/lib/python3.10/site-packages/scipy/optimize/_optimize.py:353: RuntimeWarning: Values in x were outside bounds during a minimize step, clipping to bounds
  warnings.warn("Values in x were outside bounds during a "
      Observed CLs: nan
Expected CLs(-2 σ): 0.0000
Expected CLs(-1 σ): 0.0452
Expected CLs( 0 σ): 0.1756
Expected CLs( 1 σ): 0.4375
Expected CLs( 2 σ): 1.0000
/usr/local/venv/lib/python3.10/site-packages/pyhf/infer/calculators.py:864: RuntimeWarning: invalid value encountered in divide
  CLs = tensorlib.astensor(CLsb / CLb)
@matthewfeickert matthewfeickert added the bug Something isn't working label Aug 19, 2023
@matthewfeickert
Copy link
Member Author

Okay running this locally with track_progress=True I can see this is because the signal-like pseudo-experiments fail after less than 200 trials

Signal-like:  16%|████████████████████▌                                                                                                            | 159/1000 [00:02<00:12, 66.83toy/s]
/home/feickert/.pyenv/versions/pyhf-tutorial-dev/lib/python3.10/site-packages/scipy/optimize/_optimize.py:353: RuntimeWarning: Values in x were outside bounds during a minimize step, clipping to bounds
  warnings.warn("Values in x were outside bounds during a "

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants