Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pytest cannot be run without forked #743

Open
Gamenot opened this issue Apr 7, 2021 · 2 comments
Open

Pytest cannot be run without forked #743

Gamenot opened this issue Apr 7, 2021 · 2 comments
Assignees
Labels
bug Something isn't working

Comments

@Gamenot
Copy link
Collaborator

Gamenot commented Apr 7, 2021

BUG REPORT

High Level Description
By the problems that ULTRA team were having, pytest has stability issues running tests without forked. Tests with forked take a much longer time to run. This is due to issues with having multiple SMARTS instances.

SMARTS version
SMARTS branch: ultra-gb-record-density-data
2611ccc

Previous associated issues
#719
#597
#184

Steps to reproduce the bug
Run the CI on the 2611ccc commit
Use branch listed here: #745

Resulting and expected behaviour
:display:x11display(fatal) thrown when all tests should pass.

Error logs and screenshots
#730
https://github.com/huawei-noah/SMARTS/runs/2279454801

System information

  • Operating system & Environment [Please note if using Docker]:
    Github CI: Linux 18.04 Docker
  • Number of system cores available
    2
  • Python version [Please note if using Jupyter]
    3.7
  • Python packages [e.g. from pip freeze]
    n/a
  • SUMO version
    n/a
  • Date when bug occurred
    2021/04/07

Impact [If known]
CI is much slower due to having to run forked.

@sah-huawei
Copy link
Contributor

sah-huawei commented Apr 8, 2021

With PR #747, I was just able to run make test without --forked on my local laptop (although I had to hack -n 3 to prevent overload!). All tests passed.

@Gamenot do you think we should remove --forked from the test rule in the Makefile? Or just do that for CI? (The -n issue is still annoying when running locally.)

EDIT: Never mind. I get seemingly-random failures if I play with -n or test ordering. I guess there's still more digging to do!

@sah-huawei
Copy link
Contributor

sah-huawei commented May 7, 2021

I suspect the problems may arise when running two tests that use Ray side-by-side. For example, (after reordering the tests), I got a segmentation fault (in test_multi_instance_example()) when test_rllib_hiway_env() (from ./env/tests/test_rllib_hiway_env.py) was being executed at the same time as test_multi_instance_example() (from ./tests/tests_examples.py), both of which have calls to ray.init().

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants