Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] model.eval with CPU returns None if insufficient memory #991

Open
nick-youngblut opened this issue Jul 30, 2024 · 4 comments
Open
Labels
bug Something isn't working

Comments

@nick-youngblut
Copy link

nick-youngblut commented Jul 30, 2024

Describe the bug

model = models.Cellpose(gpu=False, model_type="nuclei")
masks, _, _, _ = model.eval(image, diameter=diameter)

If < ~80 Gb of memory provided for the job, then masks is None. If sufficient memory is provided then masks is not None.

model.eval is not throwing an insufficient memory error if there is not enough memory.

To Reproduce

See above

Run log

cellpose version: 	3.0.9
platform:       	linux
python version: 	3.11.9
torch version:  	2.1.2.post100
2024-07-30 14:15:36,303 - Masking image...
2024-07-30 14:15:36,304 - >>>> using CPU
2024-07-30 14:15:36,304 - >> nuclei << model set to be used
2024-07-30 14:15:36,344 - >>>> loading model models/nucleitorch_0
2024-07-30 14:15:36,438 - >>>> model diam_mean =  17.000 (ROIs rescaled to this size during training)
2024-07-30 14:15:36,480 - channels set to [0, 0]
2024-07-30 14:15:36,481 - ~~~ FINDING MASKS ~~~
2024-07-30 14:15:38,842 - >>>> TOTAL TIME 2.36 sec

None was returned for the masks variable for this logged run. I provided 8 Gb of memory. If I instead provide 80 Gb of memory, a mask object is returned instead of None.

Conda env

name: cellpose
channels:
  - conda-forge
  - bioconda
dependencies:
  - python=3.11.9
  - pip=24.0
  - bokeh=3.4.2
  - opencv=4.8.1
  - holoviews=1.19.0
  - ipython=8.25.0
  - matplotlib=3.8.4
  - numpy=1.26.4
  - psutil=6.0.0
  - scikit-image=0.24.0
  - tifffile=2024.6.18
  - xmltodict=0.13.0
  - tqdm=4.66.4
  - scipy=1.10.1
  - pims=0.7
  - cellpose=3.0.9
  - pip:
    - aicspylibczi==3.1.2
    - glob2
    - pprintpp
@nick-youngblut nick-youngblut added the bug Something isn't working label Jul 30, 2024
@carsen-stringer
Copy link
Member

sorry I think you must be using another wrapper on top of cellpose (our code doesn't say "Masking image...") please try again without the wrapper and let us know if it's still not working

@nick-youngblut
Copy link
Author

@carsen-stringer 2024-07-30 14:15:36,303 - Masking image... is just a loggin.info call prior to running:

model = models.Cellpose(gpu=False, model_type="nuclei")
masks, _, _, _ = model.eval(image, diameter=diameter)

There is no wrapper.

@carsen-stringer
Copy link
Member

Thanks hmm let me see if I can replicate this in order to debug it. How big is the image?

@nick-youngblut
Copy link
Author

Thanks for reopening the issue. I believe the image was ~10 Gb.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants