Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

why hard code the model name? couldn't this be dynamic? #52

Open
johndpope opened this issue Dec 6, 2022 · 3 comments
Open

why hard code the model name? couldn't this be dynamic? #52

johndpope opened this issue Dec 6, 2022 · 3 comments

Comments

@johndpope
Copy link

cog-stable-diffusion/predict.py

Line 26 in 4b5af4a

"runwayml/stable-diffusion-v1-5",

@johndpope
Copy link
Author

background
I'm new to cog - so maybe overlooking something.

so my use case is I'm running this repo
https://github.com/scristobal/botatobot

and the instructions are -

Run Replicate's stable diffusion image locally with docker run -d -p 5001:5000 --gpus=all r8.im/stability-ai/stable-diffusion@sha256:a9758cbfbd5f3c2094457d996681af52552901775aa2d6dd0b17fd15df959bef.

but this image is opaque to me - I attempted to run dedockify to break it open -


FROM False
ADD file:7633003155a1059419aa1a6756fafb6e4f419d65bff7feb7c945de1e29dccb1e in /
CMD ["bash"]
RUN ENV NVARCH=x86_64
RUN ENV NVIDIA_REQUIRE_CUDA=cuda>=11.6 brand=tesla,driver>=418,driver<419 brand=tesla,driver>=450,driver<451 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471
RUN ENV NV_CUDA_CUDART_VERSION=11.6.55-1
RUN ENV NV_CUDA_COMPAT_PACKAGE=cuda-compat-11-6
RUN ARG TARGETARCH
RUN LABEL maintainer=NVIDIA CORPORATION <[email protected]>
RUN RUN |1 TARGETARCH=amd64 /bin/sh -c apt-get update \
    && apt-get install -y --no-install-recommends     gnupg2 curl ca-certificates \
    &&     curl -fsSL https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/${NVARCH}/3bf863cc.pub | apt-key add - \
    &&     echo "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/${NVARCH} /" > /etc/apt/sources.list.d/cuda.list \
    &&     apt-get purge --autoremove -y curl     \
    && rm -rf /var/lib/apt/lists/* # buildkit
RUN ENV CUDA_VERSION=11.6.2
RUN RUN |1 TARGETARCH=amd64 /bin/sh -c apt-get update \
    && apt-get install -y --no-install-recommends     cuda-cudart-11-6=${NV_CUDA_CUDART_VERSION}     ${NV_CUDA_COMPAT_PACKAGE}     \
    && ln -s cuda-11.6 /usr/local/cuda \
    &&     rm -rf /var/lib/apt/lists/* # buildkit
RUN RUN |1 TARGETARCH=amd64 /bin/sh -c echo "/usr/local/nvidia/lib" >> /etc/ld.so.conf.d/nvidia.conf     \
    && echo "/usr/local/nvidia/lib64" >> /etc/ld.so.conf.d/nvidia.conf # buildkit
RUN ENV PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
RUN ENV LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64
RUN COPY NGC-DL-CONTAINER-LICENSE / # buildkit
RUN ENV NVIDIA_VISIBLE_DEVICES=all
RUN ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
RUN ENV NV_CUDA_LIB_VERSION=11.6.2-1
RUN ENV NV_NVTX_VERSION=11.6.124-1
RUN ENV NV_LIBNPP_VERSION=11.6.3.124-1
RUN ENV NV_LIBNPP_PACKAGE=libnpp-11-6=11.6.3.124-1
RUN ENV NV_LIBCUSPARSE_VERSION=11.7.2.124-1
RUN ENV NV_LIBCUBLAS_PACKAGE_NAME=libcublas-11-6
RUN ENV NV_LIBCUBLAS_VERSION=11.9.2.110-1
RUN ENV NV_LIBCUBLAS_PACKAGE=libcublas-11-6=11.9.2.110-1
RUN ENV NV_LIBNCCL_PACKAGE_NAME=libnccl2
RUN ENV NV_LIBNCCL_PACKAGE_VERSION=2.12.10-1
RUN ENV NCCL_VERSION=2.12.10-1
RUN ENV NV_LIBNCCL_PACKAGE=libnccl2=2.12.10-1+cuda11.6
RUN ARG TARGETARCH
RUN LABEL maintainer=NVIDIA CORPORATION <[email protected]>
RUN RUN |1 TARGETARCH=amd64 /bin/sh -c apt-get update \
    && apt-get install -y --no-install-recommends     cuda-libraries-11-6=${NV_CUDA_LIB_VERSION}     ${NV_LIBNPP_PACKAGE}     cuda-nvtx-11-6=${NV_NVTX_VERSION}     libcusparse-11-6=${NV_LIBCUSPARSE_VERSION}     ${NV_LIBCUBLAS_PACKAGE}     ${NV_LIBNCCL_PACKAGE}     \
    && rm -rf /var/lib/apt/lists/* # buildkit
RUN RUN |1 TARGETARCH=amd64 /bin/sh -c apt-mark hold ${NV_LIBCUBLAS_PACKAGE_NAME} ${NV_LIBNCCL_PACKAGE_NAME} # buildkit
RUN ENV NV_CUDA_LIB_VERSION=11.6.2-1
RUN ENV NV_CUDA_CUDART_DEV_VERSION=11.6.55-1
RUN ENV NV_NVML_DEV_VERSION=11.6.55-1
RUN ENV NV_LIBCUSPARSE_DEV_VERSION=11.7.2.124-1
RUN ENV NV_LIBNPP_DEV_VERSION=11.6.3.124-1
RUN ENV NV_LIBNPP_DEV_PACKAGE=libnpp-dev-11-6=11.6.3.124-1
RUN ENV NV_LIBCUBLAS_DEV_VERSION=11.9.2.110-1
RUN ENV NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-11-6
RUN ENV NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-11-6=11.9.2.110-1
RUN ENV NV_NVPROF_VERSION=11.6.124-1
RUN ENV NV_NVPROF_DEV_PACKAGE=cuda-nvprof-11-6=11.6.124-1
RUN ENV NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev
RUN ENV NV_LIBNCCL_DEV_PACKAGE_VERSION=2.12.10-1
RUN ENV NCCL_VERSION=2.12.10-1
RUN ENV NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.12.10-1+cuda11.6
RUN ARG TARGETARCH
RUN LABEL maintainer=NVIDIA CORPORATION <[email protected]>
RUN RUN |1 TARGETARCH=amd64 /bin/sh -c apt-get update \
    && apt-get install -y --no-install-recommends     libtinfo5 libncursesw5     cuda-cudart-dev-11-6=${NV_CUDA_CUDART_DEV_VERSION}     cuda-command-line-tools-11-6=${NV_CUDA_LIB_VERSION}     cuda-minimal-build-11-6=${NV_CUDA_LIB_VERSION}     cuda-libraries-dev-11-6=${NV_CUDA_LIB_VERSION}     cuda-nvml-dev-11-6=${NV_NVML_DEV_VERSION}     ${NV_NVPROF_DEV_PACKAGE}     ${NV_LIBNPP_DEV_PACKAGE}     libcusparse-dev-11-6=${NV_LIBCUSPARSE_DEV_VERSION}     ${NV_LIBCUBLAS_DEV_PACKAGE}     ${NV_LIBNCCL_DEV_PACKAGE}     \
    && rm -rf /var/lib/apt/lists/* # buildkit
RUN RUN |1 TARGETARCH=amd64 /bin/sh -c apt-mark hold ${NV_LIBCUBLAS_DEV_PACKAGE_NAME} ${NV_LIBNCCL_DEV_PACKAGE_NAME} # buildkit
RUN ENV LIBRARY_PATH=/usr/local/cuda/lib64/stubs
RUN ENV NV_CUDNN_VERSION=8.4.0.27
RUN ENV NV_CUDNN_PACKAGE_NAME=libcudnn8
RUN ENV NV_CUDNN_PACKAGE=libcudnn8=8.4.0.27-1+cuda11.6
RUN ENV NV_CUDNN_PACKAGE_DEV=libcudnn8-dev=8.4.0.27-1+cuda11.6
RUN ARG TARGETARCH
RUN LABEL maintainer=NVIDIA CORPORATION <[email protected]>
RUN LABEL com.nvidia.cudnn.version=8.4.0.27
RUN RUN |1 TARGETARCH=amd64 /bin/sh -c apt-get update \
    && apt-get install -y --no-install-recommends     ${NV_CUDNN_PACKAGE}     ${NV_CUDNN_PACKAGE_DEV}     \
    && apt-mark hold ${NV_CUDNN_PACKAGE_NAME}     \
    && rm -rf /var/lib/apt/lists/* # buildkit
RUN ENV DEBIAN_FRONTEND=noninteractive
RUN ENV PYTHONUNBUFFERED=1
RUN ENV LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/lib/x86_64-linux-gnu:/usr/local/nvidia/lib64:/usr/local/nvidia/bin
RUN RUN /bin/sh -c rm -f /etc/apt/sources.list.d/cuda.list \
    &&     rm -f /etc/apt/sources.list.d/nvidia-ml.list \
    &&     apt-key del 7fa2af80 # buildkit
RUN ENV PATH=/root/.pyenv/shims:/root/.pyenv/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
RUN RUN /bin/sh -c apt-get update -qq \
    && apt-get install -qqy --no-install-recommends     make    build-essential         libssl-dev      zlib1g-dev      libbz2-dev    libreadline-dev  libsqlite3-dev  wget    curl    llvm    libncurses5-dev         libncursesw5-dev        xz-utils        tk-dev  libffi-dev     liblzma-dev     python-openssl  git     ca-certificates         \
    && rm -rf /var/lib/apt/lists/* # buildkit
RUN RUN /bin/sh -c curl -s -S -L https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer | bash \
    &&  git clone https://github.com/momo-lab/pyenv-install-latest.git "$(pyenv root)"/plugins/pyenv-install-latest \
    &&  pyenv install-latest "3.10" \
    &&  pyenv global $(pyenv install-latest --print "3.10") \
    &&  pip install "wheel<1" # buildkit
RUN COPY .cog/tmp/build1921649421/cog-0.0.1.dev-py3-none-any.whl /tmp/cog-0.0.1.dev-py3-none-any.whl # buildkit
RUN RUN /bin/sh -c pip install /tmp/cog-0.0.1.dev-py3-none-any.whl # buildkit
RUN RUN /bin/sh -c pip install   diffusers==0.7.2 torch==1.12.1 --extra-index-url=https://download.pytorch.org/whl/cu116 ftfy==6.1.1 scipy==1.9.0 transformers==4.21.1 accelerate==0.14.0 # buildkit
RUN WORKDIR /src
RUN EXPOSE map[5000/tcp:{}]
RUN CMD ["python" "-m" "cog.server.http"]
RUN COPY . /src # buildkit

so I understand I can update the predict.py to use the model I need.
but then I need to run
cog build

maybe that will get me out of jail.

@johndpope
Copy link
Author

johndpope commented Dec 6, 2022 •
edited
Loading

sudo docker run cog-botatobot

leads me here ->

INFO:     Started server process [1]
INFO:     Waiting for application startup.
Loading pipeline...
ERROR:    Traceback (most recent call last):
  File "/root/.pyenv/versions/3.10.8/lib/python3.10/site-packages/diffusers/configuration_utils.py", line 223, in get_config_dict
    config_file = hf_hub_download(
  File "/root/.pyenv/versions/3.10.8/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 124, in _inner_fn
    return fn(*args, **kwargs)
  File "/root/.pyenv/versions/3.10.8/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1150, in hf_hub_download
    with open(ref_path) as f:
FileNotFoundError: [Errno 2] No such file or directory: 'diffusers-cache/models--nitrosocke--redshift-diffusion/refs/main'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/.pyenv/versions/3.10.8/lib/python3.10/site-packages/starlette/routing.py", line 671, in lifespan
    async with self.lifespan_context(app):
  File "/root/.pyenv/versions/3.10.8/lib/python3.10/site-packages/starlette/routing.py", line 566, in __aenter__
    await self._router.startup()
  File "/root/.pyenv/versions/3.10.8/lib/python3.10/site-packages/starlette/routing.py", line 650, in startup
    handler()
  File "/root/.pyenv/versions/3.10.8/lib/python3.10/site-packages/cog/server/http.py", line 40, in startup
    predictor.setup()
  File "/src/predict.py", line 25, in setup
    self.txt2img_pipe = StableDiffusionPipeline.from_pretrained(
  File "/root/.pyenv/versions/3.10.8/lib/python3.10/site-packages/diffusers/pipeline_utils.py", line 366, in from_pretrained
    config_dict = cls.get_config_dict(
  File "/root/.pyenv/versions/3.10.8/lib/python3.10/site-packages/diffusers/configuration_utils.py", line 268, in get_config_dict
    raise EnvironmentError(
OSError: Can't load config for 'nitrosocke/redshift-diffusion'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'nitrosocke/redshift-diffusion' is the correct path to a directory containing a model_index.json file

ERROR:    Application startup failed. Exiting.

https://huggingface.co/nitrosocke/redshift-diffusion

predict.py

 self.txt2img_pipe = StableDiffusionPipeline.from_pretrained(
        "nitrosocke/redshift-diffusion",
        cache_dir=MODEL_CACHE,
        local_files_only=True,
    ).to("cuda")

@deepxmatter
Copy link

did you ever figure this out, seems fucking terrible...like why would this be so hard?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@johndpope @deepxmatter