[Issue]: Cannot set version_counter for inference tensor with face restore on directml

[zass30]

Issue Description

Tried to generate a SDXL image with face restore checked. When doing so, I get the error:

RuntimeError: Cannot set version_counter for inference tensor

I have tried this both on main branch and dev branch

Version Platform Description

No response

Relevant log output

E:\stable_diffusion_sdnext_dev\automatic>webui-user.bat
Using VENV: E:\stable_diffusion_sdnext\automatic\venv
17:07:22-683015 INFO     Starting SD.Next
17:07:22-685015 INFO     Logger: file="E:\stable_diffusion_sdnext_dev\automatic\sdnext.log" level=INFO size=40293
                         mode=append
17:07:22-686015 INFO     Python: version=3.10.6 platform=Windows
                         bin="E:\stable_diffusion_sdnext\automatic\venv\Scripts\python.exe"
                         venv="E:\stable_diffusion_sdnext\automatic\venv"
17:07:22-798753 INFO     Version: app=sd.next updated=2024-09-30 hash=684263c1 branch=dev
                         url=https://github.com/vladmandic/automatic//tree/dev ui=main
17:07:23-206670 INFO     Updating main repository
17:07:23-803329 INFO     Repository upgraded: 684263c1 Mon Sep 30 16:37:34 2024 -0400
17:07:23-808329 INFO     Platform: arch=AMD64 cpu=AMD64 Family 25 Model 97 Stepping 1, AuthenticAMD system=Windows
                         release=Windows-10-10.0.22621-SP0 python=3.10.6
17:07:23-812330 INFO     DirectML: selected
17:07:23-884840 INFO     Verifying requirements
17:07:23-887840 INFO     Verifying packages
17:07:23-888842 INFO     Startup: standard
17:07:23-888842 INFO     Verifying submodules
17:07:36-628473 INFO     Extensions enabled: ['Lora', 'sd-extension-chainner', 'sd-extension-system-info',
                         'sd-webui-agent-scheduler', 'sdnext-modernui', 'stable-diffusion-webui-rembg']
17:07:36-629473 INFO     Verifying requirements
17:07:36-629473 INFO     Updating Wiki
17:07:37-099382 INFO     Command line args: ['--upgrade', '--use-directml', '--lowvram'] lowvram=True upgrade=True
                         use_directml=True
17:07:40-299946 INFO     Load packages: {'torch': '2.0.0+cpu', 'diffusers': '0.31.0.dev0', 'gradio': '3.43.2',
                         'transformers': '4.44.2', 'accelerate': '0.34.2'}
17:07:40-605027 INFO     Engine: backend=Backend.DIFFUSERS compute=None device=cpu attention="Scaled-Dot-Product"
                         mode=no_grad
17:07:41-158093 INFO     Device: device=AMD Radeon RX 7900 XTX n=2 directml=0.2.0.dev230426
17:07:41-242521 INFO     Available VAEs: path="models\VAE" items=0
17:07:41-244518 INFO     Available UNets: path="models\UNET" items=0
17:07:41-244518 INFO     Available TEs: path="models\Text-encoder" items=0
17:07:41-245517 INFO     Disabled extensions: ['sdnext-modernui']
17:07:41-255523 INFO     Available Models: path="models\Stable-diffusion" items=2 time=0.01
17:07:41-344356 INFO     Extension: script='extensions-builtin\Lora\scripts\lora_script.py'
                         [2;36m17:07:41-342354[0m[2;36m [0m[34mINFO    [0m Available LoRAs: [33mitems[0m=[1;36m0[0m
                         [33mfolders[0m=[1;36m2[0m
17:07:41-580715 INFO     Extension: script='extensions-builtin\sd-webui-agent-scheduler\scripts\task_scheduler.py' Using
                         sqlite file: extensions-builtin\sd-webui-agent-scheduler\task_scheduler.sqlite3
17:07:41-593715 INFO     Available Upscalers: items=53 downloaded=0 user=0 time=0.01 types=['None', 'Lanczos',
                         'Nearest', 'ChaiNNer', 'AuraSR', 'ESRGAN', 'LDSR', 'RealESRGAN', 'SCUNet', 'SD', 'SwinIR']
17:07:41-602716 INFO     Available Styles: folder="models\styles" items=288 time=0.01
17:07:41-605716 INFO     UI theme: type=Standard name="black-teal"
17:07:42-437741 INFO     Extension list is empty: refresh required
17:07:43-075162 INFO     Local URL: http://127.0.0.1:7860/
17:07:43-185673 INFO     [AgentScheduler] Task queue is empty
17:07:43-186671 INFO     [AgentScheduler] Registering APIs
17:07:43-261673 INFO     Model metadata saved: file="metadata.json" items=2 time=0.01
17:07:43-769671 INFO     Torch parameters: backend=directml device=privateuseone:0 config=Auto dtype=torch.float16
                         vae=torch.float16 unet=torch.float16 context=no_grad nohalf=False nohalfvae=False upscast=False
                         deterministic=False test-fp16=True test-bf16=False optimization="Scaled-Dot-Product"
17:07:43-770671 INFO     Load model: select="animagineXLV31_v31 [e3c47aedb0]"
17:07:43-974239 INFO     Autodetect model: detect="Stable Diffusion XL" class=StableDiffusionXLPipeline
                         file="E:\stable_diffusion_sdnext_dev\automatic\models\Stable-diffusion\animagineXLV31_v31.safet
                         ensors" size=6617MB
Diffusers 12.96it/s ████████ 100% 7/7 00:00 00:00 Loading pipeline components...
17:07:44-554842 INFO     Load network: type=embeddings loaded=0 skipped=0 time=0.00
17:07:44-903878 INFO     Load model: time=0.70 load=0.58 move=0.10 native=1024 memory={'ram': {'used': 1.28, 'total':
                         63.12}, 'gpu': {'used': 0.02, 'total': 0.02}, 'retries': 0, 'oom': 0}
17:07:45-120896 INFO     Startup time: 8.02 torch=2.08 gradio=0.59 diffusers=0.15 libraries=1.23 extensions=0.29
                         ui-networks=0.11 ui-txt2img=0.05 ui-img2img=0.05 ui-control=0.26 ui-models=0.13
                         ui-settings=0.17 ui-extensions=0.31 launch=0.30 api=0.06 app-started=0.13 checkpoint=1.86
17:08:06-202480 INFO     Base: class=StableDiffusionXLPipeline
Progress  3.94s/it █████████████████████████████████ 100% 20/20 01:18 00:00 Base
17:09:28-850245 INFO     Load: type=FaceHires model=E:\stable_diffusion_sdnext_dev\automatic\models\yolo\yolov8n-face.pt
17:09:29-323634 ERROR    Exception: Cannot set version_counter for inference tensor
17:09:29-325140 ERROR    Arguments: args=('task(k5662uv0i6hptn9)', '', 'photo (medium), selfie, 8k, high quality , a man
                         and a woman sitting by a lake', 'text, watermark, word', [], 20, 14, 29, True, True, False,
                         False, 1, 1, 6, 6, 0.7, 0, 0.5, 1, 1, -1.0, -1.0, 0, 0, 0, 1248, 832, False, 0.3, 1, 1, 'Add
                         with forward', 'None', False, 20, 0, 0, 10, 0, '', '', 0, 0, 0, 0, False, 4, 0.95, False, 0.6,
                         1, '#000000', 0, [], 0, 1, 'None', 'None', 'None', 'None', 0.5, 0.5, 0.5, 0.5, None, None,
                         None, None, False, False, False, False, 0, 0, 0, 0, 1, 1, 1, 1, None, None, None, None, False,
                         '', False, 0, '', [], 0, '', [], 0, '', [], False, True, False, True, False, False, 0, 'None',
                         [], 'FaceID Base', True, True, 1, 1, 1, 0.5, True, 'person', 1, 0.5, True, 'None', 16, 'None',
                         1, True, 'None', 2, True, 1, 0, True, 'none', 3, 4, 0.25, 0.25, 'THUDM/CogVideoX-2b', 'DDIM',
                         49, 6, 'balanced', True, 'None', 8, True, 1, 0, None, None, '', 0.5, 5, None, '', 0.5, 5, None,
                         3, 1, 1, 0.8, 8, 64, True, 0.65, True, False, 1, 1, 1, True, 0.5, 600.0, 1.0, True, None, 1, 0,
                         0, 0, 0, 0, 0, 0, 1, 1, 1, 0.5, 0.5, 'OpenGVLab/InternVL-14B-224px', False, 0.7, 1.2, 128,
                         False, False, 'positive', 'comma', 0, False, False, '', 'None', '', 1, '', 'None', 1, True, 10,
                         'None', True, 0, 'None', 2, True, 1, 0, 0, '', [], 0, '', [], 0, '', [], False, True, False,
                         False, False, False, 0) kwargs={}
17:09:29-329140 ERROR    gradio call: RuntimeError
╭───────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────╮
│ E:\stable_diffusion_sdnext_dev\automatic\modules\call_queue.py:31 in f                                               │
│                                                                                                                      │
│   30 │   │   │   try:                                                                                                │
│ ❱ 31 │   │   │   │   res = func(*args, **kwargs)                                                                     │
│   32 │   │   │   │   progress.record_results(id_task, res)                                                           │
│                                                                                                                      │
│ E:\stable_diffusion_sdnext_dev\automatic\modules\txt2img.py:94 in txt2img                                            │
│                                                                                                                      │
│    93 │   if processed is None:                                                                                      │
│ ❱  94 │   │   processed = processing.process_images(p)                                                               │
│    95 │   processed = scripts.scripts_txt2img.after(p, processed, *args)                                             │
│                                                                                                                      │
│ E:\stable_diffusion_sdnext_dev\automatic\modules\processing.py:201 in process_images                                 │
│                                                                                                                      │
│   200 │   │   │   with context_hypertile_vae(p), context_hypertile_unet(p):                                          │
│ ❱ 201 │   │   │   │   processed = process_images_inner(p)                                                            │
│   202                                                                                                                │
│                                                                                                                      │
│ E:\stable_diffusion_sdnext_dev\automatic\modules\processing.py:363 in process_images_inner                           │
│                                                                                                                      │
│   362 │   │   │   │   │   p.ops.append('face')                                                                       │
│ ❱ 363 │   │   │   │   │   sample = face_restoration.restore_faces(sample, p)                                         │
│   364 │   │   │   │   │   if sample is not None:                                                                     │
│                                                                                                                      │
│ E:\stable_diffusion_sdnext_dev\automatic\modules\face_restoration.py:17 in restore_faces                             │
│                                                                                                                      │
│   16 │   face_restorer = face_restorers[0]                                                                           │
│ ❱ 17 │   return face_restorer.restore(np_image, p)                                                                   │
│   18                                                                                                                 │
│                                                                                                                      │
│ E:\stable_diffusion_sdnext_dev\automatic\scripts\face_details.py:112 in restore                                      │
│                                                                                                                      │
│   111 │   │   image = Image.fromarray(np_image)                                                                      │
│ ❱ 112 │   │   faces = self.predict(image)                                                                            │
│   113 │   │   if len(faces) == 0:                                                                                    │
│                                                                                                                      │
│ E:\stable_diffusion_sdnext_dev\automatic\scripts\face_details.py:57 in predict                                       │
│                                                                                                                      │
│    56 │   │   self.model.to(device)                                                                                  │
│ ❱  57 │   │   predictions = self.model.predict(                                                                      │
│    58 │   │   │   source=[image],                                                                                    │
│                                                                                                                      │
│ E:\stable_diffusion_sdnext\automatic\venv\lib\site-packages\ultralytics\engine\model.py:555 in predict               │
│                                                                                                                      │
│    554 │   │   │   self.predictor.set_prompts(prompts)                                                               │
│ ❱  555 │   │   return self.predictor.predict_cli(source=source) if is_cli else self.predictor(source=source, stream= │
│    556                                                                                                               │
│                                                                                                                      │
│                                               ... 8 frames hidden ...                                                │
│                                                                                                                      │
│ E:\stable_diffusion_sdnext\automatic\venv\lib\site-packages\ultralytics\nn\tasks.py:147 in _predict_once             │
│                                                                                                                      │
│    146 │   │   │   │   self._profile_one_layer(m, x, dt)                                                             │
│ ❱  147 │   │   │   x = m(x)  # run                                                                                   │
│    148 │   │   │   y.append(x if m.i in self.save else None)  # save output                                          │
│                                                                                                                      │
│ E:\stable_diffusion_sdnext\automatic\venv\lib\site-packages\torch\nn\modules\module.py:1501 in _call_impl            │
│                                                                                                                      │
│   1500 │   │   │   │   or _global_forward_hooks or _global_forward_pre_hooks):                                       │
│ ❱ 1501 │   │   │   return forward_call(*args, **kwargs)                                                              │
│   1502 │   │   # Do not call functions when jit is used                                                              │
│                                                                                                                      │
│ E:\stable_diffusion_sdnext\automatic\venv\lib\site-packages\ultralytics\nn\modules\conv.py:54 in forward_fuse        │
│                                                                                                                      │
│    53 │   │   """Perform transposed convolution of 2D data."""                                                       │
│ ❱  54 │   │   return self.act(self.conv(x))                                                                          │
│    55                                                                                                                │
│                                                                                                                      │
│ E:\stable_diffusion_sdnext\automatic\venv\lib\site-packages\torch\nn\modules\module.py:1501 in _call_impl            │
│                                                                                                                      │
│   1500 │   │   │   │   or _global_forward_hooks or _global_forward_pre_hooks):                                       │
│ ❱ 1501 │   │   │   return forward_call(*args, **kwargs)                                                              │
│   1502 │   │   # Do not call functions when jit is used                                                              │
│                                                                                                                      │
│ E:\stable_diffusion_sdnext\automatic\venv\lib\site-packages\torch\nn\modules\conv.py:463 in forward                  │
│                                                                                                                      │
│    462 │   def forward(self, input: Tensor) -> Tensor:                                                               │
│ ❱  463 │   │   return self._conv_forward(input, self.weight, self.bias)                                              │
│    464                                                                                                               │
│                                                                                                                      │
│ E:\stable_diffusion_sdnext\automatic\venv\lib\site-packages\torch\nn\modules\conv.py:459 in _conv_forward            │
│                                                                                                                      │
│    458 │   │   │   │   │   │   │   _pair(0), self.dilation, self.groups)                                             │
│ ❱  459 │   │   return F.conv2d(input, weight, bias, self.stride,                                                     │
│    460 │   │   │   │   │   │   self.padding, self.dilation, self.groups)                                             │
│                                                                                                                      │
│ E:\stable_diffusion_sdnext_dev\automatic\modules\dml\amp\autocast_mode.py:43 in <lambda>                             │
│                                                                                                                      │
│   42 │   │   op = getattr(resolved_obj, func_path[-1])                                                               │
│ ❱ 43 │   │   setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: forward(op, args, kwargs))                 │
│   44                                                                                                                 │
│                                                                                                                      │
│ E:\stable_diffusion_sdnext_dev\automatic\modules\dml\amp\autocast_mode.py:15 in forward                              │
│                                                                                                                      │
│   14 │   if not torch.dml.is_autocast_enabled:                                                                       │
│ ❱ 15 │   │   return op(*args, **kwargs)                                                                              │
│   16 │   args = list(map(cast, args))                                                                                │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: Cannot set version_counter for inference tensor

Backend

Diffusers

UI

Standard

Branch

Dev

Model

StableDiffusion XL

Acknowledgements

• I have read the above and searched for existing issues
• I confirm that this is classified correctly and its not an extension issue

[vladmandic]

this is happening in ultralytics package which is used to execute yolo face detection model
per ultralytics/ultralytics#13620 and ultralytics/ultralytics#907 directml is not supported with ultralytics. i'll see what can be done from sdnext side, but i'm affraid its pretty limited.

[zass30]

Thank you for looking into this. Older builds of SDNext had a different face fix button that did work under directml. Is it possible to bring this back for directml users?

[vladmandic]

what do you mean by "a different face fix button"?

[zass30]

I've always used directml, and in older builds I thought there was a button that fixed faces similar to the current face restore. And this button didn't have any issues on directml.

[vladmandic]

there was never any button. there was and is a checkbox to enable/disable and what it does did not change recently.

[brknsoul]

It's highly recommended you switch to ZLUDA over DirectML; better speeds, better memory management.
https://github.com/vladmandic/automatic/wiki/ZLUDA

[zass30]

It's highly recommended you switch to ZLUDA over DirectML; better speeds, better memory management. https://github.com/vladmandic/automatic/wiki/ZLUDA

I've done that, you're right! Thanks, it's so much better.

[vladmandic]

what's the status with face restore now?