RuntimeError: Input type (torch.FloatTensor) and weight type (XPUFloatType) should be the same or input should be a MKLDNN tensor and weight is a dense tensor

[AndrewRainfall]

Your question

Error happens when I try to use Depth_anything_v2 preprocessor on GPU Intel Arc A770.
Open_pose and Canny work fine.

ChatGPT advice this, but I don't understand how to implement it:

import torch

= Set device to XPU (Intel Arc A770)
device = torch.device("xpu")

= Move model to XPU (after loading the model)
model = model.to(device)

= Move input tensor to XPU
input_tensor = input_tensor.to(device)

= Perform the inference
output = model(input_tensor)

Logs

!!! Exception during processing !!! Input type (torch.FloatTensor) and weight type (XPUFloatType) should be the same or input should be a MKLDNN tensor and weight is a dense tensor
Traceback (most recent call last):
  File "C:\ComfyUI\execution.py", line 327, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
  File "C:\ComfyUI\execution.py", line 202, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
  File "C:\ComfyUI\execution.py", line 174, in _map_node_over_list
    process_inputs(input_dict, i)
  File "C:\ComfyUI\execution.py", line 163, in process_inputs
    results.append(getattr(obj, func)(**inputs))
  File "C:\ComfyUI\custom_nodes\comfyui-art-venture\modules\controlnet\__init__.py", line 104, in detect_controlnet
    image = apply_preprocessor(image, preprocessor, resolution=resolution)
  File "C:\ComfyUI\custom_nodes\comfyui-art-venture\modules\controlnet\preprocessor.py", line 99, in apply_preprocessor
    res = getattr(aio_preprocessor, function_name)(**args)
  File "C:\ComfyUI\custom_nodes\comfyui_controlnet_aux\__init__.py", line 120, in execute
    return getattr(aux_class(), aux_class.FUNCTION)(**params)
  File "C:\ComfyUI\custom_nodes\comfyui_controlnet_aux\node_wrappers\depth_anything_v2.py", line 24, in execute
    out = common_annotator_call(model, image, resolution=resolution, max_depth=1)
  File "C:\ComfyUI\custom_nodes\comfyui_controlnet_aux\utils.py", line 85, in common_annotator_call
    np_result = model(np_image, output_type="np", detect_resolution=detect_resolution, **kwargs)
  File "C:\ComfyUI\custom_nodes\comfyui_controlnet_aux\src\custom_controlnet_aux\depth_anything_v2\__init__.py", line 44, in __call__
    depth = self.model.infer_image(cv2.cvtColor(input_image, cv2.COLOR_RGB2BGR), input_size=518, max_depth=max_depth)
  File "C:\Users\Rain\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\utils\_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "C:\ComfyUI\custom_nodes\comfyui_controlnet_aux\src\custom_controlnet_aux\depth_anything_v2\dpt.py", line 189, in infer_image
    depth = self.forward(image, max_depth)
  File "C:\ComfyUI\custom_nodes\comfyui_controlnet_aux\src\custom_controlnet_aux\depth_anything_v2\dpt.py", line 179, in forward
    features = self.pretrained.get_intermediate_layers(x, self.intermediate_layer_idx[self.encoder], return_class_token=True)
  File "C:\ComfyUI\custom_nodes\comfyui_controlnet_aux\src\custom_controlnet_aux\depth_anything_v2\dinov2.py", line 308, in get_intermediate_layers
    outputs = self._get_intermediate_layers_not_chunked(x, n)
  File "C:\ComfyUI\custom_nodes\comfyui_controlnet_aux\src\custom_controlnet_aux\depth_anything_v2\dinov2.py", line 272, in _get_intermediate_layers_not_chunked
    x = self.prepare_tokens_with_masks(x)
  File "C:\ComfyUI\custom_nodes\comfyui_controlnet_aux\src\custom_controlnet_aux\depth_anything_v2\dinov2.py", line 214, in prepare_tokens_with_masks
    x = self.patch_embed(x)
  File "C:\Users\Rain\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1751, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\Rain\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1762, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\ComfyUI\custom_nodes\comfyui_controlnet_aux\src\custom_controlnet_aux\depth_anything_v2\dinov2_layers\patch_embed.py", line 76, in forward
    x = self.proj(x)  # B C H W
  File "C:\Users\Rain\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1751, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\Rain\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1762, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\Rain\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\conv.py", line 554, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "C:\Users\Rain\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\conv.py", line 549, in _conv_forward
    return F.conv2d(
RuntimeError: Input type (torch.FloatTensor) and weight type (XPUFloatType) should be the same or input should be a MKLDNN tensor and weight is a dense tensor

Other

No response

[AndrewRainfall]

Depth Leres works, so I guess I'll just use it instead

[simonlui]

You shouldn't report a custom node's bug inside ComfyUI, you should open a bug in the custom node repository. That being said, this is more or less a result of XPU being less mature than CUDA. There are a bunch of custom nodes that don't exactly sit right or work well with Intel's Extension for Pytorch or Pytorch nightly, the most popular of these custom nodes not working I know of is some IPAdapter custom nodes for example. It will take some time before it is fine, and even then, don't be surprised that some nodes won't be hardcoded to run on CUDA only.