Description
Is there an existing issue for this problem?
- I have searched the existing issues
Operating system
Windows
GPU vendor
Nvidia (CUDA)
GPU model
RTX 4050
GPU VRAM
6gb
Version number
5.12.0
Browser
community launcher
Python dependencies
No response
What happened
my invokeai.yaml contains these args:
enable_partial_loading: true
keep_ram_copy_of_weights: false
clear_queue_on_startup: true
lazy_offload: true
pytorch_cuda_alloc_conf: "backend:cudaMallocAsync"
using object detection now gives me OOM error, I don't remember since which invokeai version exactly this started to happen (I guess around last February/March not sure) , but I remember it was running normally on an older version since the transition to the launcher app.
no matter the input and output size of the image, it was running normally back then and now it doesn't.
I tried some basic workarounds in the original config file, changed torch_dtype = float32 and changed it to float16, changed use_bfloat16 = false into true, both gave me errors either when changing both values together or changing one of them at a time.
object detection downloads sam-vit-huge model and its config files into "\models.download_cache" folder, I really don't think this method is right since the app doesn't detect anything inside that folder in manual search mode, downloaded models in this folder can't be copied into a new machine for example and use them with a new installation of the invokeai app.
also, it's very limiting regarding the choice of specific models, in this case object detection models.
I deleted sam-vit-huge, downloaded sam-vit-large and its config, and put it inside sam-vit-huge folder and object detection runs without an issue, except it's less precise than vit-huge obviously.
What you expected to happen
- .download_cache models could be handeld the same way as .temp models; downloaded then sorted to proper folders.
-fix the newly-introduced OOM error when using sam-vit-huge segmentation model.
-maybe give some liberty to use different implementation of sam-vit, or better: segmentation models in general.
How to reproduce the problem
No response
Additional context
No response
Discord username
No response