not marked as vision in LM Studio
For whatever reason this isn't marked as vision capable in LM studio.
This one you did is though:
https://model.lmstudio.ai/download/mradermacher/Qwen2.5-VL-7B-Abliterated-Caption-it-GGUF
This repository is indeed missing the mmproj files containing the vision stack. Thanks a lot for letting us know. They will be generated shortly and if everything goes well should appear within the next hour after which the vision capabilities of this model can be used.
The mmproj files of this model are now available. This model should now be correctly detected as vision model.
- Vision stack in Q8: https://huggingface.co/mradermacher/Qwen2.5-VL-7B-Instruct-abliterated-GGUF/blob/main/Qwen2.5-VL-7B-Instruct-abliterated.mmproj-Q8_0.gguf
- Vision stack in F16: https://huggingface.co/mradermacher/Qwen2.5-VL-7B-Instruct-abliterated-GGUF/blob/main/Qwen2.5-VL-7B-Instruct-abliterated.mmproj-f16.gguf
@nicoboss Could you please also provide mmproj's for the imatrix variant of the model? Thanks!
@Iniquitatis The importance matrix is only used to quantize the LLM stack but not the vision stack. llama.cpp doesn't support an imatrix for MMPROJ extraction which makes perfect sense given that the smallest officially supported MMPROJ quant is Q8_0 which even for the LLM stack lacks imatrix support. This is also the reason why we only upload the MMPROJ vision stack inside the static repository. Please use the MMPROJ files from the static repository even if you decide to use the much better imatrix quants for the LLM stack. They are identical to the ones we would upload to this repo but decided not to do as doing so would be a waste of everyone’s resources.
well, thanks to xet, it would just be disk I/O overhead on our side to upload it twice. it's conceptually very ugly, though.