AttributeError: 'HybridCache' object has no attribute 'sliding_window'
During the tests of E4B with transformers faced error:
AttributeError: 'HybridCache' object has no attribute 'sliding_window'
Passed messages = [
{
"role": "system",
"content": [{"type": "text", "text": "You are a helpful assistant."}]
},
{
"role": "user",
"content": [
{"type": "image", "image": },
{"type": "text", "text": ""}
]
}
]
(Replaced real data with placeholders)
Same error with transformers==4.54.0 and accelerate==1.9.0 using the following prompt messages = [ { "role": "system", "content": [{"type": "text", "text": "You are a helpful assistant."}] }, { "role": "user", "content": [ {"type": "text", "text": "Hello, where is New York?"} ] } ]
Disabling HybridCache with model.generate(**inputs, use_cache=False, max_new_tokens=512, do_sample=False) solves the error, but generates text non related with the prompt.
set "cache_implementation="static" in generate method
