vllm serve的部分问题
400: {"object":"error","message":"As of transformers v4.44, default chat template is no longer allowed, so you must provide a chat template if the tokenizer does not define one. None","type":"BadRequestError","param":null,"code":400}
在vllm serve后，使用沉浸式翻译调用该接口，返回这个。请问是否可以提供一个可以给沉浸式翻译使用的jinja模板呢？
Same issue, and it appears that ByteDance's relevant developers aren't particularly engaged with Hugging Face, as their responses are notably absent from many discussions.
@pty819 Thanks for your feedback. Could you please provide us with your configuration environment and code? This will help us reproduce the problem and identify a solution.
Thank you for your response. I first pulled the Hugging Face files to /data/Seed-X-PPO-7B, and then deployed using the latest vllm-openai image with Docker. The deployment command is as follows:
docker run --runtime nvidia --gpus=all --restart always --name SeedXPPO7B -d -v /data/Seed-X-PPO-7B:/mnt/Seed-X-PPO-7B -e PUID=1000 -e PGID=1000 -e VLLM_API_KEY="xxxxxxx" -e VLLM_USE_MODELSCOPE=false -p 8610:8000 --ipc=host vllm/vllm-openai:latest --served-model-name SeedXPPO7B --model /mnt/Seed-X-PPO-7B --gpu_memory_utilization 0.7 --max-model-len 8192
@pteromyini You can try to upgrade the Transformers version to 4.51.3 to avoid providing a chat template.
@pteromyini You can try to upgrade the Transformers version to 4.51.3 to avoid providing a chat template.
upgrade the Transformers version to 4.51.3 doesn't work for me,
got "ERROR 08-04 04:34:01 [serving_chat.py:222] File "/home/ubuntu/anaconda3/envs/vllm/lib/python3.12/site-packages/vllm/entrypoints/chat_utils.py", line 1212, in apply_hf_chat_template
ERROR 08-04 04:34:01 [serving_chat.py:222] raise ValueError(
ERROR 08-04 04:34:01 [serving_chat.py:222] ValueError: As of transformers v4.44, default chat template is no longer allowed, so you must provide a chat template if the tokenizer does not define one.
INFO: 192.168.100.231:54762 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request"