Add handler.py to resolve “no handler.py file found” deployment error
#2
by
colesmcintosh
- opened
Title: Add handler.py to resolve “no handler.py file found” deployment error
Why
Deploying allenai/olmOCR‑7B‑0725 on Hugging Face Inference Endpoints failed because the repository lacked a handler.py. A handler is required so the service knows how to run inference.
What changed
Added
handler.py- Loads the model and processor from
allenai/olmOCR‑7B‑0725 - Accepts an image under the
inputskey - Reads
max_new_tokensfromdata["parameters"]and defaults to256if not supplied - Returns the OCR text in
generated_text
- Loads the model and processor from
Hey, thank you for the PR. I'm not too familiar with the handler.py aspect of Hugging Face inference.
The one you suggest is a good start, but not 100% right, since you'd have to render an image to a very certain size and provide our default prompt to the model, as in:
https://github.com/allenai/olmocr/blob/c1061146e52407228b30283e811a2efc3571b9d0/olmocr/pipeline.py#L107