a \8d@s>ddlmZddlZddlmZmZddlZGdddZdS))ImageN)Blip2ProcessorBlip2ForConditionalGenerationc@s2eZdZddddZddZddZd d ZdS) ImageCaptioningN)returncCs|\|_|_dS)N)initialize_model processormodel)selfr 6/mnt/home/wangjp/Code/Image2Text/models/blip2_model.py__init__szImageCaptioning.__init__cCsJd}|dkrtj|_ntj|_td}tjd|jd}||||fS)NcpuzSalesforce/blip2-opt-2.7b)Z torch_dtype)torchZfloat32 data_typeZfloat16rZfrom_pretrainedrto)r devicerr r r r r s   z ImageCaptioning.initialize_modelcCslt|}d}|j|dd||j}|jjfi|}|jj|ddd}t dt |t d|S) NrZpt)ZimagesZreturn_tensorsT)Zskip_special_tokensrzz**************************************************************************************************** Step1, BLIP2 caption:ze ****************************************************************************************************) ropenrrrr ZgenerateZ batch_decodestripprint)r image_srcimagerZinputsZ generated_idsgenerated_textr r r image_captions zImageCaptioning.image_captioncCsdS)Nz3A dish with salmon, broccoli, and something yellow.r )r rr r r image_caption_debug&sz#ImageCaptioning.image_caption_debug)__name__ __module__ __qualname__r rrrr r r r rs r)PILrZrequestsZ transformersrrrrr r r r s