U 2A=d @s6ddlmZddlmZddlmZGdddZdS))SegmentAnything)SemanticSegment) EditAnythingc@s:eZdZdddZddZddd Zdd d Zd d ZdS)RegionSemantic edit_anythingvit_bcCs$||_||_||_||_|dS)N)devicesam_archimage_caption_modelregion_classify_model init_models)selfrr r r rV/mnt/home/wangjp/Code/Image2Text_HuggingFace/Image2Paragraph/models/region_semantic.py__init__s zRegionSemantic.__init__cCsVt|j|jd|_|jdkr*t|j|_n(|jdkrJt|j|_ t dnt ddS)N)archssarzinitalize edit anything model5semantic_class_model must be 'ssa' or 'edit_anything') rrr segment_modelr rsemantic_segment_modelrr edit_anything_modelprint ValueError)r rrrr s    zRegionSemantic.init_modelscCsnt|dddd}t|}|dt||}d}|D]$}||ddt|d d 7}q4t|td |S) a fliter too small objects and objects with low stability score anns: [{'class_name': 'person', 'bbox': [0.0, 0.0, 0.0, 0.0], 'size': [0, 0], 'stability_score': 0.0}, ...] semantic_prompt: "person: [0.0, 0.0, 0.0, 0.0]; ..." cSs|dS)Narear)xrrrz4RegionSemantic.semantic_prompt_gen..T)keyreverseN class_namez: bboxz; o****************************************************************************************************)sortedlenminstrr)r annstopkZsorted_annotationsZanns_lenZtop_10_largest_regionsZsemantic_promptregionrrrsemantic_prompt_gens"z"RegionSemantic.semantic_prompt_gencCstdtdtd|j|}td|dkrTtd|j||}tdn0|dkr|td|j||}tdntd ||S) Nr#z Step3, Semantic Prompt:z/extract region segmentation with SAM model.... z finished... rz1generate region supervision with blip2 model.... rz9generate region supervision with edit anything model.... r)rrZ generate_maskrZsemantic_class_w_maskrrr+)r img_srcr r(Z anns_w_classrrrregion_semantic*s   zRegionSemantic.region_semanticcCsdS)Nregion_semantic_debugr)r r,rrrr.<sz$RegionSemantic.region_semantic_debugN)rr)r)r)__name__ __module__ __qualname__rr r+r-r.rrrrrs    rN)Z,models.segment_models.semgent_anything_modelrZ5models.segment_models.semantic_segment_anything_modelrZ)models.segment_models.edit_anything_modelrrrrrrs