o VfQ @sddlZddlZddlZddlmZmZmZmZmZm Z m Z m Z m Z m Z ddlZdejd<ddZeee e e e e edZejdd ZdS) N) BitsAndBytesConfig AutoConfigAutoModelForCausalLM!LlavaNextForConditionalGenerationLlavaForConditionalGeneration!PaliGemmaForConditionalGeneration Idefics2ForConditionalGenerationOwlv2ForObjectDetectionGroundingDinoForObjectDetectionSamModel1ZHF_HUB_ENABLE_HF_TRANSFERcCstjdddidddS)Nz+pip install flash-attn --no-build-isolationZFLASH_ATTENTION_SKIP_CUDA_BUILDTRUET)envshell) subprocessrunrr4/home/donbr/aie3-bootcamp/omniscience/model_utils.pyinstall_flash_attn s  r)rrrrr r r rc Csztj|dd}|jd}t|dd}|rHt|dd|dd|dtj|d d |d d|d d|d d|dd|ddd }nd}t|t }|j||dd}|rk|sk| t tj rgdnd}|rqt|nd}|ry|nd}d|d|dfWSty} z dd| fWYd} ~ Sd} ~ wty} z dd| fWYd} ~ Sd} ~ wty} z dt| fWYd} ~ Sd} ~ ww)NT)trust_remote_coderquantization_config load_in_4bitF load_in_8bitbnb_4bit_compute_dtypebnb_4bit_quant_typeZnf4bnb_4bit_use_double_quant llm_int8_enable_fp32_cpu_offloadllm_int8_has_fp16_weightllm_int8_skip_modulesllm_int8_thresholdg@) rrrrrrrrr)configrcudacpuzModel architecture not found.zConfiguration not found.z## Model Architecture z ## Configuration z ValueError: zEnvironmentError: )rZfrom_pretrainedZ architecturesgetattrrgettorchfloat16ARCHITECTURE_MAPrtodevicer! is_availablestrZto_json_string ValueErrorEnvironmentError Exception) model_namer architecturerZ bnb_configZ model_classmodel model_summaryZconfig_contentveeeerrrget_model_summarysH            r7)rosr& transformersrrrrrrrr r r spacesenvironrr(GPUr7rrrrs$0