Mgv ddlmZddlmZddlmZddlmZddlm Z  dde eeffdZ dS)N)FrozenCLIPEmbedder)Switti)VQVAE)SwittiPipeline r  r rT'F?h㈵>{Gz?ư>openai/clip-vit-large-patch14(laion/CLIP-ViT-bigG-14-laion2B-39B-b160kreturncZ|}|dz}|dkr||zdz }tjtjtjtjtjtjtjtjfD]}t|ddt|||d|| |}t|||||||| |||| | | |  |}| |||| t|}t|}t|||||}|||fS) N@rreset_parameterscdS)N)selfs 2/home/notantonvoron/switti_demo/models/__init__.pyzbuild_models..7sdT) vocab_size z_channelsch test_modeshare_quant_resi v_patch_nums)depth embed_dim num_heads drop_rateattn_drop_ratedrop_path_ratenorm_eps attn_l2_norm patch_numsrope rope_theta rope_sizeuse_swiglu_ffnuse_ar use_crop_cond) init_adalninit_adaln_gamma init_headinit_std)nnLinear LayerNorm BatchNorm2d SyncBatchNormConv1dConv2dConvTranspose1dConvTranspose2dsetattrrtor init_weightsrr) devicer8VCvaer,r.r0r9r:r;r<r=r>r7r?r@rArBr3r4dprr6text_encoder_pathtext_encoder_2_pathheadswidthclz vae_local switti_wo_ddp text_encodertext_encoder_2pipes r' build_modelsr] s}< E BJE QwEkB          < < '):):;;;; )  bjj%! %#  bjj!$)  &&788L'(;<rds$$$$$$$$$$$$ 1      5B9V*V*: 5&=;V*V*V*V*V*V*r)