B c@sddlZddlZddlZddlZddlmmZddl m Z m Z ddl m Z ddlmZddlmZddZd d Zd d Zd$eejd ddZd%eejd ddZddZddddgddgddddgddddfddZd&ddZd'd d!Zd(d"d#ZdS))N)forward_adapted_unflattenmake_backbone_default)gen_relative_position_index) checkpoint)OptionalcCs t||dS)Nforward_features)r) pretrainedxr Q/home/vaishanth/workspace/independent_study_project/MiDaS/midas/backbones/beit.py forward_beitsr cCs0||}|jr"|ddd}||}|S)zr Modification of timm.models.layers.patch_embed.py: PatchEmbed.forward to support arbitrary window sizes. r)projflatten transposenorm)selfr r r r patch_embed_forwards   rcCsfd|jdd}d|jdd}d|dd}d|dd}|j}|j}||d}|d|d} | d||ddddd} tj| ||fdd} | dddd|dd} t| ||ddg} t |dd t |d} | |j kr t ||j | <| |j |  d |d|dd|d|ddd} | ddd} | dS) zm Modification of timm.models.beit.py: Attention._get_rel_pos_bias to support arbitrary window sizes. rrrNbilinear)sizemode,) window_sizerelative_position_bias_tableZnum_relative_distancereshapepermuteF interpolatetorchcatstrrelative_position_indiceskeysrview contiguous unsqueeze)rrZ old_heightZ old_widthZ new_heightZ new_widthZ old_relative_position_bias_tableZold_num_relative_distanceZnew_num_relative_distanceZ old_sub_tableZ new_sub_tableZ new_relative_position_bias_tablekeyZrelative_position_biasr r r _get_rel_pos_biass, r*)shared_rel_pos_biascCs|j\}}}|jdk r,t|j|j|jfnd}tj||jj |d}| ||d|j d ddddd}| d\} } } | |j} | | d d} |jdk rtt|d } | || } |dk r| |} | jdd } || } | | dd ||d}||}||}|S) zc Modification of timm.models.beit.py: Attention.forward to support arbitrary window sizes. N)inputweightbiasrrrrr)dim)shapeZq_biasr!r"Zk_biasZv_biasrlinearqkvr-r num_headsrunbindscalerrtuplenparrayr*softmaxZ attn_droprZ proj_drop)rr resolutionr+BNCZqkv_biasr5qkvattnrr r r attention_forwardAs$ $"      rEc Cs|jdkrF|||j||||d}|||||}nF|||j|j||||d}|||j|||}|S)z_ Modification of timm.models.beit.py: Block.forward to support arbitrary window sizes. N)r+)Zgamma_1Z drop_pathrDnorm1Zmlpnorm2Zgamma_2)rr r=r+r r r block_forward^s   rHcCs|jdd}||}tj|j|jddd|fdd}|jdk rP||j}||}|jdk rl|nd}x<|j D]2}|j rtj st |||d}qx||||d}qxW||}|S)zg Modification of timm.models.beit.py: Beit.forward_features to support arbitrary window sizes. rNrrr)r2)r+)r3 patch_embedr!r"Z cls_tokenexpandZ pos_embedZpos_drop rel_pos_biasblocksZgrad_checkpointingjit is_scriptingrr)rr r=rKZblkr r r beit_forward_featuresls $     rO`iir/ ignorec Cst||||||||}tt|jj|jj_tt|j|j_xF|jj D]:} | j } tt | | _ tt | | _i| _ tt| | _qHW|S)N)rtypes MethodTypermodelrIforwardrOrrLrDr*rEr$rH) rWfeaturesrhooks vit_features use_readout start_indexZstart_index_readoutbackboneblockrDr r r _make_beit_backbones r`cCsJtjd|d}|dkr"ddddgn|}ddd d g}t||ddg|d |d S) NZbeit_large_patch16_512)r rSii)rYrrZr[r\)timm create_modelr`)r r\rZrWrYr r r _make_pretrained_beitl16_512s rgcCs@tjd|d}|dkr"ddddgn|}t|ddd d g|d |d S) NZbeit_large_patch16_384)r rarSrbrcrdii)rYrZr[r\)rerfr`)r r\rZrWr r r _make_pretrained_beitl16_384s rhcCs>tjd|d}|dkr"ddddgn|}t|ddd d g||d S) NZbeit_base_patch16_384)r rrarRrSrPrQii)rYrZr\)rerfr`)r r\rZrWr r r _make_pretrained_beitb16_384s ri)N)N)rTN)rTN)rTN)rer!rUnumpyr:Ztorch.nn.functionalnn functionalrutilsrrZtimm.models.beitrZtorch.utils.checkpointrtypingrr rr*TensorrErHrOr`rgrhrir r r r s0    $