čpe< dZddlZddlmZddlmZddlZddlmZddl mcm Z ddl m cmZddlmZddlmZmZddlmZdd lmZmZmZmZdd lmZd gZd)d Zeded ededededed ed edd Z Gddej!Z"Gddej#Z$Gddej%Z&Gdd ej!Z'dZ(dZ)ed*d Z*ed*d!Z+ed*d"Z,ed*d#Z-ed*d$Z.ed*d%Z/ed*d&Z0ed*d'Z1ed*d(Z2dS)+zPytorch Densenet implementation w/ tweaks This file is a copy of https://github.com/pytorch/vision 'densenet.py' (BSD-3-Clause) with fixed kwargs passthrough and addition of dynamic global avg/max pool. N) OrderedDict)partial)ListIMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STD)build_model_with_cfg)BatchNormAct2dcreate_norm_act BlurPool2dcreate_classifier)register_modelDenseNetc 0|dddddttddd S) N)r)rg?bicubiczfeatures.conv0 classifier) url num_classes input_size pool_sizecrop_pct interpolationmeanstd first_convrrrs G/home/terry/CeaseFire/code_minimal_pytorch_ITTI/timm/models/densenet.py_cfgr$s+4}SYI%.B&l   zlhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/densenet121_ra-50efcf5c.pthr"zqhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/densenetblur121d_ra-100dcfbc.pthzzDenseLayer.__init__0s j$((*** ,>!?!?@@AA  + 51QUZ"\"\"\ ] ] ^ ^ Gk,A!B!BCCDD  k !;AaQRY^"`"`"` a a b by)) 0r%ctj|d}|||}|SNr )torchcatr4r3)rExsconcated_featuresbottleneck_outputs r# bottleneck_fnzDenseLayer.bottleneck_fn<s9!Ib!,, JJtzz2C'D'DEE  r%c$|D] }|jrdS dS)NTF) requires_grad)rExtensors r#any_requires_gradzDenseLayer.any_requires_gradCs-  F# tt ur%c2fd}tj|g|RS)Nc.|SN)rR)rOrEs r#closurez6DenseLayer.call_checkpoint_bottleneck..closureMs%%b)) )r%)cp checkpoint)rErUr[s` r#call_checkpoint_bottleneckz%DenseLayer.call_checkpoint_bottleneckJs6 * * * * *}W)q))))r%cdSrZrErUs r#forwardzDenseLayer.forwardR  r%cdSrZr`ras r#rbzDenseLayer.forwardWrcr%ct|tjr|g}n|}|jrX||rCtjrtd||}n| |}| | |}|j dkr!tj||j |j}|S)Nz%Memory Efficient not supported in JITr)ptraining) isinstancerMTensorrDrWjit is_scripting Exceptionr^rRr:r9rCFdropoutrg)rErU prev_featuresrQ new_featuress r#rbzDenseLayer.forward^s a & & CMMM   BT%;%;M%J%J By%%'' I GHHH $ ? ? N N   $ 2 2= A A zz$**->"?"?@@ >A  9\T^dm\\\Lr%)__name__ __module__ __qualname__r r>rRrWrMrjunusedr^_overload_methodrb __classcell__rJs@r#r0r0/sLZ 1 1 1 1 1 1!!! Y*** Y     Y    r%r0c:eZdZdZejddffd ZdZxZS) DenseBlockr1Fc tt|t|D]9}t |||zz|||||} |d|dzz| :dS)N)rGrHrIrCrDz denselayer%dr )r=ryr>ranger0r?) rE num_layersrFrHrGrIrCrDilayerrJs r#r>zDenseBlock.__init__ts j$((***z"" = =A"Q_4'%#!1 E OONa!e4e < < < < = =r%c|g}|D]%\}}||}||&tj|dSrL)itemsappendrMrN)rE init_featuresfeaturesnamerrps r#rbzDenseBlock.forwardsX!?::<< * *KD% 5??L OOL ) ) ) )y1%%%r%) rqrrrs_versionr@ReLUr>rbrvrws@r#ryryqs\HXZX_ = = = = = =&&&&&&&r%ryc.eZdZejdffd ZxZS)DenseTransitionNc tt||d|||dt j||ddd|#|d||ddS|dt jdd dS) Nnormconvr Fr5poolrz)r7)r6r7)r=rr>r?r@rA AvgPool2d)rErFnum_output_featuresrIaa_layerrJs r#r>zDenseTransition.__init__s ot$$--///  += > >???   31SX!Z!Z!Z [ [ [   OOFHH-@$K$K$K L L L L L OOFBLQq$I$I$I J J J J Jr%)rqrrrsr@ BatchNorm2dr>rvrws@r#rrsOKM>dhKKKKKKKKKKr%rc VeZdZdZddddddded d d d f fd ZdZddZdZdZ xZ S)raDensenet-BC model class, based on `"Densely Connected Convolutional Networks" `_ Args: growth_rate (int) - how many filters to add each layer (`k` in paper) block_config (list of 4 ints) - how many layers in each pooling block bn_size (int) - multiplicative factor for number of bottle neck layers (i.e. bn_size * k features in the bottleneck layer) drop_rate (float) - dropout rate after each dense layer num_classes (int) - number of classification classes memory_efficient (bool) - If True, uses checkpointing. Much more memory efficient, but slower. Default: *False*. See `"paper" `_  rrravgNrFTc ||_| |_tt|d|v} |dz}| t jddd}n0t jt jddd| |dg}| r|x}}d|vrd|dzz}d |vr|nd |dzz}t jtd t j ||dddd fd||fdt j ||dddd fd||fdt j ||dddd fd||fd|fg|_ nPt jtd t j ||dddd fd||fd|fg|_ t|dd| rdndg|_ d}|}t|D]\}}t|||||| | }d|dz}|j |||||zz}| rdn| }|t!|dz kre|xj t||d|zgz c_ |dz}t#||dz||}|j d|dz||dz}|j d|||xj t||dgz c_ ||_t'|j|j| \|_|_|D]}t/|t j r%t j|jAt/|t jrKt j|jdt j|jdt/|t jr%t j|jddS)!Ndeeprzrr )r6r7r<)channelsr7tieredrnarrowrconv0F)r7r<r8norm0r4r3r:r9pool0rr;z features.normr)num_chs reductionmodule)r}rFrHrGrIrCrD denseblockz features.)rFrrIr transitionnorm5zfeatures.norm5 pool_type)rrCr=rr>r@ MaxPool2d SequentialrrArdict feature_info enumerateryr?lenr num_featuresr global_poolrmodulesrhinitkaiming_normal_weightr constant_r8Linear)rErG block_configrH stem_typerin_chansrrIrrCrD aa_stem_only deep_stemnum_init_features stem_pool stem_chs_1 stem_chs_2current_striderr~r}block module_nametransition_aa_layertransmrJs r#r>zDenseNet.__init__s'" h&&(((i' '!O   1aHHHII  1a@@@"3A>>>(@AI  &1 1J9$$+"23 2:i2G2G..QR]abRbMc M+")Hj!AqW\]]]^**Z001")J AaQRY^___`**Z001")J0A1QXY`efffg**%6778)$7++DMMM+")H.?QWXbcjopppq**%6778)$7++DM *a@gU^PePQPQde@g@g h h hj) &|44 1 1MAz%#/'%#!1E1A00K M $ $[% 8 8 8'*{*BBL*6"D$$H C %%)))!!P[^iPijjj&ll!!!#''3YZIZ)4GIII (()=a!e)=)=uEEE+q0    **\*B*BCCC d<>Zjkkkll(->  t/;-H-H-H)$/ - -A!RY'' -''1111Ar~.. -!!!(A...!!!&!,,,,Ary)) -!!!&!,,, - -r%c|jSrZ)r)rEs r#get_classifierzDenseNet.get_classifiers r%cf||_t|j|j|\|_|_dS)Nr)rrrrr)rErrs r#reset_classifierzDenseNet.reset_classifiers<&,=  t/;-H-H-H)$///r%c,||SrZ)rras r#forward_featureszDenseNet.forward_featuress}}Qr%c||}||}||}|SrZ)rrrras r#rbzDenseNet.forwards@  ! !! $ $   Q   OOA  r%)r) rqrrrs__doc__r r>rrrrbrvrws@r#rrs  $&OQZ\!A5*TQY^"U-U-U-U-U-U-nHHHH    r%ctjd}t|D]R}||}|r9|d|dz}||||<||=S|S)Nz]^(.*denselayer\d+\.(?:norm|relu|conv))\.((?:[12])\.(?:weight|bias|running_mean|running_var))$r rz)recompilelistkeysmatchgroup) state_dictpatternkeyresnew_keys r#_filter_torchvision_pretrainedrsjhjjGJOO%%&&  mmC    iillSYYq\\1G",S/Jw 3 r%c ||d<||d<tt||ft|tdtd|S)NrGrT)flatten_sequential) default_cfg feature_cfgpretrained_filter_fn)r r default_cfgsrr)variantrGr pretrainedkwargss r#_create_densenetrs\'F=)F> ':  )D111Hf      r%Fc (t ddd|d|}|S)tDensenet-121 model from `"Densely Connected Convolutional Networks" ` r&rrrGrr)r&rrrmodels r#r&r&(:  f#%OPZ f f^d f fE Lr%c 6t ddd|dtd|}|S)rr(rrr)rGrrrr)r()rr rs r#r(r(2s=  '(*U_kq ' '% ' 'E Lr%c *t dddd|d|}|S)rr'rrr)rGrrr)r'rrs r#r'r'=s:  )$&_PV ) )!' ) )E Lr%c (t ddd|d|}|S)ztDensenet-169 model from `"Densely Connected Convolutional Networks" ` r)r)rrrrr)r)rrs r#r)r)Hrr%c (t ddd|d|}|S)ztDensenet-201 model from `"Densely Connected Convolutional Networks" ` r*r)rr0rr)r*rrs r#r*r*Rrr%c (t ddd|d|}|S)ztDensenet-161 model from `"Densely Connected Convolutional Networks" ` r+r)rr$rr)r+rrs r#r+r+\rr%c (t ddd|d|}|S)ztDensenet-264 model from `"Densely Connected Convolutional Networks" ` r,rrr@rr)r,rrs r#r,r,frr%c 2d}t dddd||d|}|S)z6Densenet-264 model with deep stem and Inplace-ABN ctd|fi|S)Niabn)r )rrs r# norm_act_fnz&densenet264d_iabn..norm_act_fntsv|>>v>>>r%r-rrr)rGrrrIr)r-r)rrrrs r#r-r-psN???  A)+/U[: A A9? A AE Lr%c (t ddd|d|}|S)zDensenet-121 model with original Torchvision weights, from `"Densely Connected Convolutional Networks" ` r.rrr)r.rrs r#r.r.|s:  i&(S] i iag i iE Lr%)r)F)3rr collectionsr functoolsrrMtorch.nnr@torch.nn.functional functionalrmtorch.utils.checkpointutilsr]r\torch.jit.annotationsr timm.datarrhelpersr layersr r r rregistryr__all__r$rModuler0 ModuleDictryrrrrrr&r(r'r)r*r+r,r-r.r`r%r#r s ###### #########&&&&&&AAAAAAAA))))))RRRRRRRRRRRR$$$$$$ ,4 z|||DRLLL AAA4Z[[[4Z[[[4Z[[[4B<<<"d]^^^   ????????D&&&&&&&&2 K K K K Kbm K K Kxxxxxryxxxv   r%