o g5C@sdZddlZddlZddlmZddlmmZddlm Z m Z ddl m Z ddl mZddlmZdgZd@d d Zed d edd edd edd edd edd edd edd edd edd edd edd d ZGdddejZGdddejZGdddejZGd d!d!ejZGd"d#d#ejZGd$ddejZdAd&d'ZedAd(d)ZedAd*d+ZedAd,d-ZedAd.d/ZedAd0d1Z edAd2d3Z!edAd4d5Z"edAd6d7Z#edAd8d9Z$edAd:d;Z%edAdd?Z'dS)BaF Deep Layer Aggregation and DLA w/ Res2Net DLA original adapted from Official Pytorch impl at: DLA Paper: `Deep Layer Aggregation` - https://arxiv.org/abs/1707.06484 Res2Net additions from: https://github.com/gasvn/Res2Net/ Res2Net Paper: `Res2Net: A New Multi-scale Backbone Architecture` - https://arxiv.org/abs/1904.01169 NIMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STD)build_model_with_cfg)create_classifier)register_modelDLAc Ks|dddddttddd |S) N)r )rg?bilinearz base_layer.0fc) url num_classes input_size pool_sizecrop_pct interpolationmeanstd first_conv classifierr)rkwargsr(/home/terry/ogs_model/timm/models/dla.py_cfgsrz6http://dl.yf.io/dla/models/imagenet/dla34-ba72cf86.pth)rz8http://dl.yf.io/dla/models/imagenet/dla46_c-2bfd52c3.pthz9http://dl.yf.io/dla/models/imagenet/dla46x_c-d761bae7.pthz9http://dl.yf.io/dla/models/imagenet/dla60x_c-b870c45c.pthz6http://dl.yf.io/dla/models/imagenet/dla60-24839fc4.pthz7http://dl.yf.io/dla/models/imagenet/dla60x-d15cacda.pthz7http://dl.yf.io/dla/models/imagenet/dla102-d94d9790.pthz8http://dl.yf.io/dla/models/imagenet/dla102x-ad62be81.pthz9http://dl.yf.io/dla/models/imagenet/dla102x2-262837b6.pthz7http://dl.yf.io/dla/models/imagenet/dla169-0914e092.pthznhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-res2net/res2net_dla60_4s-d88db7f9.pthzohttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-res2net/res2next_dla60_4s-d327927b.pth) dla34dla46_cdla46x_cdla60x_cdla60dla60xdla102dla102xdla102x2dla169 dla60_res2netdla60_res2nextcs,eZdZdZdfdd Zd ddZZS) DlaBasicz DLA Basicrc srtt|tj||d||d|d|_t||_tjdd|_ tj||dd|d|d|_ t||_ ||_ dS)Nr F kernel_sizestridepaddingbiasdilationTinplacer) superr+__init__nnConv2dconv1 BatchNorm2dbn1ReLUreluconv2bn2r.)selfinplanesplanesr.r1_ __class__rrr56s   zDlaBasic.__init__NcCsT|dur|}||}||}||}||}||}||7}||}|SN)r8r:r<r=r>r?xshortcutoutrrrforwardAs      zDlaBasic.forwardrrrE)__name__ __module__ __qualname____doc__r5rJ __classcell__rrrCrr+3s r+cs0eZdZdZdZd fdd Zd dd ZZS) DlaBottleneckzDLA/DLA-X Bottleneckr@c stt|||_tt||d|}||j}tj ||ddd|_ t ||_ tj ||d||d||d|_ t ||_tj ||ddd|_t ||_tjdd|_dS) NrSrFr-r0r )r-r.r/r0r1groupsTr2)r4rQr5r.intmathfloor expansionr6r7r8r9r:r=r>conv3bn3r;r<)r?r@ outplanesr.r1 cardinality base_width mid_planesrCrrr5Vs     zDlaBottleneck.__init__NcCsr|dur|}||}||}||}||}||}||}||}||}||7}||}|SrE)r8r:r<r=r>rZr[rFrrrrJfs         zDlaBottleneck.forward)rrrrSrErLrMrNrOrYr5rJrPrrrCrrQRs rQcs0eZdZdZdZd fdd Zd d d ZZS) DlaBottle2neckzj Res2Net/Res2NeXT DLA Bottleneck Adapted from https://github.com/gasvn/Res2Net/blob/master/dla.py rRrc s*tt||dk|_||_tt||d|}||j}||_ t j |||ddd|_ t |||_td|d} g} g} t| D]} | t j ||d||||dd| t |qGt | |_t | |_|jrzt jd|dd|_t j |||ddd|_t ||_t jdd |_dS) NrrSFrTr )r-r.r/r1rUr0)r-r.r/Tr2)r4rar5is_firstscalerVrWrXrYwidthr6r7r8r9r:maxrangeappend ModuleListconvsbns AvgPool2dpoolrZr[r;r<) r?r@r\r.r1rer]r^r_num_scale_convsrkrlrBrCrrr5s0      zDlaBottle2neck.__init__Nc Cs|dur|}||}||}||}t||jd}g}tt|j|j D])\}\}}|dks5|j r9||n| ||} || } || } || } | | q(|j dkrh| |j rc| |dn|dt|d}||}||}||7}||}|S)Nrr)r8r:r<torchsplitrf enumerateziprkrlrdrirerncatrZr[) r?rGrHrIspxspoiconvbnsprrrrJs*   "   "    zDlaBottle2neck.forward)rrrbrcrbrEr`rrrCrra{s racs$eZdZfddZddZZS)DlaRootcsRtt|tj||ddd|ddd|_t||_tjdd|_ ||_ dS)NrFrR)r.r0r/Tr2) r4r|r5r6r7ryr9rzr;r<rH)r? in_channels out_channelsr-rHrCrrr5s  zDlaRoot.__init__cGs@|}|t|d}||}|jr||d7}||}|S)Nrr)ryrqrurzrHr<)r?rGchildrenrrrrJs   zDlaRoot.forwardrLrMrNr5rJrPrrrCrr|s r|cs.eZdZ   d fdd Zd dd ZZS) DlaTreerrSFrc  sJtt|| dkrd|} | r| |7} |dkr tj||dnt|_t|_t|||d} |dkrb||||fi| |_ |||dfi| |_ ||krat tj ||ddddt ||_n,| t| | dt|d||||fd di| |_ t|d|||fd | |i| |_ |dkrt| || | |_| |_| |_||_dS) NrrRrr.)r1r]r^F)r-r.r0)root_kernel_size root_shortcutroot_dim)r4rr5r6 MaxPool2dIdentity downsampleprojectdicttree1tree2 Sequentialr7r9updater|root level_rootrlevels)r?rblockr}r~r.r1r]r^rrrrcargsrCrrr5sJ    zDlaTree.__init__NcCs|durgn|}||}||}|jr|||||}|jdkr6||}|j||g|R}|S|||j||d}|S)Nr)r)rrrrirrrr)r?rGrHrbottomx1x2rrrrJs       zDlaTree.forward)rrrrSFrrF)NNrrrrCrrs !rc s\eZdZdddddedddf fd d Zdd d Zd dZdddZddZddZ Z S)r r r rrSFavgc  stt|||_||_||_||_| |_|dksJt tj ||ddddddt |dtj dd |_ ||d|d|d|_|j|d|d|dd d |_t||| d } t|d ||d|d d fd di| |_t|d||d |dd fd di| |_t|d||d|dd fd di| |_t|d||d|dd fd di| |_t|ddddt|dd ddt|d dddt|ddddt|ddddt|ddddg|_|d|_t|j|j| dd\|_|_| rtdnt|_|D]:} t | tj r,| j!d| j!d| j"}| j#j$%dt&'d|qt | tj r@| j#j$(d| j)j$*qdS)Nrrrrr F)r-r.r/r0Tr2rRr)r]r^rrrblevel0)num_chs reductionmodulelevel1level2rclevel3level4level5rp pool_typeuse_convg@)+r4r r5channelsrr]r^ drop_rater6rr7r9r; base_layer_make_conv_levelrrrrrrrr feature_info num_featuresr global_poolrFlattenrflattenmodules isinstancer-r~weightdatanormal_rWsqrtfill_r0zero_)r?rr output_striderin_chansr]r^r shortcut_rootrrrmnrCrrr5sP    ****    z DLA.__init__c Cs^g}t|D]#}|tj||d|dkr|nd|d|dt|tjddg|}qtj|S)Nr rrFr,Tr2)rhextendr6r7r9r;r)r?r@rArkr.r1rrxrrrr-s   zDLA._make_conv_levelcCs|jSrE)r)r?rrrget_classifier8szDLA.get_classifiercCsD||_t|j|j|dd\|_|_|rtd|_dSt|_dS)NTrr) rrrrrr6rrr)r?rrrrrreset_classifier;s  "zDLA.reset_classifiercCsJ||}||}||}||}||}||}||}|SrE)rrrrrrrr?rGrrrforward_featuresAs       zDLA.forward_featurescCsJ||}||}|jdkrtj||j|jd}||}||}|S)Nr)ptraining)rrrFdropoutrrrrrrrrJKs     z DLA.forwardrK)r) rLrMrNrar5rrrrrJrPrrrCrr s ,  FcKs&tt||ft|dtddd|S)NF)rrRr rbr) out_indices) default_cfgpretrained_strict feature_cfg)rr default_cfgsr)variant pretrainedrrrr _create_dlaUsrcK,tdddtddd|}td|fi|S)NrrrrRr rrrirrrrr]r^r)rrrarrr model_kwargsrrrr)^r)cKr)Nrrrcrbrr*rrrrrrr*frr*cK0tdgdgdtd|}td|fi|S)NrrrrRrRr)rrrSrrrrrrrr)rr+rrrrrrn rcKr)NrrrrSrSrrrr rrrQrrrrrr vrr cK4tdgdgdtddd|}td|fi|S)Nrrrrbrr!rrrrrrr!~ r!cKr)Nrrrrbrr"rrrrrrr"rr"cKr)Nrrrr#rrrrrrr#rr#cKr)Nrrrrbrr$rrrrrrr$rr$cK2tdgdgdtdd|}td|fi|S)Nrrrr rbrrTrrrrr%rrrrrrr% r%c K6tdgdgdtdddd|}td|fi|S) NrrrrbTrrrr]r^rr&rrrrrrr& r&c Kr) NrrrSrbTrr'rrrrrrr'rr'cKr)N)rrrRr rrrTrr(rrrrrrr(rr()r )F)(rOrWrqtorch.nnr6torch.nn.functional functionalr timm.datarrhelpersrlayersrregistryr__all__rrModuler+rQrar|rr rr)r*rr r!r"r#r$r%r&r'r(rrrrsv      )?2 U