U 9¢}d˜ã@sPddlZddlmZmZmZddlmZGdd„dejƒZGdd„dejƒZdS)éN)ÚnnÚsinÚpow)Ú Parametercs*eZdZdZd ‡fdd„ Zdd„Z‡ZS) ÚSnakeaÓ Implementation of a sine-based periodic activation function Shape: - Input: (B, C, T) - Output: (B, C, T), same shape as the input Parameters: - alpha - trainable parameter References: - This activation function is from this paper by Liu Ziyin, Tilman Hartwig, Masahito Ueda: https://arxiv.org/abs/2006.08195 Examples: >>> a1 = snake(256) >>> x = torch.randn(256) >>> x = a1(x) çð?TFcs\tt|ƒ ¡||_||_|jr6tt |¡|ƒ|_ntt  |¡|ƒ|_||j_ d|_ dS)a  Initialization. INPUT: - in_features: shape of the input - alpha: trainable parameter alpha is initialized to 1 by default, higher values = higher-frequency. alpha will be trained along with the rest of your model. ç•Ö&è .>N) ÚsuperrÚ__init__Ú in_featuresÚalpha_logscalerÚtorchÚzerosÚalphaÚonesÚ requires_gradÚno_div_by_zero©Úselfr rZalpha_trainabler ©Ú __class__©ú7/mnt2/liuhuadai.lhd/maa2/vocoder/bigvgan/activations.pyr s zSnake.__init__cCsJ|j d¡ d¡}|jr"t |¡}|d||jtt||ƒdƒ}|S)uŽ Forward pass of the function. Applies the function to the input elementwise. Snake ∶= x + 1/a * sin^2 (xa) réÿÿÿÿré)rÚ unsqueezer r Úexprrr)rÚxrrrrÚforward0s  $z Snake.forward)rTF©Ú__name__Ú __module__Ú __qualname__Ú__doc__r rÚ __classcell__rrrrr srcs*eZdZdZd ‡fdd„ Zdd„Z‡ZS) Ú SnakeBetaam A modified Snake function which uses separate parameters for the magnitude of the periodic components Shape: - Input: (B, C, T) - Output: (B, C, T), same shape as the input Parameters: - alpha - trainable parameter that controls frequency - beta - trainable parameter that controls magnitude References: - This activation function is a modified version based on this paper by Liu Ziyin, Tilman Hartwig, Masahito Ueda: https://arxiv.org/abs/2006.08195 Examples: >>> a1 = snakebeta(256) >>> x = torch.randn(256) >>> x = a1(x) rTFcsŒtt|ƒ ¡||_||_|jrJtt |¡|ƒ|_tt |¡|ƒ|_ n(tt  |¡|ƒ|_tt  |¡|ƒ|_ ||j_ ||j _ d|_ dS)aÍ Initialization. INPUT: - in_features: shape of the input - alpha - trainable parameter that controls frequency - beta - trainable parameter that controls magnitude alpha is initialized to 1 by default, higher values = higher-frequency. beta is initialized to 1 by default, higher values = higher-magnitude. alpha will be trained along with the rest of your model. rN) r r%r r r rr rrÚbetarrrrrrrr Os zSnakeBeta.__init__cCsf|j d¡ d¡}|j d¡ d¡}|jr>t |¡}t |¡}|d||jtt||ƒdƒ}|S)u’ Forward pass of the function. Applies the function to the input elementwise. SnakeBeta ∶= x + 1/b * sin^2 (xa) rrrr) rrr&r r rrrr)rrrr&rrrrks  $zSnakeBeta.forward)rTFrrrrrr%>sr%) r rrrÚtorch.nnrÚModulerr%rrrrÚs 5