crested.tl.zoo.utils.ffn_block_enf

Contents

crested.tl.zoo.utils.ffn_block_enf#

crested.tl.zoo.utils.ffn_block_enf(inputs, filters, expansion_rate=2, dropout=0.4, activation='relu', residual=True, ln_epsilon=1e-05, name_prefix=None)#

Construct a feedforward block (for Enformer), consisting of Residual(LayerNorm+PointwiseConv+Dropout+ReLU+PointwiseConv+Dropout).

Parameters:

  • inputs – Input tensor.

  • filters (int) – Pointwise convolution filters.

  • expansion_rate (int (default: 2)) – Scaling factor of base filters inside the FFN.

  • dropout (int (default: 0.4)) – Dropout rate.

  • activation (str (default: 'relu')) – Which activation function to use.

  • residual (bool (default: True)) – Whether to wrap the entire block in residual structure.

  • ln_epsilon (float (default: 1e-05)) – Epsilon to use in the layer normalisation layer.

  • name_prefix (str | None (default: None)) – Prefix for layer names.

Return type:

KerasTensor

Returns:

Output tensor after applying the feedforward block.

Contents