Feedforward Layers

Gluon provides a large number of build-in feedforward neural network layers in two modiules, mxnet.gluon.nn and mxnet.gluon.contrib.nn.

Basic Layers

nn.Dense	Just your regular densely-connected NN layer.
nn.Activation	Applies an activation function to input.
nn.Dropout	Applies Dropout to the input.
nn.Flatten	Flattens the input to two dimensional.
nn.Lambda	Wraps an operator or an expression as a Block object.
nn.HybridLambda	Wraps an operator or an expression as a HybridBlock object.

Convolutional Layers

nn.Conv1D	1D convolution layer (e.g.
nn.Conv2D	2D convolution layer (e.g.
nn.Conv3D	3D convolution layer (e.g.
nn.Conv1DTranspose	Transposed 1D convolution layer (sometimes called Deconvolution).
nn.Conv2DTranspose	Transposed 2D convolution layer (sometimes called Deconvolution).
nn.Conv3DTranspose	Transposed 3D convolution layer (sometimes called Deconvolution).

Pooling Layers

nn.MaxPool1D	Max pooling operation for one dimensional data.
nn.MaxPool2D	Max pooling operation for two dimensional (spatial) data.
nn.MaxPool3D	Max pooling operation for 3D data (spatial or spatio-temporal).
nn.AvgPool1D	Average pooling operation for temporal data.
nn.AvgPool2D	Average pooling operation for spatial data.
nn.AvgPool3D	Average pooling operation for 3D data (spatial or spatio-temporal).
nn.GlobalMaxPool1D	Gloabl max pooling operation for one dimensional (temporal) data.
nn.GlobalMaxPool2D	Global max pooling operation for two dimensional (spatial) data.
nn.GlobalMaxPool3D	Global max pooling operation for 3D data (spatial or spatio-temporal).
nn.GlobalAvgPool1D	Global average pooling operation for temporal data.
nn.GlobalAvgPool2D	Global average pooling operation for spatial data.
nn.GlobalAvgPool3D	Global average pooling operation for 3D data (spatial or spatio-temporal).
nn.ReflectionPad2D	Pads the input tensor using the reflection of the input boundary.

Normalization Layers

nn.BatchNorm	Batch normalization layer (Ioffe and Szegedy, 2014).
nn.InstanceNorm	Applies instance normalization to the n-dimensional input array.
nn.LayerNorm	Applies layer normalization to the n-dimensional input array.
contrib.nn.SyncBatchNorm	Cross-GPU Synchronized Batch normalization (SyncBN)

Embedding Layers

nn.Embedding	Turns non-negative integers (indexes/tokens) into dense vectors of fixed size.
contrib.nn.SparseEmbedding	Turns non-negative integers (indexes/tokens) into dense vectors of fixed size.

Advanced Activation Layers

nn.LeakyReLU	Leaky version of a Rectified Linear Unit.
nn.PReLU	Parametric leaky version of a Rectified Linear Unit.
nn.ELU	Exponential Linear Unit (ELU)
nn.SELU	Scaled Exponential Linear Unit (SELU)
nn.Swish	Swish Activation function