.. _crash_course_nn: Step 2: Create a neural network =============================== In this step, you learn how to use NP on MXNet to create neural networks in Gluon. In addition to the ``np`` package that you learned about in the previous step :ref:`crash_course_ndarray`, you also import the neural network ``nn`` package from ``gluon``. Use the following commands to import the packages required for this step. .. code:: python from mxnet import np, npx from mxnet.gluon import nn npx.set_np() # Change MXNet to the numpy-like mode. Create your neural network’s first layer ---------------------------------------- Use the following code example to start with a dense layer with two output units. .. code:: python layer = nn.Dense(2) layer .. parsed-literal:: :class: output Dense(-1 -> 2, linear) Initialize its weights with the default initialization method, which draws random values uniformly from :math:`[-0.7, 0.7]`. You can see this in the following example. .. code:: python layer.initialize() Do a forward pass with random data, shown in the following example. We create a :math:`(3,4)` shape random input ``x`` and feed into the layer to compute the output. .. code:: python x = np.random.uniform(-1,1,(3,4)) layer(x) .. parsed-literal:: :class: output array([[-0.02524132, -0.00874885], [-0.06026538, -0.01308061], [ 0.02468396, -0.02181557]]) As can be seen, the layer’s input limit of two produced a :math:`(3,2)` shape output from our :math:`(3,4)` input. You didn’t specify the input size of ``layer`` before, though you can specify it with the argument ``in_units=4`` here. The system automatically infers it during the first time you feed in data, create, and initialize the weights. You can access the weight after the first forward pass, as shown in this example. .. code:: python # layer.weight.data() # FIXME Chain layers into a neural network ---------------------------------- Consider a simple case where a neural network is a chain of layers. During the forward pass, you run layers sequentially one-by-one. Use the following code to implement a famous network called `LeNet `__ through ``nn.Sequential``. .. code:: python net = nn.Sequential() # Add a sequence of layers. net.add(# Similar to Dense, it is not necessary to specify the input channels # by the argument `in_channels`, which will be automatically inferred # in the first forward pass. Also, we apply a relu activation on the # output. In addition, we can use a tuple to specify a non-square # kernel size, such as `kernel_size=(2,4)` nn.Conv2D(channels=6, kernel_size=5, activation='relu'), # One can also use a tuple to specify non-symmetric pool and stride sizes nn.MaxPool2D(pool_size=2, strides=2), nn.Conv2D(channels=16, kernel_size=3, activation='relu'), nn.MaxPool2D(pool_size=2, strides=2), # The dense layer will automatically reshape the 4-D output of last # max pooling layer into the 2-D shape: (x.shape[0], x.size/x.shape[0]) nn.Dense(120, activation="relu"), nn.Dense(84, activation="relu"), nn.Dense(10)) net .. parsed-literal:: :class: output Sequential( (0): Conv2D(-1 -> 6, kernel_size=(5, 5), stride=(1, 1), Activation(relu)) (1): MaxPool2D(size=(2, 2), stride=(2, 2), padding=(0, 0), ceil_mode=False, global_pool=False, pool_type=max, layout=NCHW) (2): Conv2D(-1 -> 16, kernel_size=(3, 3), stride=(1, 1), Activation(relu)) (3): MaxPool2D(size=(2, 2), stride=(2, 2), padding=(0, 0), ceil_mode=False, global_pool=False, pool_type=max, layout=NCHW) (4): Dense(-1 -> 120, Activation(relu)) (5): Dense(-1 -> 84, Activation(relu)) (6): Dense(-1 -> 10, linear) ) .. raw:: html Using ``nn.Sequential`` is similar to ``nn.Dense``. In fact, both of them are subclasses of ``nn.Block``. Use the following code to initialize the weights and run the forward pass. .. code:: python net.initialize() # Input shape is (batch_size, color_channels, height, width) x = np.random.uniform(size=(4,1,28,28)) y = net(x) y.shape .. parsed-literal:: :class: output (4, 10) You can use ``[]`` to index a particular layer. For example, the following accesses the first layer’s weight and sixth layer’s bias. .. code:: python (net[0].weight.data().shape, net[5].bias.data().shape) .. parsed-literal:: :class: output ((6, 1, 5, 5), (84,)) Create a neural network flexibly -------------------------------- In ``nn.Sequential``, MXNet will automatically construct the forward function that sequentially executes added layers. Here is another way to construct a network with a flexible forward function. Create a subclass of ``nn.Block`` and implement two methods by using the following code. - ``__init__`` create the layers - ``forward`` define the forward function. .. code:: python class MixMLP(nn.Block): def __init__(self, **kwargs): # Run `nn.Block`'s init method super(MixMLP, self).__init__(**kwargs) self.blk = nn.Sequential() self.blk.add(nn.Dense(3, activation='relu'), nn.Dense(4, activation='relu')) self.dense = nn.Dense(5) def forward(self, x): y = npx.relu(self.blk(x)) print(y) return self.dense(y) net = MixMLP() net .. parsed-literal:: :class: output MixMLP( (blk): Sequential( (0): Dense(-1 -> 3, Activation(relu)) (1): Dense(-1 -> 4, Activation(relu)) ) (dense): Dense(-1 -> 5, linear) ) In the sequential chaining approach, you can only add instances with ``nn.Block`` as the base class and then run them in a forward pass. In this example, you used ``print`` to get the intermediate results and ``nd.relu`` to apply relu activation. This approach provides a more flexible way to define the forward function. The following code example uses ``net`` in a similar manner as earlier. .. code:: python net.initialize() x = np.random.uniform(size=(2,2)) net(x) .. parsed-literal:: :class: output [[0. 0. 0.00014992 0. ] [0. 0. 0.00025604 0. ]] .. parsed-literal:: :class: output array([[ 2.0575192e-06, -6.8181575e-06, -7.2768480e-06, -4.4091166e-06, 8.4296198e-06], [ 3.5140570e-06, -1.1644797e-05, -1.2428200e-05, -7.5303733e-06, 1.4397029e-05]]) Finally, access a particular layer’s weight with this code. .. code:: python net.blk[1].weight.data() .. parsed-literal:: :class: output array([[-0.0204011 , 0.00283971, -0.03490724], [-0.05335442, 0.05835057, -0.03953977], [ 0.00657327, 0.04791138, 0.01269311], [-0.02781162, 0.04208799, -0.04010999]]) Next steps ---------- After you create a neural network, learn how to automatically compute the gradients in :ref:`crash_course_autograd`.