**************************

第九章 激活函数

**************************



9.8 激活函数实践

======================



以经典的Sigmoid型函数为例，其在PyTorch中的调用方式为：



.. code-block:: python

   :linenos:



   # Sigmoid (PyTorch)

   import torch

   import torch.nn as nn



   sigmoid = nn.Sigmoid()

   inputs = torch.randn(2)

   output = sigmoid(inputs)



在MindSpore中的调用方式为：



.. code-block:: python

   :linenos:



   # Sigmoid (MindSpore)

   import mindspore as ms

   import mindspore.nn as nn

   import numpy as np



   x = ms.Tensor(np.array([-1, -2, 0, 2, 1]), ms.float16)

   sigmoid = nn.Sigmoid()

   output = sigmoid(x)



其他常用激活函数如ReLU、PReLU和Leaky ReLU（其式中代表当x≤0时斜率的超参数一般为0.1）的PyTorch调用方式分别为：



.. code-block:: python

   :linenos:



   # ReLU (PyTorch)

   relu = nn.ReLU()

   inputs = torch.randn(2)

   output = relu(inputs)



.. code-block:: python

   :linenos:



   # PReLU (PyTorch)

   prelu = nn.PReLU()

   inputs = torch.randn(2)

   output = prelu(inputs)



.. code-block:: python

   :linenos:



   # Leaky ReLU (PyTorch)

   leaky_relu = nn.LeakyReLU(negative_slope=0.1)

   inputs = torch.randn(2)

   output = leaky_relu(inputs)



在MindSpore中的调用方式分别为：



.. code-block:: python

   :linenos:



   # ReLU (MindSpore)

   x = ms.Tensor(np.array([-1, -2, 0, 2, 1]), ms.float16)

   relu = nn.ReLU()

   output = relu(x)



.. code-block:: python

   :linenos:



   # PReLU (MindSpore)

   x = ms.Tensor(np.array([-1, -2, 0, 2, 1]), ms.float16)

   prelu = nn.PReLU()

   output = prelu(x)



.. code-block:: python

   :linenos:



   # Leaky ReLU (MindSpore)

   x = ms.Tensor(np.array([-1, -2, 0, 2, 1]), ms.float16)

   leaky_relu = nn.LeakyReLU(0.1)

   output = leaky_relu(x)