SigmoidBinaryCrossEntropyLoss

class SigmoidBinaryCrossEntropyLoss(from_sigmoid=False, weight=None, batch_axis=0, **kwargs)

The cross-entropy loss for binary classification. (alias: SigmoidBCELoss)

BCE loss is useful when training logistic regression. If from_sigmoid is False (default), this loss computes:

(37)\[ \begin{align}\begin{aligned}prob = \frac{1}{1 + \exp(-{pred})}\\L = - \sum_i {label}_i * \log({prob}_i) * pos\_weight + (1 - {label}_i) * \log(1 - {prob}_i)\end{aligned}\end{align} \]

If from_sigmoid is True, this loss computes:

(38)\[L = - \sum_i {label}_i * \log({pred}_i) * pos\_weight + (1 - {label}_i) * \log(1 - {pred}_i)\]

A tensor pos_weight > 1 decreases the false negative count, hence increasing the recall. Conversely setting pos_weight < 1 decreases the false positive count and increases the precision.

pred and label can have arbitrary shape as long as they have the same number of elements.

Parameters:

from_sigmoid : bool, default is False

Whether the input is from the output of sigmoid. Set this to false will make the loss calculate sigmoid and BCE together, which is more numerically stable through log-sum-exp trick.

weight : float or None

Global scalar weight for loss.

batch_axis : int, default 0

The axis that represents mini-batch.

Inputs:

• pred: prediction tensor with arbitrary shape
• label: target tensor with values in range [0, 1]. Must have the same size as pred.
• sample_weight: element-wise weighting tensor. Must be broadcastable to the same shape as pred. For example, if pred has shape (64, 10) and you want to weigh each sample in the batch separately, sample_weight should have shape (64, 1).
• pos_weight: a weighting tensor of positive examples. Must be a vector with length equal to the number of classes.For example, if pred has shape (64, 10), pos_weight should have shape (1, 10).

Outputs:

• loss: loss tensor with shape (batch_size,). Dimenions other than batch_axis are averaged out.