In this article

Normalization

|
  • Share via
  •  Edit this article

Normalizations

[class] BatchNorm

(BatchNorm dims &key (eps 1e-5) (affine t) (bias t))
Batch Normalization Layer

[function] !batch-norm

(!batch-norm x &optional weight bias mean invstd (axis 1) (eps 1e-5))
Computes the batch normalization of a tensor x with optional weight and bias tensors.

[class] LayerNorm

(LayerNorm dims &key (eps 1e-5) (affine t) (bias t))
Layer Normalization Layer

[function] !layer-norm

(!layer-norm x normalized-shape &key (weight) (bias) (eps 1e-5))
Computes the layer normalization of a tensor x with optional weight and bias tensors.

[class] RMSNorm

(RMSNorm dims &key (eps 1e-5) (weight t))
Root Mean Square Normalization Layer

[function] !rms-norm

(!rms-norm x normalized-shape &key (weight) (eps 1e-5))
Computes the root mean square normalization of a tensor x with optional weight tensor.