In this article

Tensor

|
  • Share via
  •  Edit this article

Tensor

[struct] Tensor

A struct tensor is a multi-dimensional, and strided matrix (and nrank=0 to scalar value) containing elements of the single dtype. Also, the tensor has the following slots:

  • shape[list] The shape as a list of elements of: number, symbol, or Tensor.
  • buffer[AbstractBuffer] The buffer of the tensor. Realized arrays are stored here.
  • dtype[keyword] The dtype of the tensor.
  • order[order] The memory layout of the tensor, selected from either :row or :column.
  • id[symbol] The unique identifier of the tensor. (usually created via gensym)
  • op[Func] The Func object that represents the operation of the tensor.
  • views[list] A list of ViewRange objects, determining the bound of loops.
  • requires-grad[boolean] A flag to determine whether the tensor requires a gradient.
  • grad[Tensor] A gradient and realized tensor of the tensor.
  • grad-id[symbol] A unique identifier of the gradient tensor.
  • variables[list] A list of Tensor objects that are used in the operation of the tensor. Tensors listed here are involved in the compilation.

[function] make-tensor

(make-tensor shape &key (dtype *default-float*) (order *default-order*) (id (gensym "TID")) (requires-grad nil) (initial-element nil) (views nil) (from nil))
Create a new lazy tensor.

  • shape[list] The shape as a list of elements of: number, symbol, or Tensor.
  • dtype[keyword] The dtype of the tensor.
  • order[order] The memory layout of the tensor, selected from either :row or :column.
  • id[symbol] The unique identifier of the tensor. (usually created by gensym)
  • requires-grad[boolean] A flag to determine whether the tensor requires a gradient.
  • initial-element[null|number|symbol|ScalarTensor] An initial value of the tensor.
  • views[list] A list of ViewRange objects, determining the bound of loops.
  • from[null|AbstractBuffer|Symbol] A buffer used to initialize the tensor. If a symbol is given, then the buffer is taken from the variable table.

[function] make-scalar

(make-scalar value &key (dtype *default-float*) (order *default-order*) (id (gensym "SID")) (requires-grad nil))
Create a new scalar tensor. A ScalarTensor in Caten is a tensor with a rank of 0.

  • value[number|symbol] The value of the scalar tensor.
  • dtype[keyword] The dtype of the tensor.
  • order[order] The memory layout of the tensor, selected from either of :row or :column.
  • id[symbol] A unique identifier of the tensor (usually created by gensym).
  • requires-grad[boolean] A flag to determine whether the tensor requires a gradient.

Hint: You can use fconst, uconst, and iconst to create a scalar tensor with a default dtype (arguments are the same as make-scalar.)

[function] grad

(grad tensor)
Returns a gradient of the tensor

[function] shape

(shape tensor)
Returns a copy of the shape of the tensor

[function] ndim

(ndim tensor)
Returns a rank of the tensor

[function] dtype-of

(dtype-of tensor)
Returns a dtype of the tensor

[function] order

(order tensor)
Returns a memory-layout of the tensor.

[function] caten

(caten tensors &key (backend (ctx:getenv :BACKEND)) (rewriters nil) (simplifiers *external-simplifiers*))

An entry point for compiling the given tensors, returning GraphRuntime.

  • tensor[Tensor|List] toplevel tensors.
  • backend[keyword] a keyword of the backend to use. (assumed to be defined by define-backend)
  • rewriters[list] a list of graph rewriters called each time the compiler will lower the module.
  • simplifiers[list] a list of external simplifiers used in the graph-level compilation (defined by defsimplifier). Pass the function name.

[function] tensor-graph

(tensor-graph tensor)

Lowers the given tensors into an aasm graph, only constant folding is applied.

For some convenience, this function returns the first tensor if the input is not a tensor.

[function] tensor-lowered-graph

(tensor-lowered-graph &rest tensors)

Creates a lowered graph from the given tensors (i.e., an input graph to JIT=1).



### [method] forward
(forward runtime &rest params) 
Compute the forward pass of the compiled computational graph (GraphRuntime). The params are additional parameters that are used to initialize the GraphRuntime variable table, passed in the following format:


- `(symbol . number)` Loaded as a `*default-int*` scalar tensor, used to determine the shape of dynamic shaped tensors.
- `(symbol . buffer)` or `(symbol . tensor)` Used to assign the initial elements of the tensor specified by the name in `form` in `(make-tensor ... :from x)`

or `(forward runtime place1 value1 place2 value2 ...)` is added recently.

Here's an example.


#### Examples


```lisp title="Example of forward with dynamic shaped"

(let ((model (caten (!randn `(:a :b)))))
  (forward model :a 10 :b 10))

Result
Result
{Tensor{LISPBUFFER}[float32] :shape (A B) :id STC102882
   ((0.4678763    -0.4861329   -0.94086486  1.04754      2.2546127    0.50788605   -0.10552773  0.049141496  -1.5750943   -1.7679455)
    (1.3089938    -0.23512708  1.1692804    -0.28485575  0.3088914    -0.6845822   2.1176116    0.66664845   -0.45067576  -0.27492777)
    (-0.12159313  -1.3346667   -0.5996805   -0.47102702  0.5524228    0.35672027   -0.60367817  -0.21228786  -0.14100133  -1.1875058)
    (1.1116551    1.8247204    -1.0528817   -0.3054723   0.29911244   0.13449419   0.8812368    -0.08793502  -1.4413477   -0.2823714)
    (-0.8456068   0.20695838   -0.8928872   0.34807467   0.64626217   -1.1382821   1.5031484    -1.5459219   -0.7998622   -1.2453525)
    (0.33562827   -1.4916059   0.5784393    -1.9095349   1.2488891    0.40152097   -0.33903646  1.8610382    -0.22350395  -0.94860715)
    (1.2874851    1.0844897    -1.9168866   -0.042802844 -0.93771714  1.3879867    -0.46589965  2.1235034    -1.4425595   0.19452804)
    (-1.3601065   -1.061901    0.8746508    -0.7386104   1.9441216    0.72194284   -0.17606297  -0.17282888  2.7211494    0.41256538)
    (-1.3365294   0.41671795   0.7274572    -1.0201341   0.07424722   1.0455105    2.0199647    0.07859164   -0.81086075  0.3092433)
    (-0.56345034  1.2789423    -0.6200388   0.95400995   -1.3767093   -1.0824282   0.4128665    0.6920807    -0.22273287  -2.4520183))
  :op #<PROCEEDNODE {100397A8B3}>
  :requires-grad NIL
  :variables (STC80700)
  :tracker #<TRACKER :order={row(0 1)} :shape=(A B) :contiguous-p=T>}

[method] backward

(forward runtime &optional prev-dout)

Compute the backward pass of the compiled computational graph (GraphRuntime). Note that the prev-dout is ignored. Forward pass must be computed first. Gradients are automatically reset to zero before the forward pass.

[function] proceed

(proceed &rest tensors)

Compiles the given tensors, returning an evaluated tensors.

[macro] with-no-grad

(with-no-grad &body body)

Within the scope of with-no-grad, gradient computation is disabled. This parameter should be set during inference, such that the compiler will generate more efficient code.

[variable] inference-mode

When *inference-mode* is set to T, it explicitly indicates that the code is being executed in inference mode. Additionally, it has the following effects:

  • Static Random Generation (e.g., rand) with :requires-grad=T does nothing (it is expected to load a parameter from the state dictionary).

  • The behavior of certain NN operations, such as BatchNorm and Dropout, changes.

[macro] with-inference-mode

(with-inference-mode (() &body body))
Sets *inference-mode*=T and *no-grad*=T within the scope of the body.

[function] get-global-runtime

(get-global-runtime)
Returns a temporary runtime object just used for allocation global buffer.

Examples

Tensor Creation

lisp
CATEN-USER> (make-tensor `(30 30))
Result
Result
{Tensor[float32] :shape (30 30) :id TID102883
  :buffer nil
  :op #<ALLOCATE {100397B2E3}>
  :requires-grad NIL
  :variables NIL
  :tracker #<TRACKER :order={row(0 1)} :shape=(30 30) :contiguous-p=T>}

Realize

lisp
CATEN-USER> (proceed (make-tensor `(30 30)))
Result
Result
{Tensor{LISPBUFFER}[float32] :shape (30 30) :id STC102946
   ((0.0 0.0 0.0 0.0 0.0 ~ 0.0 0.0 0.0 0.0 0.0)
    (0.0 0.0 0.0 0.0 0.0 ~ 0.0 0.0 0.0 0.0 0.0)
    (0.0 0.0 0.0 0.0 0.0 ~ 0.0 0.0 0.0 0.0 0.0)
    (0.0 0.0 0.0 0.0 0.0 ~ 0.0 0.0 0.0 0.0 0.0)
    (0.0 0.0 0.0 0.0 0.0 ~ 0.0 0.0 0.0 0.0 0.0)
                     ...
    (0.0 0.0 0.0 0.0 0.0 ~ 0.0 0.0 0.0 0.0 0.0)
    (0.0 0.0 0.0 0.0 0.0 ~ 0.0 0.0 0.0 0.0 0.0)
    (0.0 0.0 0.0 0.0 0.0 ~ 0.0 0.0 0.0 0.0 0.0)
    (0.0 0.0 0.0 0.0 0.0 ~ 0.0 0.0 0.0 0.0 0.0)
    (0.0 0.0 0.0 0.0 0.0 ~ 0.0 0.0 0.0 0.0 0.0))
  :op #<PROCEEDNODE {10039C19D3}>
  :requires-grad NIL
  :variables (TID102887)
  :tracker #<TRACKER :order={row(0 1)} :shape=(30 30) :contiguous-p=T>}

Creating a computational graph (Lazy)

lisp
CATEN-USER> (!add (ax+b `(3 3) 0 1) (ax+b `(3 3) 0 1))
Result
Result
{Tensor[float32] :shape (3 3) :id STC102961
  :buffer nil
  :op #<ADD {1003A28F53}>
  :requires-grad NIL
  :variables (STC102953 STC102960)
  :tracker #<TRACKER :order={row(0 1)} :shape=(3 3) :contiguous-p=T>}

Evaluating a computational graph.

lisp
CATEN-USER> (proceed (!add (ax+b `(3 3) 0 1) (ax+b `(3 3) 0 1)))
Result
Result
{Tensor{LISPBUFFER}[float32] :shape (3 3) :id STC103126
   ((2.0 2.0 2.0)
    (2.0 2.0 2.0)
    (2.0 2.0 2.0))
  :op #<PROCEEDNODE {1003BAEBB3}>
  :requires-grad NIL
  :variables (STC102976)
  :tracker #<TRACKER :order={row(0 1)} :shape=(3 3) :contiguous-p=T>}

Floating Features

[function] inf

(inf &key (dtype *default-float*))
Returns positive infinity of the dtype for the current Common Lisp implementation.

This feature is supported by float-features

lisp
CATEN-USER> (inf)
Result
Result
#.SB-EXT:SINGLE-FLOAT-POSITIVE-INFINITY
lisp
CATEN-USER> (proceed (!full `(3 3) (inf)))
Result
Result
{Tensor{LISPBUFFER}[float32] :shape (3 3) :id STC103188
   ((#.SB-EXT:SINGLE-FLOAT-POSITIVE-INFINITY #.SB-EXT:SINGLE-FLOAT-POSITIVE-INFINITY #.SB-EXT:SINGLE-FLOAT-POSITIVE-INFINITY)
    (#.SB-EXT:SINGLE-FLOAT-POSITIVE-INFINITY #.SB-EXT:SINGLE-FLOAT-POSITIVE-INFINITY #.SB-EXT:SINGLE-FLOAT-POSITIVE-INFINITY)
    (#.SB-EXT:SINGLE-FLOAT-POSITIVE-INFINITY #.SB-EXT:SINGLE-FLOAT-POSITIVE-INFINITY #.SB-EXT:SINGLE-FLOAT-POSITIVE-INFINITY))
  :op #<PROCEEDNODE {1003BC6F23}>
  :requires-grad NIL
  :variables (TID103127)
  :tracker #<TRACKER :order={row(0 1)} :shape=(3 3) :contiguous-p=T>}

[function] -inf

(-inf &key (dtype *default-float*))

Returns negative infinity of the dtype for the current Common Lisp implementation.

This feature is supported by float-features

lisp
CATEN-USER> (-inf)
Result
Result
#.SB-EXT:SINGLE-FLOAT-NEGATIVE-INFINITY
lisp
CATEN-USER> (proceed (!full `(3 3) (-inf)))
Result
Result
{Tensor{LISPBUFFER}[float32] :shape (3 3) :id STC103250
   ((#.SB-EXT:SINGLE-FLOAT-NEGATIVE-INFINITY #.SB-EXT:SINGLE-FLOAT-NEGATIVE-INFINITY #.SB-EXT:SINGLE-FLOAT-NEGATIVE-INFINITY)
    (#.SB-EXT:SINGLE-FLOAT-NEGATIVE-INFINITY #.SB-EXT:SINGLE-FLOAT-NEGATIVE-INFINITY #.SB-EXT:SINGLE-FLOAT-NEGATIVE-INFINITY)
    (#.SB-EXT:SINGLE-FLOAT-NEGATIVE-INFINITY #.SB-EXT:SINGLE-FLOAT-NEGATIVE-INFINITY #.SB-EXT:SINGLE-FLOAT-NEGATIVE-INFINITY))
  :op #<PROCEEDNODE {1003BE7283}>
  :requires-grad NIL
  :variables (TID103189)
  :tracker #<TRACKER :order={row(0 1)} :shape=(3 3) :contiguous-p=T>}

[function] nan

(nan &key (dtype *default-float*))

Returns NaN of the dtype for the current Common Lisp implementation.

This feature is supported by float-features

lisp
CATEN-USER> (nan)
Result
Result
#<SINGLE-FLOAT quiet NaN>
lisp
CATEN-USER> (proceed (!full `(3 3) (nan)))
Result
Result
{Tensor{LISPBUFFER}[float32] :shape (3 3) :id STC103312
   ((#<SINGLE-FLOAT quiet NaN> #<SINGLE-FLOAT quiet NaN> #<SINGLE-FLOAT quiet NaN>)
    (#<SINGLE-FLOAT quiet NaN> #<SINGLE-FLOAT quiet NaN> #<SINGLE-FLOAT quiet NaN>)
    (#<SINGLE-FLOAT quiet NaN> #<SINGLE-FLOAT quiet NaN> #<SINGLE-FLOAT quiet NaN>))
  :op #<PROCEEDNODE {1003BFF5F3}>
  :requires-grad NIL
  :variables (TID103251)
  :tracker #<TRACKER :order={row(0 1)} :shape=(3 3) :contiguous-p=T>}

[function] float-type-of

(float-type-of x)

Returns :INF if the number is negative infinity, :-INF if the number is negative infinity, :nan if the number is NaN, or T otherwise.