Jorge Valle

Jorge Valle

Jorge Valle is a front end developer with a particular passion for, and expertise in, JavaScript and user interfaces. Lately, he's also been diving into machine learning.

The difference between rank and shape in TensorFlow.js

When beginning to use TensorFlow.js, specially without a strong background in linear algebra, it might be hard to appreciate the difference between ranks and shapes. To help clarify the confusion, for myself and others, I put together this short reference.

Rank refers to dimensions

In TensorFlow.js and in linear algebra, the rank of a tensor refers to the number of dimensions of its vector space.

For example, let's take a scalar: the number $3$. A scalar is a single numeric value, but because the concept of a tensor is very generalized, a scalar could also be described as a tensor of rank 0. Said another way, a scalar has no dimensions.

Contrast that to a vector, which has one dimension. That dimension can have two values, or it can have thousands. Regardless of the size of the dimension, the dimension remains one, so vectors can also be described as tensors of rank 1.

Shape refers to dimensions and size

In TensorFlow.js and in linear algebra, the shape of a tensor refers to both the number of dimensions, and the size of each dimension. Therefore, if you know the shape of the tensor, you can infer its rank.

Vectors, for example, have one dimension. They have a shape of $[n]$, where $n$ is the number of values (size, or cardinality) of the dimension.

Matrices, on the other hand, have two dimensions. They have a shape of $[n, m]$, where $n$ and $m$ are the number of values in each dimension.

References for different spaces

Here's a quick reference section for all of the lower-dimensionality spaces. Please note that, as implemented in the TensorFlow docs, I'll be using the verbose parameter in the print function to print out the shape, values and rank.

Scalars

A scalar is a single numeric value. It has no dimensions, so it has a rank of 0. Because it has no dimensions, it also has no shape.

$$ 3 $$
Figure 1: A scalar represented mathematically.

Scalars are instantiated in TensorFlow.js as follows.

Vectors

A vector is a data structure with one dimension. It has a rank of 1, and a shape of [n], where n is the size (number of values) in the dimension. Vectors can be column vectors or row vectors: regardless, they only have one dimension.

A vector is also a tensor of rank 1.

$$ \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix} \\ \begin{bmatrix} 1 & 2 & 3 \end{bmatrix} $$
Figure 2: A column vector, and a row vector.

The equivalent declaration in TensorFlow is as follows.

Matrices

A matrix is a data type of two dimensions, and as such can also be described as a rank 2 tensor. It has a shape of [n, m].

$$ \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} $$
Figure 3: A matrix represented mathematically.

It's equivalent to the following in TensorFlow.js.

Tensors

For the purposes here, I will use the term tensor when talking about any space larger than 2 dimensions, although as stated previosuly, it would be accurate to describe a scalar as a rank 0 tensor.

Notice how the rank corresponds to the number of dimensions, and the shapes detail the number of values in each rank.


Books, patrons and coffee in Caffè San Marco, Trieste