Comprehensive Guide To Tensors

Tensors are like containers that are used in holding and manipulating numerical data.

In TensorFlow all computations involve tensors, therefore, having a basic understanding of tensors makes the deep learning journey much easier.

What are tensors?

Tensors are multi-dimensional arrays of numeric values. Tensor is a generalization of matrices which can have any number of dimensions such as scalar, and vector among others.

A proper understanding of tensors helps one to be able to work with deep learning frameworks such as PyTorch and TensorFlow.

Tensor vs Matrix: What's the Difference?

People may be confused about the difference between a tensor and a matrix. The diagram below explains a bit the difference between a tensor, matrix, vector and scalar.

A matrix is a 2D array which comprises rows and columns which means it has two dimensions while a tensor is more general such that it encompasses scalar, vectors, matrices and higher dimension structures. Let's have a look at the picture below which explains the relationship between a tensor and a matrix

Types of tensors

There are several types of tensors which include

Zero dimensional (0D) tensor

It is also called a scalar. It is a tensor that represents a single numeric value, therefore it has no shape, that is it has no rows and columns.

The output shows that zero_d tensors have no shape, that is it has no rows and columns.

One dimensional (1D) tensor

It is also called a vector which is made up of a combination of zero-dimensional tensors, it is an array of numerical values along a single dimension.

Two dimensional(2D) tensor

It is also called a matrix, it is a 2D matrix of numeric values with rows

And columns

The output shows that there are three rows and three columns

shape(3,3))

Three dimension(3D) tensor

it is a three dimensional array that represents a data cube

A data cube is said to have length, width and height, which represent a three dimension tensor

High dimensional tensor

It is a type of a tensor that can have more than three dimensions which can be used in representing data with complex structures

There are other types non_standard tensors or specialized tensor which include

Ragged tensors
Sparse tensors
String tensors

TensorFlow mathematical operations

TensorFlow can perform complex mathematical computations that are applicable in deep learning and machine learning applications. There are so many TensorFlow mathematical operations, in this article, I will cover some of the operations used in TensorFlow.

tf.math.abs: It is a TensorFlow math operation that computes the absolute value of a tensor. That is if there is a negative value it returns it as positive.
tf.math.add: it is an operation that add multiple tensors
tf.math.argmax: an operation returns the index with the largest value across axes of a tensor. When the axis is equal to one, the operation is done column-wise; when the axis is equal to zero operation is done row-wise.
tf.math.maximum : The operation returns the maximum value of two tensors by comparing the tensors element-wise, the maximum operation and the argmax may seem to be similar but they differ in terms of output where the argmax returns the index while the maximum operation returns the value of a tensor.
tf.math.minimum:

It returns the minimum value of two tensors, that is it is the opposite of the maximum operation.
tf .math.power:

The operation returns computes the power of one value to another.

Common TensorFlow functions

TensorFlow provides various functions that are used in developing, training and optimizing models. In this article, i will cover common TensorFlow functions that are used in the development of neural network architectures

tf.expand_dims:

It is a function that adds an extra axis to the input tensor, where the extra axis has a length of one.
tf.squeeze:

It is the reverse of the expand_dims function, where the function reduces the dimension of size one from the shape of a tensor.
tf.reshape:

it is a function that reshapes a tensor, from its original shape to the specified shape
tf.concat: It concatenates tensor along one dimension. Tensors can be concatenated row wise or column-wise, when the axis is equal to one, the concatenation will be column wise and when the axis is equal to zero the concatenation is row wise.
tf.stack:

It works as almost the same as the concat function but the difference is that it does not stack tensors in axis unlike in concatenation.
tf.pads:

It is a function that pads a tensor where by default TensorFlow pads with zeros,, however, one can specify the constant value that you need to use for padding.

Specifying the constant value for padding as 3.

Conclusion

Tensors play a crucial role in TensorFlow, they help in performing complex computations through TensorFlow mathematical operations and linear algebra operations and offer a wide variety of functions which help in solving problems in the world of deep learning.

Comprehensive Guide To Tensors And Fundamental Operations For Deep Learning Mastery