PyTorch : A Deep Learning Framework

Part 1 — What are Tensors and Gradients?

PyTorch is a open source, deep learning framework developed by Facebook.

INSTALLATION OF PYTORCH

The installation of PyTorch Package is done either through pip manager or conda command.

I would recommend using Google Colab as our IDE . PyTorch can be used by directly importing torch package

import torch
# importing pytorch library in Google Colab
# pip install torch===1.5.0 torchvision===0.6.0 -f https://download.pytorch.org/whl/torch_stable.html
# Through pip installation
# !conda install pytorch cpuonly -c pytorch -y
# Through Conda Installation

WHAT IS A TENSOR?

The main element of PyTorch is PyTorch Tensor.

Tensor is a type of data structure in Linear Algebra represented in the form of multi-dimensional array. It can be a scalar, vector , matrix or any n dimension array.

Whenever a library is imported in Python it is treated as an object. Object has 2 important features.

1. Methods
2. Attributes

NOTE: To define a PyTorch tensor, torch.tensor() method is used and .ndim, .dtype are some of its attributes.

DEFINING A 0D TENSOR — ONLY A NUMBER

Fig 2 : Zero Dimension Tensor(Only a Single Number)

Scalar consists of a single value. It is then printed to view the tensor. Also, ndim and dtype attributes are used to check the dimension of the tensor and the datatype of that respective tensor

Fig 3: One Dimension Tensor(Vector)

One Dimension Tensor can also be called as a Vector. It has got one dimension.

Fig 4: Accessing the values of Tensor

The contents of the Tensor can be accessed just like how we access the values of a list.

Fig 5: 2 Dimension Tensor(Matrix)

Fig 6 : Comparison of shape attribute between 0D, 1 D and 2 D Tensor

Shape of N — Dimension Array

Here the shape of the Tensor is (4,2,3). Lets break this torch.size value.

4 indicates the total number of values in the outer dimension.

2 indicates the total number of values in 2nd Dimension. In this case it mentions the number of rows in the inner matrix.

3 indicates the number of values in the inner dimesion. In this case it is the number of columns in the inner matrix.

So to conclude, the tensor consists of totally 4 matrices with 2 rows and 3 columns.

.requires_grad = True

We need to calculate the partial derivative on a respective tensor to change the values of weights and bias in Backpropogation method. Now, to calculate the derivative of tensors with large number of dimension is impractical to calculate. Hence, we use the parameter .requires_grad to find the gradient.

Fig : Parameter for calculating Gradient of a Tensor

NOTE: Please make sure that the dtype of Tensor is floating point for calculating Gradients.

HOW TO PRINT THE VALUES OF GRADIENTS?

In Neural Network, we often come across the terms weights and bias. They can be compared to slope and co-efficient value in an equation of a straight line. We basically multiply weight with the independent value or input and and the combined value with the bias.

Let us take 3 sensors names input_value, weight_value and bias_value with scalar value

input_value = torch.tensor(2.3,requires_grad=True)
weight_value = torch.tensor(2.0,requires_grad=True)
bias_value = torch.tensor(0.45,requires_grad=True)

Lets calculate the output by taking product of weight with input and adding the combined value with bias as in the case of equation of straight line.

output_value = weight_value*input_value+bias_value
print(output_value)

Fig : Use Backward method to display gradients

The output_value is as shown, but to print the gradients of each value we have call for backward method. This can be written as

output_value.backward()

Now, time to print the output of gradients.

# Display gradients
print('dy/dx:', input_value.grad)
print('dy/dw:', weight_value.grad)
print('dy/db:', bias.grad)

In the next part, we will discuss few interesting methods of PyTorch library.