Gradient Descent :
It is an optimization technique. It is the most commonly used optimization technique when dealing with machine learning.
Two mainly classified methods on the basis of data ingestion are as follows:
- Full Batch Gradient Descent Algorithm (FBGDA)
- Stochastic Gradient Descent Algorithm (SGDA)
Full Batch Gradient Descent Algorithm:
In it we take whole data at once to compute the gradient descent.
Stochastic Gradient Descent Algorithm:
In it we take a sample while computing the gradient descent.
Challenges in Executing Gradient Descent :
- Data Challenges
- Gradient Challenges
- Implementation Challenges
Variants of Gradient Descent Algorithms:
Most commonly used gradient descent algorithms and their implementations.
- Vanilla Gradient Descent
- Simplest Form of Gradient Descent