What is the vanishing gradient problem?
When gradients become extremely small as they travel backwards, preventing early layers from updating their weights and learning.
Verify This Answer
Cross-check this information using these trusted sources:
More FAQs in Understanding Backpropagation and Gradient Descent Simply
A hyperparameter that controls how much we adjust the model weights in response to the estimated error each time.
The algorithm might overshoot the minimum error point and fail to converge, leading to unstable training.
A variation of gradient descent that updates model weights using only a single training example or mini-batch at a time, speeding up updates.
Still have questions?
Browse all our FAQs or reach out to our support team