One of the most important parts of Artificial Neural Networks is minimizing the loss functions which tells us how good or bad our model is. To minimize these losses we need to tune the weights and biases. Also to calculate the minimum value of a function we need gradient. And to update our weights we need gradient descent. But there are some problems with regular gradient descent ie. it is quite slow and not that accurate. This article aims to give an introduction to optimization strategies to gradient descent. In addition, we shall also discuss the architecture of these algorithms and further optimization of Neural Networks in general

翻译：人工神经网络最重要的部分之一是 将损失函数最小化, 这告诉我们模型是好还是坏。 为了将损失最小化, 我们需要调整重量和偏差。 同时, 为了计算一个我们需要梯度的函数的最小值, 我们需要梯度。 并且更新我们的权重。 但是, 正常的梯度下降存在一些问题 。 它相当缓慢, 并不准确 。 文章旨在引入一种优化梯度下降战略 。 此外, 我们还要讨论这些算法的架构, 并进一步优化一般的神经网络 。