Dl4j学习率衰减策略.pdf资源-CSDN下载

需积分: 10 42 浏览量 2019-07-26 10:34:07 上传评论收藏 603KB PDF 举报

资源推荐

资源详情

资源评论

Dl4j学习率衰减策略

1. package org.deeplearning4j.nn.conf;

2.

3. /**

4. * Learning Rate Policy

5. *

6. * How to decay learning rate during training.

7. *

8. * <p><b>None</b> = do not apply decay policy aka fixed in Caffe <br>

9. * <p><b>Exponential</b> = applies decay rate to the power of the # batc

hes <br>

10. * <p><b>Inverse</b> = divide learning rate by negative (1 + decay rate

* # batches)^power <br>

11. * <p><b>Poly</b> = polynomial decay that hits 0 when iterations are com

plete <br>

12. * <p><b>Sigmoid</b> = sigmoid decay rate <br>

13. * <p><b>Step</b> = decay rate to the power of the floor (nearest

integer) of # of batches by # of steps <br>

14. * <p><b>Schedule</b> = rate to use at a specific iteration <br>

15. * <p><b>Score</b> = apply decay when score stops improving <br>

16. */

17.

18. // TODO provide options using epochs in addition to iterations

19.

20. public enum LearningRatePolicy {

21. None, Exponential, Inverse, Poly, Sigmoid, Step, TorchStep, Schedule,

Score

22. }

dl4j的学习率衰减策略应用部分是在反向传播计算完地图之后，调用Updater.update()方法对梯

度进行更新并且进行梯度的衰减。

调用学习率衰减的为 package org.deeplearning4j.optimize.solvers 包下

的 BaseOptimizer 抽象类中

的 updateGradientAccordingToParams(Gradient gradient, Model model, int batchSize) 方

法中的 updater.update(layer, gradient, getIterationCount(model), batchSize); 语

句。

本内容试读结束，登录后可阅读更多

下载后可阅读完整内容，剩余4页未读，立即下载

内容反馈

Jayxp

粉丝: 6

最新资源

资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！点击此处反馈

feedback-tip