[Day 123] Optimization algorithms chapter from Dive into DL
Hello :) Today is Day 123! A quick summary of today: read about optimization algorithms, and maths for DL from Dive into Deep Learning Firstly, my notes from the optimization algorithms chapter Covered topics: Common challenges in DL optimization, convexity, convexity properties, SGD, momentum, Adagrad, RMSProp, Adadelta, Adam, LR schedulers As for the maths for DL This is actually in the apendix, but I decided to read through it, just to check if there is any interesting math. The content is as follows: Having this appendix is very nice and definitely helps to answer questions like for example when I went to the roots of multicollinearity (on Day 121 ) - understanding the math helps understand the root of problems. As for the MLx Fundamentals it started just as I am finishing this blog. The first day is from 9pm to 3am in my local time, so I will write about it in tomorrow's blog. Schedule (time is in BST) That is all for today! See you tomorrow :)