[Day 143] Forward, backward prop and param update by hand
Hello :) Today is Day 143! A quick summary of today: did forward and backprop of a toy neural net by hand and confirmed with python set up my station at the student lab at uni The idea of doing backprop manually again came back yesterday night. But this time I wanted to do it 'as in the old days'. In one of the lectures of CS109 Professor Chris Piech said he used to do forward/backprop manually back before autograd, so I wanted to experience that as well (in a toy example). Below are my handwritten notes of the forward, backprop and param update for the 1st iteration of a toy neural net. My tablet battery died towards the end so I wrote it on the board. I confirmed all the numbers with python - the used notebook is here . As for my 1st day at the lab I liked it. It was mostly waiting for downloads and setting up cuda/torch/python/etc because it was a new PC. There was a seminar where 2 of the students talked a bit about what they are doing. At the end of the day, I had a cha