[Day 39] Reading papers of powerful CNN models + going back to basics (+ some more Andrej Karpathy lectures)
Hello Today is day 39! Read a few research papers Watched Stanford's CS231n Winter 2016 lectures delivered by Andrej Karpathy Drew a nice "poster" for my wall thanks to Samson Zhang First I read about some regularization technique that I saw on paperswithcode called - Label Smoothing . I was curious what other options are there besides Dropout and L2, and that was amongst the top in the regularization category. It involves smoothing the one-hot encoded target labels by replacing some of the elements with values slightly different from 0 and 1. For example, in a classification task with 5 classes, a sample belonging to class 3 would have the target label [0, 0, 1, 0, 0]. Instead of using hard 0s and 1s in the one-hot encoded vectors, label smoothing replaces them with slightly adjusted values. For example, instead of [0, 0, 1, 0, 0], the smoothed label for the same sample might be [0.05, 0.05, 0.85, 0.05, 0.05]. It helps with overfitting and does not allow our model t