[Day 62] Stanford CS224N (NLP with DL): Transformers, Pretraining, RLHF
Hello :) Today is Day 62! A quick summary of today: Continuing with CS224N: Lecture 8 : Self-Attention and Transformers Lecture 9 : Pretraining Lecture 10 : Prompting, Reinforcement Learning from Human Feedback I watched this State of GPT talk by Andrej Karpathy a week or so ago and saw the below slide: And that was the first time I saw this GPT pipeline. Today, I got to learn about each of these steps. Feels nice knowing how GPT/ChatGPT works. Anyway ~ my notes are below :) Lecture 8: Self-Attention and Transformers Lecture 9: Pretraining Lecture 10: Prompting, Reinforcement Learning from Human Feedback Tomorrow we continue diving deeper into the world of NLP. That is all for today! See you tomorrow :)