50+ days of Machine Learning

 Hello :) Today is Day 115! A quick summary of today: submitted 3rd homework of XCS224W:ML with Graphs explored different huggingface capabilities with DeepLearning.AI As for the homework , we are not allowed to share anything from it. But I can happily share I got full marks ^^ As for the huggingface tutorial It showcased the different type of models that were available. Below is a summary.  Building a chat pipeline Text translation Text summarization Zero-shot audio classifier Apparently, the model seens the audio differently -  1 second of high resolution audio appears to the model as if it is 12 seconds of audio. Text to speech Object detection (code before the pic: od_pipe = pipeline("object-detection", "./models/facebook/detr-resnet-50")) We can use gradio as a sample interface I passed a picture of mine to check haha We can also get natural language descriptions Image captioning Example image Using the dog and woman pic again for multimodal QA Zero-shot image...

 Hello :) Today is Day 69! A quick summary of today: tried to build upon the built LLM from the book from the days before, and write a training loop in the hopes of generating some Harry Potter text ( kaggle notebook ) Firstly I will provide pictures of the implementation (then share my journey today) The built transformer is based on this configuration Dataset is Harry Potter book 1 (Harry Potter and the Philosopher's Stone) text file from kaggle. Used batch size 16, and a train:valid ratio 9:1 Model architecture code: Multi-head attention GELU activation function Feed forward network Layer normalization Transformer block GPT model (+generate function) Optimizer: AdamW with learning rate 5e-4 and 0.01 weight decay. Ran for 1000 epochs.  Final number of model parameters: 1,622,208 million Now, about my journey today ~ The easy part of was the dataset choice - harry potter. The hard part was choosing hyperparams that would end up in not perfect, but somewhat readable and s...

Hello :) Today is Day 68! A quick summary of today:  Covered chapter 4 of Build a LLM from scratch by Sebastian Raschka Below is an overview of the content with not much code. For the full code version of every step - it is on this github repo . This chapter is the 3rd and final step from the 1st state towards a LLM 4.1 Coding a LLM architecture The book will build the smallest version of GPT-2 that has 124m parameters, with the below config. The final architecture is a combination of a few steps, presented below After creating and initializing the model, and the gpt-2 tokenizer, on a batch of 2 sentences: 'Every effort moves you' and 'Every day holds a', the output is: 4.2 Normalizing activations with layer normalization Taking an example without layer norm The mean and var are Apply layer norm, and the data has 0 mean and unit variance. We can put the code in a proper class to be used later for the GPT model 4.3 Implementing a feed forward network with GELU activation...