50+ days of Machine Learning

 Hello :) Today is Day 78! A quick summary of today: Out of the 4 sessions I joined from NVIDIA GTC, the AI careers in Europe one was the most interesting my application and thoughts about Stanford's AI professional certificate program  Research paper about PERL: Parameter Efficient Reinforcement Learning from Human Feedback The 4 sessions that I had marked to join today were 1. Retrieval Augmented Generation: Overview of Design Systems, Data, and Customization 2. Navigating AI Careers in Europe 3. Customizing Foundation Large Language Models in Diverse Languages With NVIDIA NeMo 4. LLM Inference Sizing: Benchmarking End-to-End Inference Systems But besides the 2nd one about AI careers in Europe, the other did not manage to catch my interest. They were talking about research on RAGs, how many GPUs are needed to do LLM inference, based on NeMo which I had not used up until now. Nevertheless, the 2nd talk was very good because I could ask questions to an HR r...

 Hello :) Today is Day 77! A quick summary of today: I wanted to review all the content again from the ACL 2023 talk on retrieval-based models (all my notes can be found in the posts from the last 3 days) Also decided to cover lecture 10 : Retrival and RAG from CMU's 11-711 Advanced NLP course RAG seems to be an important topic these days, and recently I tried Gemini for the first time too, and on some topics it does give the source, so it is kind of cool and 'hot' in the LLM sphere, and I just read over my notes one more time to clear up any spots. In regards to lecture 10 on retrieval and RAG by professor Neubig, below are my notes (some content was covered twice, so I am just pointing to the appropriate section of my notes from the ACL talk that covers the particular topic in depth) Tomorrow is the NVIDIA conference. So I am looking forward to that ^^ That is all for today! See you tomorrow :)

 Hello :) Today is Day 76! Finished section 6 and 7 about multilingual retrieval-based LMs and retrieval-based LMs' challenges and opportunities ( ACL 2023 ) Covered  lecture 11  of CMU 11-711 Advanced NLP - Distillation, Quantization, and Pruning Section 6: Section 7: Lecture 11: Distillation, Quantization, and Pruning Problem: The best models for NLP tasks are massive. So how can we cheaply, efficiently and equitably deploy NLP systems at the expense of performance? Answer: Model compression Quantization -  keep the model the same but reduce the number of bits Pruning -  remove parts of the model while retaining performance Distillation -  train a smaller model to imitate the larger model Quantization - no parameters are changed, up to k bits of precision Amongst other methods, we can use post-training quantization We can binarize the parameters and activations  Pruning - a number of parameters are set to zero, the rest are unchanged There is Ma...

 Hello :) Today is Day 75! A quick summary of today: covered Section 4: retrieval-based LMs training of ACL 2023 covered Section 5: Applications Found out about NVIDIA GTC 2024 which is next week and registered for some of the events Firstly, as for the NVIDIA GTC courses that I registered for over the course of the conference: There is also a paid workshop on Building Transformer-Based Natural Language Processing Applications but it is sold out, and even though I have joined the waiting list, my hopes are not high haha. As for my notes on section 4 and 5 of the retrieval-based LM talk Section 4: Retrieval-based LMs: Training Section 5: Retrieval-based LMs: Applications There is ~30mins left, which I skimmed through and are not that long, but will cover tomorrow :) That is all for today! See you tomorrow :)