Posts

Showing posts from July 4, 2024

[Day 185] Using prefect as my orchestrator for my MLOps project

Image
 Hello :) Today is Day 185! A quick summary of today: creating a flow for model training creating a flow for uploading data to GCS Well, I decided to go with Prefect as my orchestrator. I have little experience using it and I liked it, so went with it.  All the code from today is on my repo . As seen from the pic, I ran jobs many times and have many successes and failures.  The first flow (as they are called in prefect) I created is for model training.  This is an example visualisation of the flow from one of the runs: It contains 4 tasks: read data (not from GCS, I need to make it read from GCS) split data get best params for the model from mlflow train the model, and produce reports on it The reports are quite nice as we can use markdown and python f-strings together to make them: Model report: MLflow info: Deploy instructions: Instead of going to mlflow to get info, I tried to make it a bit easier and clearer on how to get a model, and also showing basic performance stats.  I am usi