Posts

Showing posts from June 21, 2024

[Day 171] Data engineering zoomcamp by DataTalksClub

Image
 Hello :) Today is Day 171! A quick summary of today: started Data engineering zoomcamp by DataTalksClub completed IEUK by Bright Network Firstly, about the data engineering zoomcamp I know they deliver good content, and ends up with a solo project that will be a great addition to my portfolio.  This is the camp overview: Tons of new things to learn, some old friends (in mage.ai), kafka, dlt, spark, dbt. It was a no brainer decision. So I guess at the moment on different days of the week (I need to set up a schedule) I am taking their data engineering camp, their MLOps camp, and LLM camp.  As for covered material for today~ Learned about docker, and using postgres on docker.  Created this dockerfile And then below are some of the ran comands and steps covered. Start the postgres server on docker And access it through the terminal pgcli -h localhost -p 5432 -u root -d ny_taxi Then, using python, I uploaded data and I could query it from the terminal where I ran pgclie For better user ex