[Day 8] 시계열 - Trend 모델 구성
안녕하세요!
오늘은 Day 8입니다!
오늘은 네팔에서 2013년부터 2021년까지 채소 및 과일(Provisions)의 가격에 대한 기본적인 Trend 모델을 만들어보았다
데이터는
전처리 한 뒤 시각화 만들었다
Trend 모델 구성한 후, Trend 예측은
오늘 구성한 모델도 간단한데 연습하는 데 도움이 되었다.
오늘 이상입니다
내일 뵐게요 ^^
[Day 198] Transactions Data Streaming Pipeline Porject [v1 completed]
Hello :) Today is Day 198! A quick summary of today: data streaming pipeline project [v1 done] Here is a link to the project's repo. Well ... I did not know I can do it in a day (~14 hours) after yesterday's issues but here we are. Turns out in order to insert the full (~70 variables with nested/list structure), I need the proper pyspark schema. And yesterday I did not have that and that is why when I was reading data in the kafka producer I was getting NULL in the columns - my schema was wrong. Well today I not only fixed the schema for the 4 variables I had yesterday, but included *all* the variables that come from the Stripe API ~ 70 (for completeness). When I run docker-compose, the data streams and is input into the postgres db (and is still running). Unfortunately, the free Stripe API for creating realistic transactions has a limit of 25, so every 3 seconds, 25 new transactions are sent to the db. It has been running half the day (since I got that set up) and as I am wr
