#data-engineering

Articles tagged with #data-engineering

The Moment I Realised a Database Is Not a Data Warehouse
This article is part of a series documenting the development of a real-time crypto analytics platform.
Jun 11, 20266 min read
Building a Real-Time Crypto Analytics Platform
From Streaming Pipeline to Analytics Platform When I started learning data engineering, I wanted a project that would force me to use the technologies I was studying in a realistic setting. Tutorials
Jun 10, 20266 min read16
Week 1: Building the Dataset (Harder Than Expected)
We’re still figuring out what this project is going to become. I’m working with Zoe (data scientist), and my role is to build the dataset — daily stock data for a selection of energy companies so we c
Apr 19, 20266 min read5
I Thought I Understood Databases — Then I Built One Properly
I thought I understood databases. I had written SQL queries, built tables, and used them in projects. But everything I knew was limited to querying data - not managing it. So I decided to build a comp
Apr 15, 20265 min read13
Building an ETL Pipeline in Bash (and What I Learned)
As part of my data engineering learning, I built a small project to understand how data pipelines work end-to-end using only Bash and standard Unix tools. The idea was simple: collect weather data for
Apr 8, 20265 min read19