denis256 notes and projects

Previous: Data Preparation and Orchestration

Book: AWS Certified Data Engineer Associate Study Guide Authors: Sakti Mishra, Dylan Qu, Anusha Challa Publisher: O’Reilly Media ISBN: 978-1-098-17007-3

Chapter 5 is where the book gets into data store management. Domain 2 territory on the exam, and a big one. How do you pick the right storage? What file format should you use? How do you keep your S3 bill from growing out of control?

The chapter title is “The Utopia of Greed” and it is pure irony. What Dagny finds in the valley is the exact opposite of what “greed” looks like in the outside world. No politicians skimming off the top. No bureaucrats deciding who gets what. Just people doing honest work and trading the results fairly.

Everyone Works, Everyone Contributes

The morning after her crash landing, Dagny wakes up in Galt’s house. He is already up, heading to the powerhouse because her crash knocked the ray screen off key. Tells her he will cook breakfast when he gets back. The man who built a motor that could change the world – fixing power lines at dawn and making eggs.

Previous: Data Transformation

Book: AWS Certified Data Engineer Associate Study Guide Authors: Sakti Mishra, Dylan Qu, Anusha Challa Publisher: O’Reilly Media ISBN: 978-1-098-17007-3

Final part of Chapter 4. We move from raw transformation into two topics: data preparation for people who don’t write code, and orchestrating the whole pipeline end to end. Both matter for the exam. Both matter in real life.

Data Preparation for Nontechnical Personas

AWS Glue DataBrew is a low-code, visual tool for data cleaning and preparation. It targets data analysts, data scientists, and business users who need to work with data but don’t want to write PySpark or SQL.

Previous: Part II, Chapter 10 - The Sign of the Dollar

Part III begins. The section is called “A Is A” and we are finally inside the hidden valley. After twenty chapters of watching the world fall apart, we get to see what the people who left have been building instead. Honestly, it reads like a startup pitch deck written by someone who really, really believes in it.

Waking Up in Another World

Dagny crashed her plane chasing the mystery man’s aircraft into the mountains. She wakes up in a green valley, sunlight on her face, looking up at a stranger. Rand spends a long paragraph describing this man’s face and body in almost absurd detail. Metal-green eyes, aluminum-copper skin, hair like liquid gold. Most over-the-top character introduction in the entire book.

Previous: Data Ingestion Patterns

Book: AWS Certified Data Engineer Associate Study Guide Authors: Sakti Mishra, Dylan Qu, Anusha Challa Publisher: O’Reilly Media ISBN: 978-1-098-17007-3

Second part of Chapter 4, covering data transformation. The first part was about ingestion. Now we look at what happens after the data lands. You need to clean it, reshape it, enrich it, and get it into a format that analysts and applications can actually use.