Dataengineering

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Cover image for Building a Football Analytics Pipeline: Patterns, Tradeoffs, and What Production Would Look Like

ayoabass777

Apr 12

Building a Football Analytics Pipeline: Patterns, Tradeoffs, and What Production Would Look Like

#dataengineering #python #aws #dbt

9 min read

Mingyu Chen

Apr 12

How Hard Is It to Add an Index to an Open Format? Lessons from the Apache Iceberg Community

#architecture #database #dataengineering #opensource

16 min read

Cover image for The Data Engineering Lifecycle

Mirina-Gonzales

Apr 11

The Data Engineering Lifecycle

#architecture #data #dataengineering

3 min read

Blaine Elliott

Apr 11

You Don't Need to Write Data Tests

#dataengineering #dataquality

8 min read

Blaine Elliott

Apr 11

Data Anomaly Detection: The Complete Guide for Data Engineers

#dataengineering #dataquality

7 min read

Cover image for El Ciclo de Vida de la Ingeniería de Datos

Mirina-Gonzales

Apr 11

El Ciclo de Vida de la Ingeniería de Datos

#dataengineering #data #architecture

4 min read

Cover image for Why Shannon Entropy Catches What Schema Validation Misses

Anthony Johnson II

Apr 11

Why Shannon Entropy Catches What Schema Validation Misses

#dataquality #databricks #dataengineering #opensource

5 min read

Philip Hern

Apr 10

dbt snapshots: moving from merges to native history

#dbt #dataengineering #snowflake #snapshots

5 min read

Nyson Markus

Apr 10

PySpark to Pandas/scikit-learn: A Practical Migration Guide for Data Engineers Learning ML

#dataengineering #datascience #machinelearning #python

7 min read

Cover image for Apache Parquet File Anatomy: Row Groups, Column Chunks, Pages, and Metadata Explained 🧱📦

Kumaravelu Saraboji Mahalingam

Apr 10

Apache Parquet File Anatomy: Row Groups, Column Chunks, Pages, and Metadata Explained 🧱📦

#dataengineering #apacheparquet #iceberg #analytics

8 min read

Cover image for ETL vs ELT: Which One Should You Use and Why?

Lawrence Murithi

Apr 11

ETL vs ELT: Which One Should You Use and Why?

#architecture #data #database #dataengineering

7 min read

Ashish Srivastava

Apr 10

🚀 DB Explorer 3.0.1 — The AI‑First SQL Editor You’ll Want to Try

#sql #database #postgres #dataengineering

1 min read

Ajay M

Apr 10

My first data pipeline

#showdev #beginners #dataengineering #sideprojects

1 min read

John Wakaba

Apr 10

ETL vs ELT: Which One Should You Use and Why?

#architecture #beginners #data #dataengineering

6 min read

Daniel Rozin

Apr 10

Entity Resolution at Scale: Matching Products Across Amazon, Reddit, and RTINGS

#ai #webdev #dataengineering #tutorial

4 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Forem

# dataengineering

Building a Football Analytics Pipeline: Patterns, Tradeoffs, and What Production Would Look Like

How Hard Is It to Add an Index to an Open Format? Lessons from the Apache Iceberg Community

The Data Engineering Lifecycle

You Don't Need to Write Data Tests

Data Anomaly Detection: The Complete Guide for Data Engineers

El Ciclo de Vida de la Ingeniería de Datos

Why Shannon Entropy Catches What Schema Validation Misses

dbt snapshots: moving from merges to native history

PySpark to Pandas/scikit-learn: A Practical Migration Guide for Data Engineers Learning ML

Apache Parquet File Anatomy: Row Groups, Column Chunks, Pages, and Metadata Explained 🧱📦

ETL vs ELT: Which One Should You Use and Why?

🚀 DB Explorer 3.0.1 — The AI‑First SQL Editor You’ll Want to Try

My first data pipeline

ETL vs ELT: Which One Should You Use and Why?

Entity Resolution at Scale: Matching Products Across Amazon, Reddit, and RTINGS