DENIS IL.
Engineering Architectures

Architecture Library

Reusable data architecture patterns — from traditional analytics stacks to AI-native platforms. Each diagram shows a distinct engineering approach and its trade-offs.

Diagram 01

Traditional Analytics Stack

The foundational pattern: structured ETL pipelines move data from operational sources into a centralised DWH, where BI tools deliver reports. Reliable, well-understood, but batch-oriented and not AI-ready.

ETLDWHSQLBI Tools
SOURCE
Operational Sources
Databases · APIs · Files
extract
PIPELINE
ETL Pipeline
Extract · Transform · Load
load
WAREHOUSE
Data Warehouse
Structured · Governed · Centralised
query
REPORTING
BI & Reporting
Dashboards · Reports · Exports
Diagram 02

Modern Data Platform

The lakehouse pattern decouples storage from compute, enabling open formats like Apache Iceberg to serve both BI queries and AI workloads. A semantic layer adds governed metric access on top.

LakehouseApache IcebergdbtSemantic Layer
SOURCE
Data Sources
Streaming · Batch · APIs
ingest
WAREHOUSE
Lakehouse
Apache Iceberg · S3 · Open Formats
transform
DBT
dbt Transformations
Medallion Architecture · Metric Definitions
expose
SEMANTIC
Semantic Layer
Governed Metrics · Business Vocabulary
consume
REPORTING
Analytics & BI
Dashboards · Self-service · AI Queries
Diagram 03

AI-Native Analytics Platform

The next evolution: a governed data platform designed specifically for AI reasoning. The semantic layer becomes the contract between data and AI, enabling explainable, consistent, trustworthy answers.

Apache IcebergdbtSemantic LayerClaudeAI Analyst
SOURCE
Raw Data
Operational Systems · Event Streams
land
WAREHOUSE
Apache Iceberg Lakehouse
S3 · ACID · Time Travel · Schema Evolution
transform
DBT
DBT
Medallion Layers · Metric Contracts · Tests
govern
SEMANTIC
Semantic Layer
Business Context · Metric Definitions · NL Mappings
reason
AI AGENT
AI Analyst
Claude · Governed Queries · Explainable Answers
deliver
REPORTING
Business User
Natural Language · Trusted Metrics · Decisions
Diagram 04

Medallion Architecture

The dbt-enforced three-layer data model that separates raw ingestion from transformation from business logic. Raw preserves history. Staging cleans and types. Marts serve governed metrics.

dbtRawStagingMartData Modeling
SOURCE
Source Systems
Operational databases · APIs · Event streams
extract
WAREHOUSE
Raw Layer
Insert-only · Full history · No transformations
clean
DBT
Staging Layer
Typed · Deduplicated · No business logic
model
SEMANTIC
Mart Layer
Business logic · Governed metrics · KPI aggregations
serve
REPORTING
Consumers
BI tools · AI agents · Self-service queries
Diagram 05

Lakehouse Architecture

Storage decoupled from compute using Apache Iceberg on object storage. Open table format enables ACID transactions, schema evolution, and time-travel queries — serving both BI and AI workloads from the same data.

Apache IcebergS3LakehouseACIDTime Travel
SOURCE
Data Sources
Streaming · Batch · CDC · APIs
land
WAREHOUSE
Object Storage (S3)
Raw files · Parquet · Scalable · Cost-efficient
catalog
WAREHOUSE
Apache Iceberg
ACID transactions · Schema evolution · Time travel
transform
DBT
dbt Transformation
Medallion layers · Metric definitions · Tests
expose
SEMANTIC
Query Engine
Spark · Trino · Athena · DuckDB
serve
REPORTING
Consumers
BI dashboards · AI analysts · Ad-hoc queries
Diagram 06

AI Agent over DWH

The autonomous analytics pattern: a scheduled AI agent monitors governed metrics, detects anomalies, generates natural language insights, and escalates to human review only when thresholds are crossed.

AI AgentSemantic LayerClaudeAutonomous Analytics
WAREHOUSE
Data Warehouse
Governed mart tables · Tested metrics · Full history
query via
SEMANTIC
Semantic Layer
Metric definitions · Business context · NL mappings
reason
AI AGENT
AI Analyst Agent
Claude · Anomaly detection · Insight generation
schedule
PIPELINE
Orchestration
Airflow · Scheduled runs · Retry logic
deliver
REPORTING
Decision Maker
Insights · Alerts · Recommended actions