ingestion_materializationΒΆ

@startuml ' Explicitly enforce sequence diagram !define RECTANGLE class skinparam monochrome true skinparam shadowing false title Ingestion & Materialization Sequence !pragma layout smetana ' top to bottom direction actor Producer as P participant "X2 Gateway" as GW participant "X2 Sequencer" as SQ participant "X2 Storage" as ST participant "X2 Materializer" as MAT participant "CH Kafka Engine Table" as CHK database "CH MergeTree\n(ReplacingMergeTree x2_seq)" as CHMT database "Iceberg on S3/MinIO" as ICE participant "CH Iceberg Reader" as CHI P -> GW: Kafka Produce (records) GW -> SQ: Append (records) SQ -> ST: Persist + Replicate ST -> CHK: Stream (Kafka pull) CHK -> CHMT: MV INSERT (dedupe by x2_seq) ST -> MAT: Notify window/watermark MAT -> ICE: Write snapshot + manifests CHI -> ICE: SELECT ... (historical) @enduml