arch_live_hist

@startuml title Architecture (Dual-Path: Live + History) top to bottom direction skinparam shadowing false skinparam componentStyle rectangle package "Producers" as P { [Apps/Services] } package "AlgoX2 Data Plane" as X2 { [Kafka-compatible Gateway] as GW [Sequencer] as SQ [Storage (NVMe/Object)] as ST [Materializer\n(Iceberg Writer)] as MAT } cloud "S3 / MinIO" as S3 database "Iceberg Tables" as ICE package "ClickHouse Cluster(s)" as CH { [Kafka Engine Tables] as CHK [MergeTree / ReplacingMergeTree] as CHMT [External Iceberg Reader] as CHI } ' Live path P --> GW : events (Kafka API) GW --> SQ : append SQ --> ST : commit ST --> CHK : consume (Kafka engine) ' History path ST --> MAT : window/notify MAT --> ICE : write snapshots ICE ..> S3 ' Query paths CHK --> CHMT : MV ingest → dedupe via x2_seq CHI --> ICE : historical scans ' Fan-out (multicast) GW -down-> [CH Cluster #2] GW -down-> [CH Cluster #3] note right of CHMT - Use ReplacingMergeTree(x2_seq) - idempotent producers - at-least-once → duplicate-safe end note note bottom of GW QoS, quotas, namespaces, ACLs, TLS Multicast → no per-consumer duplication end note @enduml