high-level_architecture_components

@startuml skinparam componentStyle rectangle skinparam shadowing false skinparam ArrowColor #888 title AlgoX2 ↔ Tracto/Nebius — High-Level Architecture package "Producers / Upstream Apps" { [Producers\n(Kafka/HTTP/NATS/MQTT)] } package "AlgoX2 Data Plane" { [X2 Gateways\n(Kafka API, NATS, MQTT, Redis Streams)] [X2 Sequencers\n(ordering, replication)] [X2 Storage/Commit\n(journals, policies)] [X2 Materializer\n(Iceberg/Delta/Hudi)] } package "Object Storage\n(S3/MinIO)" as OBJ package "Tracto / Nebius" { [Tracto Notebooks\n(Python, HF, etc.)] [Spark/SPYT Jobs] [CHYT/ClickHouse SQL] [Workflows/Orchestrator] [Metadata/Catalog] } [Producers\n(Kafka/HTTP/NATS/MQTT)] --> [X2 Gateways\n(Kafka API, NATS, MQTT, Redis Streams)] [X2 Gateways\n(Kafka API, NATS, MQTT, Redis Streams)] --> [X2 Sequencers\n(ordering, replication)] [X2 Sequencers\n(ordering, replication)] --> [X2 Storage/Commit\n(journals, policies)] [X2 Storage/Commit\n(journals, policies)] --> [X2 Materializer\n(Iceberg/Delta/Hudi)] [X2 Materializer\n(Iceberg/Delta/Hudi)] --> OBJ ' Online path to Tracto via Kafka API [X2 Gateways\n(Kafka API, NATS, MQTT, Redis Streams)] --> [Tracto Notebooks\n(Python, HF, etc.)] : Kafka API (TLS/mTLS) [X2 Gateways\n(Kafka API, NATS, MQTT, Redis Streams)] --> [Spark/SPYT Jobs] : Kafka API ' Offline path via tables OBJ --> [Spark/SPYT Jobs] : Iceberg/Delta/Hudi OBJ --> [CHYT/ClickHouse SQL] : Iceberg/Parquet ' Control/trigger [X2 Storage/Commit\n(journals, policies)] --> [Workflows/Orchestrator] : control topic / webhook [Metadata/Catalog] .. [Spark/SPYT Jobs] [Metadata/Catalog] .. [Tracto Notebooks\n(Python, HF, etc.)] @enduml