Skip to content
Eon-Labs KB

Overview - System Design

Source: Notion | Last edited: 2025-10-31 | ID: 29d2d2dc-3ef...

Phase 1 — Monorepo (fastest to MVP)

Section titled “Phase 1 — Monorepo (fastest to MVP)”

Repo: alpha-forge/alpha-forge

Top-level folders

alpha-forge/
├─ dsl/                 # schema, parser, validation
├─ compiler/            # DSL -> DAG
├─ runner/              # DAG runtime, context, checkpoints
├─ plugin-kits/         # base interfaces & helpers (python, grpc)
├─ plugins/             # built-ins: datasource/feature/model/loss/trainer/backtest
│  ├─ datasources/
│  ├─ features/
│  ├─ trainers/
│  └─ ...
├─ registry/            # in-memory or sqlite registry MVP (manifest + versions)
├─ exec-gateway/        # rust service (grpc) mock → later real adapters
├─ io/                  # sinks/sources: save_csv/save_json, plotting
├─ examples/            # DSL examples + notebooks + escape bundles
├─ tools/               # codegen, scaffolding, lint hooks
├─ api/                 # protobuf/IDL (grpc + json schemas)
├─ tests/               # unit & e2e
└─ docs/                # developer guide, RFCs

Why: single place to iterate quickly on DSL/Compiler/Runner/Plugins contracts; minimal release overhead.

Phase 2 — Split Repos (clean boundaries & versioning)

Section titled “Phase 2 — Split Repos (clean boundaries & versioning)”

When interfaces stabilize (DSL schema, plugin ABI, gRPC), split into 7 core repos. Suggested names + roles:

  1. alpha-forge/dsl (Python)
  • DSL schema (JSON/YAML), validators, lints, canonical examples
  • Outputs: alphaforge-dsl Py package
  • Interfaces: JSON Schema; semver for schema (1.x)
  1. alpha-forge/compiler (Python)
  • DSL → DAG planner, static checks, param binding, materialize hooks
  • Outputs: alphaforge-compiler Py package
  • Depends on: dsl
  • Contract: DAG JSON schema (api/dag.schema.json)
  1. alpha-forge/runner (Python)
  • DAG runtime, context store, checkpointing, orchestrated plugin calls
  • Outputs: alphaforge-runner Py package/CLI
  • Contracts: Plugin runtime ABI (call signatures), checkpoint format
  1. alpha-forge/plugin-kits (Python + gRPC)
  • Plugin base classes, manifest spec, loader, sandbox, version checks
  • gRPC stubs for remote plugins (Python ↔ Rust)
  • Outputs: alphaforge-plugin-kit-py, alphaforge-plugin-kit-proto
  • Contract: plugin.manifest.schema.json, plugin.api.proto
  1. alpha-forge/plugins (Python; optional multi-package)
  • Official plugin collection (datasource, features, models, losses, trainers, backtests)
  • Can split into subpackages later: af-plugins-features, af-plugins-trainers…
  • Contract: Each plugin has Manifest + tests + golden snapshots
  1. alpha-forge/exec-gateway (Rust)
  • Execution & Risk Engine gateway (gRPC), adapters to venues, risk guards, order router
  • Outputs: docker image + alphaforge-exec-proto stubs
  • Contract: execution.api.proto, order/position/risk types; strict semver
  1. alpha-forge/registry (Service: Python/FastAPI + Postgres + S3/MinIO)

  • Plugin registry & artifact store metadata; feature/data asset metadata

  • Outputs: REST/gRPC service + CLI (af-registry)

  • Contracts: OpenAPI registry.openapi.json; auth tokens; signing/sha256 policy Shared contracts repo (optional but recommended):

  • alpha-forge/apis

    • All protobufs, JSON Schemas, OpenAPI in one place
    • Versioned independently; other repos vendor or depend via package

Phase 3 — Ecosystem Repos (optional, when growing)

Section titled “Phase 3 — Ecosystem Repos (optional, when growing)”
  1. alpha-forge/sdk-py — Python SDK for researchers (escape bundle helpers, dataset loader)
  2. alpha-forge/sdk-js — JS/TS SDK for Console or web tooling
  3. alpha-forge/examples — public examples, tutorials, sample datasets
  4. alpha-forge/community-plugins — curated third-party plugins
  5. alpha-forge/console — (later) web UI for runs, assets, lineage, audits

Responsibilities & Interfaces (quick map)

Section titled “Responsibilities & Interfaces (quick map)”
  • DSL ↔ Compiler: strategy.schema.json
  • Compiler ↔ Runner: dag.schema.json
  • Runner ↔ Plugins: Plugin Manifest + Python call ABI (or gRPC if remote)
  • Runner ↔ Exec-Gateway (Rust): execution.api.proto
  • Runner/Compiler/Plugins ↔ Registry: registry.openapi.json (auth, signatures, versions)
  • Data/Feature Assets: asset.schema.json (immutable id + version + hash + lineage)

Versioning & Releases

Section titled “Versioning & Releases”
  • Semver everywhere (MAJOR.MINOR.PATCH).
  • Schema repos (DSL/DAG/Proto): bump MAJOR on breaking changes; publish migration notes.
  • Runner & Compiler pinned to schema versions; CI ensures compatibility matrix.
  • Plugins: strict version pinning in DSL (plugin@=1.2.3) for prod; allow ranges (^1.2) in research.
  • Exec-Gateway image tags = protocol version + build hash.