Local-first AI orchestration

Autonomous agent orchestration for procedural production pipelines.

A distributed local/cloud execution engine that powers a working PBR texture pipeline today, and a deterministic animation scene graph in active R&D.

Launch Live Demo Explore the Engine
🟢 LIVE PRODUCT Asset Factory sample output -- packaged PBR texture set
PBR Textures Unity Automation Tier-0 Local GPU

Asset Factory

Production PBR Pipeline: Procedural texture synthesis with multi-stage tiling verification and 4K upscaling for store-ready game assets.

  • PBR synthesis: Albedo, Sobel-filter Normal, cavity-based AO, Metallic/AO packed per Unity's MaskMap convention -- computed on the 4K material to preserve micro-detail.
  • Packaging: .unitypackage archive with hand-authored .meta/.mat YAML -- no Unity Editor process invoked in this step.
  • Manual step remaining: a visual pass inside the Unity Editor before publishing -- not yet automated.
Fab · planned, not yet published itch.io · planned, not yet published Product Hunt · not yet submitted

Full production pipeline & sample output → · ArtStation portfolio →

🟣 R&D // SCENE DAG
Kids Cartoon Engine Scene DAG Deterministic AI

Generator Bajek

Powered by the Portable Animator engine

A modular, shot-by-shot generation engine for kids' story content, built on a scene dependency graph instead of one long prompt holding an entire story's continuity.

  • Data model: Scene DAG, JSON Schema draft-07 -- each scene is an independent node with explicit depends_on.
  • Deterministic regeneration: an input fingerprint (hash of prompt + assets + timing + camera) decides whether a node needs re-rendering -- unchanged scenes are skipped instead of reset on every pass.
  • Inspector API: planned -- per-node property edits (transform, keyframes) rather than full-project re-renders.
itch.io · planned, not yet published Product Hunt · not yet submitted

This is a specification and data model today, not a rendering engine -- see engine preview & technical prototype status.

Product

The system in action

Illustrative renderings of what the operator panel prints, with sample data -- not screenshots. The panel itself is behind the Launch Live Demo button.

localhost:8000 · Dashboard sample output
Operator panel Eight tabs over one FastAPI backend -- chat, dashboard, mail, social, projects, queue, database, staging. No external CDN: every stylesheet, script and icon is served locally.
localhost:8000 · Queue sample output
Task queue The live SQLite queue workers lease from -- status, priority, retry count and lease expiry per task, rendered from the same rows the runner reads.
localhost:8000 · FinOps sample output
FinOps budget guard Per-provider spend against daily and total limits, read from the Shadow Ledger state file. Figures flagged as estimated are labelled as such in the interface rather than presented as exact billing.
Architecture

How it works under the hood

Three layers of the AIOS engine -- cost routing for models, core mechanisms, and what's live today versus what we're asking the grant for.

Every model call is routed through one of three cost/capability tiers -- verified against core/config/model_routing.json and core/finops_router.py, not just described.

Tier 0 -- Local

Zero API cost

  • Ollama models running on local hardware.
  • PBR map synthesis (Albedo, Sobel-filter Normal, Roughness, Metallic, AO) and .meta/.mat YAML generation -- pure Python, no network call.
  • Time-critical / latency-sensitive tasks.
Tier 1 -- Free-tier cloud

Batching & validation

  • Cascade across free-tier providers: Google AI Studio (Gemini Flash), GitHub Models (GPT-4o-mini), Cloudflare Workers AI, OpenRouter.
  • Bulk prompt batching, JSON schema validation, mail assistant drafts, telemetry summarization.
  • Azure does not appear at this tier in the current routing table -- it is Tier-2 only (see below).
Tier 2 -- Paid cloud, budget-gated

Deep reasoning & self-correction

  • Azure AI Foundry and AWS Bedrock are the primary Tier-2 providers active today (per-role model assignment). A Google Vertex AI path exists in the routing configuration; its rollout status is still being finalized internally.
  • Reserved for two allow-listed purposes today: task decomposition and texture generation -- gated by a per-call spend cap, an audit trail, and a per-provider daily/total budget guard that fails closed if its state file is missing.
  • The bounded self-correction retry (Core Engine tab) runs on whichever tier a task was routed to -- it is not exclusive to Tier-2.

Budget guard: hard cutoff on daily/total limit breach, tested against a live state file -- not an automatic "rollback" of already-recorded spend, which does not exist in the code today.

Four mechanisms everything else stands on. Not a demo — working code, running today against a real task queue.

🔒 Distributed task leasing (SQLite WAL)

Tasks are claimed atomically (UPDATE … RETURNING in a single transaction) and stamped with a time-boxed lease — multiple workers, local and cloud, can safely pull from the same queue without race conditions or double-executing the same task.

🔁 Self-correction loop (Reflexion)

Code that fails its own test gets one disciplined self-correction pass from an independent reasoning model — not an unbounded loop. If the fix doesn't fully resolve the problem, the system rolls back to the last safe version instead of leaving something worse in place.

🧑‍⚖️ Human-in-the-loop gates

Changes to production files never land silently. Every result passes through an independent model review and/or an explicit approval gate before it's considered ready to ship — patches, not silent overwrites.

📈 Lightweight tracing

Every model call leaves a JSONL trace (time, model, tokens, cost) in a format compatible with a subset of the OpenTelemetry convention — cost and performance observability without a heavy monitoring stack.

🟢 LIVE / FUNCTIONAL
  • Tier-0 local runtime: PBR map generation (Sobel-filter normal maps, roughness, metallic, cavity-based ambient occlusion) and .meta/.mat YAML synthesis -- pure Python, zero API cost.
  • FastAPI backend serving the command-center panel and the public portal.
  • SQLite WAL task queue with transactional, atomic leasing -- multiple local and cloud workers pull from the same queue without double-execution.
  • Telemetry panel -- per-call model, tier, tokens and cost logged as JSONL.
  • Cloud routing, today's scope: Tier-1 free-tier cascade fully active in production; Tier-2 paid cloud conditionally available under a per-call budget guard for two allow-listed purposes.
  • Hard budget cutoff on daily/total spend limits -- tested against a live state file.
  • Bounded self-correction loop (Harness v2, max 2 attempts) -- wired into the live task pipeline.
🟣 ARCHITECTURE ROADMAP / GOOGLE CLOUD GRANT
  • Scaling cloud routing to GKE: worker nodes with GPU (NVIDIA L4 / G2), autoscaling under batch load.
  • Managed Vertex AI inference for texture and frame generation at volume -- beyond today's narrow, allow-listed Tier-2 usage.
  • Broader FinOps guardrails beyond today's per-call cutoff -- including an automatic rollback of already-recorded spend, which does not exist in the code today.
  • A second, broader self-healing loop (pytest-traceback driven, up to 3 attempts) exists in code but is not yet wired into the main task dispatcher -- used only by an isolated sandbox executor today.
  • Batch processing target: 50+ asset packages per day, scheduled nightly runs.

Everything on this side is a funding target, not a claim of current capability.

AIOS Core / Orchestrator

Completed

  • HUD Command Center with SQLite-backed state.
  • Background Mail Watcher with Edge-TTS voice notifications.
  • Asset Factory pipeline: PBR map generation (Albedo/Normal/Roughness/Metallic/AO) and packaging into .unitypackage with hand-written .meta/.mat files (no Unity Editor process invoked in this step -- a visual pass in the editor remains a manual step).
  • Admin panel with zero external CDN dependencies (no Tailwind CDN, Font Awesome CDN, or Google Fonts).
  • Native task-DAG renderer in inline SVG, zero external libraries.
  • Controlled runner self-restart mechanism with a code-fingerprint check (detects a process running a stale version).

Planned

  • Automated nightly batch scheduling (50 packages/day target).
  • Full Playwright integration for submission automation.
  • Bringing the 4K upscaling stage into the application itself, so the pipeline runs end to end without an external tool.

Portable Animator

Completed

  • Working pipeline: script → images → voice → video assembly, fully local (5 offline models).

Specification (not yet implemented)

  • Scene isolation as independent units in a dependency graph (DAG) — targeted regeneration instead of recomputing the whole project.
  • Scene Inspector — a granular panel for editing a single scene's properties (prompt, assets, timing, camera).

AICodeReview

Concept stage — not started

  • Idea: an AI-assisted code review service for solo/indie developers -- upload a repository, get a structured report of bugs, risks, and suggestions.
  • No production code exists today. Nothing here is deployed, running, or available to try.

Cloud Architecture & Grant Roadmap

Planned scaling of Asset Factory and Portable Animator with Google Cloud grant support -- running today locally / at small cloud scale (see the Execution Tiers tab), not yet on the target infrastructure below.

  • GKE with GPU (NVIDIA L4 / G2): worker nodes for Asset Factory and Portable Animator, autoscaling under batch load.
  • Vertex AI: managed model inference for texture and frame generation, as an alternative/complement to today's Tier-1/Tier-2 routing.
  • Batch processing: a target of 50+ asset packages per day, scheduled nightly runs.