SyFI-ServingStudio
@SyFI-ServingStudio Organization
Public Skills
dev-calculate-kv-cache-capacity
by SyFI-ServingStudio
"Use when the user asks for KV cache size, attention-state size, bytes per token, max cached tokens, max concurrent requests, or KV capacity of a model or deployment. For a model ServingStudio Sim already has an arch for, read the numbers from the simulator (launcher <preset> --kv-report, a run's raw/kv_cache.json, simulator cost-trees' kv_cache) instead of calculating by hand. For a model without an arch, derive its cache part by part (per-token, sliding-window, per-request) from config/public sources and vLLM's allocator, in the KvCacheLayout form its arch will declare, and show every step."
servingstudio-simulate
by SyFI-ServingStudio
Simulate an LLM serving deployment (model, GPU, parallelism, worker, replicas) on a request workload through the ServingStudio public API: a recorded capture, a generated trace or an uploaded CSV, with TTFT, TPOT, end-to-end latency and throughput. Not for the time of one batch (that is a timing prediction).
operate-run-alignment
by SyFI-ServingStudio
Run, resume, or inspect the shared ServingStudio Sim↔framework evidence pipeline for vLLM or SGLang. Its artifacts support both simulator validation through top-align-with-framework and framework optimization through top-compose-real-framework-from-sim.
top-add-new-arch
by SyFI-ServingStudio
Use as the top entry point when the user wants to add a whole new model architecture to ServingStudio Sim end to end — explore it, split its forward into kernels, categorize every element into L1/L2/L3/L4, build the kernels, ops, worklets, and model_arch, then add an independent model.work necessary-work label. The one criterion running through the timing layers is the timing-prediction wiring — where each element's timing comes from: L1 is the only layer that MEASURES a number (profile.db); L2/L3/L4 only COMPOSE measured L1 timings. Adding an arch is paired with alignment: capture a real vLLM/SGLang run first and let the measured kernel inventory — not source-reading guesswork — decide which kernels to add, largest first. Skipping the capture requires the user's explicit confirmation. This is a routing/sequencing skill; it delegates each phase to a lower skill and does not implement leaves itself. Landing the finished model on main is dev-finalize-model-addition.
dev-parallelize-arch-addition
by SyFI-ServingStudio
Use in single-agent mode when one session runs a top-add-new-arch build by launching its own parallel subagents: how to cut the work into lanes and when the serial steps run. Not for the orchestrator–implementer loop, and not for one writing agent's worktree (that is dev-orchestrate-parallel-subagents).
dev-orchestrate-parallel-subagents
by SyFI-ServingStudio
"Use when you are about to launch two or more subagents that will each MODIFY the same repository concurrently, such as several independent feature/kernel additions in one fan-out. Mandates git worktree isolation per agent so concurrent edits never race on shared files, every agent can run its task skill unmodified and self-validate end to end, and the orchestrator integrates afterward."
skill-of-skills
by SyFI-ServingStudio
Add, rename, organize, or select repo-local ServingStudio Sim skills. Defines naming levels, workflow routing, and writing quality.
operate-run-simulation
by SyFI-ServingStudio
Use when the user wants to run a ServingStudio Sim simulation from a preset (YAML preferred, JSON also accepted). This skill locates the repo logs root, names the experiment as date_index_short_name, prefers expressing requested variations as internal sweeps inside one copied preset, always does a dry run to inspect the expanded run plan before launching, and runs through the ServingStudio Sim launcher.
top-align-with-framework
by SyFI-ServingStudio
Align ServingStudio Sim to a real framework across kernel timing, GPU duty cycle, and TTFT/TPOT. Routes execution and fixes; does not run phases or edit code.
top-add-kernel
by SyFI-ServingStudio
Use as the entry point whenever a ServingStudio Sim L1 kernel is to be added. It sequences two child skills for one kernel: Python profiling, then Rust timing/cache wiring. A kernel is the unit of delegation: an orchestrator hands each kernel over whole, and whoever owns it runs this skill end to end.
dev-finalize-model-addition
by SyFI-ServingStudio
Use when a new model's arch is built and aligned and the user wants it landed on main: check the alignment results, publish presets and captures, finish profiling, merge main, review, draft and open the PR. Not for building the arch (top-add-new-arch) or only walking a diff (dev-present-changes-for-review).
servingstudio-timing-predict
by SyFI-ServingStudio
Predict how long one LLM serving iteration takes for given batch shapes on a published deployment (model, GPU, parallelism, MoE routing) through the ServingStudio public API, with a per-kernel breakdown. Not for request queueing or end-to-end latency.
dev-compose-worker
by SyFI-ServingStudio
"Use when planning or estimating a new ServingStudio Sim worker combination, or when implementing, composing, reviewing, renaming, moving, or tidying production L5 worker code under simulator/src/worker/: KV stores and capability traits, admission lifecycle/selection policies, execution adapters, cadence shells, family directories, build_*_worker recipes, and L6 wiring. Enforces the canonical four-axis ownership, share-vs-new and difficulty analysis, pending-membership rule, shell discriminator, construction/file boundaries, complete changed-file coverage, and behavior-preserving validation. Not for L4 kernel/cost-model implementation or an unrelated general file review."
impl-compose-arch
by SyFI-ServingStudio
Use when implementing an L4 model_arch under simulator/src/arch/ after its L3 worklets build — the full-model wire file for one worker type that forwards ModelCfg into worklet configs, compiles the per-iter CostTree once, and exposes IterwiseUnifiedModel. Covers the live code contract: build_configs/resolve_configs/ build + cost_tree/eval_iter + the Scale{n} layer fold + the config.rs selector. Follows the code, not design.md's retired dry_run/JitPlan/Describe. NOT for L3 worklets (impl-compose-worklet).
operate-profile-existing-kernel
by SyFI-ServingStudio
Query, fill, refresh, validate, or measure a registered ServingStudio Sim L1 kernel profile, including DB-row provenance and preservation.
orchestrator-add-kernel-to-python-profile
by SyFI-ServingStudio
Use when owning the Python-side ServingStudio Sim L1 profiling work for a new kernel kind or a new backend of an existing kind, either as the coordinator who writes implementer briefs or as the agent a whole kernel was handed to. Defines the steps, their completion checks, and the order of Torch reference work before framework backends. It does not cover Rust timing/cache wiring.
impl-validate-kernel-cache
by SyFI-ServingStudio
"Use when implementing the cache-fidelity validation leaf for a ServingStudio Sim L1 kernel Rust cost cache. Measure or compare INTERPOLATION / EXTRAPOLATION / QUANTIZATION fidelity against perf_api ground truth at off-grid, extrapolated, or bucket-boundary shapes. Covers any registered KernelSpec. NOT for profiling existing rows or batch-aggregation fidelity."
orchestrator-wire-kernel-to-rust
by SyFI-ServingStudio
Use when owning the Rust timing/cache wiring for a ServingStudio Sim L1 kernel after the Python profiling side is defined, either as the coordinator who briefs an implementer or as the agent a whole kernel was handed to. Turns a Python-to-Rust handoff into Rust steps, verifies KernelSpec/cache/bridge behavior, and decides when cache fidelity is good enough. It does not implement Python runners or profile new backend sources.
top-split-model-into-kernels
by SyFI-ServingStudio
Use as the entry point when the user wants to break a Transformer model's forward pass into the sequence of ServingStudio Sim kernels it costs as — the per-op boundary decision. First calls top-explore-models to establish the architecture, then for each op routes to dev-lookup-transformers-model (what math) and dev-explore-kernel (whether a real fused kernel exists at what launch granularity), then applies the boundary rules to assign each op a home: reuse an existing kernel, a new dedicated kernel (hand to top-add-kernel), an elementwise byte-placeholder, or fold into a neighbor. Never skips a cost-bearing op. When a measured vLLM/SGLang capture is available it is the primary evidence for launch granularity, and the table is ordered by each op's measured share of iteration time. Produces a per-op decision table, not code.
operate-run-timing-predict
by SyFI-ServingStudio
Run offline timing-predict for explicit iteration, attention, or FFN batch shapes. Not for scheduler or workload simulation.
impl-register-kernel
by SyFI-ServingStudio
Use when implementing the Python profiling registration for a ServingStudio Sim L1 kernel kind or backend after the orchestrator has classified the task. Covers the current profiling/ contracts: per-kind kernel module, co-located Args schema, lazy RunnerRef, BackendSupport, runner contract, tests, and no shared profile.db writes unless authorized. Does not cover Torch semantic reference design, framework research, or Rust timing/cache wiring.
dev-run-tests
by SyFI-ServingStudio
Use when asked to run, select, or understand ServingStudio Sim's tests — which tier needs a GPU / the built binary / an agent runtime / is just CPU, how the auto-skip works, and how the per-GPU throughput & sim-speed goldens are recorded. Covers the just test-* recipes, pytest markers, and the GPU-type-tagged golden store. NOT for writing a new kernel's cache-fidelity check (that is impl-validate-kernel-cache).
servingstudio-kernel-performance
by SyFI-ServingStudio
Look up measured GPU kernel performance for LLM serving (GEMM, attention, MoE, normalization, communication) from the ServingStudio Kernel Library API. Not for end-to-end serving latency.
impl-compose-worklet
by SyFI-ServingStudio
Use when implementing an L3 worklet under simulator/src/worklet/ after its L2 ops build — one model-module sync section that derives the per-rank partition and composes ops with sum/max over the CostTree. Covers the live code contract: Config/Resolved/Input + resolve_config/build/compile/eval, and the group suffix (Local/TP/HP/EP). Follows the code, not design.md's retired init_ops/dry_run_init_ops/lookup/Describe. NOT for L2 ops (impl-compose-op) or L4 arch (impl-compose-arch).