langwatch
@langwatch Organization
Public Skills
datasets
by langwatch
Generate realistic synthetic evaluation datasets by analyzing the user's codebase, prompts, production traces, and reference materials. Interactive and consultant-style. Asks clarifying questions, proposes a plan, generates a preview for approval, then delivers a complete dataset uploaded to LangWatch. Use when user asks to generate, create, or build a dataset for evaluation, testing, or benchmarking.
evaluate-multimodal
by langwatch
Evaluate multimodal AI agents that process images, audio, PDFs, or other files. Sets up evaluations using LangWatch's LLM-as-judge with image inputs, Scenario's multimodal testing, and document parsing evaluation patterns. Use when your agent handles non-text inputs.
langwatch
by langwatch
Read LLM traces back from LangWatch with the langwatch CLI. Use when asked what an agent, prompt or model call actually did in production, when debugging a failed or slow LLM run, when looking up a trace or session by id, or when checking whether this coding session's own activity was captured.
code-review
by langwatch
"Project-level code review: check changed files against LangWatch codebase rules (IDs, multitenancy, layering, naming, SRP)."
browser-pair
by langwatch
"Collaborative headed browser session for UI work. Launch Playwright Chromium visible to the user, handle auth, then interactively drive the browser while the user watches and gives real-time visual feedback. Edit code and refresh to verify fixes live. Use when the user says 'browser pair', 'paired browser', 'let's look at this together', 'open chromium', or wants to iterate on UI with live visual feedback."
browser-test
by langwatch
"Validate a feature works by driving a real browser with Playwright MCP. No test files — just interactive verification."
feature-map
by langwatch
"Maintain the canonical LangWatch feature map (/feature-map.json). Use when adding features, APIs, MCP tools, CLI commands, or skills — to update the central registry and keep surfaces in sync."
haven-setup
by langwatch
"Bring up the LangWatch dev stack via thuishaven (make haven up) — one-time proxy/CA setup, reusing existing local ClickHouse/Postgres/Redis, WSL2/no-colima fallback for langyagent, and the known gotchas that silently break it."
test-cli-usability
by langwatch
Write scenario tests that verify your CLI tool is usable by AI agents. Ensures commands work non-interactively, provide clear output, and don't hang on prompts. Use when you want to prove your CLI is agent-friendly.
github
by langwatch
Open a real pull request on the user's behalf — clone a repo, branch, commit, push, and open a PR authored by the requesting user. Use when the user asks to open a PR, fix something in a repo and submit it, send a patch, raise a pull request, or otherwise land a code change on GitHub.
langwatch-kanban
by langwatch
"Manage the LangWatch Kanban GitHub project board — sync statuses, view your board, find stale items, move issues, assign work."
watch-ci
by langwatch
"Watch CI for the current branch's PR. Blocks until CI completes, then fixes failures or addresses review comments. Loops until green."
launch
by langwatch
Create worktrees, tmux sessions, and Claude sessions for GitHub issues. Use when spinning up parallel implementation work.
plan
by langwatch
"Create a feature file with acceptance criteria before implementation. Use when no specs/features/*.feature file exists for the work."
orchestrate
by langwatch
"Orchestration mode for implementation tasks. Manages the plan → code → review loop. Use /orchestrate <requirements> or let /implement invoke it."
review
by langwatch
"Run parallel code reviews: uncle-bob-reviewer (SOLID/TDD), cupid-reviewer (CUPID properties), test-reviewer (pyramid placement), and pii-reviewer (security/secrets). Surfaces conflicts for orchestrator resolution."
challenge
by langwatch
"Stress-test an architecture proposal, plan, or technical idea. Invokes the devils-advocate agent to find weaknesses before you commit."
learn
by langwatch
"Learn from mistakes by updating AGENTS.md. Use when a mistake was made that should be prevented in future sessions."
test-review
by langwatch
"Review specs and tests for pyramid placement and quality."
langwatch
by langwatch
The platform for LLM evaluations and AI agent testing
e2e
by langwatch
"Generate and verify E2E tests for a feature. Explores live app, creates test plan, generates tests, runs and fixes until passing."
sherpa
by langwatch
"Delegate repository, agent, or documentation questions to the repo-sherpa. Use for onboarding, DX improvements, or meta-layer changes."
implement
by langwatch
"Start implementation of a GitHub issue. Usage: /implement #123 or /implement <issue-url>"
code
by langwatch
"Delegate implementation work to the coder agent. Provide requirements or feature file path."