gaia-research

evidence-verification-pipeline

Canonical entry point for the full Gaia evidence verification pipeline. Invoke as /evidence-verification-pipeline or /ev-pipeline — both names trigger this skill. Use whenever you need to run the evidence phases end-to-end: collecting raw evidence from the data lake, verifying live GitHub star counts, running Phase 2B benchmark-source verification, running adversarial auditing for noise and URL errors, and checking link health via Firecrawl. Also use when someone says "run the evidence pipeline", "verify the evidence lake", "audit evidence", "run ev-pipeline", "full evidence check", "pipeline run", "prepare evidence for ingestion", or "refresh the data lake". This is the pre-ingestion step — run it before importing any evidence into the registry.

gaia-research 23 8 Updated 2mo ago
GitHub

Install

npx skillscat add gaia-research/gaia-skill-tree/evidence-verification-pipeline

Install via the SkillsCat registry.

SKILL.md

Evidence Verification Pipeline

Alias entry point for /ev-pipeline. It operates on the evidence/ data lake only and does not mutate registry files.

Type-First Evidence Lake Contract (#1148)

The evidence lake is type-first. The primary working set for every phase is evidence/by-type/<canonical-evidence-type>.md. Legacy evidence/tier_*.md files may still exist as coexistence artifacts for older tooling, but they are not the semantic routing key.

Phase Responsibilities

  • Phase 0 — ev-discovery (skippable): searches for new Stage-2 evidence on declared need and appends discovered rows into source inputs for Phase 1.
  • Phase 1 — ev-collection: materializes/compiles by type. evidence/by-type/<type>.md is primary; tier_*.md is coexistence-only.
  • Phase 2 — ev-star-verification: verifies live GitHub stars without repartitioning by rank.
  • Phase 2B — ev-benchmark-verification: verifies benchmark-result rows and candidate manifests against registry/benchmark-sources.json; unknown/candidate/registered/mirrored/incomplete sources do not score. Registered sources may declare appliesToGenericSkillRefs for generic capability applicability, but that metadata is non-scoring until a named benchmark-result row satisfies reproducibility and approved provenance.
  • Phase 3 — ev-adversarial-audit: audits the by-type files, including benchmark catalog misuse and vendor-claim leakage.
  • Phase 4 — ev-link-validation: validates URLs. validate_sources.py remains a temporary coexistence URL-health shim.

Generated evidence outputs are review artifacts. Do not commit generated by-type files, tier files, source reports, validation reports, unified lake files, seeds, collector rows, or benchmark candidate manifests without a human gate. After Phase 2B + Phase 3 + Phase 4, humans must approve any benchmark catalog promotion or /gaia-ingest-batch; machines classify, humans promote. A registered benchmark source is generic-applicable catalog metadata only: no named-skill score, Trust Magnitude contribution, or scoring evidence row exists until a named benchmark-result row satisfies reproducibility and approved provenance.