Open Source Project Investment Scorecard

Purpose

Produce a rigorous, consistent, reusable investment evaluation for any open source project/company being considered by a USD VC fund — specifically calibrated for the AI technology acceleration cycle (2023-onwards).

Built from: Bessemer Venture Partners Data 3.0 Roadmap, Oxx VC, Basis Set Ventures, Linux Foundation / COSSA, Unusual VC, Matrix VC, and two live case studies (Eigent.AI / CAMEL-AI and Datastrato / Apache Gravitino).

---

Step 1 — Macro Gate (Non-Scoring Pre-Check)

Before scoring, answer these three binary questions. If any answer is NO, stop and recommend Pass.

1. Is the sub-sector still in its window-of-opportunity phase?

   • Yes if: no single open-source project has monopolised the niche yet, OR the target IS that emerging monopolist.
   • No if: a dominant closed-source or open-source player already owns >60% mindshare AND the target has no credible differentiation.

2. Does open-source mode confer structural advantage here?

   • Yes if: vendor-neutral governance, community data flywheel, standards control, or ecosystem lock-in applies.
   • No if: the project is essentially a wrapper / prompt-engineering layer with no community moat.

3. Is the AI-cycle value premium applicable?

   • Higher than cloud-era because: the project sits on a structural chokepoint in the AI stack — examples include inference throughput optimisation, training data infrastructure, model/experiment lifecycle management, AI-native metadata governance, or hardware abstraction layers where the open-source project becomes the compatibility standard across chips and cloud platforms.
   • If purely a cloud-era Open Core play with no AI-cycle differentiation, note this as a risk factor (not automatic veto).

If all three pass → proceed to the five-dimension scorecard.

---

Step 2 — Five-Dimension Scorecard

Score each dimension 0–10. Apply weights. Sum for a weighted total out of 10.

#	Dimension	Weight
A	Open-Source Ecosystem & Community Health	25%
B	Team & Globalisation Capability	20%
C	Technical Moat & Market Positioning	20%
D	Commercialisation Path & PMF	20%
E	Capital Exit Path	15%

---

Dimension A — Open-Source Ecosystem & Community Health (25%)

Core principle: Keyboard Metrics > Mouse Metrics.
GitHub Stars are the most manipulable vanity metric. Prioritise the following in descending importance:

Signal	What to Look For	Strong (8-10)	Weak (<5)
Dependent Repositories	Projects that depend on this one in production	≥1,000	<10
Monthly Active Contributors	Unique devs with commits in last 30 days	≥50	<5
External Contributor %	Non-core-team share of commits	≥40%	<10%
PR Merge Latency	Avg days open→merged	≤7 days	>30 days
Issue Close Rate (90d)	% issues resolved	≥60%	<20%
Release Cadence	Regularity of versioned releases	Weekly/bi-weekly	Sporadic
ADOPTERS.md / Enterprise Logos	Named production deployments	5+ named logos	None
Governance Tier	ASF TLP > ASF Incubator > CNCF > standalone	ASF TLP	No governance
Stars — same-sector Share of Voice	Not absolute; compare to 3 closest rivals	Top-2 in niche	Bottom half
Prestigious Backing	GitHub SOS Fund, CNCF Sandbox, LF project	Yes	No

Scoring guide: Average the signals above. An ASF Top-Level Project graduation is worth +1 bonus point (rare, non-manipulable).

One-Vote Veto for A: External contributor % <5% (pure self-directed project) → automatic Pass.

---

Dimension B — Team & Globalisation Capability (20%)

Two sub-components weighted equally: Engineering Depth and GTM/Global Reach.

Engineering Depth signals:

• Founders are Apache/CNCF/Linux Foundation committers or PMC members in the relevant stack
• Verifiable open-source contribution history (not just the company repo)
• Top-tier academic papers with reproducible benchmarks (NeurIPS / ICML / ICLR / VLDB)
• Prior experience at foundational data/AI infrastructure companies (Cloudera, Databricks, Hortonworks, Confluent, Anyscale equivalent)

GTM / Global Reach signals:

• English-first documentation and GitHub presence (Day 1)
• International (non-founding-country) contributors ≥20% of community
• US paying customers or US-based enterprise pilots
• Founder network includes: top VC relationships, Fortune 500 engineering leaders, or prior role as LF/ASF committee chair
• Cayman/Singapore holding structure already in place (or clear plan)

Scoring guide:

• 9-10: World-class engineers who are also natural community builders with proven US enterprise access
• 7-8: Strong engineering + partial GTM (needs one key hire)
• 5-6: Strong engineering, weak GTM — flag as "Series A condition"
• <5: Academic team with no commercial execution evidence

One-Vote Veto for B: Zero verifiable open-source contribution history outside the company's own repo → automatic Pass.

---

Dimension C — Technical Moat & Market Positioning (20%)

Technology Layer Assessment (use highest applicable):

Level	Description	VC Signal
L1	New algorithm / architecture (e.g., DeepGCNs, PagedAttention)	Strongest moat
L2	Significant engineering innovation on known approach	Strong moat
L3	Differentiated system integration / toolchain	Moderate moat
L4	Prompt engineering / fine-tuning only	Pass — no moat

Market Positioning:

• Is this project on track to be the de facto standard in its niche?
  • Evidence: independent benchmarks, neutral analyst reports, competitor integrations pointing TO this project
• Does vendor-neutrality create structural lock-in? (Apache governance = enterprise procurement preference)
• Is the sub-sector one of the high-value AI-cycle niches?
  • AI Toolchain (RAG, eval, data synthesis) ✅
  • Edge AI / on-device inference ✅
  • Vertical Models with proprietary data ✅
  • AI-native metadata / data governance ✅
  • General-purpose LLM (competing with OpenAI/Google) ⚠️ very high bar

Narrative Consistency Check:
Count how many times the company's core value proposition has changed in public materials. ≥2 pivots in <24 months = -1 point penalty.

One-Vote Veto for C: Core product is L4 (Prompt Engineering / fine-tuning wrapper) with no underlying algorithmic differentiation → automatic Pass.

---

Dimension D — Commercialisation Path & PMF (20%)

Revenue Quality Hierarchy (highest = best for VC):

Rank	Type	VC Multiple	Note
1	Product ARR / Subscription	8-15x	Best — scales without headcount
2	Usage-based / API billing	6-10x	Good — correlates with value delivered
3	Infrastructure embedding / OEM licensing	Strategic premium	Your engine embedded inside cloud or hardware vendor stacks (e.g., vLLM inside AWS, NVIDIA); licensing or rev-share model; valued on strategic control, not pure ARR multiple
4	Proprietary data & model asset monetisation	High ceiling, emerging multiple	Selling curated training datasets, benchmark suites, or evaluation infrastructure to AI labs and enterprises; structurally valued in AI cycle but comp set is thin
5	Professional Services	1-3x	⚠️ Not scalable — PS revenue caps out with team size; triggers mandatory product ARR conversion condition in term sheet
6	Grants / non-dilutive only	0x	⚠️ Not VC-grade revenue

Key metrics to gather and evaluate:

Metric	Healthy Signal	Risk Signal
ARR / revenue (last 12m)	Growing ≥50% YoY	Flat or declining
Largest customer concentration	No single customer >30%	One customer >50%
Customer geography	US-paying customers present	100% non-US
Gross margin	≥70% (product), ≥50% (PS)	<40%
Inbound % of pipeline	≥50% inbound (community-driven)	100% outbound
Revenue type	Product ARR dominant	PS dominant
Runway	≥18 months post-raise	<12 months

PS Revenue Special Rule:
PS revenue is not a veto, but it triggers a mandatory condition: in the term sheet, require conversion to ≥$Xk product ARR within 18 months. The threshold X = 50% of current PS ARR.

Scoring guide:

• 9-10: Product ARR ≥$1M with US enterprise customers, ≥50% inbound, no single customer >30%
• 7-8: Early product ARR + strong PMF signals (Uber/Apple-calibre logos paying, even if small)
• 5-6: PS revenue with credible enterprise logos OR product revenue <$500K
• <5: No paid customers, or 100% grant-funded, or single customer >70%

One-Vote Veto for D: Revenue entirely unverified (LoI/MOU only, no signed contracts) AND current valuation >2× sector median → automatic Pass.

---

Dimension E — Capital Exit Path (15%)

Exit Path Matrix:

Path	Probability Triggers	Typical Valuation Driver
Strategic M&A	Project = de facto standard OR team = acqui-hire grade	Strategic control premium (often >ARR multiple)
IPO	ARR ≥$50M, growth ≥30%/yr, category leadership	ARR × 8-15x
Secondary (VC→PE)	Stable growth + clear path, not IPO-ready	DCF + option value
Follow-on rounds	Good progress, not yet exit-ready	Mark-up on next round

Strategic M&A value checklist — score higher if:

• Project is the "Tabular to Databricks" analogue in its niche (infrastructure standard creator)
• Acquirer has clear "must have or competitor gets it" urgency
• Apache / CNCF governance means acquirer gets community credibility, not just code
• Named enterprise customers are logos the acquirer wants in their annual report

Comparable exit anchors to use:

Comparable	Exit	Key Logic
Tabular → Databricks	~$2B	Creator of Iceberg standard → catalog control
Red Hat → IBM	$34B	Enterprise Linux standard → platform lock-in
GitHub → Microsoft	$7.5B	Developer workflow monopoly
HashiCorp → IBM	$6.4B	Infra toolchain standard
Databricks	$43B (private)	Data + AI platform standard

Scoring guide:

• 9-10: Clear "must acquire" logic for 2+ named potential buyers; comparable exits >$1B
• 7-8: Credible M&A story with 1-2 named buyers; or strong IPO path visible at Series B/C
• 5-6: Acqui-hire probable; or M&A possible but buyer urgency low
• <5: No credible exit path; project likely to be forked rather than acquired

---

Step 3 — Compute Weighted Total

Total = (A × 0.25) + (B × 0.20) + (C × 0.20) + (D × 0.20) + (E × 0.15)

Decision thresholds:

Score	Decision	Action
8.5 – 10.0	🟢 Strongly Recommend	Fast-track IC; move to term sheet
7.0 – 8.4	🟡 Recommend with Conditions	Proceed with milestone-linked terms
5.5 – 6.9	🟠 Watch / Track	Add to pipeline; re-evaluate in 6 months
< 5.5	🔴 Pass	Decline; note reason for future reference

One-Vote Vetoes (any = automatic Pass, overrides total score):

1. External contributor % <5%
2. Zero verifiable engineering contribution history outside company repo
3. Core product is L4 (Prompt Engineering only)
4. Narrative pivot ≥3 times in <24 months
5. Revenue entirely LoI/MOU only + valuation >2× sector median
6. No English documentation AND zero international contributors — applies to any single-country project claiming global ambition

---

Step 4 — Required Output Format

Always produce output in this order:

1. Macro Gate Result

One sentence per question. State whether the project passes the gate.

2. Scorecard Table

Dimension	Weight	Score	Weighted
A. Open-Source Ecosystem	25%	X/10	X.XX
B. Team & Globalisation	20%	X/10	X.XX
C. Technical Moat	20%	X/10	X.XX
D. Commercialisation & PMF	20%	X/10	X.XX
E. Exit Path	15%	X/10	X.XX
Total	100%	—	X.XX/10

3. Verdict Line

🟢/🟡/🟠/🔴 [Decision] — [One sentence rationale]

4. Dimension Narrative

For each dimension: 2-4 sentences covering the key evidence, the strongest signal, and the main risk. Be direct — do not soften risks.

5. One-Vote Veto Check

Explicitly confirm whether any veto condition is triggered.

6. IC Thesis (one paragraph, ≤100 words)

Suitable for verbal delivery in an Investment Committee. Structure: ① why now ② why this project ③ exit path conviction.

7. DD Priority List

Top 3-5 open questions that, if answered positively, would raise the score by ≥0.5 points. Ranked by importance.

8. Watch Triggers (if verdict is Watch/Track)

Specific, measurable milestones that, if hit, would upgrade to Recommend.

---

Calibration Reference: Scored Examples

See references/scored-examples.md for:

• **vLLM/Inferact: Total 8.9/10 → Strongly Recommend. Category-defining inference standard; 2,000+ contributors; a16z + Lightspeed validation. Strong open-source (7.5) and team (6.5) undercut by PMF (4.5) and three narrative pivots.
• **Hugging Face: Total 8.5/10 → Strongly Recommend. $130M ARR, 10,000+ enterprise customers, Google/Amazon/NVIDIA as investors and distribution partners.

These two cases define the calibration anchors for the scoring scale.