Paper Digest

${var} — Research topic to search. If empty, uses topics from MEMORY.md.

0. Config check

If ${var} is set, treat it as the only topic (skip MEMORY.md topic parsing).

Otherwise read memory/MEMORY.md for tracked topics — look for ## Interests, ## Research topics, or ## Tracked topics. Accept bullet lists or comma-separated lines.

If no topics are configured AND ${var} is empty, abort:

• Log PAPER_DIGEST_NO_TOPICS to memory/logs/${today}.md with a note
• Notify once via ./notify: paper-digest: no topics configured. Add ## Interests bullets to memory/MEMORY.md or pass var="topic".
• Exit. An empty digest is worse than a skip.

Read the last 7 days of memory/logs/ → extract arXiv IDs already shipped (search for patterns like arxiv.org/abs/XXXX.XXXXX or explicit dedup lists). Treat these as skip keys.

1. Gather candidate pool

For each topic, fetch from two independent sources:

Hugging Face Papers (community signal):

curl -s "https://huggingface.co/api/papers/search?q=${TOPIC}&limit=15"

Fields used: paper.id (arXiv ID), title, summary (abstract), authors[].name, publishedAt, upvotes.

arXiv API (recency + breadth, last 14 days):

FROM=$(date -u -d '14 days ago' +%Y%m%d0000)
TO=$(date -u +%Y%m%d2359)
curl -s "https://export.arxiv.org/api/query?search_query=all:${TOPIC}+AND+submittedDate:[${FROM}+TO+${TO}]&sortBy=submittedDate&sortOrder=descending&max_results=15"

Parse the Atom XML: <entry> → <id> (URL with arXiv ID), <title>, <summary>, <author><name>, <published>. Space between calls (arXiv asks for ≥3s between requests).

Also fetch HF daily trending once for serendipity:

curl -s "https://huggingface.co/api/daily_papers?limit=15"

Deduplicate the pool by arXiv ID. Drop any ID already shipped in the last 7 days of logs.

2. Skip-gate (drop before summarizing)

Drop a paper unless it passes at least one of:

• Direct keyword match to a tracked topic in title or abstract
• ≥5 HF upvotes (community signal overrides topic match) — 5 is a tunable threshold; adjust if HF upvote distributions shift or if the digest is consistently over/under-selecting. Values of 3–10 are reasonable.
• Published in last 14 days AND author or lab is listed in MEMORY.md

HF fallback: if the Hugging Face API fails (curl + WebFetch both error) or returns an empty result set across every topic, skip the upvote gate entirely and fall back to arXiv recent: top 5 from cs.LG and top 5 from cs.AI submitted in the last 7 days, sorted by submittedDate descending. Note (HF unavailable — arXiv-only fallback) in the digest header.

Also drop if ANY of:

• Survey/review with no new empirical result (unless a tracked topic explicitly tracks surveys)
• Pure leaderboard/benchmark-number paper with no method novelty
• Abstract looks like an extended previous work (v2, v3 — check arXiv version)
• Already in the 7-day log dedup set

3. Rank survivors

Signal score = (upvotes * 2) + (1 if ≥1 citation and <30d old) + (2 if tracked-topic keyword in title). Ties broken by publishedAt descending.

4. Select — fewer is better

Pick up to 5, but only include a paper if you can write a non-generic "so-what" sentence. If signal is thin, ship 3, or 2. If the pool is genuinely empty after the skip-gate, ship zero and write a one-line "no new papers of substance this cycle" digest — padding is worse than brevity.

5. Summarize — required moves and banned phrases

For each selected paper, write two sentences:

• What's new (1 sentence): the concrete method or result, named specifically. Must cite at least one of: a number, a dataset/benchmark name, a mechanism, or a delta vs. a named prior system. Not "proposes a novel approach".
• So what (1 sentence): why a reader tracking this specific topic should care this week. Must reference a tracked interest explicitly OR name a paper/approach it supersedes, replicates, or contradicts.

Banned (rewrite or drop the paper):

• "This paper proposes", "We introduce", "In this work we..."
• "Novel", "state-of-the-art" without a number
• "Could have implications" / "may be useful for" / "opens avenues"
• Generic "advances the field" / "improves performance"

Self-edit pass: re-read each summary. If "what's new" could apply to 5+ other papers in the field, rewrite or drop the paper.

6. Output

Save to articles/paper-digest-${today}.md:

# Paper Digest — ${today}
> **Verdict:** <one line: shape of the week, e.g. "3 solid LLM agent results, 1 surprising RL finding, no new alignment work worth reading">
> Pool: HF ${n_hf} + arXiv ${n_arxiv} → ${n_deduped} deduped → ${n_shipped} shipped

## ${topic_or_cluster}
1. **Title** — First Author et al. (2026) · ↑${upvotes}
   **What's new:** specific method + number/delta.
   **So what:** connection to tracked interest or paper superseded.
   [abs](https://arxiv.org/abs/ID) | [pdf](https://arxiv.org/pdf/ID)

2. ...

Send abbreviated version via ./notify (<4000 chars) — keep the verdict line:

*Paper Digest — ${today}*
${verdict}

1. "Title" — what's new in ≤12 words (↑${upvotes})
2. ...

Full: articles/paper-digest-${today}.md

7. Log

Append to memory/logs/${today}.md:

### paper-digest
- Verdict: <verdict line>
- Shipped: <arXiv IDs, comma-separated>  <!-- used for next-cycle dedup -->
- Pool: hf=${n_hf}, arxiv=${n_arxiv} → deduped=${n_deduped} → skip-gated=${n_dropped} → shipped=${n_shipped}

Sandbox note

Curl may be blocked. For any failed fetch, retry with WebFetch against the same URL. If HF and arXiv both fail for every topic, notify PAPER_DIGEST_ERROR — all sources unreachable, log, and exit without writing a digest. If only one source works, proceed and note (partial: 1/2 sources) in the digest header.

Constraints

• Never ship a paper without a specific "what's new" sentence (number, name, or delta).
• Never pad to reach 5 — cap is 5, but fewer is better.
• Never ship a paper that doesn't tie to a tracked topic — the whole point is filtering.
• Never re-ship an arXiv ID that appears in the last 7 days of logs.