Run the Stevens Creek Toyota (SCT, dealer ST/876) Alignment-by-Advisor report from the LIVE Tekion OpenAPI — counting BOTH dedicated alignment opcodes (ALIGN/OKAL) AND alignments bundled inside a TEK service-menu op. Produces the Joe-approved 2-page Toyota-red scorecard (page 1 advisor ranking PNG, page 2 RO-level chip detail PDF) and hands to Stacey to DRAFT in Joe's inbox. Supports full-month (last month) or closed month-to-date windows. This is THE canonical AMG alignment-report FORMAT Joe reuses a lot going forward.
Resources
1Install
npx skillscat add joecastelino/jay-skill-pack/sct-alignment-by-advisor-report Install via the SkillsCat registry.
SCT Alignment-by-Advisor Report (LIVE OpenAPI)
Counts alignments SCT sold in a period, broken down by service advisor, capturing:
- Dedicated alignment opcodes sold on an RO:
ALIGN,OKAL,ALIGN00BRA. - Bundled alignments performed inside a TEK service-menu op (any
TEK*opcode
whose operation story contains "align").
Joe approved this exact report + format on 2026-07-01 ("Alignment report is
perfect!") and said the FORMAT will be reused a lot across stores. The generic
per-store version is skill tekion-alignment-by-advisor-report; THIS skill is the
SCT-specific, proven pipeline with the frozen SCT opcode set + scripts.
Scripts (built, proven)
- Scan:
/home/itadmin/tekion-reports/sct_align_full_june.py— the full-month
scanner (edit MS0/MS1 window at top for a different month). Two-tier, rate-limit-safe. - MTD scan:
/home/itadmin/tekion-reports/sct_align_mtd.py— closed month-to-date
(1st of current month → end of today, Pacific). Same architecture; window auto-computed.
STALE SAME-DAY INDEX TRAP (hit 2026-07-10): the scan cachesdata/sct-mtd-<YYYY-MM-DD>-closed-index.jsonkeyed by DATE only. If anything ran the
scan earlier the SAME day (e.g. a 9:31am probe), the 7pm run reuses that morning index
and silently misses every RO closed since (observed 1,354 vs 1,461 ROs → 128 vs 141
alignments — wrong report, exit 0, failed=[]). Before the nightly scan, check the index
file's mtime; if it exists and is hours old,mvit to a.bakand re-run. The
align-scan checkpoint still resumes, so only the NEW candidates fan out (~30 ROs, fast).
A "loaded cached index: N ROs" log line at 7pm is the tell — always question it. - Render:
/home/itadmin/tekion-reports/render_sct_align_full_june.py— reads the
self-contained by-advisor JSON, emits the 2-page scorecard. (A generalizedrender_sct_align.pyreads anysct-*-align-by-advisor*.json+ a period label.) - Interpreter:
/home/itadmin/.hermes/hermes-agent/venv/bin/python3.11. Data dir:/home/itadmin/tekion-reports/data/.
Scan architecture (why it's built this way — DO NOT simplify)
- Window + status: filter
closedTimeBTW [start, end-of-today] +status IN CLOSED,INVOICED. NEVERmodifiedTime(overcounts). Pacific tz (-07:00). - Two-tier, rate-limit-safe: Pass 1 enumerates all closed ROs, capturing per-RO
the FREE opcodetags({field:"OPCODE", value:<code>}) +assignee.advisor.id.
Pass 2 fans out jobs→operations ONLY on CANDIDATES = ROs whose opcode tags include a
dedicated ALIGN code OR anyTEK*code (a bundled alignment can hide inside any menu).
Full June: 5,382 closed → 1,169 candidates. This ~5x reduction keeps the run under
the app-wideOVERALL_RATELIMIT. - DEALER_QUOTA exhaustion — PARTIAL/deep-endpoint variant (hit 2026-08-01): a THIRD
quota type. Signature:/repair-orders:searchand/repair-orders/{id}/jobsreturn 200,
but/jobs/{id}/operations429s withLimit exhausted for type : DEALER_QUOTA.— so the
skill's standard quota probe (search pageSize=1) returns 200 and LIES. The scan builds the
index fine, then hangs silently in the fan-out retry/backoff loop forever (no checkpoint
file ever appears — checkpoint only writes every 20 done ROs). Tells: >15 min with index
built but NO align-scan.json checkpoint; thesct-closed/opened-quota-recoverycron logs
showOPS_429. Diagnose with a probe on the OPS endpoint, not search:O.call('GET','/repair-orders/<rid>/jobs/<jid>/operations'). Both backoff guards in
sct_align_mtd.py now also match "DEALER_QUOTA" (patched 2026-08-01). Playbook: kill the
hung scan, arm the dated self-heal pair (selfheal_sct_align_20260801.shprobes the OPS
endpoint, not search) + handoff watcher; the handoff script now bakes on-disk byte sizes
into the Stacey build ask. Outage observed 19:00 → past 21:15 PDT with no recovery.
DEALER_QUOTA can persist >22h + the sync-all hog + expired-watcher re-arm (2026-08-02):
the same outage was STILL 429 at 5 PM the next day. Three lessons:- Hunt the hog FIRST: the dealer-detail nightly
sync:all(cron 23:00,cron-sct-sync.sh→npm run sync:all→ tsx) had been running 17 hours, stuck
retrying SCT against the 429 wall — it grabs any refill instantly. Diagnose:pgrep -af "sync-all|cron-sct-sync|tsx --conditions"+ps -o lstart=on the flock
pid (a start time from last night = stuck). Fix: kill the WHOLE tree (flock, .sh, npm,
tsx, node pids) and confirm/tmp/dealerdetail-sct-sync.lockis gone. Do this before
re-arming anything. - The 21h self-heal DEADLINE expires mid-outage: an outage starting ~7 PM outlives a
watcher armed at ~8:53 PM. Re-arm with fresh DATED copies — the sed date-swap works:
`sed -e 's/2026-08-01/2026-08-02/g; s/20260801/20260802/g' selfheal_sct_align_20260801.shselfheal_sct_align_20260802.sh
(LOG/LOCK lines carry dates so both rename cleanly),bash -n` both, chmod +x in its own foreground call, kill the old watcher pair, launch
each new one via terminal(background=true, /usr/bin/bash explicit). - Month rollover during an outage: when recovery lands after the 1st, the MTD scan
produces a tiny new-month report. The prior month's FINAL data (through the 31st) is
already on disk (sct-mtd-<mm>-31-align-by-advisor.json) — offer Joe a July-final
render+draft that needs ZERO API quota while waiting.
If still 429 at the next nightly refill, escalate to Joe: multi-day DEALER_QUOTA
non-reset is a Tekion-side problem (raise quota / investigate reset).
Confirmed recurrence 2026-08-03 (7pm nightly, 3rd/4th consecutive day): OPS probe
still 429 DEALER_QUOTA at 19:22 PDT (search+jobs 200, /operations 429 — same signature).
No sync:all hog and no quota_recovery/backfill runner was active this time (checkedpgrep -af "sync-all|cron-sct-sync|tsx --conditions"andpgrep -af "quota_recovery|bt_seed_watcher|sct_closed_backfill"— both empty), so the
non-reset is NOT being caused by a local competing consumer this round — points more at
a Tekion-side quota bucket that simply isn't refilling daily. Armed fresh dated pairselfheal_sct_align_20260803.sh+selfheal_sct_align_handoff_watch_20260803.sh(probe
RO/job ids swapped to a fresh 2026-08-03 candidate: RO 6a710a6bcb86dd4c535a2836 / job
6a710afbcb86dd4c535a9e8f). This is now a MULTI-DAY unresolved outage (8/1→8/2→8/3) —
worth escalating to Joe directly as a Tekion support ticket rather than re-arming
nightly forever.
Confirmed recurrence 2026-08-04 (7pm nightly, 4th consecutive day, still 429 at
next-day 7pm): OPS probe (same RO/job pair6a727c3d84f7f40f9a42f396/6a727e1f84f7f40f9a43c9bereused from the 8/3 probe, confirmed still returns 200 on
search/jobs) still 429 DEALER_QUOTA at 19:04 PDT. Re-checked for a local competing
consumer again (sync-all|cron-sct-sync|tsx --conditionsandquota_recovery|bt_seed_watcher|sct_closed_backfill|sct_align_mtd|selfheal_sct_align)
— both empty, confirming (2nd time in a row) this is NOT a local hog; the Tekion-side
DEALER_QUOTA bucket for SCT is simply not resetting daily. Armed fresh dated pairselfheal_sct_align_20260804.sh+selfheal_sct_align_handoff_watch_20260804.shvia
the standard `sed -e 's/2026-08-03/2026-08-04/g; s/20260803/20260804/g' .sh >
.sh
date-swap (LOG/LOCK lines carry dates so both rename cleanly) — reused the SAME probe RO/job ids as 8/3 since they still validate (200 on jobs, 429 only on operations); no need to hunt a fresh candidate every day if the old probe pair still round-trips through search/jobs.bash -nboth before chmod, chmod in its own foreground call, launched each via terminal(background=true, /usr/bin/bash explicit). No prior-outage watchers were still alive to kill (8/3 pair had already hit its 21h/22h TIMEOUT and exited cleanly per its own log). At month rollover (this outage started right at 8/1) there is no partial-month fallback to render meanwhile — unlike the 7/31→8/1 case, there was no "prior month final" data to offer since the July report had already shipped. This is now a 4-DAY continuous unresolved outage (8/1→8/2→8/3→8/4) — each nightly run should keep re-arming the self-heal pair (it costs nothing and will catch a mid-outage recovery) but should ALSO reiterate the Tekion-side escalation in its report every time, rather than treating it as routine. Recommend Joe open a formal Tekion support ticket if this persists past day 4-5, since local self-heal cannot fix a server-side quota bucket that never refills. **Confirmed continuing 8/5 and 8/6 (days 5 and 6, still 429 both nights):** same OPS probe signature (search/jobs 200,/operations429 DEALER_QUOTA) held straight through — the 8/5 self-heal watcher logged a 429 every ~10 min for its full run with no recovery (never hit "quota restored"), and the fresh 8/6 probe at 19:01 PDT was still 429. No local competing consumer found either night (checked the same process list both times). Reused the SAME validated probe RO/job pair unchanged since 8/3 (still round-trips 200 on search/jobs) — don't bother hunting a new candidate as long as the old one still validates. **This is now a 6-CONSECUTIVE-DAY unresolved outage (8/1→8/6) with zero August MTD reports shipped** — at this length, stop treating "keep re-arming nightly" as sufficient; the report to Joe should explicitly recommend a formal Tekion support ticket rather than soft-pedaling it as routine self-heal. **Confirmed continuing 8/7 (day 7, still 429):** same signature (search/jobs 200,/operations429 DEALER_QUOTA) at 19:01 PDT, cross-checked against the independentsct-closed-quota-recoveryjob's own probes logged 18:03-18:54 the same evening — also all 429. No local competing consumer found (checkedsync-all|cron-sct-sync|tsx --conditionsandquota_recovery|bt_seed_watcher|sct_closed_backfill|sct_align_mtd|selfheal_sct_align— both empty). Reused the standard sed date-swap (sed -e 's/2026-08-06/2026-08-07/g; s/20260806/20260807/g' .sh > .sh) for BOTH the probe script and its handoff-watch pair,bash -nboth, chmod in its own foreground call, launched each via terminal(background=true, /usr/bin/bash explicit) — confirmed alive via the log's "watcher started" line within 5s. Before arming, asked Stacey a terse read-only Gmail search ("subject substring 'SCT Alignment' in Drafts and Sent, last 10 days") to confirm the last thing that actually shipped: July full-month report, Sent 2026-08-02 — zero August MTD reports have gone out. **7 CONSECUTIVE DAYS (8/1-8/7) is well past routine self-heal territory** — every nightly report in this window should explicitly recommend Joe open a formal Tekion support ticket, not just re-arm and move on. **When the current scan is blocked, confirm what's already in Drafts before assuming nothing shipped:** ask Stacey a terse read-only Gmail search for a SUBSTRING (not exact subject match, per the exact-subject false-zero trap above) like "SCT Alignment" to find the most recent draft already sitting in Joe's Drafts (e.g. the July MTD-final report from before the outage began) — useful context to include in the blocked-night report so Joe knows the last successful data point even while August is stuck. **Confirmed continuing 8/8 and 8/9 (days 8 and 9, still 429):** the 8/8 self-heal watcher ran its full 21h deadline probing every ~10 min (124 log lines, all 429) and hit "TIMEOUT — quota never restored" at 8/9 16:05 PDT with zero recovery. Fresh 8/9 probe at 19:01 PDT (same validated RO/job pair, unchanged since 8/3) also 429. No local competing consumer either night (checked the standard process list). Re-armed the standard dated pair via the sed date-swap,bash -n+ chmod + terminal(background=true) launch, confirmed alive via pgrep + log "watcher started" line — same recipe as prior days, no changes needed. Stacey Gmail check (read-only, "SCT Alignment" substring, Drafts+Sent, last 10 days) confirmed the last actually-SENT report is July Full Month (sent Aug 2, to Joe) and the July MTD-through-7/31 draft is still sitting unsent in Drafts — zero August reports (draft or sent) exist as of day 9. **9 CONSECUTIVE DAYS (8/1-8/9)** — this is well past the point where "keep re-arming nightly" should be reported as routine; explicitly recommend a formal Tekion support ticket in every blocked-night report at this point, not just on first crossing day 4-5. **Confirmed continuing 8/10 (day 10, still 429):** 19:01 PDT probe on the same validated RO/job pair (unchanged since 8/3) — search/jobs 200,/operations429 DEALER_QUOTA, identical signature to every prior day. No local competing consumer (sync-all|cron-sct-sync|tsx --conditionsandquota_recovery|bt_seed_watcher|sct_closed_backfill|sct_align_mtd|selfheal_sct_alignboth empty — the 8/9 watcher pair had already exited cleanly at its own TIMEOUT, nothing to kill). Re-armed the standard dated pair via the sed date-swap (sed -e 's/2026-08-09/2026-08-10/g; s/20260809/20260810/g' .sh > .sh),bash -nboth, chmod in its own foreground call, launched each via terminal(background=true, /usr/bin/bash explicit), confirmed alive via pgrep + the log's "watcher started" line within 5s. Stacey Gmail check (read-only, "SCT Alignment" substring, Drafts+Sent, last 10 days) reconfirmed: last SENT = July Full Month (Aug 2), last DRAFT = July MTD-through-7/31 (still unsent) — zero August reports (draft or sent) as of day 10. **10 CONSECUTIVE DAYS (8/1-8/10) with zero August MTD reports shipped** — every nightly report in this window must explicitly and prominently recommend Joe open a formal Tekion support ticket for the SCT DEALER_QUOTA bucket; this is no longer routine self-heal territory by any measure. **OUTAGE RESOLVED 2026-08-11 (day 11, first clean run after 10 straight 429 nights):** the standard OPS probe (/repair-orders/{rid}/jobs/{jid}/operationson the same validated RO/job pair reused since 8/3) returned a clean 200 at 19:01 PDT. Ran the MTD scan immediately — 1,403 closed ROs, 405 candidates, 0 failed, completed in a few minutes (no rate-limit backoff triggered at all). First successful August MTD report shipped: 132 alignments (118 dedicated + 14 bundled), 132 ROs, 16 advisors. Lesson: the quota DOES eventually self-heal on the Tekion side without any local fix — always probe fresh each night before assuming the outage continues; don't skip straight to "re-arm and escalate" without a live check. No self-heal watcher was needed this run. **Confirmed stable 2026-08-12 (2nd consecutive clean night post-outage):** OPS probe 200 immediately, no self-heal needed. Full run clean: 1,658 closed ROs, 513 candidates, 0 failed, ~19 min. 154 alignments (137 dedicated + 17 bundled), 154 ROs, 16 advisors, top Cristian Gonzalez (25). Stacey's build ask timed out (exit 124) on the first try as usual — the standard fix (sleep 60s, then a single subject-anchored verify ask) resolved it cleanly: DRAFTS_COUNT=1, SENT=no, PDF part non-zero (303,528B for a 233,121B file, within normal encoding variance), HTML part 171,532B comfortably above the PNG*4/3 floor. First-try clean draft, no rebuild needed. The DEALER_QUOTA outage looks resolved for good at this point — no further escalation language needed unless it recurs. Confirmed again 2026-08-13 (3rd consecutive clean night): no quota issues at all — index built 1,903 closed ROs, 596 candidates, 0 failed, ~23 min scan (steady ~80 ROs/3min pace, checkpoint mtime advancing throughout — no backoff triggered). 165 alignments (148 dedicated + 17 bundled), 165 ROs, 16 advisors, top Cristian Gonzalez (25). Stacey's build ask did NOT time out this time — completed in 114s on the first try with sizes baked into the build ask (per the 07-22 prevention note), full verification in her own reply (HTML 127,110B >= PNG*4/3=126,483; PDF part exactly 238,779B matching the file). A shortsleep 15(not the usual 60s) before the subject-anchored verify ask was enough — it answered in 44s with DRAFTS_COUNT=1, correct TO, SENT_FOLDER_COUNT=0. Confirms the pipeline is now reliably stable post-outage; a Stacey build timeout is no longer "usual", just occasional. **Confirmed again 2026-08-14 (4th consecutive clean night, quota fully healthy):** index built 2,145 closed ROs, 682 candidates, 0 failed, ~25 min scan (19:02→19:27, no backoff triggered). 190 alignments (168 dedicated + 22 bundled), 190 ROs, 16 advisors, top Cristian Gonzalez (26). Stacey's build completed clean on the first ask (38s) with sizes baked in per 07-22 prevention. BUT the note-6 dedupe trap fired anyway — a PARTS-list verify ask surfaced **3 draft UIDs** with that subject (59/67/72, PDF parts 303,528B → 326,752B → 344,454B, growing each revision) even though only ONE build ask was ever sent — Stacey's internal retry/revision process silently creates extra draft copies without a distinct build request each time. Size math confirmed the newest (UID 72, largest PDF) was the good one: HTML part 175,878B >= PNG*4/3=127,743B; PDF part 344,454B is +2.6% over PDF*4/3=335,617B (normal encoding variance). **Effective dedupe technique:** don't just say "delete duplicates" — give her the EXACT UIDs to keep vs delete ("Keep ONLY UID 72, delete UID 59 and UID 67") from a PARTS-list ask that already enumerated them; this resolved cleanly in one ask (59.9s) with a one-line confirm (DRAFTS_COUNT=1). Final verify showed PDF_PART_BYTES=251,713 exactly matching the on-disk PDF file size — a clean, exact byte-count match is the strongest possible confirmation, stronger than the ~4/3 encoded-size heuristic. Lesson: even on a fast, non-timing-out build night, ALWAYS run the PARTS-list verify (not just a generic DRAFTS_COUNT ask) — it's the only ask that reveals multiple UIDs so you can dedupe by exact UID instead of a vague "delete the old ones" request. **Confirmed again 2026-08-17 (7th consecutive clean night, quota fully healthy) — UID INSTABILITY across sequential verify asks:** scan clean (2,618 closed, 869 candidates, 0 failed, ~34 min). 247 alignments (224 dedicated + 23 bundled), 247 ROs, 16 advisors, top Cristian Gonzalez (30). Stacey's build completed clean on the FIRST ask (115s) with sizes baked in. Verify step 1 (subject-anchored DRAFTS_COUNT/ TO/SENT_FOLDER_COUNT) came back clean instantly: 1 | kstapp@sctoyota.com | 0. Verify step 2 (combined PARTS/RAW_SIZE ask) timed out (124) as usual; per note 11's 07-27 pattern, degraded to the terse two-line form ("TO=? RAW_SIZE=?") after a 60s pause — but this answer cited **"UID 88"**, while the original build confirmation had cited **"UID 42368"** — two different UID numbers for what should be the same single draft. Rather than assume a duplicate was created, ran ONE more explicit ask: "List all drafts with subject substring X dated today, reply UID=<uid> SIZE=<bytes> per draft, then DRAFTS_COUNT=<n>" — this came back with a THIRD UID (42251) and DRAFTS_COUNT=1, plus a clarifying note that 4 drafts total match the subject substring but only 1 is dated today (others are from different nights, per the 8/16 multi-night-match note). **Lesson: Stacey's reported UID numbers are NOT stable/reliable across separate asks, even referring to the same single draft in the same conversation turn window** — don't panic-dedupe on a UID mismatch across asks. The authoritative signal is a DATE-ANCHORED DRAFTS_COUNT (not the UID value itself); if that count is 1, there is one draft, regardless of which UID number she quotes in different replies. Only trust UID values for dedupe INSTRUCTIONS within the SAME ask/reply pair (per the 8/14 "keep UID X delete UID Y" pattern), never across separate asks. **Confirmed again 2026-08-18 (8th consecutive clean night) — SENT_FOLDER_COUNT>0 is NOT a send-trap alarm once Joe starts sending prior drafts:** pre-flight OPS probe 200. Index 2,786 closed ROs, 932 candidates, 0 failed, ~29 min (19:01→19:30). 264 alignments (239 dedicated + 25 bundled), 264 ROs, 16 advisors, top Cristian Gonzalez (31). Stacey's build clean on the FIRST ask (163s) with paths + on-disk sizes baked in; her own reply gave HTML part 176,412B >= PNG*4/3=129,825B and PDF part 288,540B = EXACT on-disk size. The subject-anchored verify returnedDRAFTS_COUNT=1 | TO=kstapp@sctoyota.com |
SENT_FOLDER_COUNT=3— the 3 are PRIOR NIGHTS' reports Joe reviewed and sent himself in the morning (the whole point of draft-only), NOT evidence Stacey sent anything. Do NOT panic on a nonzero Sent count now that the nightly has been running for weeks. Correct tiebreaker = a TODAY-scoped one-liner: "In Sent Mail, how many with subject substring 'SCT Alignment' were sent TODAY? Reply: SENT_TODAY=<n>" → came back 0. (The richer "list every Sent match with subject+date" ask timed out exit 124; per note 11, a 60s pause then the ultra-terse single-number form answered fine. Keep re-asks to ONE question with ONE number.) **Confirmed again 2026-08-19 (9th consecutive clean night, quota fully healthy):** pre-flight OPS probe (same validated RO/job pair, unchanged since 8/3) returned 200 — keep doing this probe even deep into a healthy streak, it costs one call. Index 3,046 closed ROs, 1,008 candidates, 0 failed, ~38 min scan (19:01→19:39, no backoff). 287 alignments (256 dedicated + 31 bundled), 287 ROs, 16 advisors, top Cristian Gonzalez (33) — note Jaime Sanchez and Artist Battle tied at 30, so Cristian's usual #1 spot is narrowing. Stacey's build clean on the FIRST ask (67s) with paths + on-disk sizes baked in; her own reply gave HTML part 176,942B and PDF part 405,746B. Standard two-ask verify both answered on the first try with only a short sleep (15s then 10s, not 60s):DRAFTS_COUNT=1 | TO=kstapp@sctoyota.com | SENT_FOLDER_COUNT=0, then PARTSRAW_SIZE=591,957 | text/html=179,240 | application/pdf=411,016— HTML >= PNG*4/3 (130,233) confirms inline PNG, PDF part +2.6% over PDF*4/3 (400,476) is the normal CRLF-wrap variance, and RAW_SIZE ≈ html+pdf parts cross-checks. Note her reply strips underscores (DRAFTSCOUNT,PDFPART_BYTES`) — same answer, don't re-ask. Also note the
part sizes she reports in the BUILD confirmation vs the later PARTS ask differ slightly
(176,942 vs 179,240 html; 405,746 vs 411,016 pdf) — both pass the lower-bound math;
small discrepancies between her two reports of the same part are NOT a duplicate-draft
signal (the date-anchored DRAFTS_COUNT=1 is authoritative, per the 8/17 UID-instability
note).
Confirmed again 2026-08-20 (10th consecutive clean night) — FLAT-DAY / STORE
POSTING-LAG false alarm: OPS probe 200. Index 3,065 closed ROs, 1,008 candidates,
0 failed, ~42 min (19:01→19:43). Result 287 alignments (256 ded + 31 bundled), 287
ROs, 16 advisors — byte-for-byte the same candidate count (1,008) and the same
total (287) as 8/19, which looks exactly like the stale-same-day-index trap. It was
NOT. Diagnosis recipe when today's MTD numbers don't move:- Confirm the index is FRESH, not a cached reuse — compare the
windowend
timestamp insct-mtd-<today>-closed-index.jsonvs yesterday's (must advance one
day: 1787209199999 → 1787295599999) and diff the RO id sets
(new in 8/20: 19, dropped: 0). A reused index would have an IDENTICAL window. - Then confirm at the API level with a single-day closed count per day rather than
trusting the MTD delta: closed 8/18=167, 8/19=261, 8/20=19. Only 19 ROs closed
store-side all day (cashiering/posting lag), and none of the 19 were alignment
candidates → new candidates today = 0 → totals legitimately unchanged.
So identical day-over-day totals can be REAL. Always run the two checks above before
suspecting a stale index or a broken scan; conversely a matchingwindowend
timestamp is the definitive tell that you DID hit the stale-index trap.
Also note:sct-mtd-latest-align-by-advisor.jsonis a COPY, not a symlink — arealpathcomparison against the dated file will (correctly) return False. Don't
treat that as a stale-pointer bug; verify by reading itsperiod_label/totals
instead.
Stacey's build clean on the FIRST ask (70s) with paths + on-disk sizes baked in
(HTML part 131,086 >= PNG4/3=130,319; PDF part exactly 300,357B = on-disk size).
Standard two-ask verify, short sleeps (15s then 10s), both first-try:DRAFTS_COUNT=1 | TO=kstapp@sctoyota.com | SENT_FOLDER_COUNT=0, thenRAW_SIZE=592,097 | text/html=179,384 | application/pdf=411,016(+2.6% CRLF variance,
RAW_SIZE ≈ html+pdf). Underscore-stripped keys again — same answer, don't re-ask.
Confirmed again 2026-08-21 (11th consecutive clean night) — TIGHT HTML-part pass is
still a PASS: OPS probe 200 (same validated RO/job pair, unchanged since 8/3). Index
3,089 closed ROs, 1,012 candidates, 0 failed, ~37 min (19:01→19:38, no backoff).
Window end 1787381999999 advanced one day from 8/20's 1787295599999 — the definitive
not-a-stale-index check. 290 alignments (259 dedicated + 31 bundled), 290 ROs, 16
advisors, top Cristian Gonzalez (33). Stacey's build clean on the FIRST ask (52s) with
paths + on-disk sizes baked in. Two-ask verify, short sleeps (15s then 10s), both
first-try:DRAFTS_COUNT=1 | TO=kstapp@sctoyota.com | SENT_FOLDER_COUNT=0, thenRAW_SIZE=594,374 | text/plain=434 text/html=131,618 application/pdf=301,484. Note the
HTML part cleared the PNG4/3 floor (131,060) by only 558 bytes — a far tighter pass
than the usual ~45KB headroom nights where Joe's heavy HTML signature inflates it. Per
note 13 a tight pass IS a pass (her build sometimes omits the heavy signature); do NOT
rebuild over thin headroom when the PDF part is an exact on-disk byte match (301,484)
and DRAFTS_COUNT=1. Also: RAW_SIZE (594,374) exceeding html + pdf4/3 (533,597) is
normal header/encoding overhead — only a RAW_SIZE below that sum is a red flag.
Underscore-stripped keys again (DRAFTSCOUNT,SENTFOLDER_COUNT) — same answer.
Confirmed again 2026-08-22 (12th consecutive clean night) — SECOND flat-day, and a
TRUE ZERO-CLOSE day: OPS probe 200. Index 3,089 closed ROs, 1,012 candidates, 0 failed,
~37 min (19:01→19:38). Totals 290 alignments (259 ded + 31 bundled), 290 ROs, 16 advisors,
top Cristian Gonzalez (33) — byte-identical to 8/21 in every figure, and the index file
was even the same byte size (425,567), which looks alarming. Ran the note-20 flat-day
recipe and it cleared: thewindowend advanced one day (1787381999999 → 1787468399999,
so NOT a cached index), but the RO id-set diff wasnew: 0, dropped: 0— unusual, since
8/20's flat day still had 19 new ROs. Per-day API closed counts explained it: 8/19=261,
8/20=19, 8/21=24, 8/22=0. A Saturday with literally zero ROs closed store-side means
the MTD set cannot change at all, so identical index size + identical totals are correct.
Recipe refinement:new in index = 0alone is NOT proof of a stale index — pair it with
the window-end check (definitive stale tell) and the single-day closed count; if that
count is 0, an unchanged report is the right answer. Also note the per-day count helper
must sendvalues(notvalue) as STRING lists in the search filter — avalue/int
form returns HTTP 500 with no totalCount and will mislead you into thinking the API is
broken; paginate and count results rather than trusting ameta.totalCountfield (it
isn't returned). Stacey's build clean on the FIRST ask (93s) with paths + on-disk sizes
baked in. Two-ask verify, short sleeps (15s then 10s), both first-try:DRAFTS_COUNT=1 | TO=kstapp@sctoyota.com | SENT_FOLDER_COUNT=0, thenRAW_SIZE=594,777 | text/plain=608 text/html=180,502 application/pdf=412,560— HTML part
clears PNG4/3 (131,174) with the usual heavy-signature headroom, PDF part +2.6% over
PDF4/3 (401,979) is the normal CRLF variance, RAW_SIZE ≈ html+pdf parts. Underscore-
stripped keys again.
Confirmed again 2026-08-23 (13th consecutive clean night) — THIRD flat-day, near-zero
close day, cleared by the note-20/22 recipe: OPS probe 200 (same validated RO/job pair,
unchanged since 8/3); no same-day index existed pre-run (checked mtime first per the stale-
index trap). Index 3,091 closed ROs, 1,012 candidates, 0 failed, ~38 min (19:01→19:39).
Totals 290 alignments (259 ded + 31 bundled), 290 ROs, 16 advisors, top Cristian Gonzalez
(33) — identical totals to BOTH 8/21 and 8/22 (three flat nights in a row). Cleared via
the standard recipe: window end advanced 1787468399999 → 1787554799999 (NOT a cached index),
id-set diffnew: 2, dropped: 0, and per-day API closed counts 8/20=19, 8/21=24, 8/22=0,
8/23=2 — a Sunday with only 2 ROs closed store-side, neither an alignment candidate, so
unchanged totals are correct. Note the index FILE SIZE grew slightly (425,567 → 425,823)
which is a cheap extra confirmation the index is fresh even when totals don't move. Also
note the leaderboard tightened further: Jaime Sanchez 31, Chris Mai 30, Artist Battle 30 —
Cristian's lead is now 2. Stacey's build clean on the FIRST ask (163s) with paths + on-disk
sizes baked in; her reply gave HTML part 132,033B and PDF part 301,487B (exact on-disk
match). Two-ask verify, short sleeps (15s then 10s), both first-try:DRAFTS_COUNT=1 | TO=kstapp@sctoyota.com | SENT_FOLDER_COUNT=0, thenRAW_SIZE=595,032 | text/plain=630 text/html=180,678 application/pdf=412,564— HTML clears
PNG4/3 (131,308) with heavy-signature headroom, PDF part +2.6% CRLF variance over
PDF*4/3 (401,983), RAW_SIZE ≈ html+pdf. Underscore-stripped keys again. Note her BUILD
confirmation reported the PDF part as the exact decoded file size (301,487) while the later
PARTS ask reported the encoded size (412,564) — per note 9/8 BOTH forms are valid passes;
the discrepancy between her two reports of the same part is not a duplicate signal.
Confirmed again 2026-08-24 (14th consecutive clean night) — VOLUME RESUMED + a NEW
"TO=" verify false alarm: OPS probe 200 (same validated RO/job pair,
unchanged since 8/3); no same-day index existed pre-run. Index 3,320 closed ROs (up 229
from 8/23's 3,091 — the three-day flat spell ended, Monday catch-up posting), 1,079
candidates, 0 failed, ~35 min (19:01→19:36). Window end advanced 1787554799999 →
1787641199999 and index file grew 425,823 → 458,053 B — both fresh-index confirmations.
317 alignments (285 dedicated + 32 bundled), 317 ROs, 16 advisors. Leaderboard is now a
THREE-WAY TIE at #1 — Jaime Sanchez, Chris Mai, Cristian Gonzalez all 33 (Cristian's
long-running solo #1 finally gone; Artist Battle 30 close behind). When the top spot is
tied, phrase the email summary as "a three-way tie between X, Y and Z with N each" rather
than arbitrarily picking one — Joe's report reads as a ranking. Stacey's build clean on
the FIRST ask (81s) with paths + on-disk sizes baked in; her reply gave HTML part 176,347B
= PNG4/3=129,751B and PDF part 427,654B (+1.3% over PDF4/3=422,100 — normal CRLF
variance).
NEW TRAP — the subject-anchored verify can answerTO=<a PERSON NAME>(the From/account
identity), not the recipient address: tonight's clean verify returnedDRAFTSCOUNT=1 | TO=Joe Castelino | SENTFOLDER_COUNT=0. "Joe Castelino" is the mailbox
OWNER, not the To: header — it directly contradicted the build confirmation'sTO=kstapp@sctoyota.com. Do NOT rebuild or panic-dedupe on this. Tiebreaker = one
ultra-terse read-only ask for the RAW HEADER: "print its raw To: header line exactly as
stored. Reply ONE line: TO_HEADER=" → came backTO_HEADER=kstapp@sctoyota.com. Lesson: when the verify's TO field looks like a human
name rather than an email address, she's read the account/From identity; ask forTO_HEADERexplicitly. (Same family as the note-15 un-anchored garble and note-17 UID
instability — her field labels are not trustworthy, the raw header is.)
Also note: her first Drafts search MISSED the draft entirely and she self-corrected —
"himalaya uses ISO dates (2026-08-24) not text dates (Aug 24)". A zero/miss on a
dated-today search that she then retries is a search-syntax artifact, not a missing
draft; let her self-correct before re-asking. Short sleeps sufficed throughout (15s
then 10s), no timeouts at all this run.
Confirmed again 2026-08-25 (15th consecutive clean night) — NEW TRAP: PARTS-reported
PDF size can equal a PRIOR NIGHT'S file size: OPS probe 200 (same validated RO/job pair,
unchanged since 8/3); no same-day index pre-run. Index 3,514 closed ROs (up 194), 1,079+
candidates, 0 failed, ~43 min (19:01→19:44). Window end advanced 1787641199999 →
1787727599999 and index grew 458,053 → 484,759 B — both fresh-index confirmations; id diff
new: 194, dropped: 0. 325 alignments (293 dedicated + 32 bundled), 325 ROs, 16 advisors.
Two-way tie at #1: Jaime Sanchez and Cristian Gonzalez, 34 each (Chris Mai / Artist
Battle 33) — phrase ties as ties per the 8/24 note. Stacey's build clean on the FIRST ask
(102s) with paths + on-disk sizes baked in.
The trap: the PARTS verify returnedapplication/pdf:316575— which was NOT today's
on-disk PDF (320,623) but was an EXACT match for the PREVIOUS night's (8/24) PDF file size,
which looks exactly like she attached the wrong/stale file. She did not. The note-14 decode
tiebreaker settled it in one 27s ask — extend it to also request the FILENAME, which is the
strongest single signal: "PDF_FILENAME= | PDF_DECODED_BYTES="
→SCT-Alignment-By-Advisor-MTD-2026-08-25.pdf | 320623, i.e. correct file, exact byte
match. Lesson: a PARTS PDF size matching a PRIOR night's file size is a coincidental
misreport, not a stale attachment — ALWAYS resolve with filename+decoded-bytes before
rebuilding, since the filename carries the date and removes all ambiguity. Also note the
HTML part passed the PNG4/3 floor by only 117 bytes (130,540 vs 130,423) — even
tighter than the 8/21 558-byte pass; a tight pass is still a PASS (note 13).
Two-ask verify with short sleeps (15s then 10s), both first-try:DRAFTS_COUNT=1 | TO=kstapp@sctoyota.com | SENT_FOLDER_COUNT=0. Underscore-stripped keys
again. Also: her PARTS list included zero-size structural parts
(multipart/mixed:0, multipart/alternative:0) — those are container parts, NOT the
note-13 zero-byte-PDF trap; only a zero-sizeapplication/pdfpart is a failure.
Confirmed again 2026-08-26 (16th consecutive clean night) — NEW TRAP:DRAFTS_COUNT=2
from a LOOSE "dated today" match is a FALSE duplicate alarm: OPS probe 200 (same validated
RO/job pair, unchanged since 8/3); no same-day index pre-run. Index 3,751 closed ROs (up 237),
1,248 candidates, 0 failed, ~55 min (19:01→19:56 — longest of the month, late-month window
ballooning as expected). Window end advanced 1787727599999 → 1787813999999 and index grew
484,759 → 515,615 B (both fresh-index confirmations). 333 alignments (300 dedicated + 33
bundled), 333 ROs, 16 advisors. Three-way tie at #1 again: Jaime Sanchez, Chris Mai,
Cristian Gonzalez, 35 each (Artist Battle 33) — phrase as a tie per the 8/24 note.
Stacey's build clean on the FIRST ask (78s) with paths + on-disk sizes baked in; HTML part
130,060B cleared PNG4/3 (129,304) by only 756 bytes (tight pass = PASS, note 13) and
PDF part 324,005B was an EXACT on-disk match.
The trap: the terse dated-today ask returnedDRAFTS_COUNT=2, which looks exactly like
the note-6 duplication norm — but there was only ONE draft for tonight. Two follow-ups timed
out (exit 124) before the resolution; the ask that settled it definitively was a date-free
subject-substring enumeration: "Gmail Drafts, subject substring 'SCT Alignment Sales by
Advisor'. Reply one line per match: SUBJECT= DATE=" → returned 14 matches,
one per night (8/14 through 8/26 plus the July MTD leftover), each with a DISTINCT through-date
in the subject and its own date. Only one said "(through 8/26)". Lesson: Stacey's "dated today"
filter is unreliable and can pull in the adjacent night's draft, inflating DRAFTS_COUNT by one.
Before deduping on a count of 2, ALWAYS run the date-free SUBJECT+DATE enumeration and check
the through-date in each subject — the through-date is the authoritative per-night
discriminator (stronger than DRAFTS_COUNT, stronger than UIDs per note 17). Deleting on a bare
count of 2 here would have destroyed the previous night's draft. Also note the count-versus-
enumeration disagreement is the same family as notes 12/15/17: trust the enumeration, not the
aggregate. Final verify:TO_HEADER=Kevin Stapp <kstapp@sctoyota.com>(raw-header form per
note 8/24) andSENT_TODAY=0. Timeout pattern held to note 11 — 60s pause then a ONE-number
ask answered every time.
Confirmed again 2026-08-27 (17th consecutive clean night) — PUT THE THROUGH-DATE IN THE
VERIFY SUBSTRING and the note-26 DRAFTS_COUNT=2 trap never fires: OPS probe 200 (same
validated RO/job pair, unchanged since 8/3); no same-day index pre-run; no competing
consumer. Index 3,975 closed ROs (up 224 from 8/26's 3,751), 0 failed, ~49 min
(19:01→19:50). Window end advanced 1787813999999 → 1787900399999 and index grew
515,615 → 544,982 B (both fresh-index confirmations). 345 alignments (312 dedicated + 33
bundled), 345 ROs, 16 advisors. Two-way tie at #1: Jaime Sanchez and Chris Mai, 36 each
(Cristian Gonzalez 35 — he has now slipped off the top spot entirely after a month of
leading); phrase ties as ties per the 8/24 note.
Refinement that avoids the 8/26 false-duplicate entirely: anchor the verify substring
with the FULL subject INCLUDING the through-date —'SCT Alignment Sales by Advisor - August MTD (through 8/27)'— rather than the generic'SCT Alignment Sales by Advisor - August MTD'. The through-date is the authoritative per-night discriminator (note 26), so
baking it into the substring makes DRAFTS_COUNT inherently per-night: returned a cleanDRAFTS_COUNT=1on the first ask with zero enumeration follow-up needed, where the
date-free form would have matched ~14 nights. Note the parens/dashes did NOT break the
search here because it was explicitly framed as a SUBSTRING match ("substring match, not
exact") — the note-11 exact-subject false zero is about EXACT matching, not punctuation
per se. Also foldTO_HEADER(note 8/24) andSENT_TODAY(note 8/18) into that same
first ask instead ofTO/SENT_FOLDER_COUNT— both preempt their own false alarms in
one round-trip:DRAFTSCOUNT=1 | TOHEADER=kstapp@sctoyota.com | SENT_TODAY=0.
Recommended verify ask #1 going forward: subject substring WITH through-date →DRAFTS_COUNT=<n> | TO_HEADER=<exact To: header> | SENT_TODAY=<n>.
Stacey's build clean on the FIRST ask (125s) with paths + on-disk sizes baked in. PARTS
ask (with PDF_FILENAME folded in per the 8/25 note) answered first-try after a 10s sleep:RAW_SIZE=633,032 | text/plain:430, text/html:131,922, application/pdf:329,436 | PDF_FILENAME=SCT-Alignment-By-Advisor-MTD-2026-08-27.pdf. HTML part cleared PNG4/3
(131,145) by only 777 bytes (third tight pass of the month — a tight pass is a PASS,
note 13), PDF part was an EXACT on-disk match, and the filename carried today's date
(strongest possible attachment confirmation). Her BUILD confirmation had reported the HTML
part as 178,305 vs the PARTS ask's 131,922 — the familiar two-reports-of-the-same-part
discrepancy (notes 8/9/8-23), both pass, not a duplicate signal. Short sleeps throughout
(15s then 10s), no timeouts. Underscore-stripped keys again.
Confirmed again 2026-08-28 (18th consecutive clean night) — the note-27 through-date
substring anchor is NOT enough on its own; the ENUMERATION is: OPS probe 200 (same
validated RO/job pair, unchanged since 8/3); no same-day index pre-run; no competing
consumer. Index 4,214 closed ROs (up 239 from 8/27's 3,975), 1,389 candidates, 0 failed,
~51 min (19:01→19:52). Window end advanced 1787900399999 → 1787986799999 and index grew
544,982 → 579,593 B (both fresh-index confirmations); id diff new: 239, dropped: 0.
361 alignments (327 dedicated + 34 bundled), 361 ROs, 16 advisors. Chris Mai is now
SOLO #1 with 40 (Jaime Sanchez 38; Artist Battle and Cristian Gonzalez tied 35) — the
multi-way tie at the top has broken.
Trap refinement: the note-27 recipe (bake the through-date into the verify substring)
did NOT prevent the false count this time — the ask used the full'SCT Alignment Sales by Advisor - August MTD (through 8/28)'substring and STILL came
backDRAFTS_COUNT=15, i.e. she matched every August night despite the date being in
the substring. So the through-date anchor helps but is not reliable; the date-free
SUBJECT+DATE enumeration (note 26) is the only trustworthy per-night discriminator.
It resolved in 30s: 16 matches listed, each with a DISTINCT through-date, exactly ONE
reading "(through 8/28)". Never dedupe on a raw DRAFTS_COUNT — always enumerate first.
Also note that enumeration incidentally reveals Joe has 15 August drafts still sitting
unsent in Drafts (8/14 → 8/28) plus the July MTD leftover; that's his review backlog,
not a pipeline fault.
Stacey's build completed on the FIRST ask (119s) with paths + on-disk sizes baked in,
though her transcript showed several internal self-corrections (typo'd addheader, a
missed regex, "syntax mangling") before succeeding — internal retries do NOT imply
duplicate drafts (enumeration proved 1). Her build reply: HTML part 177,575B, PDF part
337,054B (exact on-disk). Final PARTS ask (162s, no timeout):RAW_SIZE=642,755 | text/plain:385 text/html:131,451 application/pdf:337,054 | PDF_FILENAME=SCT-Alignment-By-Advisor-MTD-2026-08-28.pdf— HTML cleared PNG4/3
(130,731) by only 720 bytes (fourth tight pass of the month, still a PASS per note
13), PDF part an exact on-disk match, filename carrying today's date.SENT_TODAY=0,TO_HEADER=kstapp@sctoyota.com. Note again her build reply vs PARTS reply disagreed on
the HTML part size (177,575 vs 131,451) — the familiar two-reports-of-the-same-part
discrepancy, both pass, not a duplicate signal.
Confirmed again 2026-08-29 (19th consecutive clean night) — NEW TRAP: MISSPELLED
ATTACHMENT FILENAME (content perfect, name wrong): OPS probe 200 (same validated RO/job
pair, unchanged since 8/3); no same-day index pre-run; no competing consumer. Index 4,240
closed ROs (up only 26 from 8/28's 4,214 — light Saturday), 1,393 candidates, 0 failed,
~53 min (19:01→19:54). Window end advanced 1787986799999 → 1788073199999 and index grew
579,593 → 583,458 B (both fresh-index confirmations); id diff new: 26, dropped: 0.
365 alignments (331 dedicated + 34 bundled), 365 ROs, 16 advisors. Chris Mai extends his
solo #1 to 43 (Jaime Sanchez 39; Artist Battle and Cristian Gonzalez tied 35).
The new trap: Stacey's build was clean on the FIRST ask (60s) and every size check
passed — but the PDF ATTACHMENT FILENAME came backSCT-A**ll**ignment-By-Advisor-MTD-2026-08-29.pdf(double L). Confirmed real, not a reply
typo, by re-asking forPDF_DECODED_BYTES+FILENAME_EXACTin one line: bytes were an
exact on-disk match (338,883) while the filename still read "Allignment". So the note-25
filename check is not only an anti-stale-attachment signal — it also catches cosmetic
misspellings in the name Kevin actually sees when he opens the mail. Always read the
filename character-by-character, don't skim it; "Alignment" vs "Allignment" is easy to miss.
Fix recipe (worked in ONE 90s ask, no duplicates): tell her to imap.append() ONE fresh
draft identical to the current '(through 8/29)' one but with the correct filename, then
DELETE the misspelled one — and explicitly scope-fence the other nights: "there are 16
other SCT Alignment drafts (through 8/14 to 8/28) plus a July one, leave every one of them
alone, only the '(through 8/29)' one is in scope." Without that fence a delete instruction
risks her trimming Joe's review backlog. Post-fix enumeration returned TOTAL_MATCHES=17
(unchanged from before the fix) with exactly one "(through 8/29)" — proving the old copy
was replaced, not added, and no prior night was touched. Final parts:RAW_SIZE=645,926 | text/plain=530 text/html=180,554 application/pdf=463,736 | PDF_FILENAME=SCT-Alignment-By-Advisor-MTD-2026-08-29.pdf | TO_HEADER=kstapp@sctoyota.com | SENT_TODAY=0— HTML clears PNG4/3 (131,224) with heavy-signature headroom, PDF part
+2.6% CRLF variance over PDF4/3 (451,844). Note her build-confirmation HTML part
(131,943, a 719-byte tight pass) vs the PARTS ask (180,554) disagreed again — the familiar
two-reports-of-the-same-part discrepancy, both pass. Underscore-stripped keys again.
Recommended final verify ask going forward: foldPDF_FILENAMEinto the PARTS ask
every night (RAW_SIZE | PARTS | PDF_FILENAME | TO_HEADER | SENT_TODAY) — one round-trip
that catches stale attachments, misspelled filenames, wrong recipients and accidental
sends at once.
Confirmed again 2026-08-30 (20th consecutive clean night, final full day of August):
OPS probe 200 (same validated RO/job pair, unchanged since 8/3); no same-day index pre-run;
no competing consumer. Index 4,286 closed ROs (up 46 from 8/29's 4,240 — light Sunday),
1,409 candidates, 0 failed, ~53 min (19:01→19:54). Window end advanced 1788073199999 →
1788159599999 and index grew 583,458 → 589,909 B (both fresh-index confirmations); id diff
new: 46, dropped: 0. 376 alignments (340 dedicated + 36 bundled), 376 ROs, 16 advisors.
Chris Mai solo #1 with 45 (Jaime Sanchez 42, Cristian Gonzalez 36, Artist Battle 35).
Stacey's build clean on the FIRST ask (89s) with paths + on-disk sizes baked in: HTML part
132,155B cleared PNG4/3 (131,398) by only 757 bytes (another tight pass = PASS, note 13),
PDF part 345,219B an EXACT on-disk match.
Verify sequencing that worked best (use this order going forward): (1) run the date-free
SUBJECT+DATE enumeration FIRST rather than any DRAFTS_COUNT ask — it returned 18 matches, one
per night 8/14→8/30 plus the July leftover, with exactly ONE "(through 8/30)", proving
no-duplicate in a single 58s round-trip and completely sidestepping the note-26/note-28
false-count trap (no DRAFTS_COUNT ask was needed at all tonight). (2) Then the combined
5-field final ask — which TIMED OUT (exit 124) as usual when it bundles RAW_SIZE+PARTS+
PDF_FILENAME+TO_HEADER+SENT_TODAY. Per note 11, a 60s pause then two two-field asks
answered cleanly:PDF_FILENAME | PDF_DECODED_BYTES→ correct spellingSCT-Alignment-By-Advisor-MTD-2026-08-30.pdf(checked character-by-character for the
note-29 "Allignment" trap) with decoded bytes 345,219 = exact on-disk; thenTO_HEADER | SENT_TODAY→kstapp@sctoyota.com | 0. Two-field asks are the reliable
granularity — five fields times out, one field works but wastes round-trips.
Underscore-stripped keys again throughout.
Confirmed again 2026-08-31 (21st consecutive clean night — FINAL day of August, biggest
single-day catch-up of the month): OPS probe 200 (same validated RO/job pair, unchanged
since 8/3); no same-day index pre-run. NOTE: a dealer-detailsync:alltsx process HAD been
running 7h43m (started 11:18 AM) — per the 8/2 hog note this normally warrants killing, but
since the OPS probe returned a clean 200 the quota was NOT being starved, so it was left
alone and the scan ran fine. Lesson: a long-running sync:all is only a problem if the OPS
probe is 429 — probe FIRST, kill only if quota is actually blocked. Index 5,094 closed ROs
(up 808 from 8/30's 4,286 — month-end posting surge, by far the largest daily jump; index
file 589,909 → 699,924 B), 1,593 candidates, 0 failed, ~63 min (19:01→20:04 — longest run
to date, as expected for a full-month window). Window end advanced 1788159599999 →
1788245999999; id diff new: 811, dropped: 3 (a few ROs reverted out of CLOSED — normal).
443 alignments (399 dedicated + 44 bundled), 443 ROs, 16 advisors. Jason Sulon takes solo
#1 with 50 (Chris Mai 49, Jaime Sanchez 44, Artist Battle 43, Cristian Gonzalez 41) — a
new leader on the final day after Chris Mai led 8/28-8/30. Stacey's build clean on the FIRST
ask (97s) with paths + on-disk sizes baked in: HTML part 178,062B >= PNG4/3=131,017B, PDF
part 379,886B = EXACT on-disk match. Note-30 verify sequencing worked perfectly again and is
now the settled recipe: (1) date-free SUBJECT+DATE enumeration FIRST (104s, 19 matches —
18 August nights 8/14→8/31 plus the July leftover, exactly ONE "(through 8/31)") — no
DRAFTS_COUNT ask needed at all; (2)PDF_FILENAME | PDF_DECODED_BYTES(61s) → correct
one-L spelling + exact bytes; (3)TO_HEADER | SENT_TODAY(95s) →To: kstapp@sctoyota.com
| 0. All three answered first-try with only short sleeps (15s/10s/10s), zero timeouts.
Underscore-stripped keys again throughout. Month rollover reminder: the 9/1 run will
produce a tiny September MTD report (1 day) — that is CORRECT, not a bug; the MTD window
auto-resets to 1st-of-month. August's final figure of record is 443.
Confirmed again 2026-08-16 (6th consecutive clean night, quota fully healthy):
pre-flight OPS probe (same validated RO/job pair) returned 200. Index built 2,364
closed ROs, 771 candidates, 0 failed, ~30 min scan (19:01→19:31, no backoff
triggered). 221 alignments (199 dedicated + 22 bundled), 221 ROs, 16 advisors, top
Cristian Gonzalez (28). Stacey's build completed clean on the FIRST ask (~138s)
with file path + on-disk byte sizes baked in. Verify used the standard two-ask
minimal set: subject-substring-anchored DRAFTS_COUNT/TO/SENT_FOLDER_COUNT (clean:
1/kstapp@sctoyota.com/0) then a PARTS ask — PDF part 266,726B exact byte match to
file; HTML part 128,131B >= PNG4/3=127,280B. Note: the PARTS ask reported "3
drafts match that substring" — these were 3 DIFFERENT NIGHTS' drafts (8/14, 8/15,
8/16, all with distinct through-dates in the subject), not duplicates of the same
night; the earlier DRAFTS_COUNT=1 (dated-today filter) already confirmed only one
draft for tonight specifically. Don't confuse a multi-night substring match with
same-night duplication — check the through-date in each match before assuming a
dedupe is needed.
Confirmed again 2026-08-15 (5th consecutive clean night, quota fully healthy,
no dedupe needed): pre-flight OPS probe (same validated RO/job pair) returned 200
before launching — good habit to keep even on a healthy streak. Index built 2,272
closed ROs, 749 candidates, 0 failed, ~28 min scan (19:01→19:29, no backoff
triggered). 206 alignments (184 dedicated + 22 bundled), 206 ROs, 16 advisors, top
Cristian Gonzalez (26). Stacey's build completed clean on the FIRST ask (~122s) with
file paths + on-disk byte sizes baked in per 07-22 prevention — no rebuild needed.
Skipped the generic 5-field verify per note 15 and went straight to a
subject-substring-anchored ask ("DRAFTS_COUNT=1 | TO=kstapp@sctoyota.com |
SENT_FOLDER_COUNT=0") — clean, single draft, no duplicate-UID trap this time (unlike
8/14). Final PARTS check: HTML part 129,442B >= PNG4/3=128,704B (inline PNG
confirmed); PDF part 259,684B is an EXACT byte-for-byte match to the on-disk PDF file
size (strongest possible confirmation per note 14). Two-ask verify (subject-anchored
count/recipient, then PARTS/RAW_SIZE) is now reliably sufficient — no need to run the
older combined 5-field ask at all on a healthy night. - Hunt the hog FIRST: the dealer-detail nightly
- OVERALL_QUOTA exhaustion (hit 2026-07-07): distinct from OVERALL_RATELIMIT — this is
the store's DAILY API quota being fully spent (other pipelines, e.g. a TOL backfill loop +
caliber-ops scrapers, can burn it). EVERY call 429s
(Limit exhausted for type : OVERALL_QUOTA) for HOURS (observed 20:12 PDT → past 05:35
next morning). No amount of in-run backoff fixes it; the nightly report cannot ship on
time. Playbook: patch backoffs to also match "OVERALL_QUOTA", then deploy the self-heal
pairselfheal_sct_align_20260707.sh(probe every 10 min → on 200 run scan+render) +selfheal_sct_align_handoff_watch.sh(waits for "render exit=0 — DONE" in the selfheal
log → runsselfheal_sct_align_handoff.pywhich verifies the JSON and asks Stacey for the
DRAFT-ONLY Kevin email with all trap language baked in). Logs indata/sct-align-selfheal-*.log/data/sct-align-handoff-*.log. Both scripts have
flock guards + deadlines — adjust LOG/LOCK/DEADLINE lines when reusing.
Reuse notes (2026-07-08, second consecutive quota night):selfheal_sct_align_handoff.pyis DATE-AGNOSTIC (readssct-mtd-latest-align-by-advisor.json,
globs newest PNG/PDF, derives subject from period_label) — reuse untouched. Only the two .sh
wrappers need new dated copies with fresh LOG/LOCK paths.- Fast diagnosis: don't wait on a silent scan. If the background scan shows ZERO output after
a couple minutes, DO NOT kill it yet — probe quota in a SEPARATE call first:O.call('POST','/repair-orders:search',{filters:[status IN CLOSED],pageSize:1})— a 429 body
withOVERALL_QUOTAconfirms exhaustion in seconds; a 200 means the scan is fine, just silent.
(2026-07-11: even WITHpython3.11 -u, the Hermes process log showed ZERO lines for the entire
~13-min run — all output appeared only at exit.-udoes NOT guarantee live output in the
process pipe. HEALTHY-SCAN tells while the pipe is silent: probe returns 200, AND the data files
are moving —data/sct-mtd-<date>-closed-index.jsonappears after pass 1 and the checkpointdata/sct-mtd-<date>-align-scan.jsonmtime advances every ~20 ROs. Check mtimes before assuming
a dead scan; killing a healthy run wastes the checkpoint pacing.) - Launch trap: Hermes rejects shell background wrappers (setsid/nohup/&) AND aborts the whole
combined command — so achmod +x && setsid ...line leaves the scripts non-executable (exit 126
on the next try). chmod in its own foreground call, then launch each watcher with
terminal(background=true). Verify both alive via the log's "watcher started" line + pgrep count. - Quota can stay dead >24h: the 2026-07-08am self-heal probed 05:35→18:42 (13h, all 429) and
TIMED OUT — the through-7/7 draft never shipped. Other queued recovery pipelines (BC/TOL/BTquota_recovery_runner.sh, caliber-ops) burn the bucket the moment it refills. If a self-heal
times out, ESCALATE the quota-hog problem to Joe rather than just re-arming watchers — two
consecutive missed nightly drafts means the systemic consumer needs to be found/paced or the
Tekion limit raised. When quota restores after midnight, the MTD scan window auto-extends to the
new "today" — acceptable; the subject/period label self-adjusts.
PRIORITIZE the nightly over the quota hogs (2026-07-09, THIRD consecutive quota night —
outage continuous since 7/07 14:12, ~53h): re-arming a self-heal alone loses the refill race —
the queued backfill runners grab the bucket first. Fix: at self-heal launch, SIGSTOP the
competing consumers (pgrep -af "quota_recovery|bt_seed_watcher"→kill -STOP <pids>;
a runner holding/tmp/tekion-quota-recovery.lockalso freezes anything flock-waiting behind
it, e.g.sct_closed_backfill_runner.sh— good). Then baketrap resume_paused EXITinto the
self-heal .sh (kill -CONTeach paused pid, logged) so they resume automatically no matter how
the scan exits — never leave processes stopped. Set the probe deadline LONG (21h) to cover a
mid-day refill. STILL escalate the systemic issue to Joe in the same report (options: pace the
backfill queue / drop stale backfill days / ask Tekion to raise OVERALL_QUOTA) — the pause trick
is triage, not the fix.
- THE 429 TRAP (hit 2026-07-01):
sct_menu_sales_api.call()returns a STRING
body on any non-200 (e.g.{"message":"Limit exhausted ... OVERALL_RATELIMIT"}), NOT
a dict. Any.get()on it throws'str' object has no attribute 'get'and the scan
dies. The_get()helper MUST guardisinstance(body, dict)and, on a string body
containingOVERALL_RATELIMIT, back off HARD (60*(att+1)s). Regular 429/0/5xx →12*(att+1)s. Inter-RO pacingtime.sleep(0.5). Checkpoint every 20 ROs so a mid-run
failure resumes (data/sct-<period>-align-scan.json), plus a serial retry pass for the
failed set, andfailed[]in the output so truncation can't hide. - Classify each op: DEDICATED if opcode in {ALIGN,OKAL,ALIGN00BRA}; BUNDLED if
opcode startsTEKAND story (" ".join(corrections[].text)+ opcodeDescription,
lowercased) contains "align". - Advisor:
assignee.advisor.idis FREE on the search result (no fan-out) → resolve
viaO.user_name(aid)(public OpenAPI/users/{id}). No browser needed. - Output JSON is SELF-CONTAINED:
rows[]each{advisor, dedicated, bundled, total, ros, detail[]}where each detail ={ro, opcode, kind}; plustotals+chip_total.
VERIFYchip_total == totals.total(sum of chips == total alignments) before shipping.
Render + verify
Renderer = Toyota-red (#EB0A1E) header with the real Toyota logo (logo_0.png), 2 pages:
- Page 1 (PNG, inline in email): 4 KPI cards (Total Alignments / Unique ROs / Advisors
/ Daily Pace) + ranked advisor table (Dedicated, Bundled, Total, ROs, red bar) + TOTAL row. - Page 2 (PDF only): RO-level chips per advisor — red chip = dedicated, blue = bundled.
Per-advisor header count line usesr['total']labeled "N ROs" (matches page-1 bar total). - PNG = page-1 only; PDF = full 2-page. ALWAYS
vision_analyzethe PNG and confirm the
TOTAL row matches the KPI total before shipping. (Small dataset PNGs read fine full-page;
for a very tall table crop the top band and 2x-upscale like the menu-sales skill.)
Verified result (June 2026 full month, 2026-07-01)
516 alignments (469 dedicated + 47 bundled) across 510 ROs, 19 advisors. Jon Vu #1 (110),
Alex Anderson (50), William Dominguez & Cristian Gonzalez (39). chip_total == total == 516,
0 failed. Joe called it "perfect."
Email via Stacey — DRAFT by default
Files land in /home/itadmin/tekion-reports/data/:SCT-Alignment-By-Advisor-<period>.png + .pdf. Hand to Stacey viatimeout 170 ~/bin/ask-agent stacey "...". Format = the joe-email-jay-report base64
inline-PNG layout: greeting / summary line with the total BOLD / PNG inline IN THE MIDDLE
/ "Sent from Tekion Open API — live data" / Joe's HTML signature. Attach the PDF by full path.
Recipients (task-spec wins over defaults):
- Daily MTD report → TO Kevin Stapp (kstapp@sctoyota.com), greeting "Kevin," (Joe's
standing instruction 2026-07-01: nightly 7pm MTD-closed alignment report to Kevin). - A one-off "drafted for Joe to review" → TO jcastelino@americanmotorscorp.com, "Joe,".
⚠️ STACEY DRAFT-ONLY TRAP (hit HARD 2026-07-01 — read before every handoff)
Joe asked for a DRAFT; Stacey's rebuild/retry loop tripped her SMTP/X-GM-RAW send path
and SENT 7 copies, the newest addressed to Kevin via her hardcoded "SCT report → Kevin"
default — instead of leaving a draft. To DRAFT-ONLY safely:
- Give an EXPLICIT hard stop: "Create the draft via imap.append() to Drafts ONLY. DO NOT
call any send/SMTP/X-GM-RAW path. DO NOT send." State the exact TO + greeting and tell
her to override any hardcoded recipient default. - Her FIRST build often MISSES the inline PNG (
INLINE_PNG=no) even when the PDF attaches
— explicitly demand a base64 data-URI<img>inline in the MIDDLE of the body (NOT CID —
CID shows broken in Gmail draft view). - Do NOT ask her to "rebuild" in a way that re-fires her send pipeline. If a build is wrong,
tell her to create ONE fresh draft via imap.append() and delete the bad one — still no send. - Verify with a TERSE one-line read-only ask (verbose multi-field asks silently time out →
empty):TO=<addr> | INLINE_PNG=<y/n> | PDF=<y/n> | IN_DRAFTS=<y/n> | SENT=<y/n>.
An empty reply is a timeout, NOT proof of failure — re-ask the same question shorter.
This applies to ACTION asks too (hit 2026-07-04): a delete-duplicates ask returned
exit 124/empty but HAD completed — confirm with a fresh read-only count, don't re-fire
the action blindly. - NO EMOJI in the ask-agent message (hit 2026-07-04): characters like ⚠️ contain
Unicode variation selectors and trip the terminal security scanner
(tirith:variation_selector), blocking the command for approval — fatal in a headless
cron run. Use plain text like "HARD STOP:" instead. - Check DRAFT_COUNT (hit 2026-07-04): her fix-the-PNG rebuild leaves the old bad
draft(s) behind — 4 drafts piled up. After the fresh draft verifies good, ask her
(read-only count first) and have her DELETE duplicates, keeping only the newest good
one. Final state must be DRAFT_COUNT=1 | SENT=no. (2026-07-05: even a single clean
build produced DRAFT_COUNT=2; 2026-07-14: a single clean first-try build produced
DRAFT_COUNT=3 — duplication is the NORM, not the exception. ALWAYS count + dedupe
even when everything verified good on the first ask.) - INLINE_PNG false-negative (hit 2026-07-05): a generic "PNG=<y/n>" ask can come
backPNG=neven when the draft is CORRECT — she checks MIME attachments, and a
base64 data-URI<img>lives in the HTML body, not as an attachment. Before
rebuilding anything, verify with the precise body check:
"Does the HTML body contain an with data:image/png;base64? Reply:
DATAURI_IMG=<y/n>". Only rebuild if DATAURI_IMG=n. This avoids destroying a good
draft over a shallow read. - DATAURI check can false-negative EVEN on the precise raw-MIME ask (hit
2026-07-06): after a rebuild whose own confirmation said DATAURI_IMG=y, a
follow-up "fetch raw MIME, does html contain 'data:image/png;base64'" ask still
came back false. Definitive tiebreaker = SIZE MATH, which needs no substring
check at all: ask read-only "Reply one line: RAW_SIZE=<RFC822.SIZE> |
HTML_SIZE=<bytes of text/html part>". base64 inflates ~4/3, so
HTML_SIZE ≈ PNG_bytes * 4/3 (+ a few KB of text) proves the inline image is
embedded (e.g. 84,756-byte PNG → ~113,008 b64 vs HTML_SIZE 113,603 — match).
Treat it as a LOWER BOUND, not an equality (2026-07-12): Joe's HTML
signature can add ~48KB of extra HTML (96,338-byte PNG → expected ~128,451
b64, actual HTML part 176,638 — still a GOOD draft). Pass condition:
HTML_SIZE >= PNG_bytes4/3; only a few-KB HTML part means the PNG is missing.
PDF part likewise runs ~2.5% over PDF_bytes4/3 from base64 CRLF line-wrapping
(239,206-byte PDF → 327,338 part — normal).
RAW_SIZE ≈ HTML_SIZE + PDF_bytes*4/3 cross-checks the attachment. If HTML_SIZE
is only a few KB, the PNG is genuinely missing — rebuild then. Trust the math
over her y/n substring answers; never rebuild a draft the math says is good. - Best size-verify phrasing = per-part list (2026-07-10): a bare
"HTML_SIZE=" ask once returned 595 on a draft whose HTML part was actually
122,705 bytes. The reliable ask: "reply one line: RAW_SIZE=<RFC822.SIZE> |
PARTS=" — e.g.PARTS=text/plain=398 text/html=122705 application/pdf=226903proves inline PNG
(html ≈ PNG4/3) AND the exact PDF byte size in one shot.
NOTE (2026-07-13): PARTS sizes may come back as DECODED bytes — a clean run reported
application/pdf=249548, exactly the raw PDF file size (not the ~4/3-inflated encoded
size note 8 predicts). Don't false-alarm on that: a PDF part equal to the file's
byte size is a PASS. Cross-check with RAW_SIZE ≈ HTML_part + PDF_bytes4/3 (observed
478,162 ≈ 130,058 + 332,731 — consistent). The HTML-part lower-bound test
(HTML_SIZE >= PNG_bytes*4/3) is unaffected. - Telegram detour (2026-07-10): a read-only ask can come back "I sent the
summary to your Telegram" instead of answering — the bridge gets nothing. Add
"reply in THIS chat as plain text, do not message Telegram" to verification asks. - One combined verify ask works best (2026-07-15): merge notes 4/6/9 into a
SINGLE read-only ask — "Reply one line: TO= | IN_DRAFTS=<y/n> | SENT=<y/n> |
DRAFT_COUNT= | RAW_SIZE=<RFC822.SIZE> | PARTS=". Returned everything cleanly in one round-trip;
size math (HTML part 177,507 >= 97,9804/3; PDF part ~+1.4% over PDF4/3) verified
the inline PNG + attachment without any y/n substring answers. That run was also a
clean first-try DRAFT_COUNT=1 — rare but possible; still always check the count.
(2026-07-17: the ultra-short re-ask form "PARTS=?" can come back as just a part
COUNT, e.g.PARTS=3, not the per-part type=size list. Don't re-ask endlessly —
RAW_SIZE alone still verifies: RAW_SIZE ≈ HTML-with-inline-PNG (~PNG*4/3 + sig)
- PDF4/3. Observed 552,379 ≈ ~174K + ~378K for a 97,845 B PNG + 283,334 B PDF —
pass. Also that night: the full note-11 combined ask timed out (exit 124) but the
shortened one-line form answered instantly — keep the re-ask TERSE.)
(2026-07-27: the combined ask timed out TWICE — even a moderately shortened form.
The pattern that worked: two ultra-terse asks, each ONE line — first
"TO=? RAW_SIZE=?" (answered instantly), then the note-12 folder search
"SENT_FOLDER_COUNT=? DRAFTS_COUNT=?". Skipped PARTS entirely; verified via
RAW_SIZE >= PNG4/3 + PDF*4/3 (682,124 vs 139.6K+476.0K for a 104,691 B PNG +
356,988 B PDF) + DRAFTS_COUNT=1/SENT_FOLDER_COUNT=0. When combined asks keep
timing out, degrade straight to this two-ask minimal set — it is sufficient.
Confirmed again 2026-07-28: skipped the combined ask entirely, went straight to
the two-ask minimal set — both answered instantly, clean first-try DRAFT_COUNT=1.
Note her reply may strip underscores ("SENTFOLDERCOUNT=0") — same answer.
EXACT-SUBJECT SEARCH FALSE ZERO (2026-07-31): the folder-search ask with the
EXACT full subject ('SCT Alignment Sales by Advisor - July MTD (through 7/31)')
returned DRAFTS_COUNT=0 even though a prior ask had just found the draft (TO +
RAW_SIZE answered fine). Punctuation/encoding in the subject (dashes, parens)
can break her exact-match search. Do NOT rebuild on a zero — re-ask with a
SUBSTRING: "Search subject substring 'SCT Alignment Sales by Advisor' (not
exact match), only messages dated today" → came back DRAFTS_COUNT=1,
SENT_FOLDER_COUNT=0. A zero count contradicting an earlier successful fetch is
a search-syntax problem, not a missing draft.
2026-07-29: two-ask minimal set clean again (first-try DRAFTS_COUNT=1, SENT=0) —
this is now the DEFAULT verify path; don't bother with the combined ask. Also:
(2026-07-31: even the terse two-ask forms timed out THREE times in a row right
after the build ask — Stacey was likely still busy. Asleep 60before the next
re-ask got an instant clean answer. On consecutive timeouts, pause ~60s instead
of hammering.)
baking file paths + on-disk byte sizes into the initial build ask (note 13
prevention) again produced a clean one-shot draft. 2026-07-30: third consecutive
clean first-try night with the same recipe (sizes baked into build ask + two-ask
verify) — the pipeline is stable; treat any deviation as anomalous and re-read
the trap notes.)
SENT=y / DRAFT_COUNT FALSE ALARM in the combined ask (hit 2026-07-23): the
note-11 combined verify returnedSENT=y | DRAFT_COUNT=7on a clean first-try
build — BOTH wrong. Size math in the same reply was perfect (HTML 139,506 >=
PNG4/3; PDF part ~+2.6% over PDF4/3). Tiebreaker before panicking/rebuilding:
one terse read-only ask — "Search [Gmail]/Sent Mail for that exact subject — how
many? Search [Gmail]/Drafts — how many? Reply: SENT_FOLDER_COUNT= |
DRAFTS_COUNT=". Came back 0 | 1 → draft was fine, nothing sent, no dedupe
needed. Her SENT flag can reflect the \Seen/session state, and DRAFT_COUNT can
count the whole Drafts folder, not the subject. Trust folder searches + size math
over flag answers.ZERO-BYTE PDF PART (hit 2026-07-21): her first build can attach the PDF as an
EMPTY part — combined verify returnedapplication/pdf:0B(and RAW_SIZE ~275K, far
below HTML + PDF4/3) while her own confirmation claimed the PDF was attached. The
PARTS list is the only thing that catches this — always check the PDF part size is
nonzero and ≈ the on-disk PDF byte size (decoded, per note 9) or ~4/3 of it. Fix ask:
ONE fresh imap.append() draft, explicitly "read and base64-attach the PDF file bytes
from ( bytes on disk — verify the part is non-zero)", then delete the bad
draft. That fix ask TIMED OUT (exit 124) but HAD completed — per note 4, re-verify
read-only (DRAFT_COUNT went 1→2, new draft PDF_PART_BYTES=321889 exact file size)
before re-firing, then have her delete the older 0-byte-PDF draft. Final
DRAFT_COUNT=1 | SENT=n. Also seen that night: an image/jpeg part alongside a
healthy-sized HTML part — the data-URI PNG lives in the HTML (size math passed);
don't treat an extra image part as a failure.
PREVENTION (2026-07-22, clean first-try run): bake the fix language into the
INITIAL build ask — state both file paths WITH their on-disk byte sizes and say
"read and base64-attach the actual file bytes ... verify the PDF MIME part is
NON-ZERO and matches the file size". Night after the 0-byte trap, this produced a
clean DRAFT_COUNT=1 with PDF part = exact on-disk size on the first ask (confirmed
again 2026-07-25: clean first-try, DRAFT_COUNT=1, PDF part exactly file size,
HTML part 133,546 >= PNG4/3=132,828 — the bake-sizes-into-build-ask prevention
is reliably producing clean one-shot drafts). Also note:
the HTML-part lower-bound test can pass with only ~1KB headroom
(135,136 vs PNG*4/3=134,349) when her build omits the heavy HTML signature — a
tight pass is still a PASS.UN-ANCHORED short verify ask can return GARBLED/CROSS-CONTAMINATED data from
OTHER stores' pending drafts (hit 2026-08-11): a generic first-shot verify
("Reply ONE line: TO=? | INLINE_PNG=? | PDF=? | IN_DRAFTS=? | SENT=?" with no
subject anchor) came back nonsense mixing unrelated stores:
TO=TOL | INLINEPNG=? | INDRAFTS=0 | SENT=SCT+BC (TOL missing)— Stacey appears to
pull from whatever draft/task context is freshest in her working memory when the ask
doesn't pin down WHICH draft, especially if she has other stores' report drafts
pending around the same time. Do not treat this as evidence of a failed/missing
draft. Fix: ALWAYS anchor the verify ask with the specific subject SUBSTRING (per
the exact-subject false-zero trap, note 11's 07-31 entry) — e.g. "Search Gmail
Drafts for subject substring 'SCT Alignment Sales by Advisor - August MTD', dated
today. Reply: DRAFTS_COUNT= | TO= |
SENT_FOLDER_COUNT=". This came back clean (DRAFTS_COUNT=1, TO=kstapp@sctoyota.com,
SENT_FOLDER_COUNT=0) immediately after the garbled generic ask — same draft, same
minute, opposite answer. A follow-up subject-anchored PARTS/RAW_SIZE ask then
confirmed the inline PNG and PDF cleanly (HTML 125,348B >= PNG*4/3; PDF part exactly
221,808B matching the file). Lesson: skip the bare 5-field generic ask entirely on
the first verify attempt — go straight to a subject-substring-anchored ask.PDF part as application/octet-stream with AMBIGUOUS size + DECODE tiebreaker
(2026-07-24): the combined verify listed the PDF part as
application/octet-stream:387722— content-type NOT application/pdf (don't
false-alarm on that), and the size matched NEITHER the decoded file (344,616)
NOR ~4/3 encoded (~459K). Same reply also had DRAFT_COUNT=8 (note-12 whole-folder
false alarm; folder search returned SENT_FOLDER_COUNT=0 | DRAFTS_COUNT=1 — fine).
When part sizes are ambiguous, the DEFINITIVE check is a terse read-only ask:
"decode the PDF attachment part of the newest matching draft and give decoded
byte count. Reply: PDF_DECODED_BYTES=" — came back exactly the on-disk file
size (344,616). Decoded-bytes-equals-file-size beats all size-math heuristics;
use it whenever PARTS numbers don't reconcile. (The 3-question follow-up ask
timed out exit 124; splitting into two one-line asks answered instantly — keep
re-asks to ONE question each.)
Cron (LIVE)
Job 25ec117cfe72 "SCT Alignment MTD Closed — nightly 7pm draft to Kevin", schedule0 19 * * * (7pm Pacific), skills=[sct-alignment-by-advisor-report, agent-to-agent-bridge],
deliver=origin (status back to the Slack thread). It runs sct_align_mtd.py (background,
notify_on_complete), renders via render_sct_align.py, vision-verifies, then hands to Stacey
as a DRAFT ONLY to Kevin (kstapp@sctoyota.com), greeting "Kevin," in Joe's Gmail Drafts —
Joe reviews + sends in the morning. Reset at month rollover is automatic (the MTD scan
computes 1st-of-month → today, so nothing to reset). Scan is paced — early-month it runs
~15-25 min, but LATE-MONTH the MTD window balloons (2026-07-25: 3,941 closed ROs, 1,252
candidates → ~47 min, draft ~7:50pm; 2026-07-27: 1,330 candidates → ~58 min; 2026-07-28:
2026-07-28: 4,488 closed, 1,409 candidates → ~61 min; 2026-07-29: 1,468 candidates → ~58 min;
2026-07-30: 4,914 closed, 1,539 candidates → ~55 min, clean). A 30-65 min
silent scan late in the month is NORMAL —
check checkpoint mtime (advances every ~20 ROs) before assuming it's stuck. Established
2026-07-01 on Joe's instruction.
To flip it to AUTO-SEND to Kevin later: update the cron prompt to have Stacey SEND (SMTP
template-send) instead of draft-only, and drop the DRAFT-ONLY hard-stop language.
STALE BACKGROUND-NOTIFICATION REPLAY AFTER THE DAILY RESET (2026-08-28)
The daily ~4 AM session reset can RE-DELIVER the previous night'sBackground process ... completed (exit code 0) message for sct_align_mtd.py into the
fresh morning session, making it look like a scan just finished and needs rendering +
a Stacey handoff. It does not. Before acting on any align-scan completion notice,
check the clock and the file mtimes:ls -la --time-style='+%m-%d %H:%M' data/ | grep -E 'sct-mtd-<date>|SCT-Alignment'.
If the by-advisor JSON + PNG/PDF are already stamped ~19:50 the PRIOR evening, the run
already shipped — do NOTHING. Re-running rebuilds the whole index and burns OpenAPI
quota for identical data. Cross-check this skill's own dated confirmation notes too: if
the night in question is already written up here with matching totals, it's a replay.
Recurred 2026-08-29 (~04:04 AM reset) — the 8/27 run's completion message
(TOTALS {'dedicated': 312, 'bundled': 33, 'total': 345}) was re-delivered TWO days later,
so the replayed notice is not necessarily from the immediately-preceding night. The totals
line in the replayed message is the fastest discriminator: match it against this skill's
dated notes (8/27 = 345 / 312 + 33) — if it's already written up, it shipped. Correct
response is a one-line "stale replay, nothing to do" plus the evidence (file mtimes,
Stacey's verified DRAFTS_COUNT=1 | TO_HEADER=... | SENT_TODAY=0); do NOT re-render, do
NOT re-ask Stacey (a second handoff would create a duplicate draft for that night).
⚠️ READ FIRST — WHICH SCRIPT DOES JOE ACTUALLY WANT? (2026-09-01)
Condensed one-page version of this section + the menu-dollar trap:
references/alacarte-vs-menu-dollars.md— read that if you only need the
"which script / à la carte vs menu / don't sum the dollars" rules.
When Joe asks for "alignments sold, service menus and a la carte", he means the report that already
exists: sct_align_mtd.py (scan) + render_sct_align.py (render) — "Alignment Sales by Advisor",
closed MTD, Dedicated / Bundled / Total / ROs. Do NOT build a new renderer. He said plainly:
"I need it MTD. for the prior month. I think you had that as a skill prior" — and he was right.
- "a la carte" = the existing Dedicated column (ALIGN / OKAL / ALIGN00RBA)
- "service menus" = the existing Bundled column (TEK*BNM/VNM/PSM)
His vocabulary changed; the report did not. Map his words onto the existing columns instead of
inventing a variant. To re-cut a prior month from an alignbg scan with no re-scrape, reshape the
JSON to {advisor, dedicated, bundled, total, ros, detail} (drop kind=="bg") and feedrender_sct_align.py. Back up any file you overwrite (.bak-<oldtotal>).
BUG THIS UNCOVERED — the 8/31 nightly emailed 443, truth was 448
sct_align_mtd.py was patched at 19:23 Aug 31 for the ALIGN00RBA vs ALIGN00BRA typo, but the
nightly cron had already fired at 19:01 — so Kevin's emailed August number came from the buggy
build and dropped 5 ROs (577147, 578500, 579093, 579597, 581768), all tagged ALIGN00RBA and present
in the closed-index the whole time. After ANY opcode-set patch, re-run the current period's report
and reissue it — a same-day patch does not retroactively fix an already-sent email.
RULE: never hand-wave a count discrepancy
I first explained 448-vs-443 as "different candidate scope (ALIGN ∪ BG ∪ TEK)". That was plausible
and WRONG — it was the opcode typo. Always run the two-way set diff on (ro, opcode) and then look up
the offending ROs in the closed-index to see their real tags:
sB - sA # extra rows -> inspect idx tags for each RO before theorizing
sA - sB # missing rows; if empty, one side is a strict supersetJoe accepts "I don't know yet"; he does not accept a confident wrong explanation.
VARIANT: ALIGNMENTS ONLY WITH DOLLARS, A LA CARTE vs SERVICE MENU (secondary — only if $ requested)
Joe explicitly does NOT want BG in the alignment report: "I don't need bg sales, I need just the
alignments sold, INCLUDING SERVICE MENUS AND A LA CARTE." Default to THIS cut, not the BG companion.
Renderer: render_sct_alignments_only.py <by-advisor.json> — accepts an alignbg-shaped JSON and
ignores kind=="bg", so an existing alignbg scan can be re-cut with no re-scrape / zero quota.
Writes SCT-Alignments-Sold-By-Advisor-<date>.{png,pdf} + sct-alignments-only-<date>.json.
Terminology (use Joe's exact words as column labels):
- A la carte =
kind=="dedicated"= ALIGN / OKAL / ALIGN00RBA - In a service menu =
kind=="bundled"= alignment inside a TEK*BNM/VNM/PSM package
⚠️ THE MENU-DOLLAR TRAP — NEVER SUM THE TWO DOLLAR COLUMNS
Tekion bills a service menu as ONE line, so a menu alignment has no separately priced alignment
amount. cents on a bundled row is the entire package price, not the alignment's share.
Tell — run this before trusting any menu dollars:
statistics.median([x['cents'] for x in dedicated])/100 # -> 139.99 one clean price
statistics.median([x['cents'] for x in bundled])/100 # -> 553.80 up to $1,553.81 (TEK60000PSM)Bundled median several multiples of the a-la-carte price = you are looking at package revenue.
Aug 2026: summing them would have overstated alignment revenue by $28,354 on $55,970 (+51%).
Renderer therefore shows a greyed "Menu Package $ (not align $)" column, keeps "A La Carte $"
as the only money-styled column, prints a warning note, and labels menu chips pkg $N.
True alignment revenue = the a-la-carte column ONLY.
Menu rows still count as UNITS — a menu alignment IS an alignment sold. Units add; dollars do not.
Reconciling 448 (this cut) vs 443 (Kevin's nightly align-only)
alignbg candidate set is ALIGN ∪ BG ∪ TEK, so it opens ROs the align-only scan never touches.
Delta = exactly 5 ALIGN00RBA lines (ROs 577147, 578500, 579093, 579597, 581768), andset(nightly) - set(alignbg) == ∅. 448 is the more complete number. Always run that two-way set
diff on (ro, opcode) before explaining a discrepancy — never guess at it.
Coaching angle Joe responds to
Report per-advisor menu attach share (menu units ÷ total). Advisors at 0% menu alignments
(Aug: Jaime Sanchez, Juan Jose Perez, Michael Parayo) have a menu-PRESENTATION gap, not an alignment
gap — that's the actionable read, not the raw ranking.
COMPANION REPORT: Alignment + BG MENU SALES (built 2026-08-31, Joe's ask)
Joe asked for "total alignment sales with the BG menus." SCT has NO BG-branded alignment
opcode — BG at SCT = the BG Products chemical/fluid service line (17 opcodes seen in Aug:
BGCF, BGBFX, BGTF, BGCVTF, BGCFX, BGRDIFF, BGTC, BGATFX, BGFDIFF, BGAC, BGPSX, BGBAT,
BG208, BGETH, BGTBINJ, BGFINJ, BGMAF). So "alignment sales WITH the BG menus" = the
alignment report PLUS the BG service line, in DOLLARS not just units, with the
cross-sell attach rate (ROs carrying both).
- Scan:
sct_align_bg_scan.py— REUSES that day'ssct-mtd-<tag>-closed-index.json
(zero extra index calls; setTAG=YYYY-MM-DD). Candidate set = ALIGN ∪ BG ∪ TEK.
Captureslabor.saleAmount+ summed operationparts saleAmount(BOTH CENTS) per line.
Checkpointdata/sct-alignbg-<tag>-scan.json, resumable,_get()has the mandatory
dict-guard + OVERALL_RATELIMIT/OVERALL_QUOTA/DEALER_QUOTA hard-backoff. - Render:
render_sct_align_bg.py→ 3 pages: p1 advisor ranking (align units/$, BG
units/$, attach ROs, combined $), p2 BG product mix ranked by revenue, p3 RO-level chips
(red=dedicated, blue=TEK-bundled, green=BG) each stamped with its dollar amount.
Output stemSCT-Alignment-BG-Sales-By-Advisor-<date>.{png,pdf}. - NEVER run it concurrently with
sct_align_mtd.py— same rate-limit bucket; a
collision breaks Kevin's nightly.pgrep -af sct_align_mtdmust be empty first. - Operations endpoint returns
data.roOperations(NOTdata.operations), andO.call()returns a(status, body)TUPLE — both bite immediately if you copy the
jobs-endpoint shape. Parts are a LINK:/repair-orders/{rid}/jobs/{jid}/operations/{oid}/parts
→data.parts[].saleAmount, one extra call per BG line. - Aug 2026 reference scale: 5,094 closed ROs → 275 with a BG opcode, 200 of those carry
NO align/TEK op (invisible to the alignment scan), 64 ROs sold BOTH. - RECIPIENT: draft to JOE (jcastelino@americanmotorscorp.com, greeting "Joe,"), NOT
Kevin — this is a NEW format Joe has not signed off on yet. Only after he approves
does it go to Kevin / join the nightly cadence / roll out to the other stores. Same
DRAFT-ONLY hard-stop language as the main report applies. - SCHEDULING: it must run AFTER the nightly
sct_align_mtd.pyfinishes, not beside
it (same rate-limit bucket; Kevin's 7 PM report has priority). Scheduled at 21:15
Pacific (job733827598a50, first run for August 2026 final on 2026-09-01) so the
~19:01→~20:04 nightly is well clear. Stillpgrep -af sct_align_mtdbefore launching.
⚠️ AGENT-BABYSAT SCAN DIED MID-WAIT — FIXED with a self-contained runner (2026-09-01)
What happened (first run of this job, Aug-2026-final): the 21:16 cron launched the scan
as a background process and then sat in process(action='wait') loops. The scan itself
finished FINE at 23:19 (5,094 ROs, 1,798 candidates, 0 failed, correct totals written todata/sct-alignbg-2026-08-31-by-advisor.json) — but the agent session hit its iteration
ceiling on the 180-s wait timeouts and ended before render + email. Joe woke up to
nothing and had to ask "were you able to finish this report?" The data was on disk for 7
hours untouched. Root cause is architectural: a 2-hour scan cannot be babysat by an agent
turn loop — every wait timeout burns an iteration, and when they run out, the render/email
tail is simply never reached. Nothing errored, nothing alerted.
FIX — never babysit again. Use the runner:
(General pattern + reusable template for ALL long report crons:
skill tekion-report-cron-self-contained-runner.)
/usr/bin/bash /home/itadmin/tekion-reports/run_sct_alignbg.sh [scan-args]It does scan → render → email as ONE flock-guarded shell process (/tmp/sct-alignbg.lock),
logging to data/_sct_alignbg_run.log. Launch it with terminal(background=true, notify_on_complete=true) and then STOP — do not poll in a wait loop. Even if the agent
session dies the instant after launch, the email still goes out.
Emailer = sct_alignbg_email.py <by-advisor.json> <png> <pdf> — uses jay_mail.send_report
(SMTP, CID inline PNG, PDF attached), so no Stacey handoff and none of the 29 Stacey traps
above apply to this report. Recipient is Joe until he approves the format.
Recovery drill if a scan JSON exists but no email went out: don't re-scan (burns quota
for identical data). Just render + email the existing file:
cd /home/itadmin/tekion-reports
PY=/home/itadmin/.hermes/hermes-agent/venv/bin/python3.11
$PY render_sct_align_bg.py data/sct-alignbg-<tag>-by-advisor.json
$PY sct_alignbg_email.py data/sct-alignbg-<tag>-by-advisor.json \
data/SCT-Alignment-BG-Sales-By-Advisor-<tag>.png \
data/SCT-Alignment-BG-Sales-By-Advisor-<tag>.pdfTotal round trip ~30 s. Always vision_analyze the PNG header first (logo trap: logo_0.png
IS Stevens Creek Toyota, correct for this store only).
Aug 2026 final figures of record (delivered 2026-09-01 06:29): 448 alignments
(404 dedicated + 44 bundled) / $84,324; 358 BG units on 275 ROs / $92,269; combined
$176,592; 65 attach ROs; 16 advisors. Top: Chris Mai $26,714, Artist Battle $19,291,
Cristian Gonzalez $16,606, Brian Keat $15,431, Robin Porter $13,723. Top BG by revenue:
BGBFX (84 / $25,680), BGCF (106 / $14,633). Note the alignment total here (448) exceeds the
main nightly's 443 because this scan's candidate set is ALIGN ∪ BG ∪ TEK — it catches
alignments on ROs the align-only scan never fans out to.
2026-09-01 (first September night) — DEALER_QUOTA 429 + THE SYNC HOG RESPAWNS
September MTD was BLOCKED: pre-flight OPS probe 429 DEALER_QUOTA at 19:01 PDT (search 200,
jobs 200, /operations 429 — the classic deep-endpoint signature). Unlike the 8/1-8/10 outage,
the hog WAS present and was the likely cause: sync:all had been running 1d 6h 44m
(started Aug 31 12:17 PM, SYNC_STORES=SCT,SCVW,TOL,VWC). Killed the tree + removed/tmp/dealerdetail-sct-sync.lock.
NEW TRAP — killing the sync hog is NOT durable; it RESPAWNS in minutes: four minutes after
the kill, pgrep showed sync:all alive again with BRAND-NEW PIDs. Walking the parent chain
(ps -o pid=,ppid=,lstart=,cmd= up the tree) showed it was NOT the 23:00 cron-sct-sync.sh
crontab entry — the launcher was Jay's own gateway process (hermes_cli.main --profile jay gateway run --replace), i.e. a CONCURRENT JAY SESSION/cron turn firingbash -lic ... npm run sync:all. So the 8/2 "kill the hog" playbook only buys minutes here.
Always walk the PPID chain to the real launcher before assuming crontab; if the parent is a
hermes gateway, another agent turn is the competitor and re-killing just fights it. Escalate to
Joe to stagger/disable that sibling job rather than kill-looping. Quota did not recover within
~25 min of the kill (probes at 19:05 and 19:15 both 429) — an exhausted bucket does not refill
just because the consumer stopped.
Armed the standard dated self-heal pair via the sed date-swap from the 8/10 copies
(sed -e 's/2026-08-10/2026-09-01/g; s/20260810/20260901/g'), bash -n both, chmod in its own
foreground call, launched each via terminal(background=true, /usr/bin/bash explicit), confirmed
alive via the log's "watcher started" line.
Confirmed continuing 2026-09-02 (day 2 of the September DEALER_QUOTA outage, still 429):
same deep-endpoint signature at 19:01 PDT (search 200, /operations 429 DEALER_QUOTA on the
validated probe pair, unchanged since 8/3). The 9/1 self-heal watcher probed 429 every ~10 min
19:05→23:04 with zero recovery (log simply ends at 23:04 — its Hermes background session
likely died with the parent; the 21h DEADLINE was NOT reached, so a truncated selfheal log
ending mid-evening means the watcher itself died, check pgrep not just the log). No sync:all
hog this time (the stale /tmp/dealerdetail-sct-sync.lock from 9/1 23:14 remains with NO
process behind it — left in place since it usefully blocks the 23:00 nightly sync from piling
on). A caliber-ops run-scraper.ts started 19:00 today — left alone per the 8/31 lesson
(probe-first; it isn't the cause since quota was already dead all of 9/1 before it started).
Re-armed the standard dated pair via sed date-swap from the 9/1 copies, bash -n + chmod
foreground + two terminal(background=true) launches, both confirmed alive via log line +
pgrep count=2 each. Stacey read-only Gmail check (SUBJECT+DATE enumeration form, Drafts+Sent,
7 days): August 2026 Final CORRECTED (448) was SENT to Kevin 9/1; the CORRECTED draft plus 6
August nightly drafts still sit in Drafts; zero September reports exist (9/1 and 9/2 both
blocked). Unlike 9/1 there is no unshipped prior-period fallback left to re-cut — the
corrected August final already went out, so tonight is a pure blocked-night report. If still
429 on 9/3, that matches the 8/1-8/11 pattern (10-day non-refill) — recommend Joe open the
Tekion support ticket early rather than waiting out another 10-day arc.
Confirmed continuing 2026-09-03 (day 3 of the September DEALER_QUOTA outage, still 429):
same deep-endpoint signature at 19:01 PDT (search 200, jobs 200, /operations 429
DEALER_QUOTA on the validated probe pair, unchanged since 8/3). No competing consumers (both
standard pgrep sweeps empty — the 9/1 sync-hog respawn did NOT recur). The 9/2 self-heal
watcher ran its FULL 21h deadline (unlike the 9/1 one that died mid-evening) probing every
~10 min through 16:10 PDT today — all 429, "TIMEOUT — quota never restored". So quota has
been continuously dead 9/1 19:01 → 9/3 19:03+ with not one clean probe. Re-armed the dated
pair via the standard sed date-swap from the 9/2 copies (bash -n, chmod foreground,
terminal(background=true) x2, both confirmed alive via "watcher started" + pgrep=2). Stacey
read-only enumeration (Drafts+Sent, 'SCT Alignment' substring, 7 days): August CORRECTED
(448) draft still in Drafts + 5 August nightly drafts; Sent = the two 9/1 morning sends
(alignbg final + corrected-448 note). Zero September reports exist — 9/1, 9/2, 9/3 all
blocked. No zero-quota fallback left (August final already shipped). This now matches the
August 10-day non-refill arc — the report to Joe recommends opening the Tekion support
ticket NOW rather than waiting out the arc again.
ZERO-QUOTA FALLBACK THAT SAVED THE NIGHT — ship the corrected prior-month final. Because the
8/31 nightly emailed the BUGGY 443 (patched at 19:23, after the 19:01 cron fired — see the
ALIGN00RBA section), the correct August number had never reached Kevin. The alignbg scan from
8/31 23:19 was already on disk with the CORRECT figure, so re-cut it with zero API calls
using the documented reshape (drop kind=="bg", keep {advisor,dedicated,bundled,total,ros, detail}), wrote it to a NEW file sct-mtd-2026-08-31-align-by-advisor-CORRECTED.json (do NOT
overwrite the nightly's file), and fed it to render_sct_align.py. Verified chip_total == totals.total == 448, failed=[], 16 advisors, then vision-checked the PNG. Lesson: on a
quota-blocked night, always check whether an unshipped/incorrect prior period can be re-cut and
delivered from on-disk data — a blocked night does not have to be an empty night.
August 2026 figure of record: 448 (404 dedicated + 44 bundled), 448 ROs, 16 advisors,
two-way tie at #1 Chris Mai and Jason Sulon with 50 each (Jaime Sanchez 45, Artist Battle 43,
Cristian Gonzalez 42). Subject used: SCT Alignment Sales by Advisor - August 2026 Final CORRECTED (through 8/31), with a body paragraph naming the 5 dropped ROs and stating it
supersedes the 443.
Stacey's build clean on the FIRST ask (76s) with paths + on-disk sizes baked in: HTML part
180,781B >= PNG4/3 (132,715), PDF part 517,323B (+1.3% CRLF variance over PDF4/3=510,604).
Verify sequencing per notes 30/31 worked perfectly: (1) date-free SUBJECT+DATE enumeration
first → 20 matches, exactly ONE "August 2026 Final CORRECTED" dated today, no dedupe needed;
(2) PDF_FILENAME | PDF_DECODED_BYTES → correct one-L spelling + 382,953 exact on-disk match;
(3) TO_HEADER | SENT_TODAY → kstapp@sctoyota.com | 3.
SENT_TODAY=3 needed a scoped tiebreaker: per note 8/18 a nonzero Sent count is Joe's own
morning sends of the backlog, but rather than assume, one ultra-terse SUBJECT-SCOPED ask
settled it definitively: "In Sent Mail, subject substring 'August 2026 Final CORRECTED' —
CORRECTED_IN_SENT=" → 0. Scoping the Sent search to THIS report's unique subject
substring is stronger than the date-scoped SENT_TODAY form; prefer it whenever the subject
has a distinctive token. Underscore-stripped keys again throughout.
Confirmed continuing 2026-09-04 and 2026-09-05 (days 4 and 5 of the September
DEALER_QUOTA outage, still 429 both nights): same deep-endpoint signature at 19:01 PDT
each night (search 200, jobs 200, /operations 429 DEALER_QUOTA on the validated probe
pair, unchanged since 8/3). No local competing consumer either night (standard pgrep
sweeps empty; the stale 9/1 /tmp/dealerdetail-sct-sync.lock remains with no process —
left in place, it usefully blocks the 23:00 nightly sync). NEW OBSERVATION — self-heal
watchers launched via terminal(background=true) often DIE MID-RUN when their parent
Hermes session/gateway resets: the 9/3 watcher's log ends 9/4 12:02 and the 9/4
watcher's ends 9/5 11:52, both WITHOUT a TIMEOUT line and well before their 21h
deadlines (only the 9/2 watcher ever reached its full deadline). A selfheal log ending
mid-morning with no TIMEOUT line = the watcher died with its parent, not a deadline
expiry. The nightly re-arm covers this, but don't read a truncated log as "quota might
have recovered" — probe live. Re-armed the standard dated pair each night via the sed
date-swap, bash -n + chmod foreground + terminal(background=true) x2, confirmed alive
via "watcher started" log line + pgrep. Stacey read-only enumeration (Drafts+Sent,
'SCT Alignment' substring, 8 days) on 9/5: 5 August drafts still in Drafts (8/28-8/31
nightlies + Final CORRECTED), 3 Sent = the 9/1 morning sends of the August finals —
zero September reports exist; 9/1-9/5 all blocked. This is now a 5-consecutive-day
outage tracking the August 10-day arc — every blocked-night report should prominently
recommend Joe open the formal Tekion support ticket for the SCT DEALER_QUOTA bucket.
Also note: a sleep inside execute_code before a pgrep can hang the whole script to
its 300s timeout — do the post-launch liveness check as a plain foreground terminal
call a few seconds after launch instead.
BUG FOUND + FIXED 2026-08-31 — ALIGN00RBA vs ALIGN00BRA
ALIGN_OPC had ALIGN00BRA, but the opcode SCT actually uses is ALIGN00RBA (R and
B transposed). That code appears 107 times across the July+August indexes and ZEROALIGN00BRA ever existed — so every one of those alignments was silently uncounted, and
in August 5 of them weren't even candidates (no ALIGN/OKAL/TEK op on the RO, so the
fan-out never looked at them). Patched all four scanners to accept BOTH spellings:align_scan.py, align_scan_june.py, sct_align_full_june.py, sct_align_mtd.py.
Effect is small (~5/month at SCT) but it means historical alignment totals in prior
reports are LOW by the ALIGN00RBA count for that month. Lesson: derive the opcode set
from the live index (collections.Counter over r['opc']) rather than trusting a
hardcoded literal — a transposed character produces a silent zero, not an error.
Pitfalls recap
- Opcode set is SCT-specific (ALIGN/OKAL/ALIGN00RBA — note the R-B order). Other stores differ — see
tekion-alignment-by-advisor-report+tol-alignment-by-advisor-report. - The
_get()dict-guard + OVERALL_RATELIMIT hard-backoff is MANDATORY — without it the
scan dies on a string error body. - chip_total must equal totals.total. failed[] must be empty (or re-run — it resumes).
- DRAFT-ONLY means DRAFT-ONLY: lock Stacey's send path per the trap above.