88affbeb4f
Precedent was always a valid criterion for federal grants — the index just didn't cover them. Now: ingest captures ALN/CFDA numbers from fetchOpportunity (grants.alns); nightly federalPrecedent workflow (04:45) queries USASpending award search per distinct program (free official API, 3-year NH lookback) and stamps program_state_award_count + sample recipients onto open grants (multi-ALN keeps highest). Match retrieval feeds the same funderStateGrantCount input and 25-point tiers foundations use; detail page shows the recipients-evidence table with an incumbent-renewal caution. Also: detail refresh now rotates oldest-verified-first (serverMapGrantVerification) — the Set-based partition re-fetched the same 200 every pass, leaving 365/565 grants ALN-less. Live: 564/564 grants ALN-tagged, 156 programs swept, 85 with NH history, 468 grants carrying precedent, first federal easy-wins (66pts, 25/25 precedent, Aug-24 deadline). 158 tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
32 lines
4.9 KiB
Markdown
32 lines
4.9 KiB
Markdown
# Scoring engine v1 (Stage 2)
|
||
|
||
Nightly `matchGrants` workflow (05:15 UTC, after embeddings) — the plan's "SQL gates, vectors rank" hybrid.
|
||
|
||
## Flow, per candidate org (NH + good standing + primary ICP)
|
||
|
||
1. **Profile embedding** — v0 stub: NTEE-derived mission text (`buildOrgMissionText`) embedded as `RETRIEVAL_QUERY`, stored in `org_profiles` at confidence 0.2. The Stage 4 research profiler upgrades the row in place; this workflow doesn't change.
|
||
2. **Retrieval** — `serverListEligibleGrantsForOrg`: one SQL statement enforcing the cheap hard gates (status open, embedded, deadline ≥ 21 days, ceiling ≥ $10K) with pgvector cosine ranking; top 50 per org.
|
||
3. **Remaining gates in TS** — entity eligibility (`entryAdmitsEntity`: conservative pattern matching over Grants.gov applicantTypes prose; ambiguous entries do NOT admit) and geography (word-boundary state code + full state name). `application_form_supported` is **deliberately ignored** for pass/fail (2026-07-16 manual-first decision — draftability verified by hand for top leads); its failure still lands in `rationale.gateFailures`. Failed pairs are not stored.
|
||
4. **Deterministic subscores** (`scoreMatch`, pure, tested): mission fit 30 (similarity 0.45–0.75 → 0–30) · capacity 15 (award 10–75% of revenue = sweet spot) · competition 15 (state-restricted ≫ national) · effort 10 · runway 5 (3–10 weeks ideal). **Funder precedent (25pts, live)**: tiers of repeated giving into the org's state from the 990-PF index (1→8, 3→15, 5→20, 10→25); federal/no-data funders score 0 — absence of evidence ranks below presence. Easy win = total ≥ 65 AND precedent ≥ 12 (the plan's precedent floor). `subscores` jsonb keeps the full breakdown for reweighting.
|
||
5. **Upsert + hero** — pair-keyed upsert that never touches review fields (a human's reject stands even when scores move); `serverAssignHeroMatch` marks the org's top non-rejected gate-passing match.
|
||
|
||
## First live run (2026-07-16)
|
||
|
||
64 orgs × top-50 grants → 3,200 matches, 0 gate failures (corpus was pre-filtered to nonprofit-eligible, federal = geography-unrestricted), **0 easy wins, max 39/75**. That's the system being honest: the current corpus is 200 NIH-dominated federal research grants — wrong pond for $100K–$5M NH service nonprofits (similarity ceiling ~0.58). The engine's next real gains are corpus-side: NH state agency sources, 990-PF foundation ingestion, full Grants.gov detail backlog, real effort estimates.
|
||
|
||
|
||
## v2 (2026-07-16, same day): 990-PF precedent + lead-quality gates
|
||
|
||
- `ingest990pf` (monthly, `0 6 2 * *`): IRS BMF `eo_nh.csv` discovers NH private foundations (`PF_FILING_REQ_CD=1`) → e-file index CSVs select their latest 990-PF filings → batch ZIPs (capped 4/run, most-hits-first, deferred logged) → grants-paid rows into `funder_grants` → funders with ≥2 NH grants synthesize a rolling `grants` row (source `irs_990pf`, `funder_ein` set, null close date) that flows through embed + match like any RFP.
|
||
- Rolling (null) deadlines now PASS the runway gate and score 2/5 runway.
|
||
- Candidate orgs exclude NTEE `T*` grantmakers, and matches self-gate by funder EIN plus normalized-name fallback — the first precedent run's top "leads" were foundations matched to themselves (NHDOJ registers grantmakers as charities; several lack resolved EINs).
|
||
- First full run: 747 NH foundations, 21 batches (~6.5GB processed, 1 deferred), 2,766+ grants-paid rows, 123 synthesized foundation grants → **89 easy wins across 27 orgs**, top hero 69/100 with real matches like AIDS Response-Seacoast → Foundation for Seacoast Health. Coverage grows nightly as enrichment drains the org backlog (56 candidate orgs of ~6.2K NH registrants so far).
|
||
|
||
## v3 (2026-07-16): federal precedent via USASpending
|
||
|
||
Funder precedent now covers federal grants at the **program (ALN/CFDA) level** — the criterion was always valid for federal funders; only the data was missing. Ingest captures each opportunity's ALN numbers (`grants.alns`); the nightly `federalPrecedent` workflow (04:45) queries USASpending's award search (free, official, no key) for grant awards to NH recipients over a 3-year lookback per distinct program, stamping `program_state_award_count` + sample recipients onto every open grant carrying that ALN (multi-ALN grants keep the highest count). Match retrieval feeds it through the same `funderStateGrantCount` input and 25-point tiers foundations use. The match detail page shows the recipients table ("Recent program awards to NH recipients") with an explicit caution: distinguish one incumbent's renewals from genuine spread across orgs.
|
||
|
||
Ingest refresh also now rotates oldest-verified-first (`serverMapGrantVerification`) — the plain-Set version re-fetched the same head of the search results every pass, which had left 365 of 565 grants without ALNs.
|
||
|
||
First sweep: 156 distinct programs, 85 with NH history, 468/564 open federal grants carrying precedent; first federal easy-wins appeared (score 66, precedent 25/25, real Aug-24 deadline).
|