feat(scoring): nightly grant-embedding workflow (Stage 2, step 1)
embedGrants (cron 04:15, after the ingest crons): open grants with a
synopsis and no vector → buildGrantEmbeddingText (pure, tested:
title+funder+program areas+synopsis, 8K cap) → gemini-embedding-001 @
1536 dims RETRIEVAL_DOCUMENT (org profiles will embed as
RETRIEVAL_QUERY on the other side) → grants.synopsis_embedding.
Chunked embed→store (100/chunk) so failures resume from the last
stored chunk; 500/run spend cap.
Core: serverListGrantsNeedingEmbedding (open+unembedded, closest
deadline first), serverSetGrantEmbeddings. Worker gains
@novelpad/outreach-ai dep; wired into main.ts and run-once (incl. the
missed run-once deps injection).
Live-verified: 199/199 open grants embedded in 16s; semantic probe
('after-school STEM education for youth') ranks NCI Youth Enjoy
Science R25 first at 0.639 cosine via the hnsw index.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -12,6 +12,7 @@ Grant upserts key on `grants.source_url`; org registry upserts key on case-insen
|
||||
| `ingestPndRss` | `30 3 * * *` | Philanthropy News Digest RFP feed |
|
||||
| `expireGrants` | `0 * * * *` | (sweep: `status='expired'` past `close_date`) |
|
||||
| `enrichOrgs` | `0 5 * * *` | ProPublica Nonprofit Explorer |
|
||||
| `embedGrants` | `15 4 * * *` | gemini-embedding-001 over open-grant synopses → `grants.synopsis_embedding` (first paid AI call; ~pennies/batch) |
|
||||
| `ingestNhdojOrgs` | `0 4 1 * *` | NHDOJ Charitable Trusts registry PDF |
|
||||
|
||||
## Sources
|
||||
|
||||
Reference in New Issue
Block a user