feat(scoring): nightly grant-embedding workflow (Stage 2, step 1)

embedGrants (cron 04:15, after the ingest crons): open grants with a
synopsis and no vector → buildGrantEmbeddingText (pure, tested:
title+funder+program areas+synopsis, 8K cap) → gemini-embedding-001 @
1536 dims RETRIEVAL_DOCUMENT (org profiles will embed as
RETRIEVAL_QUERY on the other side) → grants.synopsis_embedding.
Chunked embed→store (100/chunk) so failures resume from the last
stored chunk; 500/run spend cap.

Core: serverListGrantsNeedingEmbedding (open+unembedded, closest
deadline first), serverSetGrantEmbeddings. Worker gains
@novelpad/outreach-ai dep; wired into main.ts and run-once (incl. the
missed run-once deps injection).

Live-verified: 199/199 open grants embedded in 16s; semantic probe
('after-school STEM education for youth') ranks NCI Youth Enjoy
Science R25 first at 0.639 cosine via the hnsw index.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Croissant Le Doux
2026-07-16 16:26:41 -04:00
parent 8d57d57557
commit a532f0bebf
13 changed files with 329 additions and 1 deletions

View File

@@ -1540,7 +1540,7 @@ __metadata:
languageName: unknown
linkType: soft
"@novelpad/outreach-ai@workspace:packages/outreach-ai":
"@novelpad/outreach-ai@workspace:^, @novelpad/outreach-ai@workspace:packages/outreach-ai":
version: 0.0.0-use.local
resolution: "@novelpad/outreach-ai@workspace:packages/outreach-ai"
dependencies:
@@ -1605,6 +1605,7 @@ __metadata:
"@dbos-inc/dbos-sdk": "npm:4.17.6"
"@dbos-inc/drizzle-datasource": "npm:4.17.6"
"@novelpad/config": "workspace:^"
"@novelpad/outreach-ai": "workspace:^"
"@novelpad/outreach-core": "workspace:^"
"@types/node": "npm:^22"
"@types/pg": "npm:8.20.0"