Files
grant-outreach-engine/packages/outreach-core
Croissant Le Doux a532f0bebf feat(scoring): nightly grant-embedding workflow (Stage 2, step 1)
embedGrants (cron 04:15, after the ingest crons): open grants with a
synopsis and no vector → buildGrantEmbeddingText (pure, tested:
title+funder+program areas+synopsis, 8K cap) → gemini-embedding-001 @
1536 dims RETRIEVAL_DOCUMENT (org profiles will embed as
RETRIEVAL_QUERY on the other side) → grants.synopsis_embedding.
Chunked embed→store (100/chunk) so failures resume from the last
stored chunk; 500/run spend cap.

Core: serverListGrantsNeedingEmbedding (open+unembedded, closest
deadline first), serverSetGrantEmbeddings. Worker gains
@novelpad/outreach-ai dep; wired into main.ts and run-once (incl. the
missed run-once deps injection).

Live-verified: 199/199 open grants embedded in 16s; semantic probe
('after-school STEM education for youth') ranks NCI Youth Enjoy
Science R25 first at 0.639 cosine via the hnsw index.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 16:26:41 -04:00
..

@novelpad/outreach-core

Domain/data package for the HelmDocs grant-match outreach engine: the Drizzle schema for grants, orgs, org profiles, contacts, matches, and pipeline events, plus the actions/queries that read and write them.

Modeled on packages/core in novelpad-desktop, following the same action/query pattern: every read lives in a domain's queries/, every write in its actions/, both barrel-exported, and route handlers/workflows/hooks never touch db.select()/db.insert() directly.

Domains

  • grants/ — open grant opportunities (Grants.gov, NH state, 990-PF, PND RSS, Candid). Ingestion upserts on sourceUrl; an hourly sweep expires anything past its close date.
  • orgs/ — NH nonprofits, plus org_profiles (LLM/agent-researched enrichment: mission, programs, known funders, budget band, citation sources).
  • matches/ — a scored (org, grant) pair. hard-gates.ts is the deterministic, DB-free pass/fail predicate (entity eligibility, geography, ≥21-day runway, ≥$10k ceiling, application-form support) that runs before the weighted LLM subscoring pass — see its module doc for why it's kept pure.
  • pipeline/ — outreach lifecycle events (enrolledsentopened/replied/... → demo_bookedconverted), synced from Apollo and recorded internally.

Deliberate deviations from @novelpad/core

This package is server-only — there is no client app, so several things @novelpad/core does for its PGlite/OPFS client don't apply here:

  • No PGlite, no client schema, no sync. There's exactly one database handle type (NpOutreachDatabase/NpOutreachTransaction in src/db/db.ts), not the client/server pair (NpClientDatabase vs NpDatabase) @novelpad/core needs to support both a local and a remote copy of the same data.
  • No novelpad_core_dev export condition. @novelpad/core's package.json uses a custom novelpad_core_dev condition so dev builds resolve straight to src/*.ts without a build step. This package's exports map only has plain types/default conditions — run yarn build (or yarn dev for a watch build) before consuming it from a sibling workspace in dev.
  • No build:migration-files bundling step. @novelpad/core's create:migration also runs a script that bundles generated SQL into a TS module the client can import for its OPFS-based local migrator. There is no local migrator here, so bin/create-migration only runs drizzle-kit generate against the single server config.
  • Root index.ts is intentionally small. It exports only the schema, DB types, and the pure hard-gates predicate — not domain client barrels, since every domain here is server-only. Everything else lives behind index.server.ts / the ./server export condition.

Scripts

yarn build            # tsc
yarn dev              # tsc --watch
yarn typecheck        # tsc --noEmit
yarn test             # vitest run
yarn create:migration <name>   # drizzle-kit generate against drizzle.config.ts
yarn migrate           # drizzle-kit migrate
yarn studio             # drizzle-kit studio