christian/grant-outreach-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
8d57d57557435f09c8c0324d87d403cec8510d83
grant-outreach-engine/packages/outreach-core
History
Croissant Le Doux 8d57d57557 fix(ingestion): survive first contact with real data sources 
NHDOJ: parser rebuilt for the real 8-column registry layout (Reg. No. |
Charity Name | Address | City | State | Zip | Status | Report Due) with
single-letter G/X/S statuses; Reg. No. is the stable upsert key (new
orgs.registration_number column + partial unique index, enum gains
'suspended' via idempotent ADD VALUE); out-of-state registrants keep
their real state. Akamai-safe fetch headers + NHDOJ_REGISTRY_PDF_PATH
local-file override.

ProPublica: zero-hit state-scoped searches return 404, not an empty
list — map to no-candidates instead of failure (tripped the systemic-
failure breaker at 60/200 on first contact). Enrichment queue now
prioritizes NH good-standing orgs over the out-of-state tail.

PND: feed retired upstream (HTML shell on every historical path) —
documented as rework candidate, low priority.

run-once.ts: supervised one-off runner through the durable DBOS
handles (workflow modules now export run*Now accessors); drop the
double pool.end() after DBOS.shutdown().

First supervised run: 200 Grants.gov opportunities (1 auto-expired),
13,632 orgs from the 427-page registry, enrichment at failed=0 with
121/200 EIN resolution in the NH-priority batch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 16:20:47 -04:00
..
bin
feat: scaffold outreach engine monorepo on the novelpad-desktop stack
2026-07-16 11:08:24 -04:00
drizzle/server
fix(ingestion): survive first contact with real data sources
2026-07-16 16:20:47 -04:00
src
fix(ingestion): survive first contact with real data sources
2026-07-16 16:20:47 -04:00
drizzle.config.ts
feat: scaffold outreach engine monorepo on the novelpad-desktop stack
2026-07-16 11:08:24 -04:00
package.json
feat: scaffold outreach engine monorepo on the novelpad-desktop stack
2026-07-16 11:08:24 -04:00
README.md
feat: scaffold outreach engine monorepo on the novelpad-desktop stack
2026-07-16 11:08:24 -04:00
tsconfig.json
feat: scaffold outreach engine monorepo on the novelpad-desktop stack
2026-07-16 11:08:24 -04:00

README.md

@novelpad/outreach-core

Domain/data package for the HelmDocs grant-match outreach engine: the Drizzle schema for grants, orgs, org profiles, contacts, matches, and pipeline events, plus the actions/queries that read and write them.

Modeled on packages/core in novelpad-desktop, following the same action/query pattern: every read lives in a domain's queries/, every write in its actions/, both barrel-exported, and route handlers/workflows/hooks never touch db.select()/db.insert() directly.

Domains

  • grants/ — open grant opportunities (Grants.gov, NH state, 990-PF, PND RSS, Candid). Ingestion upserts on sourceUrl; an hourly sweep expires anything past its close date.
  • orgs/ — NH nonprofits, plus org_profiles (LLM/agent-researched enrichment: mission, programs, known funders, budget band, citation sources).
  • matches/ — a scored (org, grant) pair. hard-gates.ts is the deterministic, DB-free pass/fail predicate (entity eligibility, geography, ≥21-day runway, ≥$10k ceiling, application-form support) that runs before the weighted LLM subscoring pass — see its module doc for why it's kept pure.
  • pipeline/ — outreach lifecycle events (enrolled → sent → opened/replied/... → demo_booked → converted), synced from Apollo and recorded internally.

Deliberate deviations from @novelpad/core

This package is server-only — there is no client app, so several things @novelpad/core does for its PGlite/OPFS client don't apply here:

  • No PGlite, no client schema, no sync. There's exactly one database handle type (NpOutreachDatabase/NpOutreachTransaction in src/db/db.ts), not the client/server pair (NpClientDatabase vs NpDatabase) @novelpad/core needs to support both a local and a remote copy of the same data.
  • No novelpad_core_dev export condition. @novelpad/core's package.json uses a custom novelpad_core_dev condition so dev builds resolve straight to src/*.ts without a build step. This package's exports map only has plain types/default conditions — run yarn build (or yarn dev for a watch build) before consuming it from a sibling workspace in dev.
  • No build:migration-files bundling step. @novelpad/core's create:migration also runs a script that bundles generated SQL into a TS module the client can import for its OPFS-based local migrator. There is no local migrator here, so bin/create-migration only runs drizzle-kit generate against the single server config.
  • Root index.ts is intentionally small. It exports only the schema, DB types, and the pure hard-gates predicate — not domain client barrels, since every domain here is server-only. Everything else lives behind index.server.ts / the ./server export condition.

Scripts

yarn build            # tsc
yarn dev              # tsc --watch
yarn typecheck        # tsc --noEmit
yarn test             # vitest run
yarn create:migration <name>   # drizzle-kit generate against drizzle.config.ts
yarn migrate           # drizzle-kit migrate
yarn studio             # drizzle-kit studio