8d57d57557
NHDOJ: parser rebuilt for the real 8-column registry layout (Reg. No. | Charity Name | Address | City | State | Zip | Status | Report Due) with single-letter G/X/S statuses; Reg. No. is the stable upsert key (new orgs.registration_number column + partial unique index, enum gains 'suspended' via idempotent ADD VALUE); out-of-state registrants keep their real state. Akamai-safe fetch headers + NHDOJ_REGISTRY_PDF_PATH local-file override. ProPublica: zero-hit state-scoped searches return 404, not an empty list — map to no-candidates instead of failure (tripped the systemic- failure breaker at 60/200 on first contact). Enrichment queue now prioritizes NH good-standing orgs over the out-of-state tail. PND: feed retired upstream (HTML shell on every historical path) — documented as rework candidate, low priority. run-once.ts: supervised one-off runner through the durable DBOS handles (workflow modules now export run*Now accessors); drop the double pool.end() after DBOS.shutdown(). First supervised run: 200 Grants.gov opportunities (1 auto-expired), 13,632 orgs from the 427-page registry, enrichment at failed=0 with 121/200 EIN resolution in the NH-priority batch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
HelmDocs Grant-Match Outreach Engine
Automated pipeline that matches open grants to NH nonprofits, then runs a personalized email drip whose success metric is a booked demo call. The grant match is the lead magnet — match accuracy is the prime engineering directive.
Architecture doc: docs/plan.md. Build plan and phase status: ~/.claude/plans/helmdocs-outreach.md.
Relationship to novelpad-desktop
This repo deliberately mirrors the novelpad-desktop stack so a future merge is a directory move, not a rewrite:
- Yarn 4.5.0 + Turbo monorepo, Node 22.16 (volta), TypeScript ^5.9.3
- Postgres + pgvector + Drizzle (0.44.6, pinned via
resolutions), actions/queries pattern - DBOS for durable jobs/cron (worker-app pattern)
@google/genaion Vertex; embeddingsgemini-embedding-001@ 1536 dims (identical to HelmDocs RAG — never mix embedding models)- React Router v7 for the review UI
Merge rules: all packages stay in the @novelpad/ scope with names that never collide with novelpad-desktop workspaces (outreach-*). Files copied from novelpad-desktop carry a provenance header (source path @ commit) — diff against origin at merge time. No live code sharing; drift is fixed at merge, by design.
Boundary: separate repo and separate Postgres on purpose — outreach tooling and sourced contact data stay outside the compliance-scoped product repo/DB.
Workspaces
| Workspace | Purpose |
|---|---|
packages/config |
Shared tsconfig/tailwind base (copied from novelpad-desktop) |
packages/outreach-core |
Drizzle schema (grants, orgs, profiles, contacts, matches, pipeline_events), actions/queries, hard gates |
packages/outreach-ai |
Gemini client, embeddings, org-profiler + mission-fit-judge agents |
apps/outreach-worker |
DBOS executor: ingestion crons, expiry sweep, scoring runs, Apollo sync |
apps/outreach-review |
Human review queue (RR7). No auth by design — run behind IAP/localhost |
Development
yarn install
yarn build
yarn typecheck
yarn test
Migrations: yarn create:migration <name> inside packages/outreach-core (server-only — no client DB in this repo).
Local infrastructure
docker-compose.yml mirrors novelpad-desktop's compose conventions (pgvector image, healthcheck-gated startup), trimmed to this repo: Postgres 16 + pgvector on host port 54341 (novelpad's stacks use 54321/54331 — all three run side by side), plus worker and review services built from the root multi-stage Dockerfile (same image, different command — the novelpad website/auth pattern).
docker compose up -d postgres # just the database
DATABASE_URL=postgresql://postgres:password@localhost:54341/helmdocs_outreach \
yarn workspace @novelpad/outreach-core migrate
docker compose up -d # full stack (worker + review UI on :3100)
Migrations always run from the host — drizzle-kit is a devDependency and deliberately absent from the production image. The DBOS system schema initializes itself into the same database on worker boot (the dbos schema; ignore the misleading *_dbos_sys URL in DBOS's boot log — the shared pool override wins).