grant-outreach-engine

christian/grant-outreach-engine

Fork 0

Commit Graph

Author	SHA1	Message	Date
Croissant Le Doux	fe12b032af	feat(infra): dockerized Postgres+pgvector and app image, mirroring novelpad compose conventions docker-compose.yml: pgvector/pgvector:pg16 on host 54341 (side-by-side with novelpad's 54321/54331), healthcheck-gated worker + review services from one multi-stage Dockerfile (same image, different command — the website/auth pattern). No Electric/Redis: nothing client-synced here. Dockerfile adapted from novelpad-desktop @ 62c56b87, trimmed: no native modules so no build-tools layer, no pm2. Default CMD = DBOS worker. Verified end to end: migration applied against the container (vector 0.8.2, 6 tables, 2 HNSW indexes), containerized worker boots, DBOS system schema lands in helmdocs_outreach ('dbos' schema — the *_dbos_sys URL in DBOS's boot log is cosmetic; the shared pool wins). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-16 13:49:55 -04:00
Croissant Le Doux	1735ff6754	feat(ingestion): Phase 1 data spine — Grants.gov, PND RSS, NHDOJ registry, ProPublica enrichment Core: icpBandForRevenue (100K-5M primary), serverUpsertOrgFromRegistry (case-insensitive name/city/state key, new-registrant signal), serverEnrichOrg (IRS fields + re-band, all-null marks attempted), serverListOrgsNeedingEnrichment. Worker: four source verticals, each a thin fail-loud client + pure tested normalize layer + DBOS scheduled workflow: - ingestGrants (nightly): Search2 paginated (cap 1000) -> fetchOpportunity details (cap 200, logged drops, 250ms politeness) -> batched upsert - ingestPndRss (nightly): RSS via fast-xml-parser, heuristic funder/ deadline extraction, link-keyed upsert - ingestNhdojOrgs (monthly): pdfjs-dist positioned-text extraction, pure row reconstruction (multi-line names, inferred columns, fail-loud on layout change), registry upsert; no-op warn when PDF URL unset - enrichOrgs (daily): ProPublica search -> conservative name/city match (null beats guess) -> latest-filing revenue/NTEE/FYE, per-org steps for checkpointed resume, >20% batch failure rethrows 94 worker + 16 core + 5 ai tests green; docs/features/ingestion.md added. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-16 13:11:50 -04:00
Croissant Le Doux	14200edb60	feat: scaffold outreach engine monorepo on the novelpad-desktop stack Workspaces: config (copied), outreach-core (schema + actions/queries + hard gates), outreach-ai (Gemini client + embeddings copies, profiler and mission-fit-judge agent stubs), outreach-worker (DBOS executor with nightly ingest + hourly expiry workflows), outreach-review (RR7 review queue v0). Initial drizzle migration incl. pgvector extension. Stack contract: Yarn 4.5.0 + Turbo, Node 22.16, Drizzle 0.44.6 + pgvector, DBOS 4.17.6, @google/genai on Vertex, gemini-embedding-001 @1536, React Router v7. Files copied from novelpad-desktop carry provenance headers @ 62c56b87. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-16 11:08:24 -04:00

Author

SHA1

Message

Date

Croissant Le Doux

fe12b032af

feat(infra): dockerized Postgres+pgvector and app image, mirroring novelpad compose conventions

docker-compose.yml: pgvector/pgvector:pg16 on host 54341 (side-by-side
with novelpad's 54321/54331), healthcheck-gated worker + review services
from one multi-stage Dockerfile (same image, different command — the
website/auth pattern). No Electric/Redis: nothing client-synced here.

Dockerfile adapted from novelpad-desktop @ 62c56b87, trimmed: no native
modules so no build-tools layer, no pm2. Default CMD = DBOS worker.

Verified end to end: migration applied against the container (vector
0.8.2, 6 tables, 2 HNSW indexes), containerized worker boots, DBOS
system schema lands in helmdocs_outreach ('dbos' schema — the
*_dbos_sys URL in DBOS's boot log is cosmetic; the shared pool wins).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

2026-07-16 13:49:55 -04:00

Croissant Le Doux

1735ff6754

feat(ingestion): Phase 1 data spine — Grants.gov, PND RSS, NHDOJ registry, ProPublica enrichment

Core: icpBandForRevenue (100K-5M primary), serverUpsertOrgFromRegistry
(case-insensitive name/city/state key, new-registrant signal),
serverEnrichOrg (IRS fields + re-band, all-null marks attempted),
serverListOrgsNeedingEnrichment.

Worker: four source verticals, each a thin fail-loud client + pure
tested normalize layer + DBOS scheduled workflow:
- ingestGrants (nightly): Search2 paginated (cap 1000) -> fetchOpportunity
  details (cap 200, logged drops, 250ms politeness) -> batched upsert
- ingestPndRss (nightly): RSS via fast-xml-parser, heuristic funder/
  deadline extraction, link-keyed upsert
- ingestNhdojOrgs (monthly): pdfjs-dist positioned-text extraction, pure
  row reconstruction (multi-line names, inferred columns, fail-loud on
  layout change), registry upsert; no-op warn when PDF URL unset
- enrichOrgs (daily): ProPublica search -> conservative name/city match
  (null beats guess) -> latest-filing revenue/NTEE/FYE, per-org steps for
  checkpointed resume, >20% batch failure rethrows

94 worker + 16 core + 5 ai tests green; docs/features/ingestion.md added.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

2026-07-16 13:11:50 -04:00

Croissant Le Doux

14200edb60

feat: scaffold outreach engine monorepo on the novelpad-desktop stack

Workspaces: config (copied), outreach-core (schema + actions/queries +
hard gates), outreach-ai (Gemini client + embeddings copies, profiler and
mission-fit-judge agent stubs), outreach-worker (DBOS executor with
nightly ingest + hourly expiry workflows), outreach-review (RR7 review
queue v0). Initial drizzle migration incl. pgvector extension.

Stack contract: Yarn 4.5.0 + Turbo, Node 22.16, Drizzle 0.44.6 +
pgvector, DBOS 4.17.6, @google/genai on Vertex, gemini-embedding-001
@1536, React Router v7. Files copied from novelpad-desktop carry
provenance headers @ 62c56b87.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

2026-07-16 11:08:24 -04:00

3 Commits