14200edb60
Workspaces: config (copied), outreach-core (schema + actions/queries + hard gates), outreach-ai (Gemini client + embeddings copies, profiler and mission-fit-judge agent stubs), outreach-worker (DBOS executor with nightly ingest + hourly expiry workflows), outreach-review (RR7 review queue v0). Initial drizzle migration incl. pgvector extension. Stack contract: Yarn 4.5.0 + Turbo, Node 22.16, Drizzle 0.44.6 + pgvector, DBOS 4.17.6, @google/genai on Vertex, gemini-embedding-001 @1536, React Router v7. Files copied from novelpad-desktop carry provenance headers @ 62c56b87. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
62 lines
2.7 KiB
Markdown
62 lines
2.7 KiB
Markdown
# @novelpad/outreach-worker
|
|
|
|
DBOS executor process for the HelmDocs grant-match outreach engine. Runs the
|
|
scheduled workflows that keep the outreach schema's grant catalog fresh —
|
|
nightly ingestion from upstream sources and an hourly expiry sweep — decoupled
|
|
from any HTTP-facing app in this monorepo (mirrors the `workflow-worker` split
|
|
in `novelpad-desktop`).
|
|
|
|
## Boot order (load-bearing)
|
|
|
|
`src/main.ts` imports `./workflows/ingest-grants.js` and
|
|
`./workflows/expire-grants.js` **before** calling `DBOS.launch()`. Each of
|
|
those modules calls `DBOS.registerWorkflow` + `DBOS.registerScheduled` at
|
|
module-evaluation time — DBOS only dispatches scheduled/queued jobs for
|
|
functions that were registered before launch, so importing them after launch
|
|
(or not at all) silently means the cron jobs never fire.
|
|
|
|
The `db` handle is then threaded into each workflow module via its
|
|
`set*Deps` injector (`setIngestGrantsDeps` / `setExpireGrantsDeps`), also
|
|
before launch — DBOS serializes scheduled-function arguments, so a Drizzle
|
|
client can't be passed through the scheduler call itself. This is the same
|
|
module-scope-registry pattern novelpad-desktop's `workflow-worker` uses for
|
|
`setStartDeps`.
|
|
|
|
```
|
|
1. import workflow modules → registers ingestGrants / expireGrants
|
|
2. build pg Pool + drizzle(db)
|
|
3. set*Deps({ db }) → populates each workflow's registry
|
|
4. DBOS.setConfig(...)
|
|
5. DBOS.launch() → scheduler starts firing
|
|
```
|
|
|
|
## Workflows
|
|
|
|
- **`ingestGrants`** (`0 3 * * *`, nightly) — `fetchGrantsGov` (stub; real
|
|
implementation calls the Grants.gov Search2 API,
|
|
`POST https://api.grants.gov/v1/api/search2`) → `normalize` → upsert via
|
|
`serverInsertGrants` from `@novelpad/outreach-core/server`. NH state
|
|
postings and 990-PF extracts land as additional fetch+normalize steps
|
|
later.
|
|
- **`expireGrants`** (`0 * * * *`, hourly) — marks grants whose close date has
|
|
passed as closed via `serverExpireClosedGrants`, so they drop out of the
|
|
active match/scoring pool.
|
|
|
|
Both are registered as a DBOS workflow *and* a scheduled function referencing
|
|
the same function object (`DBOS.registerWorkflow` then `DBOS.registerScheduled`)
|
|
— see the doc comments in `src/workflows/*.ts` for why the dual registration
|
|
is required.
|
|
|
|
## Scripts
|
|
|
|
- `yarn dev` — `node --env-file=.env --env-file-if-exists=.env.local --import tsx/esm ./src/main.ts`
|
|
- `yarn build` — esbuild bundle to `build/main.js` (`--packages=external`)
|
|
- `yarn start` — run the built bundle
|
|
- `yarn typecheck` — `tsc --noEmit`
|
|
|
|
## Environment
|
|
|
|
Copy `.env.example` to `.env` and fill in `DATABASE_URL` (required — `main.ts`
|
|
throws on boot without it) and `GCP_SERVICE_ACCOUNT_KEY_PATH` (used by
|
|
future ingestion/scoring steps that call Google-hosted APIs).
|