1735ff6754
Core: icpBandForRevenue (100K-5M primary), serverUpsertOrgFromRegistry (case-insensitive name/city/state key, new-registrant signal), serverEnrichOrg (IRS fields + re-band, all-null marks attempted), serverListOrgsNeedingEnrichment. Worker: four source verticals, each a thin fail-loud client + pure tested normalize layer + DBOS scheduled workflow: - ingestGrants (nightly): Search2 paginated (cap 1000) -> fetchOpportunity details (cap 200, logged drops, 250ms politeness) -> batched upsert - ingestPndRss (nightly): RSS via fast-xml-parser, heuristic funder/ deadline extraction, link-keyed upsert - ingestNhdojOrgs (monthly): pdfjs-dist positioned-text extraction, pure row reconstruction (multi-line names, inferred columns, fail-loud on layout change), registry upsert; no-op warn when PDF URL unset - enrichOrgs (daily): ProPublica search -> conservative name/city match (null beats guess) -> latest-filing revenue/NTEE/FYE, per-org steps for checkpointed resume, >20% batch failure rethrows 94 worker + 16 core + 5 ai tests green; docs/features/ingestion.md added. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
36 lines
993 B
JSON
36 lines
993 B
JSON
{
|
|
"name": "@novelpad/outreach-worker",
|
|
"packageManager": "yarn@4.5.0",
|
|
"type": "module",
|
|
"version": "1.0.0",
|
|
"private": true,
|
|
"scripts": {
|
|
"dev": "node --env-file=.env --env-file-if-exists=.env.local --import tsx/esm ./src/main.ts",
|
|
"build": "esbuild src/main.ts --bundle --platform=node --format=esm --outfile=build/main.js --packages=external",
|
|
"start": "node ./build/main.js",
|
|
"typecheck": "tsc --noEmit",
|
|
"test": "vitest run"
|
|
},
|
|
"imports": {
|
|
"#~/*": "./src/*"
|
|
},
|
|
"dependencies": {
|
|
"@dbos-inc/dbos-sdk": "4.17.6",
|
|
"@dbos-inc/drizzle-datasource": "4.17.6",
|
|
"@novelpad/outreach-core": "workspace:^",
|
|
"drizzle-orm": "0.44.6",
|
|
"fast-xml-parser": "^4.5.0",
|
|
"pdfjs-dist": "^4.10.38",
|
|
"pg": "8.20.0",
|
|
"tsx": "^4.19.2"
|
|
},
|
|
"devDependencies": {
|
|
"@novelpad/config": "workspace:^",
|
|
"@types/node": "^22",
|
|
"@types/pg": "8.20.0",
|
|
"esbuild": "^0.24.0",
|
|
"typescript": "^5.9.3",
|
|
"vitest": "^3.2.4"
|
|
}
|
|
}
|