feat(profiler): Stage 4 grounded research profiler — cited profiles replace NTEE stubs

profileOrgs (nightly 05:45, 25/run, easy-win orgs best-first per the
plan's coarse-match-gates-profiling rule): two-pass Gemini flash —
Google Search-grounded research (groundingMetadata = citation universe)
then JSON-schema extraction citing only from it. Overwrites the stub
profile (mission/programs/geography/funders/staff/news/sources/
confidence); re-embeds mission fit from researched text unless identity
unconfirmed or confidence <0.5 (then the NTEE embedding stays —
unverified research must not steer scoring). Detail page renders
researched programs w/ source links, funders, staff.

Fixes: org_profiles.confidence is float4 — 0.2 stores as 0.20000000298
so 'confidence <= 0.2' excluded every stub (epsilon comparison); RR7
loader serialization turns unknown into never (hoisted casts).

Live: 55 orgs profiled (41 research-grade, 2 low-confidence, 0 failed).
Scoring effect verified both directions: Annie's Angels x NIH mammalian
models fit 19->12; Seacoast Pathways x Foundation for Seacoast Health
fit 25/30 — and the researched knownFunders independently names that
funder. 161 tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Croissant Le Doux
2026-07-17 00:08:12 -04:00
parent 88affbeb4f
commit 4f35dcdb8d
14 changed files with 765 additions and 0 deletions

View File

@@ -48,6 +48,10 @@ import {
runMatchGrantsNow,
setMatchGrantsDeps,
} from './workflows/match-grants.js';
import {
runProfileOrgsNow,
setProfileOrgsDeps,
} from './workflows/profile-orgs.js';
import {
runIngestPndRssNow,
setIngestPndRssDeps,
@@ -63,6 +67,7 @@ const RUNNERS: Record<string, () => Promise<void>> = {
matchGrants: runMatchGrantsNow,
ingest990pf: runIngest990PfNow,
federalPrecedent: runFederalPrecedentNow,
profileOrgs: runProfileOrgsNow,
};
const FIRST_RUN_ORDER = [
@@ -75,6 +80,7 @@ const FIRST_RUN_ORDER = [
'federalPrecedent',
'embedGrants',
'matchGrants',
'profileOrgs',
];
if (process.env.DATABASE_URL == null) {
@@ -113,6 +119,7 @@ async function main() {
setMatchGrantsDeps({ db });
setIngest990pfDeps({ db });
setFederalPrecedentDeps({ db });
setProfileOrgsDeps({ db });
DBOS.setConfig({
name: 'helmdocs-outreach-worker',