4f35dcdb8d
profileOrgs (nightly 05:45, 25/run, easy-win orgs best-first per the plan's coarse-match-gates-profiling rule): two-pass Gemini flash — Google Search-grounded research (groundingMetadata = citation universe) then JSON-schema extraction citing only from it. Overwrites the stub profile (mission/programs/geography/funders/staff/news/sources/ confidence); re-embeds mission fit from researched text unless identity unconfirmed or confidence <0.5 (then the NTEE embedding stays — unverified research must not steer scoring). Detail page renders researched programs w/ source links, funders, staff. Fixes: org_profiles.confidence is float4 — 0.2 stores as 0.20000000298 so 'confidence <= 0.2' excluded every stub (epsilon comparison); RR7 loader serialization turns unknown into never (hoisted casts). Live: 55 orgs profiled (41 research-grade, 2 low-confidence, 0 failed). Scoring effect verified both directions: Annie's Angels x NIH mammalian models fit 19->12; Seacoast Pathways x Foundation for Seacoast Health fit 25/30 — and the researched knownFunders independently names that funder. 161 tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
16 lines
2.1 KiB
Markdown
16 lines
2.1 KiB
Markdown
# Stage 4 — Org research profiler
|
|
|
|
Nightly `profileOrgs` workflow (05:45, after matchGrants): grounded web research for orgs that already have ≥1 non-rejected easy-win match (the plan's "coarse match gates profiling" cost rule), best-scoring first, 25/night (`PROFILE_ORGS_PER_RUN`).
|
|
|
|
**Two Gemini passes per org** (`packages/outreach-ai/src/agents/org-profiler/research.ts`):
|
|
1. **Grounded research** — `gemini-2.5-flash` with the Google Search tool; `groundingMetadata` supplies the citation universe (every page actually consulted).
|
|
2. **Structured extraction** — flash constrained to a JSON schema over the research notes, citing only from that universe. Search grounding and JSON-schema output can't share one call, hence the split.
|
|
|
|
**Output** overwrites the org's NTEE stub in `org_profiles` (mission, programs w/ populations served, service geography, known funders, staff, recent news, sources, confidence). The profile embedding is rebuilt from the researched text — unless identity wasn't confirmed or confidence < 0.5, in which case the embedding stays NTEE-derived so unverified research can't steer mission fit. An identity-unconfirmed profile is stored at ≤0.3 confidence: "we looked and couldn't confirm" beats an NTEE guess, and the review UI warns on it.
|
|
|
|
**Review integration**: match detail pages render researched programs (with per-claim source links), known funders, and staff.
|
|
|
|
**First live batches (2026-07-16/17)**: 55 orgs, 41 research-grade (>0.4), 2 low-confidence, 0 failures, ~28 min for 30 orgs (grounded search dominates). Verified effect on scoring: Annie's Angels → NIH Mammalian Models mission fit fell 19→12 (real mission: family financial crisis support), Seacoast Pathways → Foundation for Seacoast Health rose to 25/30 fit — and its researched `knownFunders` list independently named Foundation for Seacoast Health, confirming the precedent match.
|
|
|
|
**Gotcha fixed en route**: `org_profiles.confidence` is float4 — `0.2` stores as `0.20000000298`, so `confidence <= 0.2` (float8 comparison) silently excluded every stub; the needing-profile query uses an epsilon.
|