feat(scoring): v4 — peer precedent, mission-fit floor, LLM match judge

Fixes the federal mismatch class (boys' camp × NIH research center):

- Peer precedent: federalPrecedent paginates USASpending (≤500 awards/
  program) and name-matches every recipient against primary-ICP NH
  registry orgs (shared normalizeOrgNameForMatching, also used by the
  self-match gate). The 25-pt precedent tiers now key off
  program_state_peer_award_count — Dartmouth renewals and SBIR LLCs no
  longer grant precedent to community nonprofits. Raw count + peer-
  annotated award list stay as review evidence (peer badges, peers-first).
- Mission-fit floor (12/30, grants_gov only): below it a match is stored
  with fit_viable=false and hidden from the pending queue, hero selection,
  and easy-win. Foundation-synthesized grants exempt (generic synopses).
- Mission-fit judge live (judgeMatches, 06:15, 200/night best-first):
  JUDGE_MODEL reads the synopsis against the org profile with an explicit
  ignore-eligibility-breadth instruction; graded verdict with required
  citations; deterministic verdict→points map (27/18/8/0) sets missionFit,
  total, easy-win, and viability. Verdicts survive nightly re-scores via
  an upsert splice and re-enter the judge queue when the org profile is
  re-researched (org_profiles.updated_at).

First sweep: 81/149 programs have NH history, only 6 have peer history;
queue-head judging zeroes the research-mechanism garbage (mismatch) while
surfacing genuine strong fits.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Croissant Le Doux
2026-07-17 10:52:50 -04:00
parent 7cefbbbfa1
commit cbc4512ffa
37 changed files with 4379 additions and 163 deletions

View File

@@ -39,12 +39,15 @@ export interface ScoreMatchInput {
readonly now: Date;
/**
* Historical grants this funder has paid to recipients in the org's
* state (from the 990-PF index). Null = no precedent data for this
* grant's funder (e.g. federal agencies) — scores 0, not neutral: the
* plan weights precedent as the strongest single predictor, and absence
* of evidence should rank below presence.
* state (990-PF index for foundations; USASpending PEER award count for
* federal programs — recipients name-matched to primary-ICP NH orgs).
* Null = no precedent data — scores 0, not neutral: the plan weights
* precedent as the strongest single predictor, and absence of evidence
* should rank below presence.
*/
readonly funderStateGrantCount: number | null;
/** Grant source (`grants.source`) — drives the source-aware fit floor. */
readonly grantSource: string;
}
export const ACHIEVABLE_MAX_SCORE = 100;
@@ -57,6 +60,18 @@ export const ACHIEVABLE_MAX_SCORE = 100;
export const EASY_WIN_THRESHOLD = 65;
export const EASY_WIN_MIN_PRECEDENT = 12;
/**
* Mission-fit floor (federal RFPs only): below 12/30 (≈ cosine 0.57) the
* match is stored but NOT review-viable — non-mission subscores sum to 50,
* so without a floor a boys' camp scores 60 on an NIH obesity-research
* center grant purely on precedent + capacity. Foundation-synthesized
* grants are exempt: their synopses are generic by construction ("grants
* for NH nonprofits"), so embedding fit carries no signal there and the
* precedent evidence IS the case for the match.
*/
export const MISSION_FIT_VIABLE_MIN = 12;
const FIT_FLOOR_SOURCES: ReadonlySet<string> = new Set(['grants_gov']);
/** Similarity below this scores 0 fit; above the ceiling scores full fit. */
const SIMILARITY_FLOOR = 0.45;
const SIMILARITY_CEILING = 0.75;
@@ -154,6 +169,13 @@ export interface ScoredMatch {
readonly totalScore: number;
readonly subscores: MatchSubscores;
readonly easyWin: boolean;
/**
* False when mission fit is below the source-aware floor — the match is
* recorded (audit trail, re-scoring continuity) but hidden from the
* pending review queue. The LLM match judge may later override in
* either direction.
*/
readonly fitViable: boolean;
}
export function scoreMatch(input: ScoreMatchInput): ScoredMatch {
@@ -174,11 +196,17 @@ export function scoreMatch(input: ScoreMatchInput): ScoredMatch {
subscores.runway +
subscores.funderPrecedent;
const fitViable =
!FIT_FLOOR_SOURCES.has(input.grantSource) ||
subscores.missionFit >= MISSION_FIT_VIABLE_MIN;
return {
totalScore,
subscores,
easyWin:
fitViable &&
totalScore >= EASY_WIN_THRESHOLD &&
subscores.funderPrecedent >= EASY_WIN_MIN_PRECEDENT,
fitViable,
};
}