corpus: align determinability key with the schema column
The truth files used `determinableFromResponseDocument`; the schema column
and gateEvaluableFromResponse() use `determinableFromResponse`. A silent
name drift means the flag reads as its default (true), and R-B6C — "two bid
copies submitted", which no response document can ever establish — starts
failing bidders on a packaging detail.
Renamed in rfp.truth.json and validate-responses.py, and added the mirror
check to validate-truth.py so the drift cannot recur unnoticed:
gates 13, evaluable from the response 12, excluded 1
excluded: R-B6C Two bid copies submitted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -13,7 +13,7 @@ gates = {r['id'] for r in rfp['requirements'] if r['kind'] == 'mandatory_gate'}
|
||||
# A gate whose satisfaction is invisible to the response document cannot be scored
|
||||
# from it. Excluded from the silent-gate check; see requirementNotes in the RFP truth.
|
||||
undeterminable = {r['id'] for r in rfp['requirements']
|
||||
if r.get('determinableFromResponseDocument') is False}
|
||||
if r.get('determinableFromResponse') is False}
|
||||
COVERAGE = {'answered', 'not_answered', 'indeterminate'}
|
||||
|
||||
errs, warns = [], []
|
||||
|
||||
Reference in New Issue
Block a user