Covers all nine stress cases from #25, #15 and #16. The lowest bid is disqualified on two missing health documents and the highest bid is strongest on the SFA's own geographic-preference criterion, so a ranking that cannot represent absence gets the field exactly backwards. Deliberate traps: a planted $400 arithmetic error in a stated total, an unverifiable total whose unit price lives only in the external Excel, an unlabelled exception, a non-priceable exception that is really a conditional withdrawal, and an answered-but-negative response. Second schema finding, surfaced by the validator: R-B6C is a mandatory gate that no response document can establish. Coverage over it is unknowable rather than absent. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
238 lines
9.7 KiB
JSON
238 lines
9.7 KiB
JSON
{
|
|
"$comment": "Ground truth for source/chesapeake-food-service.md. The hard end of the binding gradient.",
|
|
"vendor": {
|
|
"id": "CHESAPEAKE",
|
|
"name": "Chesapeake Food Service, Inc.",
|
|
"location": "Baltimore, MD"
|
|
},
|
|
"bindingMechanism": {
|
|
"level": "free_prose",
|
|
"$comment": "#22's least reliable tier. Four sections exist but carry no requirement-level anchors, no RFP references and no matrix. Recovering which sentence answers which requirement is text classification, not structural lookup. This is the document that will expose extraction weakness."
|
|
},
|
|
"expectedOutcome": {
|
|
"gatesPassed": false,
|
|
"failedGates": [
|
|
"R-B6D",
|
|
"R-C5-INSPECTIONS"
|
|
],
|
|
"atRiskGates": [
|
|
"R-B2-LOCAL",
|
|
"R-C1-VALUES",
|
|
"R-A2"
|
|
],
|
|
"narrative": "Lowest price in the field at $541,280 and DISQUALIFIED regardless, on two missing mandatory health documents. This is the case that proves absence must be representable — a ranking that scores it on price alone puts a disqualified bid at the top."
|
|
},
|
|
"answers": [
|
|
{
|
|
"requirementId": "R-A1",
|
|
"coverage": "answered",
|
|
"evidence": "Cover block present"
|
|
},
|
|
{
|
|
"requirementId": "R-A2",
|
|
"coverage": "not_answered",
|
|
"evidence": null,
|
|
"$comment": "No checklist is reproduced anywhere."
|
|
},
|
|
{
|
|
"requirementId": "R-B6D",
|
|
"coverage": "not_answered",
|
|
"evidence": null,
|
|
"isGateFailure": true,
|
|
"$comment": "THE PRIMARY OMISSION. A current state or local health certificate is never mentioned. Not refused, not deferred — simply absent. The response lists many enclosures, which makes the absence easy for a human to miss and is exactly why coverage must be computed per requirement rather than eyeballed."
|
|
},
|
|
{
|
|
"requirementId": "R-C5-INSPECTIONS",
|
|
"coverage": "not_answered",
|
|
"evidence": null,
|
|
"isGateFailure": true,
|
|
"$comment": "Two most recent health inspections never mentioned. The response DOES discuss pest management at the same facility, which is topically adjacent and will tempt a weak extractor into a false positive."
|
|
},
|
|
{
|
|
"requirementId": "R-B2-LOCAL",
|
|
"coverage": "indeterminate",
|
|
"evidence": "'Chesapeake buys from the Maryland and Delmarva growing region in season... We would expect a meaningful proportion of produce delivered between June and October to originate within the region.'",
|
|
"$comment": "Addresses the TOPIC warmly and satisfies none of the OBLIGATION — no percentage, no certification, no acceptance of the audit right. This is KM-2's requirement meeting a response that gestures at it. The hardest coverage call in the corpus."
|
|
},
|
|
{
|
|
"requirementId": "R-C1-VALUES",
|
|
"coverage": "indeterminate",
|
|
"evidence": "'We are committed to student nutrition and to the communities we serve' — one clause in the Executive Summary, under no heading.",
|
|
"$comment": "Exercises KM-3 from the response side. Whether this constitutes a 'response to values statement' is a judgement, not a lookup."
|
|
},
|
|
{
|
|
"requirementId": "R-B6C",
|
|
"coverage": "not_answered",
|
|
"evidence": null
|
|
},
|
|
{
|
|
"requirementId": "R-B7-FORMAT",
|
|
"coverage": "answered",
|
|
"evidence": "Four sections present",
|
|
"$comment": "Section three is titled 'Method of Approach and Implementation', dropping 'Plan'. Near-miss on a required section title — a strict matcher fails, a fuzzy one passes."
|
|
},
|
|
{
|
|
"requirementId": "R-B8-QUEST",
|
|
"coverage": "answered",
|
|
"evidence": "'Three references are provided in the questionnaire' — implies completion, never states it"
|
|
},
|
|
{
|
|
"requirementId": "R-B8-LICENSE",
|
|
"coverage": "answered",
|
|
"evidence": "'Our business license and certificate of occupancy are enclosed'"
|
|
},
|
|
{
|
|
"requirementId": "R-B8-OCCUPANCY",
|
|
"coverage": "answered",
|
|
"evidence": "Same sentence as the business license"
|
|
},
|
|
{
|
|
"requirementId": "R-B8-FINANCIALS",
|
|
"coverage": "answered",
|
|
"evidence": "Two years audited, Hanlon & Reeve LLP"
|
|
},
|
|
{
|
|
"requirementId": "R-D6-INSURANCE",
|
|
"coverage": "answered",
|
|
"evidence": "$2M per occurrence / $4M aggregate, FPCS as additional insured"
|
|
},
|
|
{
|
|
"requirementId": "R-ATT-B",
|
|
"coverage": "answered",
|
|
"evidence": "'debarment, price determination and lobbying certifications are signed and enclosed'"
|
|
},
|
|
{
|
|
"requirementId": "R-ATT-C",
|
|
"coverage": "answered",
|
|
"evidence": "Same sentence"
|
|
},
|
|
{
|
|
"requirementId": "R-ATT-D",
|
|
"coverage": "answered",
|
|
"evidence": "Same sentence",
|
|
"$comment": "THREE separate certification requirements satisfied by ONE sentence naming all three. A per-requirement extractor must split one span three ways."
|
|
},
|
|
{
|
|
"requirementId": "R-E1-EEO",
|
|
"coverage": "answered",
|
|
"evidence": "'as are the equal opportunity and clean air and water certifications'"
|
|
},
|
|
{
|
|
"requirementId": "R-E3-CLEANAIR",
|
|
"coverage": "answered",
|
|
"evidence": "Same sentence"
|
|
},
|
|
{
|
|
"requirementId": "R-SCHED-C",
|
|
"coverage": "answered",
|
|
"evidence": "'submitted separately on the Excel schedule provided'"
|
|
},
|
|
{
|
|
"requirementId": "R-USDA-FOODS",
|
|
"coverage": "answered",
|
|
"evidence": "'We participate in the USDA Foods programme in Maryland... our processing relationships are stronger in the protein categories than in fruit and vegetable.'",
|
|
"$comment": "ANSWERED but WEAK. Coverage and quality are orthogonal — a correct extraction marks this answered and a correct score marks it low. Conflating them is a defect."
|
|
},
|
|
{
|
|
"requirementId": "R-CN-LABELS",
|
|
"coverage": "answered",
|
|
"evidence": "'category managers hold CN labels and formulation statements... typically within two business days of a request'"
|
|
},
|
|
{
|
|
"requirementId": "R-C6-PEST",
|
|
"coverage": "answered",
|
|
"evidence": "'licensed applicator, serviced monthly'"
|
|
},
|
|
{
|
|
"requirementId": "R-D1-DELIVERY",
|
|
"coverage": "answered",
|
|
"evidence": "Site survey of all eight locations; delivery window confirmed with authorised designee"
|
|
},
|
|
{
|
|
"requirementId": "R-B5-PRICING-TERMS",
|
|
"coverage": "answered",
|
|
"evidence": "'Prices are held firm through the base year'"
|
|
},
|
|
{
|
|
"requirementId": "R-D8-EPA",
|
|
"coverage": "answered",
|
|
"evidence": "'relief... in the event of a commodity movement exceeding eight percent'"
|
|
},
|
|
{
|
|
"requirementId": "R-C5-FOODSAFETY",
|
|
"coverage": "indeterminate",
|
|
"evidence": "Pest management is described; no GFSI, SQF or equivalent certification is claimed.",
|
|
"$comment": "Partial. The requirement asks for a food safety programme and the response supplies one component of one."
|
|
},
|
|
{
|
|
"requirementId": "R-ATT-A",
|
|
"coverage": "indeterminate",
|
|
"evidence": "'the District's Healthy Schools Act obligations' named in passing, no compliance detail"
|
|
},
|
|
{
|
|
"requirementId": "R-C7-QA",
|
|
"coverage": "not_answered",
|
|
"evidence": null
|
|
},
|
|
{
|
|
"requirementId": "R-C8-FARM2SCHOOL",
|
|
"coverage": "not_answered",
|
|
"evidence": null
|
|
},
|
|
{
|
|
"requirementId": "R-C4-MENUS",
|
|
"coverage": "not_answered",
|
|
"evidence": null
|
|
},
|
|
{
|
|
"requirementId": "R-D3-RECORDS",
|
|
"coverage": "not_answered",
|
|
"evidence": null
|
|
}
|
|
],
|
|
"priceQuote": {
|
|
"rateStructure": {
|
|
"kind": "flat_unit_plus_frequency_surcharge",
|
|
"$comment": "Exercises #29 differently from Potomac. Cost depends on a variable the SFA controls per site and may change mid-term with two weeks' notice — so derived_cost is only defined against a declared frequency assumption, and the response states one.",
|
|
"unit": "case",
|
|
"flatUnitPrice": null,
|
|
"$note": "Unit case price lives only in the external Excel sheet — the KM-1 shape again, this time making the vendor's own stated total unverifiable from the document alone.",
|
|
"perStopCharge": [
|
|
{
|
|
"deliveriesPerWeek": 2,
|
|
"chargePerStop": 34.0
|
|
},
|
|
{
|
|
"deliveriesPerWeek": 3,
|
|
"chargePerStop": 28.5
|
|
},
|
|
{
|
|
"deliveriesPerWeek": 4,
|
|
"chargePerStop": 24.0
|
|
}
|
|
],
|
|
"frequencyChangeableWithNotice": "2 weeks"
|
|
},
|
|
"statedBaseYearTotal": 541280,
|
|
"statedAssumptions": {
|
|
"deliveriesPerWeek": 3,
|
|
"sites": 8,
|
|
"calendarDays": 180
|
|
},
|
|
"$verify": "UNVERIFIABLE from this document. The case price is absent, so no computed total can be produced — only the vendor's stated figure exists. Contrast Potomac, where the arithmetic can be checked and fails."
|
|
},
|
|
"exceptions": [
|
|
{
|
|
"id": "IMPLICIT-1",
|
|
"labelled": false,
|
|
"requirementId": "R-B5-PRICING-TERMS",
|
|
"text": "We would ask only for relief under the economic price adjustment provision in the event of a commodity movement exceeding eight percent",
|
|
"$comment": "AN UNLABELLED EXCEPTION. It is phrased as a polite request inside the Pricing section, carries no heading, no price delta and no 'exception' wording. Compare Anacostia, which labels three exceptions explicitly with deltas. #22 called exceptions the thing evaluators care most about and the thing an invented corpus always forgets — this is the version that is present but hidden."
|
|
}
|
|
],
|
|
"extractionDifficulty": {
|
|
"expectRecall": "low",
|
|
"$comment": "No anchors of any kind. Expect false positives on R-C5-INSPECTIONS from adjacent pest-management prose, and expect the three-certifications-in-one-sentence span to be split incorrectly or attributed to only one requirement."
|
|
}
|
|
}
|