19e7ffe92a
Covers all nine stress cases from #25, #15 and #16. The lowest bid is disqualified on two missing health documents and the highest bid is strongest on the SFA's own geographic-preference criterion, so a ranking that cannot represent absence gets the field exactly backwards. Deliberate traps: a planted $400 arithmetic error in a stated total, an unverifiable total whose unit price lives only in the external Excel, an unlabelled exception, a non-priceable exception that is really a conditional withdrawal, and an answered-but-negative response. Second schema finding, surfaced by the validator: R-B6C is a mandatory gate that no response document can establish. Coverage over it is unknowable rather than absent. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
125 lines
5.5 KiB
Markdown
125 lines
5.5 KiB
Markdown
# Friendship PCS — extraction notes
|
||
|
||
Text extracted with pdfjs 4.10.38 (line-grouped by y-position). 52pp, 105,154
|
||
chars, **no page needed OCR** — the text layer is clean throughout.
|
||
|
||
Raw output in `extracted/`. This is a reading aid for authoring, *not* the
|
||
production pipeline's output — that runs through Document AI and is what gets
|
||
measured later.
|
||
|
||
## The rubric is real and nested three deep
|
||
|
||
Verbatim from Section K.2, "Technical Rating Scale":
|
||
|
||
```
|
||
Pricing 40
|
||
Total Bid Cost (Base Year) (no separate points)
|
||
Method of Approach and Implementation 30
|
||
Food and Packaging 15
|
||
Utilization of USDA Foods 10
|
||
Geographic preference 5
|
||
Bidder's Experience, Expertise & Reliability 20
|
||
Knowledge of USDA / District requirements 5
|
||
Dependability 5
|
||
Inspection of warehouse and facilities 5
|
||
Customer Service 5
|
||
Proven Organizational Capacity 5
|
||
(three sub-bullets, NO points)
|
||
Community Connection, Values Alignment 5
|
||
(three sub-bullets, NO points)
|
||
```
|
||
|
||
Matches what #22 recorded. Two structural facts it did not capture:
|
||
|
||
**Sub-items are inconsistent.** Under Method of Approach and Experience they are
|
||
*weighted sub-criteria* summing to the parent. Under Organizational Capacity and
|
||
Community Connection they are *unweighted guidance*. The same visual nesting
|
||
means two different things in one document.
|
||
|
||
**Geographic preference embeds a mandatory submission inside a scored
|
||
criterion:** "Bidder must submit certification of percentage of locally grown or
|
||
raised foods." A scored criterion carrying a pass/fail artifact.
|
||
|
||
## Extraction hazards confirmed here
|
||
|
||
**The rubric never uses the word "criteria" as a heading.** It lives under
|
||
"SECTION K: Evaluation Factors" → "K.2 Technical Rating Scale", with weights as
|
||
bullet suffixes (`– 40 points`). A keyword search for "evaluation criteria" or
|
||
"scoring" returns nothing. First-pass evidence for why #16's completeness sweep
|
||
cannot rely on headings.
|
||
|
||
**Schedule C is the price sheet and contains no prices.** Verbatim, the entire
|
||
section body is:
|
||
|
||
> SCHEDULE C: PRICING SHEET
|
||
> *Separate attachment to be completed in Excel.
|
||
|
||
This is #22's second extraction hazard in its purest form: a PDF-only extractor
|
||
finds **zero pricing** on a solicitation where pricing is 40% of the score. It is
|
||
also a ready-made **labelled known-miss** for the sweep — the obligation is
|
||
present, its content is not, and no amount of better PDF parsing recovers it.
|
||
|
||
## Obligation density
|
||
|
||
149 × "shall", 64 × "must" across 52 pages. An earlier ad-hoc extraction reported
|
||
zero of each; that was the extractor failing, not the document.
|
||
|
||
---
|
||
|
||
# Response set
|
||
|
||
Three synthesized responses in `responses/source/`, each paired with a truth
|
||
file. Designed so the field has a real outcome rather than three
|
||
interchangeable bids.
|
||
|
||
| Vendor | Binding | Price | Gates | Distinguishing feature |
|
||
|---|---|---|---|---|
|
||
| Potomac Provisions | tabbed compliance matrix | $589,970 | PASS | Complete; volume-banded pricing; 18% local |
|
||
| Chesapeake Food Service | free prose, no anchors | $541,280 | **FAIL** | Cheapest and disqualified; unlabelled exception |
|
||
| Anacostia Farm Collective | named sections + enumeration | $624,315 | PASS | 61% local; three labelled exceptions; seasonal pricing |
|
||
|
||
The lowest bid is disqualified and the highest bid is strongest on the
|
||
criterion the SFA wrote into its own rubric. That is deliberate: a ranking
|
||
that cannot represent absence puts Chesapeake first.
|
||
|
||
## Deliberate traps
|
||
|
||
**Planted arithmetic error (Potomac).** The stated base-year total is
|
||
$589,970; band price times scenario volume is $589,570. A $400 gap. #15
|
||
decision 2 puts all price arithmetic in code precisely so a stated total is
|
||
never trusted over a computed one — this is the case that catches an
|
||
extractor lifting the vendor's own figure.
|
||
|
||
**Unverifiable total (Chesapeake).** The case price exists only in the
|
||
external Excel sheet, so no computed total can be produced at all. Contrast
|
||
Potomac, where the arithmetic can be checked and fails.
|
||
|
||
**Unlabelled exception (Chesapeake).** A price-relief request phrased as
|
||
politeness inside the Pricing section, with no heading and no delta.
|
||
Anacostia's three exceptions are explicitly headed with deltas and act as
|
||
the control.
|
||
|
||
**Non-priceable exception (Anacostia, EX-3).** If declined, the vendor
|
||
cannot bid produce at all — a conditional withdrawal expressed as an
|
||
exception. A price-delta field cannot hold it.
|
||
|
||
**Answered-but-negative (Anacostia).** "We are not currently an approved
|
||
USDA Foods processor." Squarely answers the requirement; the substance is a
|
||
disclosure of inability. Coverage and quality are orthogonal, and a system
|
||
that scores this like Potomac's 340,000 lbs is broken.
|
||
|
||
**Three certifications in one sentence (Chesapeake).** Attachments B, C and
|
||
D are satisfied by a single clause naming all three. A per-requirement
|
||
extractor must split one span three ways.
|
||
|
||
## Finding: a gate that no response document can satisfy
|
||
|
||
`R-B6C` — "Two bid copies submitted" — is a physical submission fact.
|
||
Potomac happens to assert it; the other two do not, and no better extraction
|
||
would change that. Coverage over it is neither answered nor not_answered but
|
||
**unknowable from the artifact**.
|
||
|
||
Surfaced by `validate-responses.py`, not by design. Treating it as
|
||
not_answered would disqualify two of three bidders over a packaging detail.
|
||
Raised against #10.
|