20 fragment queries. For each: run cupertino search "<fragment>" --limit 10, extract top-10 URIs, apply a loose regex that matches URIs whose path contains the fragment as a substring in any segment. Compute P@1, P@5, P@10, and first-match rank. Aggregate as means.
Note on the initial regex bug: an earlier strict regex (requiring the terminal slug to END in the fragment) reported P@5 = 0.49, with Field and Provider showing zero matches. Investigation revealed the strict regex was rejecting valid deeply-nested results like appintents/appshortcutoptionscollectionprotocol/provider and embedded results like defaulttextfieldstyle. The corrected loose regex (substring anywhere in any path segment) measures the actual recall and is what this audit reports.
Harness source: /tmp/cupertino-search-eval-fragment.py (not yet versioned in repo). Full JSON dump (all 20 top-10 lists): /tmp/cupertino-search-eval-fragment-20260520.json.