Pipeline

flowchart TD
    Q["30 canonical-lookup query strings (corpus V2)
zero overlap with the original 50-query corpus"]:::input
    R["per-query regex pattern
accepts canonical + specific-variant URIs"]:::input
    Q --> H[scripts/eval/search-quality-phase1-extended.py]
    R --> H

    BA["Arm A: brew cupertino 1.1.0
+ ~/.cupertino/search.db (schema 13)"]:::arm
    BB["Arm B: dev cupertino 1.2.0
+ ~/.cupertino-dev/search.db (schema 18)"]:::arm
    H --> BA
    H --> BB

    BA --> SA["per-query rank → RR, P@1, P@5, NDCG@10"]
    BB --> SB["per-query rank → RR, P@1, P@5, NDCG@10"]

    SA --> PAIR["paired buckets"]
    SB --> PAIR
    PAIR --> WX["Wilcoxon signed-rank"]:::stat
    PAIR --> MC["McNemar 2×2"]:::stat
    WX --> MD["audit MD"]:::out
    MC --> MD

    classDef input fill:#0a84ff,stroke:#0040cc,color:#fff
    classDef arm fill:#5856d6,stroke:#3634a3,color:#fff
    classDef stat fill:#ff9500,stroke:#c4730a,color:#fff
    classDef out fill:#34c759,stroke:#1f7a3a,color:#fff