Radar
Top stories
Importance 2 of 5Covered by 1 source

Apple paper proposes a theory of evaluability for foundation models

Apple researchers formalize when models can be reliably evaluated without assuming the test set was held out from training. The work addresses accuracy, safety, calibration and fairness measures for large foundation models.

Why it matters

Contamination of test sets undermines benchmark claims companies rely on when choosing models.