TY  - CONF
AU  - Kofler, Florian
AU  - Wahle, Johannes
AU  - Ezhov, Ivan
AU  - Wagner, Sophia J.
AU  - Al-Maskari, Rami
AU  - Gryska, Emilia
AU  - Todorov, Mihail
AU  - Bukas, Christina
AU  - Meissen, Felix
AU  - Peng, Tingying
AU  - Ertürk, Ali
AU  - Rueckert, Daniel
AU  - Heckemann, Rolf
AU  - Kirschke, Jan
AU  - Zimmer, Claus
AU  - Wiestler, Benedikt
AU  - Menze, Bjoern
AU  - Piraud, Marie
TI  - Approaching Peak Ground Truth
PB  - IEEE
M1  - DZNE-2023-00988
SP  - 1-6
PY  - 2023
AB  - Machine learning models are typically evaluated by computing similarity with reference annotations and trained by maximizing similarity with such. Especially in the biomedical domain, annotations are subjective and suffer from low inter-and intra-rater reliability. Since annotations only reflect one interpretation of the real world, this can lead to sub-optimal predictions even though the model achieves high similarity scores. Here, the theoretical concept of Peak Ground Truth (PGT) is introduced. PGT marks the point beyond which an increase in similarity with the reference annotation stops translating to better Real World Model Performance (RWMP). Additionally, a quantitative technique to approximate PGT by computing inter- and intra-rater reliability is proposed. Finally, four categories of PGT-aware strategies to evaluate and improve model performance are reviewed. © 2023 IEEE.
T2  - 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI)
CY  - 18 Apr 2023 - 21 Apr 2023, Cartagena (Colombia)
Y2  - 18 Apr 2023 - 21 Apr 2023
M2  - Cartagena, Colombia
LB  - PUB:(DE-HGF)8 ; PUB:(DE-HGF)7
DO  - DOI:10.1109/ISBI53787.2023.10230497
UR  - https://pub.dzne.de/record/265364
ER  -