TY  - JOUR
AU  - Albarqouni, Shadi
AU  - Baur, Christoph
AU  - Achilles, Felix
AU  - Belagiannis, Vasileios
AU  - Demirci, Stefanie
AU  - Navab, Nassir
TI  - AggNet: Deep Learning From Crowds for Mitosis Detection in Breast Cancer Histology Images.
JO  - IEEE transactions on medical imaging
VL  - 35
IS  - 5
SN  - 0278-0062
CY  - New York, NY
PB  - IEEE
M1  - DZNE-2020-04873
SP  - 1313-1321
PY  - 2016
AB  - The lack of publicly available ground-truth data has been identified as the major challenge for transferring recent developments in deep learning to the biomedical imaging domain. Though crowdsourcing has enabled annotation of large scale databases for real world images, its application for biomedical purposes requires a deeper understanding and hence, more precise definition of the actual annotation task. The fact that expert tasks are being outsourced to non-expert users may lead to noisy annotations introducing disagreement between users. Despite being a valuable resource for learning annotation models from crowdsourcing, conventional machine-learning methods may have difficulties dealing with noisy annotations during training. In this manuscript, we present a new concept for learning from crowds that handle data aggregation directly as part of the learning process of the convolutional neural network (CNN) via additional crowdsourcing layer (AggNet). Besides, we present an experimental study on learning from crowds designed to answer the following questions. 1) Can deep CNN be trained with data collected from crowdsourcing? 2) How to adapt the CNN to train on multiple types of annotation datasets (ground truth and crowd-based)? 3) How does the choice of annotation and aggregation affect the accuracy? Our experimental setup involved Annot8, a self-implemented web-platform based on Crowdflower API realizing image annotation tasks for a publicly available biomedical image database. Our results give valuable insights into the functionality of deep CNN learning from crowd annotations and prove the necessity of data aggregation integration.
KW  - Breast Neoplasms: diagnostic imaging
KW  - Crowdsourcing: methods
KW  - Female
KW  - Histocytochemistry
KW  - Humans
KW  - Image Interpretation, Computer-Assisted: methods
KW  - Internet
KW  - Machine Learning
KW  - Mitosis: physiology
KW  - Neural Networks, Computer
KW  - Video Games
LB  - PUB:(DE-HGF)16
C6  - pmid:26891484
DO  - DOI:10.1109/TMI.2016.2528120
UR  - https://pub.dzne.de/record/138551
ER  -