Journal of Pathology Informatics

ORIGINAL ARTICLE
Year
: 2021  |  Volume : 12  |  Issue : 1  |  Page : 54-

Stress testing pathology models with generated artifacts


Nicholas Chandler Wang1, Jeremy Kaplan1, Joonsang Lee1, Jeffrey Hodgin2, Aaron Udager2, Arvind Rao1 
1 Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
2 Department of Pathology, University of Michigan Medical School, Ann Arbor, MI, USA

Correspondence Address:
Dr. Arvind Rao
100 Washtenaw Ave, Room 2305, Ann Arbor, MI 48109
USA

Background: Machine learning models provide significant opportunities for improvement in health care, but their “black-box” nature poses many risks. Methods: We built a custom Python module as part of a framework for generating artifacts that are meant to be tunable and describable to allow for future testing needs. We conducted an analysis of a previously published digital pathology classification model and an internally developed kidney tissue segmentation model, utilizing a variety of generated artifacts including testing their effects. The artifacts simulated were bubbles, tissue folds, uneven illumination, marker lines, uneven sectioning, altered staining, and tissue tears. Results: We found that there is some performance degradation on the tiles with artifacts, particularly with altered stains but also with marker lines, tissue folds, and uneven sectioning. We also found that the response of deep learning models to artifacts could be nonlinear. Conclusions: Generated artifacts can provide a useful tool for testing and building trust in machine learning models by understanding where these models might fail.


How to cite this article:
Wang NC, Kaplan J, Lee J, Hodgin J, Udager A, Rao A. Stress testing pathology models with generated artifacts.J Pathol Inform 2021;12:54-54


How to cite this URL:
Wang NC, Kaplan J, Lee J, Hodgin J, Udager A, Rao A. Stress testing pathology models with generated artifacts. J Pathol Inform [serial online] 2021 [cited 2022 May 23 ];12:54-54
Available from: https://www.jpathinformatics.org/article.asp?issn=2153-3539;year=2021;volume=12;issue=1;spage=54;epage=54;aulast=Wang;type=0