On the acceptance of “fake” histopathology: A study on frozen sections optimized with deep learning
Mario Siller1, Lea Maria Stangassinger2, Christina Kreutzer3, Peter Boor4, Roman D Bulow4, Theo J F Kraus5, Saskia von Stillfried4, Soraya Wolfl6, Sebastien Couillard-Despres3, Gertie Janneke Oostingh2, Anton Hittmair7, Michael Gadermayr1
1 Department of Information Technology and System Management, Salzburg University of Applied Sciences, Salzburg, Austria
2 Department of Biomedical Sciences, Salzburg University of Applied Sciences, Salzburg, Austria
3 Institute of Experimental Neuroregeneration, Spinal Cord Injury and Tissue Regeneration Center Salzburg, Paracelsus Medical University, Salzburg, Austria
4 Institute of Pathology, RWTH Aachen University Hospital, Aachen, Germany; Center for Integrated Oncology Aachen Bonn Cologne Duesseldorf (CIO ABCD), Aachen, Germany
5 Institute of Pathology, University Hospital Salzburg, Paracelsus Medical University, Salzburg, Austria
6 Patholab Salzburg, Salzburg, Austria
7 Department of Pathology and Microbiology, Kardinal Schwarzenberg Klinikum, Schwarzach, Austria
Dr. Michael Gadermayr
Department of Information Technology and System Management, Salzburg University of Applied Sciences, Urstein Sud 1, 5412 Puch.
Source of Support: None, Conflict of Interest: None
Background: The fast acquisition process of frozen sections allows surgeons to wait for histological findings during the interventions to base intrasurgical decisions on the outcome of the histology. Compared with paraffin sections, however, the quality of frozen sections is often strongly reduced, leading to a lower diagnostic accuracy. Deep neural networks are capable of modifying specific characteristics of digital histological images. Particularly, generative adversarial networks proved to be effective tools to learn about translation between two modalities, based on two unconnected data sets only. The positive effects of such deep learning-based image optimization on computer-aided diagnosis have already been shown. However, since fully automated diagnosis is controversial, the application of enhanced images for visual clinical assessment is currently probably of even higher relevance. Methods: Three different deep learning-based generative adversarial networks were investigated. The methods were used to translate frozen sections into virtual paraffin sections. Overall, 40 frozen sections were processed. For training, 40 further paraffin sections were available. We investigated how pathologists assess the quality of the different image translation approaches and whether experts are able to distinguish between virtual and real digital pathology. Results: Pathologists’ detection accuracy of virtual paraffin sections (from pairs consisting of a frozen and a paraffin section) was between 0.62 and 0.97. Overall, in 59% of images, the virtual section was assessed as more appropriate for a diagnosis. In 53% of images, the deep learning approach was preferred to conventional stain normalization (SN). Conclusion: Overall, expert assessment indicated slightly improved visual properties of converted images and a high similarity to real paraffin sections. The observed high variability showed clear differences in personal preferences.