Journal of Pathology Informatics Journal of Pathology Informatics
Contact us | Home | Login   |  Users Online: 445  Print this pageEmail this pageSmall font sizeDefault font sizeIncrease font size 

Table of Contents    
J Pathol Inform 2021,  12:13

Artificial intelligence in pathology: From prototype to product

1 Fraunhofer MEVIS, Bremen, Germany
2 Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Institute of Pathology; HTW University of Applied Sciences Berlin, Berlin, Germany

Date of Submission29-Sep-2020
Date of Decision28-Dec-2020
Date of Acceptance18-Jan-2021
Date of Web Publication22-Mar-2021

Correspondence Address:
Dr. André Homeyer
Fraunhofer MEVIS, Am Fallturm 1, 28359 Bremen
Login to access the Email id

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/jpi.jpi_84_20

Rights and Permissions

Modern image analysis techniques based on artificial intelligence (AI) have great potential to improve the quality and efficiency of diagnostic procedures in pathology and to detect novel biomarkers. Despite thousands of published research papers on applications of AI in pathology, hardly any research implementations have matured into commercial products for routine use. Bringing an AI solution for pathology to market poses significant technological, business, and regulatory challenges. In this paper, we provide a comprehensive overview and advice on how to meet these challenges. We outline how research prototypes can be turned into a product-ready state and integrated into the IT infrastructure of clinical laboratories. We also discuss business models for profitable AI solutions and reimbursement options for computer assistance in pathology. Moreover, we explain how to obtain regulatory approval so that AI solutions can be launched as in vitro diagnostic medical devices. Thus, this paper offers computer scientists, software companies, and pathologists a road map for transforming prototypes of AI solutions into commercial products.

Keywords: Artificial intelligence, business model, image analysis, integration, pathology, regulatory approval, reimbursement, technology readiness

How to cite this article:
Homeyer A, Lotz J, Schwen LO, Weiss N, Romberg D, Höfener H, Zerbe N, Hufnagl P. Artificial intelligence in pathology: From prototype to product. J Pathol Inform 2021;12:13

How to cite this URL:
Homeyer A, Lotz J, Schwen LO, Weiss N, Romberg D, Höfener H, Zerbe N, Hufnagl P. Artificial intelligence in pathology: From prototype to product. J Pathol Inform [serial online] 2021 [cited 2022 Jul 6];12:13. Available from:

∗André Homeyer, Johannes Lotz, Lars Ole Schwen, and Nick Weiss contributed equally.

   Introduction Top

Artificial intelligence (AI) has the potential to revolutionize pathology.[1],[2],[3] AI refers to the application of modern machine learning techniques to digital tissue images in order to detect, quantify, or characterize specific cell or tissue structures. By automating time-consuming diagnostic tasks, AI can greatly reduce the workload and help to remedy the serious shortage of pathologists.[4] At the same time, AI can make analyses more sensitive and reproducible and it can capture novel biomarkers from tissue morphology for precision medicine. In a survey from the year 2019 of 487 pathologists from 54 countries, a great majority looked forward to using AI as a diagnostic tool.[5]

A wide variety of diagnostic tasks in pathology benefits from AI support. Even general tasks, such as counting nuclei or classifying tumor tissue, usually must be solved in a disease- and tissue-specific manner. A PubMed search for the terms “artificial intelligence” or “machine learning” in conjunction with “pathology” resulted in more than 5000 publications, half of which were published since 2018 (as of September 2020). Several overview papers were published on how AI techniques can be applied in pathology.[6],[7],[8],[9],[10] Recent reviews discuss AI applications in pathology and specifically in breast pathology.[11],[12] Another much-studied application of AI is the extraction of novel immuno-oncology biomarkers from pathology images.[13]

Despite the extensive research developments, very few AI solutions have become commercial products for routine use.[11] As a result, a large part of the potential of AI remains untapped. Various technical, business, and regulatory challenges must be overcome in order to commercialize AI solutions in pathology. Similar challenges also arise for AI solutions in other medical domains,[14] such as radiology[15],[16],[17] and ophthalmology.[18] Research prototypes must be further developed and integrated into the IT infrastructure of clinical laboratories before they are usable in routine pathology workflows. Commercial success requires a profitable business model and the ability for pathologists to be reimbursed for using the product. AI solutions are medical devices and therefore require regulatory approval before they may be sold as a product.

Many publications about AI in pathology point to specific challenges, such as obtaining regulatory approval or reimbursement.[6],[8],[10],[13],[19] A recent paper gives a detailed outline of the regulatory steps for developing clinical AI software.[20] Still, it remains a major effort for AI solution developers to gather the wealth of knowledge required to meet the challenges of commercialization. In the present paper, we aim to provide researchers, software companies, and clinicians with a comprehensive overview on how to transform prototypes of AI solutions into commercial products. To this end, we provide basic knowledge and advice on all the challenges mentioned above.

   Software Maturity Top

The main objective in developing research prototypes is to demonstrate the feasibility of a task. Once feasibility has been verified, research prototypes still need to mature in various ways to meet the requirements for routine use: The software must be of sufficient quality to be stable and maintainable. The solution must also run efficiently enough for the laboratory workflow and reliably deliver accurate results.

Software quality

Research prototypes of AI solutions are often written quickly without considering code quality. This makes such prototypes expensive to maintain and to extend by new features, i.e., liable to “technical debt”.[21] Moreover, it makes prototypes prone to runtime errors and vulnerable to security holes. For these reasons, software quality often needs to be improved before a research prototype is ready for commercialization. Software quality is also important for obtaining regulatory approval. While regulatory approval does not explicitly prescribe a certain degree of software quality, one must adhere to a quality management system (QMS) throughout the entire development process, which should lead to high quality.

Various measures can be taken to improve software quality. Research code often has grown gradually and lacks structure. In this case, it helps to improve its architecture, e.g., by encapsulating functional units in a modular way and by implementing object-oriented design patterns.[22] For achieving stable and properly functioning code, extensive testing is indispensable. Testing should be carried out automatically, both by low-level unit tests and high-level functional and integration tests.[23] Ideally, source code should be developed in a test-driven manner right from the beginning.[24]


Research prototypes of AI solutions for pathology are usually optimized for accuracy, often at the expense of computing efficiency. Such prototypes must be further improved to be fast enough (while still accurate enough) for laboratory practice. The first step in optimizing efficiency should always be to identify the specific performance bottlenecks. In addition, there are several general strategies to make AI solutions for pathology more efficient.[25]

One strategy is to divide long-running operations into several smaller operations that can be executed in parallel. Tensor processing operations, which are performed in large quantities in deep learning-based algorithms, are parallelized automatically by all major machine learning libraries. Similarly, slide images can be divided into smaller “tiles” and processed in parallel. Another common strategy is to apply an algorithm to a lower resolution image which is substantially faster to process, if this does not affect accuracy in a relevant way.


Research prototypes of AI solutions have only been trained and tested on limited datasets provided by a clinical project partner or in the context of a challenge. Such datasets hardly cover the real-world variability of tissue images. The variability of tissue images is enormous, as their appearance is influenced by the depicted tissue types, pathologies, and the processing of the tissue section, including its cutting, staining, and scanning.[6] For this reason, research prototypes often need to be trained further on larger and more representative datasets before becoming sufficiently reliable for routine use.[11]

There are no simple criteria for deciding whether a particular dataset is representative. In general, datasets should be collected in a consistent and well-curated form from several laboratories using individual preparation and scanning procedures and covering different ethnic groups and disease subtypes.[10],[26],[27]

The need for collecting large datasets can be somewhat reduced by incorporating synthetic data in the training process. The most popular technique is data augmentation, in which several transformations are derived from the original training data according to some variability model. Besides standard transformations such as rotations and changes in contrast, more advanced data augmentation techniques can be employed, e.g., to simulate staining variability.[28],[29] Another recent approach is using Generative Adversarial Networks to automatically create realistic synthetic histology images.[30],[31]

Technology readiness levels

In addition to software maturity, there are further technical requirements to make AI solutions in pathology ready for routine use.[32] These include integrability into the laboratory IT infrastructure and compliance with regulatory technical requirements, as described in the next sections.

Technical readiness is commonly classified in technology readiness levels (TRLs) on a spectrum from an initial idea to a solution for productive use. As such, TRLs are a practical measure for both internal product management and communication with investors. Originating from aerospace engineering,[33] TRLs have been used in different domains with slightly varying definitions.[34],[35],[36],[37] We have adapted the original definition of TLRs to illustrate the steps required to turn a prototype AI solution in pathology into a product [Figure 1].
Figure 1: Technology readiness levels for artificial intelligence solutions in digital pathology

Click here to view

A typical research prototype corresponds to TRL 3 if it was evaluated in a peer-reviewed publication, or TRL 4 if it has proven itself in a community challenge.[38],[39] Product development is an iterative process requiring input and feedback from the intended users. TRL 5 provides initial information for defining efficiency and reliability requirements. Fulfilling these requirements is necessary to reach TRL 7, i.e., a clinical prototype with demonstrated utility in an operational environment. The clinical prototype is the basis for obtaining regulatory approval and ultimately achieving product status on TRL 9.

   Integration Top

Research prototypes of AI solutions in pathology are usually standalone tools with their own interfaces for execution, data input, and data output. To be usable in diagnostic routine, the tools must be integrated into the laboratory IT infrastructure and made interoperable with software from other manufacturers.

Laboratory IT infrastructure

Three pieces of software are of particular importance for using AI solutions in pathology [Figure 2]. The central component of any laboratory IT infrastructure is the Pathology Laboratory Information System (PLIS).[40] Its core functionality is case and sample management, especially storing all received and generated data on specimens and tracking specimen-related data during histological processing. Many PLISs also support work planning of laboratory personnel and coding and billing of diagnostic procedures.
Figure 2: Communication paths between an artificial intelligence solution and the main IT components in a digital pathology lab

Click here to view

In digital pathology laboratories, another important piece of software is the image archive,[41] which manages and provides access to slide images and their associated metadata. Manufacturers of slide scanner devices offer special slide server software optimized for their own image formats. In addition, there are Picture Archiving and Communications Systems (PACSs) that manage and provide access to medical image data in a standardized way. Such PACSs were originally developed for radiology images, but are now also available for pathology images. Health-care institutions are increasingly using vendor-neutral archives (VNAs) that integrate various types of medical images, including pathology images.[42] Such VNAs greatly facilitate sharing image data between departments and reduce maintenance costs by consolidating multiple storage systems into a single central solution.[43] A recent publication provides guidance on the use of VNAs in pathology.[42]

The digital pathology workstation is the software with which the pathologist interacts most when assessing a case. Its user interface is centered around a virtual microscope viewer providing means for exploring and interacting with a slide image, e.g., to make measurements or annotations. For this purpose, the workstation is closely linked to the image archive. Many workstations also provide functionality for accessing additional patient information or submitting diagnostic reports by communicating with the PLIS. Slide scanner manufacturers typically offer special workstation software for their devices. However, there is also an increasing number of vendor-neutral or open-source workstation software.[44],[45]

Communication with PLISs works according to general clinical messaging standards that define how to encode patient data, specimen data, and diagnostic reports. The most widespread standard Health Level Seven International (HL7) V2 was developed by the HL7 organization in the 1980s.[46] The latest standard called HL7 Fast Healthcare Interoperable Resources (FHIR) is based on current Internet technologies and easier to use.[47] Image archive systems offered by slide scanner manufacturers usually exhibit proprietary interfaces for accessing image data. In contrast, PACSs and VNA systems enable uniform image access through the Digital Imaging and Communications in Medicine (DICOM) standard. Originating from radiology, the DICOM standard is also becoming increasingly popular in pathology.[48]

Integration of artificial intelligence

AI solutions can be integrated into the laboratory IT infrastructure either as standalone tools or workstation extensions.[8],[45],[49],[50] Standalone tools provide their own user interface for triggering an analysis, for selecting input parameters, such as regions-of-interest or algorithm settings, and for presenting analysis results. With workstation extensions, all these functions are performed through the user interface of their host software. Workstation extensions must implement a predefined programming interface through which they are triggered and input and output data are exchanged.

The major advantage of standalone tools is flexibility. They can be tailored to the specific needs of analysis methods in terms of user interaction, parameterization, and visualization. In particular, standalone tools enable long-running methods with multiple analysis steps including preprocessing. Workstation extensions must adhere to the programming interface of their host and get along with its basic interaction and visualization capabilities. For this reason, they must work with minimal user input and finish quickly in a single analysis step.

Workstation extensions, on the other hand, have the major advantage of workflow efficiency. They can be used without leaving the familiar workstation environment and without having to become acquainted with new tools. Another advantage of workstation extensions is lower development costs. Standalone tools must be interoperable with PLIS, image archive, and workstation software in order to retrieve input data or submit results. Given the different types of interfaces, this can be a major effort. For workstation extensions, this functionality is already provided by their host and does not need to be implemented again.

Because of their respective advantages and disadvantages, standalone tools are mainly offered by well-established manufacturers with a comprehensive portfolio of solutions for different applications.[51],[52] For new companies intending to turn a prototype into a product, it can be easier to offer their solution as a workstation extension.

There is no standard for integrating AI solutions into workstation software, but each system offers its own specific interface. The interface definitions of commercial workstation software are often not public and only accessible through partnerships with the respective manufacturers. While some workstations integrate AI solutions as dynamically-linked software components,[53] most modern systems integrate them as Docker containers or RESTful Web services.[49],[50] The latter technologies have the advantage of allowing solutions to be implemented in any programming language. They also enable distributed operation of the solution on optimized hardware (e.g., GPUs), which is particularly important when applying AI technologies to large slide images.

   Business Model Top

Successful development and sale of AI solutions for pathology require a sustainable business model. A business model is defined as “the rationale of how an organization creates, delivers, and captures value”.[54] At the heart of every business model is a value proposition for customers. Around this, a business model covers the conditions required to create that value, how to deliver that value to customers, and how to generate revenue.

Business model canvas

A popular approach to developing a business model are canvases, i.e., large poster-like forms summarizing different aspects of the business model. Following the popular business model canvas template,[55] we discuss the aspects for AI solutions in pathology [Figure 3]. We here focus on technology-driven start-ups aiming to bring a technology into the clinical market rather than exit-driven business models where the product is the start-up itself.[57]
Figure 3: Example business model canvas for a pathology artificial intelligence solution: It is recommended to construct the business model starting with the right side that focuses on the customer before addressing the left side that addresses the business itself.[54] This figure is based on the Strategyzer business model canvas,[55] licensed CC-BY-SA[56]

Click here to view

Customer segments

Customers decide on the purchase of a solution. They are not always identical to users, but their interests often overlap. There are two main customer segments for image analysis solutions in pathology: the first are pathology laboratories or hospitals who are also users of the solutions. The second are vendors of pathology workstations or device manufactures who integrate and distribute third-party solutions with their software—often as white label products under their own name.

Value proposition

The value proposition is what makes the customer pay money for a solution and can differ between customer segments. Assumptions about the value proposition need to be validated as early as possible, e.g., through customer relations or market research. For AI solutions in pathology, the principal value propositions are cost reduction, quality improvement, and innovation. AI solutions can reduce costs by assisting pathologists and enabling them to perform more work in less time. Pathologists can focus more on complicated cases by saving time on simple cases. Quality improvement means higher accuracy or sensitivity of diagnostic tests. Finally, AI solutions can compute innovative computational biomarkers that are prognostic for the course of a disease or the effectiveness of a therapy.


AI solutions can be distributed through direct or indirect sales channels. It is common practice for developers of AI solutions to partner with manufacturers of scanner devices or pathology software systems. The respective products are then either cross-sold directly by both manufacturers,[58] or the AI solution is integrated into existing software and distributed indirectly through the respective manufacturer.[59] Particularly for new companies, it can be attractive to enter business development deals with established manufacturers and to benefit from their market reach and marketing.[60]

In recent years, app stores have become a common way of distributing software.[61] App stores facilitate multiple aspects of a business model such as distribution, advertising, purchase processing, and customer feedback.[62] Although there is not yet a vendor-neutral app store where developers can sell AI solutions for pathology, the first web-based directories have been launched.[63],[64]

Customer relations

Building customer relationships is important to validate and adapt the value proposition according to the customer needs. Important points of contact between developers of AI solutions and customers are scientific congresses, the company website, and social media. Furthermore, webinars and user group meetings are commonly hosted in order to showcase new products or features and collect feedback in a systematic way. Especially in the early phase of a company, it can make sense to offer freemium software[65],[66] or free trials, since testing the value proposition may be more valuable than first sales.

Revenue streams

AI solutions for pathology can be licensed for a one-time payment, as part of a subscription service, or on a pay-per-use basis. Recurring payment models are becoming increasingly popular in the software market. Subscription services ensure a continuous revenue stream for the developer while entitling the customer to the latest updates.[67] Pay-per-use minimizes the entry barrier for the customer, but also leads to an uncertain revenue stream for the developer.

The realized revenues depend strongly on the pricing strategy. In cost-based pricing, the costs of development and maintenance determine the price of the product,[68] which may result in a price below the customer's willingness to pay.[69] In competition-based pricing, the aim is to achieve the price of a competitor. Customer-based pricing estimates the value that the product brings to the customer, such as the cost savings from faster analysis or the reimbursement paid for computer assistance.[69]

Key resources, activities, and partners

Key resources for creating an AI solution for pathology are a development team with experience in machine learning and professional software development, computing resources to develop software and train machine learning models, and perhaps most importantly, data. Since the quality of an AI solution highly depends on the quality and representativeness of the training data, a large number of tissue images and high-quality annotations must be obtained from several laboratories.

Key activities include demonstrating clinical utility and robustness, establishing a distribution channel, and obtaining regulatory approval. If the value proposition focuses on quality improvement instead of cost reduction, establishing reimbursement options is also an important activity.

Key partners are pathologists who assist in identifying opportunities for computer support, who acquire tissue images and annotations, and who provide feedback on the usefulness of a solution. Other important partners are manufacturers of scanner devices or pathology software systems who integrate and distribute AI solutions as part of their software platforms.

Cost structure

The main cost factors in the development of AI solutions for pathology are personnel and the procurement of sufficient amounts of training and testing data. Other significant costs arise from office operations, computing resources, fees for consulting pathologists, and marketing. In technology-driven business models, a large part of the value lies in the skills and knowledge of the employees. Hiring highly qualified employees is therefore often a higher priority than reducing personnel costs.[57] Furthermore, in the early stages of a new business, winning more customers and growing the business is often a higher priority than cutting costs, provided that financing allows for this.[60]

Iterative business models

A novel business model is based on many assumptions. To reduce the risk of failure, one should initially focus on the quick release of a minimum viable product (MVP).[70] An MVP contains only essential features necessary to win first customers. This makes it possible to receive early feedback and validate the assumptions of the business model.

The need to obtain regulatory approval makes it very costly to develop AI solutions for routine use. A cheaper way to test the utility of a new solution and detect potential shortcomings is to provide an analysis service from a central laboratory as an intermediate business model.[71] Other laboratories can send in physical tissue slides or digital tissue images and the central laboratory returns a report about the analysis results. The central laboratory model is already established for complex or proprietary immunohistochemical (IHC) tests that cannot be conducted at local laboratories.

The central laboratory model can be implemented by offering AI solutions as laboratory-developed tests (LDTs). With some limitations, LDTs may use any hardware or software components without requiring them to be certified medical devices.[72] Instead, the laboratory as a whole must be accredited to work competently[73] and the test must be validated in-house, which is usually less costly than the approval of a medical device. Once an AI solution has proven itself within the central laboratory model, the effort of certification as a medical device is worthwhile in order to reach a larger customer group.

   Regulation Top

In most countries, medical products may only be distributed with regulatory approval. From a regulatory point of view, AI solutions for pathology are in vitro diagnostic medical devices (IVDMDs), as they evaluate images of body fluids or tissue outside the body for diagnostic or therapeutic purposes.[74] We here describe different concepts relevant in the regulation of IVDMDs, focusing on the EU and the USA as two examples. In this context, we address specific challenges and current developments in the regulation of AI solutions.

Regulation basics

The approval process for medical devices varies between legislations and some countries have separate regulations for IVDMDs.[75],[76] In the EU, IVDMDs were regulated by Directive 98/79/EC[77] until 2017. After a transition phase until May 26, 2022, IVDMDs will be regulated by Regulation 2017/746.[78] General medical devices, which are not IVDMDs, are regulated separately.[79] In the USA, regulatory procedures for IVDMDs are defined in the Code of Federal Regulations, Title 21[80] and are the same as for other medical devices.[81]

Agencies handling the regulatory process of IVDMDs are structured differently in different legislations: they can be part of the ministry of health (e.g., in China and Israel), a specialized agency (e.g., the U. S. Food and Drug Administration [FDA]), or the task can be delegated further (e. g., EU, India or Japan).[75],[76],[82] One example for delegation are the Notified Bodies in the EU, which are independent organizations handling the actual approval process.[78] Manufacturers are free to choose with which Notified Body in their country they want to work.

Medical devices are commonly classified according to their risk for the patient. In the EU, AI solutions in pathology are considered class C devices (on a scale from A to D with increasing risk) according to Annex VIII, 2.3 of.[77] The FDA considers AI solutions in pathology class II medical devices (on a scale from I to III with increasing risk).[20],[83]

Regulatory approval procedures

The approval of medical software usually requires compliance with quality standards in the development process and an assessment of the product [Figure 4].[75]
Figure 4: Overview of documents necessary for obtaining regulatory approval

Click here to view

Some legislations have different pathways for approval depending on the novelty and the risk of the respective device. As an example, the FDA uses specific terminology for different types of what is generally denoted as regulatory approval.[84] High-risk (class III) devices need to be “approved” by the FDA before entering the market in the USA, whereas class II devices are “cleared” after passing a less complex process.[85] In case of positive evaluation via the “de novo” pathway (see below), devices are “granted.”[86] In either case, the product may legally be marketed afterward (where “marketing” in the regulatory context denotes entering the market at one point in time, and pre-/postmarket refers to what needs to be done beforehand/afterward).

Having established a QMS is one requirement for regulatory approval (e. g., Annex IX of[78] in the EU or Part 820 of[80] in the USA). This is commonly demonstrated by compliance with ISO 13485[87] or similar national standards. Other relevant standards for AI solutions in pathology are ISO 14971[88] for risk management and IEC 62304[89] for software life cycle processes.

The assessment of a product requires generating and submitting extensive documentation:

  • Technical documentation, a description of requirements, architecture, and test results;
  • Performance evaluation, see below; and
  • A plan for postmarket surveillance, proactive monitoring, and addressing incidences.

In the EU, these requirements are described in Annexes II, XIII, and III of[78] and need to be submitted for the specific product.

For a device to be cleared by the FDA in the USA, the manufacturer must demonstrate that the device is substantially equivalent to another device with FDA clearance or approval. The manufacturer must submit a 510(k) premarket notification submission to the FDA, who clears the device if equivalence is demonstrated.[85],[90] Otherwise, the manufacturer must request a de novo classification[86] and demonstrate the safety and efficacy of the device.[91]

In the EU, the manufacturer may use the Conformité Européenne (CE) mark for the device after having successfully completed the approval process.[83] Other agencies (e.g., the FDA) offer a database of approved devices on their website.

Performance evaluation

In addition to the establishment of a QMS, the proper functional performance of the product is also assessed in regulatory procedures. As an example, the performance evaluation required in the EU has three aspects (Annex XIII of[78]):

  • Scientific validity: showing that there is a valid association between software output and targeted clinical condition,[92]
  • Analytical performance: showing that the software produces accurate and reliable output, e.g., by comparison to a reference approach for analysis, and
  • Clinical performance: showing that the output meets the intended purpose in a clinical context and for the target population.[92]

The performance evaluation for FDA clearance is similar, and it may be less effort if equivalence to another product is shown.

The performance evaluation is generally conducted in three steps: (a) writing a plan for each part of the evaluation, (b) assessing the tool according to the plan, and (c) writing a report on the results. These tasks require extensive amounts of validation data to be acquired by the manufacturer.

Development of AI-based solutions and the final products typically use third-party components, e.g., machine learning libraries. Such components are off-the-shelf software (OTSS) and/or software of unknown/uncertain pedigree/provenance (SOUP) from a regulatory point of view.[93] The manufacturer of the AI-based solution needs show that the OTSS or SOUP fulfills the requirements defined for the product at hand.

Scaling internationally

Requirements for national approval vary if a software has already been approved elsewhere. For instance, a CE mark or FDA clearance is sufficient for an IVDMD to enter the market in India and certain South American nations.[75] Other countries, such as Israel, only require some additional administrative steps.[75] In the USA, on the other hand, a completely new approval procedure is required, even if the devices were previously approved elsewhere.

Requirements for the QMS of the manufacturer may differ in details but are generally similar to those in the ISO 13485 standard.[87] Similarly, parts of performance analyses can be re-used even if details in the requirements differ between legislations. There are, however, initiatives to harmonize medical device regulation internationally.[75],[94]

Regulation of artificial intelligence solutions

Image analysis software for digital pathology has been developed for many years and some current solutions are already AI-based (e.g.[95]). However, AI-based software still poses certain challenges for regulation, such as novelty for specific application, opaqueness of algorithms, representativeness of training data, and evolving algorithms.

Existing scientific literature on AI solutions for specific applications may be sparse to nonexistent or of uncertain quality.[96] This makes a critical assessment of the scientific state of the art, which is necessary in the context of performance evaluation, difficult.

Training and validation of AI algorithms require data sufficiently representative for the target applications to avoid bias.[6],[96] Assessing whether data are representative, however, is subject to many assumptions. Moreover, algorithms must be robust with respect to small changes in datasets[96] and corrupt input data.[97]

The fact that AI algorithms adapt to new data makes them “uncharted territory for organizations that regulate medical devices.”[98] If only the manufacturer trains an algorithm and each new or re-trained version remains static, the performance evaluation can be carried out in advance, which is easier to handle in the regulatory approval process.[99] If, however, the algorithm is to be further trained by the user, it becomes dynamic after approval and a “moving target for regulators”.[96] Moreover, algorithms using AI (e.g., neural networks) usually act as “black boxes” lacking transparency and explainability how analysis results are obtained.[6],[96]

Regulatory authorities like the FDA are currently in the process of establishing clear guidelines and procedures for AI-based IVDMDs.[92],[100] As a result, almost no AI solutions have been approved for in vitro diagnostics to date. In March 2019, the FDA for the first time classified an AI solution for pathological cancer diagnostics as a “breakthrough device” to accelerate product development and regulatory review.[101] This solution for the detection of prostate cancer was also the first deep-learning-based solution to receive European IVD approval.[102]

Obtaining regulatory approval for IVDMDs is challenging already in one country, and even more so when scaling internationally. In addition, regulatory procedures for AI-based IVDMDs are currently being updated. To cope with this complexity, it can be necessary to seek help from specialized consultants and legal experts.

   Reimbursement Top

The payer of diagnostic pathology services is either a health insurance entity or the patient. Pathologists claim reimbursement for their services from these payers. AI solutions for pathology cause investment, maintenance, and operating costs, e.g., for licensing, IT administration, or manual supervision. For AI solutions to be used in practice, it is thus essential that pathologists can be reimbursed for their costs.

Billing medical services is extremely complex. Reimbursement systems vary from country to country. Even within countries, the procedures and amounts of reimbursement often vary between health insurance entities, insurance plans, and between regions.[103] This section provides basic knowledge about reimbursement for AI solutions in pathology, using the USA and Germany as examples, to facilitate the establishment of reimbursement opportunities for newly developed products.

Fee schedules

Most people in the USA are enrolled in private health insurance.[104] In addition, there are public health insurance programs like Medicare and Medicaid covering elderly or low-income people. In Germany, most people are enrolled in statutory health insurance funds. It is also possible to take out private health insurance with an insurance company.

Most reimbursement systems are based on a kind of fee schedule that lists the content and fees of the various medical services [Figure 5]. Diagnostic pathology services provided to Medicare patients in the USA are reimbursed according to the Medicare Physician Fee Schedule (MPFS).[105] The MPFS is often taken as the basis for the fee schedules of private health insurance plans and its rates are an upper limit on the fees reimbursed by Medicaid. The fees for clinical diagnostic laboratory tests are not listed in the MPFS but in the separate Clinical Laboratory Fee Schedule (CLFS).[106]
Figure 5: Fee schedules are negotiated between different stakeholders and determine reimbursement for diagnoses in digital pathology. This ultimately influences the revenues of artificial intelligence solution developers in digital pathology

Click here to view

Germany has two basic medical fee schedules. The “Einheitlicher Bewertungsmaßstab” (EBM) schedule applies to services billed to statutory health insurers[107] and the “Gebührenordnung für Ärzte” (GOÄ) applies to billings to private health insurers.[108] The EBM only covers services deemed necessary and economical. Patients with statutory health insurance can obtain services not listed in the EBM on a self-pay basis, which are then billed according to the GOÄ.

Most fee schedules in the USA use the Current Procedural Terminology (CPT) code system for identifying medical services.[109] Medicare or Medicaid use an extension to the CPT called Healthcare Common Procedure Coding System (HCPCS).[110] The EBM and GOÄ fee schedules in Germany use individual code systems for identifying services.

Reimbursement of novel procedures

In the USA, the CPT system and the MPFS are updated annually.[105] While the EBM in Germany is continuously being revised,[111] the GOÄ was last updated in 1996 and therefore does not consider the latest medical progress. The GOÄ, however, gives physicians the opportunity to value novel medical services not listed in the schedule “analogously,” i.e., according to a listed service that is equivalent in type, cost, and time (see,[108] § 6[2]).

The national associations of pathologists play a key role in introducing new procedures in the fee schedules. In the USA, the College of American Pathologists (CAP) administers the Pathology Coding Caucus, which makes recommendations about the inclusion of new procedures into the CPT.[105] Furthermore, the CAP advises on the fees for new diagnostic pathology services in the MPFS.[105] In Germany, the Federal Association of Pathologists maintains a dedicated fees commission,[112] which makes proposals for changes to the EBM or GOÄ and which publishes recommendations on the analog valuation of pathology services.[113]

For a new diagnostic procedure to become reimbursable, its clinical utility must be demonstrated.[114] The clinical utility of a procedure refers to its ability to improve patient outcomes, such as extensions of survival or reductions of morbidity. The clinical utility also refers to whether the benefits of a procedure are justified by its costs or effort. Diagnostic procedures can only indirectly improve patient outcomes by enabling better treatment decisions. Therefore, it can be helpful to develop a review framework to clarify the relationships between test results, clinical decisions, and patient outcomes.[115] The clinical utility of diagnostic procedures is typically demonstrated by comparison with existing tests.[114] In the USA, there is a program for the concurrent review of analytical and clinical performance for regulatory approval and clinical utility for Medicare coverage determination.[116]

Reimbursement for computer assistance

Most AI solutions in pathology offer some kind of computer assistance based on automated image analysis. None of the fee schedules in the USA or Germany contain general listings for computer assistance in pathology or, in particular, for the use of AI. However, as explained in the following three examples, it is already possible to obtain reimbursement for certain diagnostic procedures involving computer assistance. The additional valuation of computer assistance differs significantly between procedures, between countries, and often also over time.

Screening for cervical cancer

Many countries offer cytological screening services to women to detect cervical cancer early.[117] Computer assistance in the analysis of cytological samples can reduce screening errors and cause a significant increase in sensitivity.[118]

Cervical cancer screening tests in the USA are reimbursable under the CLFS. For conventional cytology, screening with computer assistance was valued US$ 16 higher than manual screening in 2020 (HCPCS G0148-P3000). For liquid-based cytology, which offers better sample quality, the use of computer assistance was valued US$ 6 higher (HCPCS G0145-G0123).[106] Both examples refer to location-guided screening, where the computer automatically identifies a certain number of points of interest which must then be reviewed manually. The German EBM and GOÄ do not distinguish between manual or computer-assisted CC or LBC and therefore do not allow for additional reimbursement of computer assistance.[119]

DNA image cytometry

DNA image cytometry is sometimes used to identify or grade tumors based on irregularities in the DNA content of cells, e.g., as a supplement to Gleason grading in prostate cancer. With computer assistance, considerably more cells can be evaluated, making the analysis much more sensitive.[120]

While the MPFS in the USA does not contain a listing for DNA image cytometry, this is one of the few procedures with computer assistance that can be billed in Germany. Conventional DNA image cytometry without computer assistance can be billed according to the EBM code 19330 at € 31.42.[107] Since 2015, the procedure with computer assistance can be billed to the patient on a self-pay basis at a price of € 193.74.[121] There is no dedicated code for the procedure, but it is valued analogously according to the GOÄ.[122]


IHC biomarkers measure the expression of proteins relevant in tumorigenesis (e.g., ER/PR, PD-L1, Ki67, or HER2). They provide valuable information about the course of cancer or the likelihood of response to therapy.[123] Computer assistance enables the automatic quantification of protein expression and thus makes the analysis of IHC biomarkers more accurate and reproducible.[124]

In the USA, the evaluation of IHC biomarkers is reimbursable under the MPFS since 2004.[125] Manual or computer-assisted assessment can be billed under CPT/HCPCS codes 88360 or 88361, respectively. The additional fees for computer assistance have steadily decreased over the years. While the fees for both codes differed by US$ 31.74 in 2009, the difference in 2020 was only US$ 1.80.[106] The German fee schedules do not list any dedicated fees for the use of computer assistance in the evaluation of IHC biomarkers.

   Conclusions Top

As described in the previous sections, a multitude of challenges must be overcome for turning prototypes of AI solutions in pathology into profitable products. These challenges can best be tackled in cooperation.

Cooperation is crucial

It is important to involve pathologists from an early stage of development. To obtain sufficient quantities of training and test data, partnerships must be established with pathologists and incentives should be provided for them to provide image data and required annotations. It is essential to enter into such partnerships with multiple laboratories in order to obtain a sample of interlaboratory variability. In order to develop a successful product, it is indispensable to take the pathologists' needs into account. Therefore, pathologists should be regularly provided with new prototypes and encouraged to test the prototypes extensively and provide systematic feedback. In view of the novelty and the high entry barrier to digital pathology, it is important to offer comprehensive support to pathologists, e.g., through user meetings and webinars.

Besides pathologists, it can be beneficial to collaborate with workstation manufacturers. The number of pathology laboratories working digitally is still small and there are no widely used distribution channels. By building strategic partnerships with workstation manufacturers who distribute AI solutions as part of their software, one can benefit from existing sales channels and customers.

Solution developers should also get involved with pathology associations. Cooperation with the national professional associations of pathologists is essential to establish reimbursement opportunities for new diagnostic procedures. In recent years, several organizations were founded specifically for the advancement of digital pathology, such as the Digital Pathology Association,[126] the Alliance for Digital Pathology,[127] and the European Society for Digital and Integrative Pathology.[128] It can pay off to join these organizations in order to participate in shaping regulatory processes and technical standards.

Remaining challenges

There are further challenges in producing AI solutions that are beyond the scope of this paper. This concerns the many algorithmic challenges associated with AI development, which have already been addressed in numerous publications.[6],[7],[8],[9],[10] Likewise, this concerns issues of usability and user experience.[129] From a business perspective, a major challenge is setting up a company and financing the initial phase while the business model is not yet profitable. There is also extensive literature on this subject, e.g., on applying for public funding[130],[131] or participating in incubator or accelerator programs.

One of the biggest barriers to the use of AI in pathology remains the inadequate IT infrastructure in most pathology laboratories. Only few laboratories have digital slide scanners at their disposal. Existing pathology software, such as PLISs, is often years old and lacks support for modern software technologies to integrate AI solutions (like web-based APIs or containerization).[132] For the use of AI in pathology to become widespread, laboratories must be equipped with state-of-the-art digital pathology systems as standard.


Besides products for clinical-diagnostic purposes, which we focused on in this paper, AI solutions can also be turned into products for research purposes, e.g., to calculate tissue parameters for scientific studies. Research solutions have much lower requirements for product quality and robustness. Moreover, obtaining regulatory approval and reimbursement are irrelevant. Compared to the clinical market, however, the research market is much smaller.

Despite the challenges, there has never been a better time to bring AI solutions for pathology to market. Thanks to the latest AI technologies, automated image analysis has reached a level of robustness required for diagnostic use. Furthermore, digital pathology is finding its way into laboratory practice. Multiple vendors offer scanners and workstation software that are technologically mature and cleared for primary diagnosis. Finally, the potential of AI in pathology is widely recognized and multiple countries are funding initiatives to establish digitization and AI-based diagnosis in pathology.[20],[133],[134],[135]


The authors would like to thank Sabrina Haase for fruitful discussions about regulatory affairs and Wolfgang Daniel Schmitt for advice on reimbursement.

Financial support and sponsorship

This work was funded by the German Federal Ministry for Economic Affairs and Energy via the EMPAIA project (grant numbers 01MK20002B, 01MK20002A).

Conflicts of interest

There are no conflicts of interest.

   References Top

Niazi MK, Parwani AV, Gurcan MN. Digital pathology and artificial intelligence. Lancet Oncol 2019;20:e253-61.  Back to cited text no. 1
Dolgin E. The AI Medical Revolution Starts Here; 2018. Available from: 20. [Last accessed on 2020 Aug 18].  Back to cited text no. 2
Parwani AV. Next generation diagnostic pathology: Use of digital pathology and artificial intelligence tools to augment a pathological diagnosis. Diagn Pathol 2019;14:138.  Back to cited text no. 3
Robboy SJ, Weintraub S, Horvath AE, Jensen BW, Alexander CB, Fody EP, et al. Pathologist workforce in the United States: I. Development of a predictive model to examine factors influencing supply. Arch Pathol Lab Med 2013;137:1723-32.  Back to cited text no. 4
Sarwar S, Dent A, Faust K, Richer M, Djuric U, Van Ommeren R, et al. Physician perspectives on integration of artificial intelligence into diagnostic pathology. NPJ Digit Med 2019;2:28.  Back to cited text no. 5
Abels E, Pantanowitz L, Aeffner F, Zarella MD, van der Laak J, Bui MM, et al. Computational pathology definitions, best practices, and recommendations for regulatory guidance: A white paper from the digital pathology association. J Pathol 2019;249:286-94.  Back to cited text no. 6
Serag A, Ion-Margineanu A, Qureshi H, McMillan R, Saint Martin MJ, Diamond J, et al. Translational AI and deep learning in diagnostic pathology. Front Med (Lausanne) 2019;6:185.  Back to cited text no. 7
Aeffner F, Zarella MD, Buchbinder N, Bui MM, Goodman MR, Hartman DJ, et al. Introduction to digital image analysis in whole-slide imaging: A white paper from the digital pathology association. J Pathol Inform 2019;10:9.  Back to cited text no. 8
[PUBMED]  [Full text]  
Tizhoosh HR, Pantanowitz L. Artificial intelligence and digital pathology: Challenges and opportunities. J Pathol Inform 2018;9:38.  Back to cited text no. 9
[PUBMED]  [Full text]  
Bera K, Schalper KA, Rimm DL, Velcheti V, Madabhushi A. Artificial intelligence in digital pathology-New tools for diagnosis and precision oncology. Nat Rev Clin Oncol 2019;16:703-15.  Back to cited text no. 10
Acs B, Rantalainen M, Hartman J. Artificial intelligence as the next step towards precision pathology. J Intern Med 2020;288:62-81.  Back to cited text no. 11
Ibrahim A, Gamble P, Jaroensri R, Abdelsamea MM, Mermel CH, Chen PC, et al. Artificial intelligence in digital breast pathology: Techniques and applications. Breast 2020;49:267-73.  Back to cited text no. 12
Koelzer VH, Sirinukunwattana K, Rittscher J, Mertz KD. Precision immunoprofiling by image analysis and artificial intelligence. Virchows Arch 2019;474:511-22.  Back to cited text no. 13
He J, Baxter SL, Xu J, Xu J, Zhou X, Zhang K. The practical implementation of artificial intelligence technologies in medicine. Nat Med 2019;25:30-6.  Back to cited text no. 14
Recht MP, Dewey M, Dreyer K, Langlotz C, Niessen W, Prainsack B, et al. Integrating artificial intelligence into the clinical practice of radiology: Challenges and recommendations. Eur Radiol 2020;30:3576-84.  Back to cited text no. 15
Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts HJ. Artificial intelligence in radiology. Nat Rev Cancer 2018;18:500-10.  Back to cited text no. 16
Dreyer KJ, Geis JR. When machines think: Radiology's next frontier. Radiology 2017;285:713-8.  Back to cited text no. 17
Abràmoff MD, Tobey D, Char DS. Lessons learnt about autonomous AI: Finding a safe, efficacious and ethical path through the development process. Am J Ophthalmol 2020;214:134-42.  Back to cited text no. 18
Mandal S, Greenblatt AB, An J. Imaging intelligence: AI is transforming medical imaging across the imaging spectrum. IEEE Pulse 2018;9:16-24.  Back to cited text no. 19
Colling R, Pitman H, Oien K, Rajpoot N, Macklin P, CM-Path AI in Histopathology Working Group, et al. Artificial intelligence in digital pathology: A roadmap to routine use in clinical practice. J Pathol 2019;249:143-50.  Back to cited text no. 20
Ramasubbu N, Kemerer CF. Managing technical debt in enterprise software packages. IEEE Trans Softw Eng 2014;40:758-72.  Back to cited text no. 21
Martin RC. Clean Code: A Handbook of Agile Software Craftsmanship. Upper Saddle River, New Jersey, USA: Prentice Hall; 2008.  Back to cited text no. 22
Myers GJ. The Art of Software Testing. Hoboken, New Jersey, USA: Wiley; 2004.  Back to cited text no. 23
Williams L, Maximilien EM, Vouk M. Test-Driven Development as a Defect-Reduction Practice. In: Proceedings of 14th IEEE International Symposium on Software Reliability Engineering ISSRE 2003; 2003. p. 34-45.  Back to cited text no. 24
Kothari S, Phan JH, Stokes TH, Wang MD. Pathology imaging informatics for quantitative analysis of whole-slide images. J Am Med Inform Assoc 2013;20:1099-108.  Back to cited text no. 25
Challen R, Denny J, Pitt M, Gompels L, Edwards T, Tsaneva-Atanasova K. Artificial intelligence, bias and clinical safety. BMJ Qual Saf 2019;28:231-7.  Back to cited text no. 26
Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC Med 2019;17:195.  Back to cited text no. 27
Tellez D, Litjens G, Bándi P, Bulten W, Bokhorst JM, Ciompi F, et al. Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology. Med Image Anal 2019;58:101544.  Back to cited text no. 28
Lafarge MW, Pluim JPW, Eppenhof KAJ, Veta M. Learning domain-invariant representations of histological images. Front Med (Lausanne) 2019;6:162.  Back to cited text no. 29
Tschuchnig ME, Oostingh GJ, Gadermayr M. Generative Adversarial Networks in Digital Pathology: A Survey on Trends and Future Potential; 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 30
Levine AB, Peng J, Farnell D, Nursey M, Wang Y, Naso JR, et al. Synthesis of diagnostic quality cancer pathology images by generative adversarial networks. J Pathol 2020;252:178-88.  Back to cited text no. 31
Eljasik-Swoboda T, Rathgeber C, Hasenauer R. Assessing Technology Readiness for Artificial Intelligence and Machine Learning Based Innovations. In: Proceedings of the 8th International Conference on Data Science, Technology and Applications (Data 2019); 2019. p. 281-8.  Back to cited text no. 32
NASA. Technology Readiness Level Definitions; 2010. Available from: [Last accessed on 2020 Jun 11].  Back to cited text no. 33
European Space Agency. Technology Readiness Levels Handbook for Space Applications; 2008. Available from: [Last accessed on 2020 Jun 11].  Back to cited text no. 34
European Commission. Horizon 2020 – Work Programme 2014/15, Annex G. Technology Readiness Levels; 2014. Available from: [Last accessed on 2020 Jun 11].  Back to cited text no. 35
Armstrong JR. 6.4.2 Applying Technical Readiness Levels to Software: New thoughts and Examples. In: INCOSE International Symposium; 2010. p. 838-45.  Back to cited text no. 36
CloudWATCH. Readiness for Market: More than Completing Software Development; 2017. Available from: [Last accessed on 2020 Jun 11].  Back to cited text no. 37
Huang CC, Lu Z. Community challenges in biomedical text mining over 10 years: Success, failure and the future. Brief Bioinform 2016;17:132-44.  Back to cited text no. 38
Friedberg I, Wass MN, Mooney SD, Radivojac P. Ten simple rules for a community computational challenge. PLoS Comput Biol 2015;11:e1004150.  Back to cited text no. 39
Park SL, Pantanowitz L, Sharma G, Parwani AV. Anatomic pathology laboratory information systems: A review. Adv Anat Pathol 2012;19:81-96.  Back to cited text no. 40
Stathonikos N, Nguyen TQ, Spoto CP, Verdaasdonk MA, van Diest PJ. Being fully digital: Perspective of a Dutch academic pathology laboratory. Histopathology 2019;75:621-35.  Back to cited text no. 41
Pantanowitz L, Sharma A, Carter AB, Kurc T, Sussman A, Saltz J. Twenty years of digital pathology: An overview of the road travelled, what is on the horizon, and the emergence of vendor-neutral archives. J Pathol Inform 2018;9:40.  Back to cited text no. 42
[PUBMED]  [Full text]  
Sirota-Cohen C, Rosipko B, Forsberg D, Sunshine JL. Implementation and benefits of a vendor-neutral archive and enterprise-imaging management system in an integrated delivery network. J Digit Imaging 2019;32:211-20.  Back to cited text no. 43
Marée R, Rollus L, Stévens B, Hoyoux R, Louppe G, Vandaele R, et al. Collaborative analysis of multi-gigapixel imaging data using Cytomine. Bioinformatics 2016;32:1395-401.  Back to cited text no. 44
Bankhead P, Loughrey MB, Fernández JA, Dombrowski Y, McArt DG, Dunne PD, et al. QuPath: Open source software for digital pathology image analysis. Sci Rep 2017;7:16878.  Back to cited text no. 45
Benson T, Grieve G. Principles of Health Interoperability: SNOMED CT, HL7 and FHIR. London, UK: Springer; 2016.  Back to cited text no. 46
Bender D, Sartipi K. HL7 FHIR: An Agile and RESTful Approach to Healthcare Information Exchange. In: Proceedings of the 26th IEEE International Symposium on Computer-Based Medical Systems, Porto, 2013. p. 326-331.  Back to cited text no. 47
Herrmann MD, Clunie DA, Fedorov A, Doyle SW, Pieper S, Klepeis V, et al. Implementing the DICOM standard for digital pathology. J Pathol Inform 2018;9:37.  Back to cited text no. 48
[PUBMED]  [Full text]  
Rollus L, Others. Cytomine ULiège R&D Documentation; 2019. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 49
APEER Docs; 2019. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 50
Visiopharm App Center – APPsolute Image Analysis; 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 51
Indica Labs Halo Modules; 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 52
QuPath Wiki; 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 53
Osterwalder A, Pigneur Y. Business Model Generation: A Handbook for Visionaries, Game Changers, and Challengers. Hoboken, New Jersey, USA: John Wiley & Sons; 2010.  Back to cited text no. 54
Stragegyzer AG. Business Model Canvas; 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 55
Creative Commons. Attribution-Sharealike 3.0 Unported (CC BY-SA 3.0); 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 56
Chammassian RG, Sabatier V. The role of costs in business model design for early-stage technology startups. Technol Forecast Soc Change 2020;157:120090.  Back to cited text no. 57
About Visiopharm; 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 58
Press Release: Corista and Indica Labs Announce Integration to Advance Digital Pathology Workflows; 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 59
Weinberg G, Mares J. Traction: How any Startup Can Achieve Explosive Customer Growth. New York, NY: Portfolio/Penguin; 2015.  Back to cited text no. 60
Jansen S, Bloemendal E. Defining App Stores: The Role of Curated Marketplaces in Software Ecosystems. In: International Conference of Software Business ICSOB 2013: Software Business. From Physical Products to Software Services and Solutions; 2013. p. 195-206.  Back to cited text no. 61
Krypczyk V, Bochkor O. App Stores-Eine Kritische Analyse des Vertriebskanals; 2016. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 62
APEER – Cloud-Based Digital Image Processing Platform; 2020.Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 63
Modelhub; 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 64
Kumar V. Making “Freemium” Work; 2014. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 65
Liu CZ, Au YA, Choi HS. Effects of freemium strategy in the mobile app market: An empirical study of Google Play. J Manag Inf Sys 2014;31:326-54.  Back to cited text no. 66
Visiopharm Subscription Editions; 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 67
Shapiro D. How to Price an AI Project; 2017. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 68
Schindler RM. Pricing Strategies: A Marketing Approach. Thousand Oaks, California, USA: SAGE Publications; 2011.  Back to cited text no. 69
Ries E. The Lean Startup: How Today's Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses. New York, USA: Crown Business; 2011.  Back to cited text no. 70
Lange H, Luengo C. Pathology AI (Artificial Intelligence) Reference Guide; 2019. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 71
Schreier J, Feeney R, Keeling P. Diagnostics reform and harmonization of clinical laboratory testing. J Mol Diagn 2019;21:737-45.  Back to cited text no. 72
Tzankov A, Tornillo L. Hands - On experience: Accreditation of pathology laboratories according to ISO 15189. Pathobiology 2017;84:121-9.  Back to cited text no. 73
European Commission. Guidance Document Medical Devices-Scope, Field of Application, Definition-Qualification and Classification of Stand Alone Software-MEDDEV 2.1/6; 2016. Available from: [Last accessed on 2020 Jul 01].  Back to cited text no. 74
Lamph S. Regulation of medical devices outside the European Union. J R Soc Med 2012;105 Suppl 1:S12-21.  Back to cited text no. 75
Gupta SK. Medical device regulations: A current perspective. J Young Pharma 2016;8(1):6–11.  Back to cited text no. 76
European Commission. Directive 98/79/EC of the European Parliament and of the Countil of 27 October 1998 on In vitro Diagnostic Medical Devices; 1998. Available from: 31998L0079&from=EN. [Last accessed on 2020 Apr 15].  Back to cited text no. 77
European Commission. Regulation (EU) 2017/746 of the European Parliament and of the Council of 5 April 2017 on In vitro Diagnostic Medical Devices and Repealing Directive 98/79/EC and Commission Decision 2010/227/EU; 2017. Available from: 32017R0746&from=EN. [Last accessed on 2020 Apr 06].  Back to cited text no. 78
European Commission. Regulation (EU) 2017/745 of the European Parliament and of the Council of 5 April 2017 on Medical Devices, Amending Directive 2001/83/EC, Regulation (EC) no 178/2002 and Regulation (EC) no 1223/2009 and Repealing Council Directives 90/385/EEC and 93/42/EEC; 2017. Available from: [Last accessed on 2020 Apr 06].  Back to cited text no. 79
Code of Federal Regulations, Title 21: Food and Drugs; 2020. Available from: [Last accessed on 2020 Jul 14].  Back to cited text no. 80
U. S. Food and Drug Administration. Overview of IVD Regulation; 2019. Available from: [Last accessed on 2020 May 07].  Back to cited text no. 81
World Health Organization. Medical Devices Regulatory Systems at Country Level (Japan); 2017. Available from: [Last accessed on 2020 Apr 15].  Back to cited text no. 82
García-Rojo M, De Mena D, Muriel-Cueto P, Atienza-Cuevas L, Domínguez-Gómez M, Bueno G. New European Union regulations related to whole slide image scanners and image analysis software. J Pathol Inform 2019;10:2.  Back to cited text no. 83
Rish T. How FDA distinguishes between clearance vs. approval vs. granted. Available from: [Last accessed on 2020 Jun 29].  Back to cited text no. 84
Mitroff S. FDA Approved vs. FDA Cleared: Why You Need to Know the Difference. c | net Health and Wellness; 2019. Available from: [Last accessed on 2020 Jun 29].  Back to cited text no. 85
U. S. Food and Drug Administration. De novo Classification Request; 2019. Available from: [Last accessed on 2020 Jun 29].  Back to cited text no. 86
International Organization for Standardization. ISO 13485:2016 Medical Devices - Quality Management Systems - Requirements for Regulatory Purposes; 2016. Available from: [Last accessed on 2020 Jul 01].  Back to cited text no. 87
International Organization for Standardization. ISO 14971:2019 Medical Devices-Application of Risk Management to Medical Devices; 2019. Available from: [Last accessed on 2020 Jul 01].  Back to cited text no. 88
International Electrotechnical Commission. IEC 62304:2006 Medical Device Software-Software Life Cycle Processes; 2006. Available from: [Last accessed on Jul 01].  Back to cited text no. 89
U. S. Food and Drug Administration. Premarket Notification 510(k); 2020. Available from: [Last accessed on 2020 Jun 29].  Back to cited text no. 90
Speer J. De novo Pathway: Explaining the Process for Medical Device Companies; 2019. Available from: [Last accessed on 2020 Jun 29].  Back to cited text no. 91
U. S. Food and Drug Administration. Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD). Discussion Paper and Request for Feedback; 2020. Available from: [Last accessed on 2020 Apr 15].  Back to cited text no. 92
Magnusson A. Regulations for the Development of Medical Device Software. Göteborg, Sweden: Department of Signal; Systems, Chalmers University of Technology; 2012. Available from: [Last accessed on 2020 Sep 18].  Back to cited text no. 93
U. S. Food and Drug Administration. Medical Device Single Audit Program (MDSAP); 2020. Available from: [Last accessed on 2020 Apr 08].  Back to cited text no. 94
Proscia. DermAI: Streamline High-Volume Derm Workflows; 2020. Available from: [Last accessed on 2020 Apr 30].  Back to cited text no. 95
Ordish J, Murfet H, Hall A. Algorithms as Medical Devices. PHG Foundation; 2019. Available from: [Last accessed on 2020 Apr 07].  Back to cited text no. 96
American Medical Informatics Association. Feedback on FDA-2019-N-1185 “Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD)-Discussion Paper and Request for Feedback”; 2019. Available from: [Last accessed on 2020 Apr 20].  Back to cited text no. 97
Wagner C. Regulating AI-Based Medical Devices: A Moving Target?; 2019. Available from: regulating-ai-based-medical-devices-a-moving-target/. [Last accessed on 2020 Apr 20].  Back to cited text no. 98
Kuan R. Adopting AI in health care will be slow and difficult; 2019. Available from: adopting-ai-in-health-care-will-be-slow-and-difficult. [Last accessed on 2020 Apr 15].  Back to cited text no. 99
European Commission. White Paper on Artificial Intelligence-A European Approach to Excellence and Trust; 2020. Available from: [Last accessed on 2020 Apr 20].  Back to cited text no. 100
Businesswire. FDA Grants Breakthrough Designation to Paige AI; 2019. Available from: [Last accessed on 2020 Apr 20].  Back to cited text no. 101
CISION PR Newswire. Paige Announces First Ever CE Mark for Deep Learning in Pathology; 2019. Available from: [Last accessed on 2020 Apr 20].  Back to cited text no. 102
LeRoy L, Kalfoglou AL, Wolman DM, editors. Medicare Laboratory Payment Policy: Now and in the Future. Washington, DC, USA: National Academies Press; 2001.  Back to cited text no. 103
United States Census Bureau. Income, Poverty and Health Insurance Coverage in the United States: 2018; 2019. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 104
College of American Pathologists. Medicare Physician Fee Schedule; 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 105
Centers for Medicare and Medicaid Services. Clinical Laboratory Fee Schedule; 2020. Available from: [Last accessed on 2020 Jun 11].  Back to cited text no. 106
Kassenärztliche Bundesvereiningung. Einheitlicher Bewertungsmaßstab (1. Juli 2020); 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 107
Bundesministerin der Justiz und für Verbraucherschutz. Gebührenordnung für Ärzte; 2019. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 108
American Medical Association. CPT® Purpose and Mission; 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 109
Centers for Medicare and Medicaid Services. HCPCS-General Information; 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 110
Bundesministerium für Gesundheit. Glossar: Einheitlicher Bewertungsmaßstab-EBM; 2020.Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 111
Bundesverband Deutscher Pathologen. Vorstand; 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 112
Bundesverband Deutscher Pathologen. Mitgliederhandbuch; 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 113
Developing Clinical Evidence for Regulatory and Coverage Assessments in In vitro Diagnostics (IVDs). Medical Device Innovation Consortium; 2019. Available from: [Last accessed on 2021 Jan 03].  Back to cited text no. 114
Samson D, Schoelles KM. Chapter 2: Medical tests guidance (2) developing the topic and structuring systematic reviews of medical tests: Utility of PICOTS, analytic frameworks, decision trees, and other frameworks. J Gen Intern Med 2012;27 Suppl 1:S11-9.  Back to cited text no. 115
Food and Drug Administration, Centers for Medicare and Medicaid Services. Program for parallel review of medical devices. Fed Regist 2016;81:73113-5.  Back to cited text no. 116
Klug SJ, Neis KJ, Harlfinger W, Malter A, König J, Spieth S, et al. A randomized trial comparing conventional cytology to liquid-based cytology and computer assistance. Int J Cancer 2013;132:2849-57.  Back to cited text no. 117
S3-Leitlinie Prävention des Zervixkarzinoms. Langversion 1.1; 2020. Available from: [Last acessed on 2020 Aug 19].  Back to cited text no. 118
Kassenärztliche Bundesvereiningung. Organisierte Früherkennung von Gebärmutterhalskrebs. Die Leistungen im Überblick; 2020.Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 119
Böcking A, Chen J, Friedrich D, Meyer-Ebrecht D. Computer-unterstützte Erkennung von Krebszellen mittels DNA-Karyometrie. Trilium Krebsmedizin 2015;2015. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 120
Landesverband Prostatakrebs-Selbsthilfe Baden-Württemberg (LPS-BW). Merkblatt für Patienten zur Diagnostischen DNA-Zytometrie/DNA-Karyometrie; 2015. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 121
Patho.punkt 03.2015; 2015. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 122
Alvi E, Gupta R, Borok RZ, Escobar-Hoyos L, Shroyer KR. Overview of established and emerging immunohistochemical biomarkers and their role in correlative studies in MRI. J Magn Reson Imaging 2020;51:341-54.  Back to cited text no. 123
Hamilton PW, Bankhead P, Wang Y, Hutchinson R, Kieran D, McArt DG, et al. Digital pathology and image analysis in tissue biomarker research. Methods 2014;70:59-73.  Back to cited text no. 124
Chromavision Medical Systems, Inc., Form 10-K for the Fiscal Year Ended December 31, 2004. [SEC Edgar File]; 2004. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 125
Digital Pathology Association; 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 126
The Alliance for Digital Pathology; 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 127
European Society for Digital and Integrative Pathology; 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 128
Molin J. Diagnostic Review with Digital Pathology: Design of Digitals Tools for Routine Diagnostic Use. PhD Thesis. Department of Computer Science; Engineering, Chalmers University of Technology; 2016. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 129
Bundesministerium für Wirtschaft und Energie. Das EXIST-Gründerstipendium; 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 130
Small Business Innovation Research (SBIR) Program; 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 131
Guo H, Birsa J, Farahani N, Hartman DJ, Piccoli A, O'Leary M, et al. Digital pathology and anatomic pathology laboratory information system integration to support digital pathology sign-out. J Pathol Inform 2016;7:23.  Back to cited text no. 132
[PUBMED]  [Full text]  
Ecosystem for Pathology Diagnostics with AI Assistance; 2020. Available from: [Last accessed on 2020 Aug 19].  Back to cited text no. 133
Marble HD, Huang R, Dudgeon SN, Lowe A, Herrmann MD, Blakely S, et al. A regulatory science initiative to harmonize and standardize digital pathology and machine learning processes to speed up clinical innovation to patients. J Pathol Inform 2020;11:22.  Back to cited text no. 134
  [Full text]  
Wiegand T, Krishnamurthy R, Kuglitsch M, Lee N, Pujari S, Salathé M, et al. WHO and ITU establish benchmarking process for artificial intelligence in health. Lancet 2019;394:9-11.  Back to cited text no. 135


  [Figure 1], [Figure 2], [Figure 3], [Figure 4], [Figure 5]

This article has been cited by
1 EMPAIA – Ökosystem zur Nutzung von KI in der Pathologie
Peter Hufnagl
Der Pathologe. 2021;
[Pubmed] | [DOI]
2 EMPAIA App Interface: An open and vendor-neutral interface for AI applications in pathology
Daniel Romberg, Klaus Strohmenger, Christoph Jansen, Tobias Küster, Nick Weiss, Christian Geißler, Tomasz Soltysinski, Michael Takla, Peter Hufnagl, Norman Zerbe, André Homeyer
Computer Methods and Programs in Biomedicine. 2021; : 106596
[Pubmed] | [DOI]




   Browse articles
    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

  In this article
   Software Maturity
   Business Model
    Article Figures

 Article Access Statistics
    PDF Downloaded993    
    Comments [Add]    
    Cited by others 2    

Recommend this journal