AI refers to the ability of machines to perform tasks that normally require human intelligence – for example, recognizing patterns, learning from experience, drawing conclusions, making predictions, or taking action – whether digitally or as the smart software behind autonomous physical systems
As of right now, there is no globally recognized standard regarding AI & ML in Medical Devices of Healthcare.
There are multiple working groups, developing standards for all industries: Foundational standards (terminology, framework); Big Data (vocabulary, reference architecture); Trustworthiness (risk, robustness, bias); Use cases & applications; AI Governance.
Software risk management activities should be implemented based on the intended use of the software (target disease, clinical use, importance, urgency), usage scenarios (applicable population, target users, places of use, clinical processes), core functions (processing objects, data compatibility, functional types) and throughout the software lifecycle process.
The risk of clinical use of software should include false negative and false positive, where false negative is missed diagnosis, which may lead to delay in follow-up diagnosis and treatment activities, especially to consider the risk of delay in diagnosis and treatment of rapidly progressing diseases. In addition to considering false positive and false negative risks, imported software should also consider the effects and risks of differences in epidemiological characteristics, clinical norms, and treatment. Companies should take adequate, appropriate, and effective risk control measures to ensure the safety and effectiveness of the software.
Demand analysis should be guided by the clinical needs and risks of use of the software, combined with the intended use of the software, use scenarios, and core functions, considering the requirements of regulations, standards, users, products, data, functions, performance, interfaces, user interfaces, network security, warnings focusing on data collection, algorithmic performance, clinical use limitations, and other requirements.
Data collection should consider compliance and diversity of data sources, epidemiological characteristics of targeted diseases, and data quality control requirements. Data sources should ensure data diversity on a compliance basis to improve the ability to generalize algorithms, such as representative clinical institutions from as many, different geographies, and levels as possible, and as many acquisition devices as possible from different acquisition parameters.
Epidemiological characteristics of the target disease include, but are not limited to, disease composition (e.g. classification, classification, station), population distribution (e.g. health, patient, sex, age, occupation, geography, lifestyle), statistical indicators (e.g. morbidity, prevalence, cure rate, mortality rate, survival rate), and the impact of complications and similar diseases of the target disease.
It is important that the quality control of the acquisition equipment should clarify the compatibility requirements and the acquisition requirements of the acquisition equipment. Compatibility requirements should be based on data generation methods (direct generation, indirect generation) to provide a list of acquisition equipment compatibility or technical requirements, clear acquisition equipment manufacturers, model specifications, performance indicators, and other requirements, if there are no specific requirements for acquisition equipment should provide appropriate support information. Acquisition requirements should specify the acquisition method (e.g. regular imaging, enhanced imaging), acquisition protocol (e.g. MRI imaging sequence), acquisition parameters (e.g. CT load voltage, load current, load time, layer thickness), acquisition accuracy (e.g. resolution, sample rate) and other requirements.
The quality control of the collection process should establish the operation specification of data collection, and make clear the requirements of the collection personnel and the requirements of the collection process. Collection personnel requirements include personnel selection, training, assessment. Acquisition process requirements include personnel responsibilities, acquisition processes (e.g. acquisition steps, operational requirements).
Labeling process quality control should establish data labeling operating norms, clearly label personnel (e.g. qualifications, quantity, responsibilities), labeling process (e.g. label objects, label forms, labeling rounds, labeling procedures, operational requirements), clinical norms (e.g. clinical guidelines, expert consensus), disagreement processing (e.g. arbitrators, arbitration methods), and traceability (e.g. data, operations).
Label quality assessment should be clear people, methods, indicators, adoption guidelines, and other requirements.
After the data is labeled, a label database is formed, and its sample type can be divided into data series, single data (composed of multiple blocks of data), and data blocks (image area, data fragments). The risk considerations for sample size, sample distribution, are the same for the underlying database.
Build training sets (for algorithm training), tuning sets (for algorithmic hyper-parameter tuning), test sets (for algorithm performance evaluation) based on label databases, and clarify the division methods, basis, and data allocation ratios of training sets, tuning sets, test sets.
The training set shall ensure that the sample distribution is balanced, the test set and the tuning set shall ensure that the sample distribution conforms to the clinical actual situation, and the samples of the training set, the tuning set, and the test set shall be two or two intersections.
Algorithm selection should specify the name, structure (e.g. number of layers, parameter size), flowchart, out-of-the-box framework (e.g. Tensorflow, PyTorch), input and output, operating environment, algorithm source basis, and other information.
The principles, methods and risk considerations of algorithm selection and design should be clarified, such as quantitative error, gradient disappearance, overfit, white boxing.
If migration learning techniques are used, summary information such as data set construction, validation, and validation of pre-trained models should be supplemented in addition to the above.
Algorithmic training needs to be based on the training set, tuning set training and tuning, should be clearly evaluated indicators, training methods, training objectives, tuning methods, training data volume - evaluation indicator curve, and other requirements.
Evaluation indicators recommend selection based on clinical needs, such as sensitivity, specificity, etc. Training methods include, but are not limited to, the set-aside method and the cross-validation method. Training objectives should meet clinical requirements and be supported by evidence such as ROC curves. The tuning method should clarify the algorithm optimization strategy and implementation method. Amount of training data - The evaluation indicator curve should be able to confirm the advent and effectiveness of algorithmic training.
Network security protection should be combined with the intended use of software, usage scenarios, and core functions, based on confidentiality, integrity, availability, and other network security characteristics, to determine the software network security capacity-building requirements to deal with network threats such as cyberattacks and data theft.
The algorithm performance evaluation needs to evaluate the algorithm design results based on the test set, and the evaluation requirements of false negative and false positive, repetitive and reproducible, robustness/robustness should be clarified to prove that the algorithm performance meets the algorithm design requirements.
At the same time, the algorithm performance factors and their degrees of influence, such as acquisition equipment, acquisition parameters, disease composition, lesions characteristics, and other factors should be analyzed in order to improve the explanatory nature of the algorithm, and as the basis for software verification and software confirmation.
Whether using a predictive (Waterfall; V- model) or adaptive methodology (Agile), it is imperative that the following processes and their corresponding outputs to be incorporated in the SaMD Development:
Manufacturers of SaMD must report any correction or removal of an SaMD if action was initiated to:
The European Commission put forward a European approach to Artificial Intelligence and Robotics. It deals with technological, ethical, legal, and socio-economic aspects to boost the EU's research and industrial capacity and to put AI at the service of European citizens and the economy.
The European Trade Association COCIR posted their position paper regarding AI in healthcare.
Also, you may want to review Knowledge Centre Data & Society (2020) “Feedback on EC White Paper on Artificial Intelligence“
We have the regulatory savvy, QMS experience, and technical expertise to help mature and emerging technologies companies meet their challenges and take full advantage of their business opportunities.
Key documentation on Artificial Intelligence and Machine Learning within Medical Device space