Preprints with The Lancet is a collaboration between The Lancet Group of journals and SSRN to facilitate the open sharing of preprints for early engagement, community comment, and collaboration. Preprints available here are not Lancet publications or necessarily under review with a Lancet journal. These preprints are early-stage research papers that have not been peer-reviewed. The usual SSRN checks and a Lancet-specific check for appropriateness and transparency have been applied. The findings should not be used for clinical or public health decision-making or presented without highlighting these facts. For more information, please see the FAQs.
22 PagesPosted: 16 Dec 2024
See all articles by Sven F. Seys
Sven F. Seys
Hippo Dx
Valérie Hox
Catholic University of Louvain (UCL) - Cliniques Universitaires Saint-Luc
Adam Chaker
Technische Universität München (TUM)
Glynnis De Greve
GZA Sint-Augustinus
Winde Lemmens
Hospital Oost-Limburg
Anne-Lise Poirrier
University of Liège
Eline Beckers
Hospital Oost-Limburg
Rembert Daems
Hippo Dx
Zuzana Diamant
KU Leuven
Carmen Dierickx
Hospital Oost-Limburg
Peter W. Hellings
KU Leuven
Caroline Huart
Catholic University of Louvain (UCL) - Cliniques Universitaires Saint-Luc
Claudia Jerin
Technische Universität München (TUM)
Mark Jorissen
UZ Leuven
Dirk Loeckx
Hippo Dx
Hanne Ocsé
GZA Sint-Augustinus
Karolien Roux
University of Groningen
Mark Thompson
Zurich University of Applied Sciences
Sophie Tombu
University of Liège
Saartje Uyttebroek
UZ Leuven
Andrzej Zarowski
GZA Sint-Augustinus
Senne Gorris
Hippo Dx
Laura Van Gerven
UZ Leuven
More...
Abstract
Background: Theskin prick test (SPT) is the gold standard for diagnosing allergic sensitization to aeroallergies. A novel device, Skin Prick Automated Test (SPAT), has previously demonstrated reduced variability and more consistent test results compared to manual SPT. The current study aimed to develop and validate an artificial intelligence (AI) assisted readout method to support physicians in interpreting skin reactions following SPAT.
Methods: 963 patients with suspected aeroallergies underwent SPT using SPAT for ten common allergens. To train and validate the AI algorithm, respectively 7812 (651 patients, 75%) and 2604 (217 patients, 25%) wheals were manually annotated by a person blinded to the outcome of the AI. The longest wheal diameter was measured by the treating physician and compared to the AI measurement. The AI-assisted readout was validated on a separate test cohort of 95 patients (1140 wheals).
Findings: The AI measurements of the longest wheal diameter exhibited a strong correlation with the physician’s measurements. The AI algorithm showed a specificity of 98.4% and sensitivity of 85·0% in determining positive or negative test results in the validation cohort. In the test cohort, physicians adjusted 5·8% of AI measurements, leading to a change in the test interpretation for only 0·5% of cases. AI-assisted readout significantly reduced inter- and intra-observer variability and readout time compared to manual physician measurements.
Interpretation: The AI-assisted readout software demonstrated high accuracy, with minimal misclassification of test results. Adding AI to SPAT further improved standardization across the SPT process, significantly reducing observer variability and time to readout.
Trial Registration: This study wasregistered online at www.clinicaltrials.gov (NCT05918354).
Funding:The study was funded by Hippocreates BV. LVG was supported by post-doctoral grants from 86 the University Hospitals Leuven (KOOR-UZ Leuven) and by the Research Foundation Flanders 87 (FWO) Senior Clinical Investigator Fellowship (18B2222N).
Declaration of Interest:MJT received consulting fees for statistical advice for the study. SFS, RD, DL are employees of Hippocreates BV. SFS, RD, DL, SG and LVG hold shares of Hippocreates BV. ZD received speaker or consultant honoraria and/or served on advisory boards (past 36 months) at: Antabio, Arcede, Biosion, Foresee Pharmaceuticals, Galenus Health, GlaxoSmithKline, Hippo43 Dx, Pleuran, QPS-Netherlands, Sanofi-Genzyme-Regeneron. During the last 3 years of her assignment as Research Director Respiratory and Allergy at QPS-Netherlands, the company received an European grant from ERA4TB and funding from Foresee Pharmaceuticals for early clinical studies. ZD serves/ed as associate editor for Allergy (2018-2023) and Respiratory Medicine (ongoing) and acted as Chair of the Asthma Section at EAACI (2017-2019) and Expert Panel at EUFOREA (2020-2024). AMC reports grants, speaker honoraria, consultancy or advisory fees and/or research support and other, all via Technical University of Munich from Allergopharma, ALK Abello, Astra Zeneca, Bencard/Allergen Therapeutics, GSK, Novartis, Hippo Dx, LETI, Roche, Zeller, Sanofi, Regeneron, Thermo Fisher, European Institute of Technology (EIT Health) and Federal Ministry of Research and Education Germany. Other authors have nothing to disclose related to this study.
Ethical Approval: The study was approved by the institutional review boards.
Keywords: Skin prick test, diagnosis, type I hypersensitivity, allergy, skin prick automated test, machine learning, artificial intelligence, medical device
Suggested Citation:Suggested Citation
Seys, Sven F. and Hox, Valérie and Chaker, Adam and De Greve, Glynnis and Lemmens, Winde and Poirrier, Anne-Lise and Beckers, Eline and Daems, Rembert and Diamant, Zuzana and Dierickx, Carmen and Hellings, Peter W. and Huart, Caroline and Jerin, Claudia and Jorissen, Mark and Loeckx, Dirk and Ocsé, Hanne and Roux, Karolien and Thompson, Mark and Tombu, Sophie and Uyttebroek, Saartje and Zarowski, Andrzej and Gorris, Senne and Van Gerven, Laura, Artificial Intelligence (AI)-Assisted Readout Method for the Evaluation of Skin Prick Automated Test Results. Available at SSRN: https://ssrn.com/abstract=5054180 or http://dx.doi.org/10.2139/ssrn.5054180
Sven F. Seys (Contact Author)
Hippo Dx ( email )
Valérie Hox
Catholic University of Louvain (UCL) - Cliniques Universitaires Saint-Luc ( email )
Belgium
Adam Chaker
Technische Universität München (TUM) ( email )
Glynnis De Greve
GZA Sint-Augustinus ( email )
Winde Lemmens
Hospital Oost-Limburg ( email )
Anne-Lise Poirrier
University of Liège ( email )
Eline Beckers
Hospital Oost-Limburg ( email )
Rembert Daems
Hippo Dx ( email )
Zuzana Diamant
KU Leuven ( email )
Carmen Dierickx
Hospital Oost-Limburg ( email )
Peter W. Hellings
KU Leuven ( email )
Caroline Huart
Catholic University of Louvain (UCL) - Cliniques Universitaires Saint-Luc ( email )
Belgium