SPC365

Artificial Intelligence-based detection of cardiovascular outcomes: from unstructured data to automatic auditing and research

Session: Sessão de Posters 47 - Dos dados às decisões: a revolução da IA em cardiologia
Speaker: Diogo Rosa Ferreira

Congress: CPC 2026
Topic: N. E-Cardiology / Digital Health, Public Health, Health Economics, Research Methodology
Theme: 33. e-Cardiology / Digital Health
Subtheme: 33.4 Digital Health
Session Type: Posters Eletrónicos
FP Number: ---
Authors: Diogo Ferreira; Daniel Cazeiro; Marta Vilela; João Cravo; Cláudia Jorge; Pedro Carrilho Ferreira; João Marques; Pedro Cardoso; Fausto Pinto; Miguel Nobre Menezes

<p style="text-align:justify"><span style="font-size:11pt"><span style="font-family:Arial,sans-serif"><span style="color:#000000"><strong>Introduction:</strong></span></span></span><br />
<span style="font-size:11pt"><span style="font-family:Arial,sans-serif"><span style="color:#000000">Most clinical information in electronic health records is unstructured, making large-scale research and quality monitoring dependent on manual review. Large Language Models (LLMs) offer a scalable alternative, though their performance on cardiology documentation is not well defined. We evaluated locally deployed LLMs for extracting key outcomes from cardiology records, using TAVI procedure and discharge reports as a representative test case.</span></span></span></p>

<p style="text-align:justify"><span style="font-size:11pt"><span style="font-family:Arial,sans-serif"><span style="color:#000000"><strong>Methods:</strong></span></span></span><br />
<span style="font-size:11pt"><span style="font-family:Arial,sans-serif"><span style="color:#000000">Procedural reports and discharge summaries from 521 consecutive TAVI cases were analysed for three endpoints: procedural success; any intraprocedural complications from procedural reports; and intra- or post-procedural complications from discharge notes. Three locally deployed general-use LLMs&mdash;one resource-intensive (Llama3-70B) and two lighter models (Mistral-7B, Mistral-8B)&mdash;were tested. A zero-shot, schema-guided prompting strategy prioritised sensitivity to avoid missed complications. Ground-truth labels came from independent manual review using VARC-3 criteria, with additional non-procedural complications included. Sensitivity, specificity, PPV, NPV, and concordance were calculated.</span></span></span></p>

<p style="text-align:justify"><span style="font-size:11pt"><span style="font-family:Arial,sans-serif"><span style="color:#000000"><strong>Results:</strong></span></span></span><br />
<span style="font-size:11pt"><span style="font-family:Arial,sans-serif"><span style="color:#000000">Procedural success was 99.8%. Manual review found complications in 64 (12.3%) procedural reports and 94 (18.1%) discharge summaries, with a major complication rate of 6.3%.</span></span></span><br />
<span style="font-size:11pt"><span style="font-family:Arial,sans-serif"><span style="color:#000000">All models showed high sensitivity and NPV for procedural success: Llama3-70B and Mistral-7B reached 100% for both, and Mistral-8B performed similarly (99% sensitivity, 100% NPV).</span></span></span><br />
<span style="font-size:11pt"><span style="font-family:Arial,sans-serif"><span style="color:#000000">For intraprocedural complications, Llama3-70B maintained high sensitivity (95%) and NPV (99%). Both Mistral models had &ge;95% NPV, though Mistral-7B showed lower sensitivity (67%). Specificity and PPV were modest, particularly for Mistral-8B (63% specificity, 27% PPV).</span></span></span><br />
<span style="font-size:11pt"><span style="font-family:Arial,sans-serif"><span style="color:#000000">Detection from discharge summaries was harder. Sensitivity remained high (90% for Llama3-70B; 100% and 98% for Mistral-7B and Mistral-8B) with NPVs of 96&ndash;100%, but specificity and PPV were low, especially for the Mistral models (3&ndash;27%). Llama3-70B provided the most balanced overall performance.</span></span></span></p>

<p style="text-align:justify"><span style="font-size:11pt"><span style="font-family:Arial,sans-serif"><span style="color:#000000"><strong>Conclusion:</strong></span></span></span><br />
<span style="font-size:11pt"><span style="font-family:Arial,sans-serif"><span style="color:#000000">Locally deployed LLMs were highly sensitive for detecting procedural success and complications, with NPVs near or at 100%. However, lower specificity and PPV&mdash;mainly from overinterpreting benign findings&mdash;limit fully autonomous use. Only the resource-intensive model delivered consistently high performance, constraining broader implementation. Still, because complications were absent in over 80% of reports, these models could have reduced manual review by more than 80%, greatly improving efficiency. Their potential for automated auditing and research is clear.</span></span></span></p>