石田憲太郎_LLMATCH第二期最終発表会

石田憲太郎 LLMを使ったカルテの構造化 〜実臨床への応用〜 ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 1

研究テーマ ー カルテの構造化 ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 2

診療文章の8割は自由記載 診療文章の例 Diagnosis: Uterine corpus, hysterectomy: - Endometrioid adenocarcinoma, Grade 1. Tumor size: 2.4 cm in greatest dimension. Lymph-Vascular invasion: Not identified. pTNM (UICC 7th): pT1a pN0. Surgical margin: Negative. - Leiomyoma. Uterine cervix, hysterectomy: - No evidence of malignancy. Adnexa, right, status post-oophorectomy, salpingectomy: - No evidence of malignancy. Adnexa, left, salpingo-oophorectomy: - No evidence of malignancy. Lymph node, dissection: - No evidence of malignancy (0/33). Microscopic Findings: [Uterine Corpus (1–11)] Grade 1 endometrioid carcinoma is identified in sections 1, 3, 5, 7, 10, and 11. The tumor is confined to the endometrium. No lymph vascular invasion is identified. Leiomyomas are identified in sections 6 and 8. [Uterine Cervix (12)] No intraepithelial lesion or carcinoma is identified. [Right Fallopian Tube (13)] No malignant findings. [Left Fallopian Tube (14)] Dilated fallopian tube with cystic changes is noted. [Left Ovary (15)] No malignant findings. [Lymph Nodes (16–24)] No tumor metastasis is identified. (16) Right common iliac nodes (0/2), (17) Right external iliac nodes (0/5), (18) Right internal iliac nodes (0/4), (19) Right obturator nodes (0/5), (20) Presacral nodes (0/5), (21) Left external iliac nodes (0/2), (22) Left common iliac nodes (0/3), (23) Left obturator nodes (0/5) ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 3 3

LLMにカルテの自由記載を構造化させたい

Machine-readable structured data
Diagnosis:
Uterine corpus, hysterectomy:
- Endometrioid adenocarcinoma, Grade 1.
Tumor size: 2.4 cm in greatest dimension.
Lymph-Vascular invasion: Not identified.
pTNM (UICC 7th): pT1a pN0.
Surgical margin: Negative.
- Leiomyoma.
Uterine cervix, hysterectomy:
- No evidence of malignancy.
Adnexa, right, status post-oophorectomy,

Unstructured
EHR data

processed by LLM

©︎MATSUO LAB, THE UNIVERSITY OF TOKYO

{”Specimen": ”hysterectomy",
“Diagnosis”: ”adenocarcinoma",
”Tumor size":{”Horizontal": "17
mm",
”Depth": "5 mm"},
“Lyv”: “absent",
”Margin": ”Positive",
"TNM":{"pT":"1b1","pN":"X","M":"X"}}

4

課題1 : 医療者にプログラミングは難しい 多くの医療関係者にとって プログラミングは難しい 使いやすいUIも必須 ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 5

LLMATCH 第1期の取組 生成AIを簡単にローカル/クラウドで扱えるアプリ開発 ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 6

課題2 : ChatGPTはそのままでは使えない When using commercial LLMs (e.g., ChatGPT) Diagnosis: Uterine corpus, hysterectomy: - Endometrioid adenocarcinoma, Grade 1. Tumor size: 2.4 cm in greatest dimension. Lymph-Vascular invasion: Not identified. pTNM (UICC 7th): pT1a pN0. Surgical margin: Negative. - Leiomyoma. Uterine cervix, hysterectomy: - No evidence of malignancy. Adnexa, right, status post-oophorectomy, Personal information Diagnosis: Uterine corpus, hysterectomy: - Endometrioid adenocarcinoma, Grade 1. Tumor size: 2.4 cm in greatest dimension. Lymph-Vascular invasion: Not identified. pTNM (UICC 7th): pT1a pN0. Surgical margin: Negative. - Leiomyoma. Uterine cervix, hysterectomy: - No evidence of malignancy. Adnexa, right, status post-oophorectomy, 日本の外に出る 学習に使われる 既報では公開データや匿名化した上での報告が多い ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 7

個人情報をそのまま入力できる環境を用意 ・2つの環境を構築 ・倫理委員会の承認を得る クラウド環境 セキュアな接続 国内サーバー、学習に使わない契約 ローカル環境 ローカルサーバーを構築 Ada 6000 *2枚→96GB VRAM 実際にカルテを処理させて精度を見る ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 8

実験内容 「腫瘍登録」というがん患者の登録情報の判定を行った LLMが自動で １、登録すべき患者を判定 ２、TNM分類と呼ばれるがんのステージを抽出・判定 個人情報をマスクしない、生の情報を扱う ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 9

Prompt Engineering for Effective LLM Performance
Example of prompt
Prompt translated into English
You are an AI assistant that determines the presence or absence of distant metastasis
based on imaging findings.
Input is provided in JSON format, including date, modality, and radiology reports.
Please output the classification and the supporting statements in JSON format, paying
attention to the following points.

タスクごとにプロンプト設計

Input format:
JSON
[{"Report Name": "Modality Name,"
"Diagnosis": "Diagnostic content,"
"Findings": "Detailed findings"}]

Output format:
{“M”: “[M0 or M1],”
“reasoning”: “[List the basis for distant metastasis determination from the findings]”}
Criteria for determination:
M classification:
M0: No distant metastasis recognized
M1: Distant metastasis recognized

Important notes:
Regional lymph nodes: Regional lymph nodes are pelvic lymph nodes (obturator, internal
iliac, external iliac, presacral, and common iliac) and para-aortic lymph nodes.
Distant metastasis: Metastasis to lymph nodes other than regional ones (e.g., inguinal,
mediastinal, and supraclavicular) is classified as distant metastasis (M1).
Do not easily judge inflammatory nodules or suspicious findings as distant metastasis
without due consideration. In cases of ovarian tumors or cancer, do not classify peritoneal
dissemination as distant metastasis.

©︎MATSUO LAB, THE UNIVERSITY OF TOKYO

10

1、登録すべき患者を判定 Secure environment Patient for registration すべてのカルテを抽出 カルテ 2024年の 入院患者606人 LLM:Qwen3-30B-A3B or LLMにジャッジさせる ・がん患者か ・治療は2024年に始まっているか ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO Others 11 11

1、結果 1人あたりの入力は平均6万字 210人が登録すべきと判断 Judgement by LLM 606人 登録すべき患者はすべて含まれていた 偽陰性=0 LLMにをスクリーニングとして使えば、 606人→210人に労力が削減できる ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 12

２、TNM分類と呼ばれるがんのステージを抽出・判定

病理レポートからの抽出

放射線レポートからの判定

Diagnosis:
Uterine corpus, hysterectomy:
- Endometrioid adenocarcinoma, Grade 1.
Tumor size: 2.4 cm in greatest dimension.
Lymph-Vascular invasion: Not identified.
pTNM (UICC 7th): pT1a pN0.
Surgical margin: Negative.
- Leiomyoma.
Uterine cervix, hysterectomy:
- No evidence of malignancy.
Adnexa, right, status post-oophorectomy,
salpingectomy:
- No evidence of malignancy.
Adnexa, left, salpingo-oophorectomy:
- No evidence of malignancy.
Lymph node, dissection:
- No evidence of malignancy (0/33).

PET-CT REPORT
Examination Date: July 15, 2025 Study Type:
18F-FDG PET-CT (Whole Body)
Clinical Information: Post-hysterectomy for
endometrial cancer. Pathology: Endometrioid
adenocarcinoma Grade 1, pT1a pN0 Indication:
Post-operative surveillance
Findings:
Primary Site: No abnormal FDG uptake in the
post-hysterectomy pelvis.
Lymph Nodes:
Hypermetabolic left supraclavicular lymph node
(SUVmax 8.2, short axis 12mm)
Mild FDG uptake in mediastinal lymph node at
carina level (SUVmax 4.5, short axis 8mm)
Multiple hypermetabolic para-aortic lymph
nodes (SUVmax 6.8-9.1, largest short axis
15mm)

"TNM":{"pT":"1a","pN":”0",”cM":”1"}

©︎MATSUO LAB, THE UNIVERSITY OF TOKYO

13

2、結果 ■ Ground truth vs pTN extracted by Gemini Accuracy 99.4% Accuracy N.S. : there are not enough information in report 99.3% ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 14

2、結果 ■ Ground truth vs cM assessed by Gemini Accuracy 90.9% 判定は90%の精度 ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 15

成果 成果2：論文の投稿(under revision) 成果1：医療系の国際学会での発表 16 アクセプトされれば共有いたしたいと思います。 ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO

将来展望 ■ 医師だけでやっていくのは結構限界 ■ コラボレーションしていくには予算が必要 ■ 今回の論文を材料に大学院生でも申請できる予算に手当り次第応募していきたい ■ 個人情報を扱う基盤はできた ■ 実際にLLMのサポートで医学研究をおこなっていく ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 17

©︎MATSUO LAB, THE UNIVERSITY OF TOKYO