Fingerprint Feature Extraction of Electronic Medical Records Based on Few-Shot Named Entity Recognition Technology

doi:10.3969/j.issn.1671-1122.2024.10.007

Abstract

Abstract:

With the promulgation and implementation of the “Personal Information Protection Law of the People’s Republic of China” “Data Security Law of the People’s Republic of China” and other relevant laws and regulations, electronic medical record data protection has attracted much attention. Fast and efficient identification of electronic medical records is the first link of data protection and an important research topic in the field of data security. This paper proposed an electronic medical record fingerprint feature extraction method based on few-shot named entity recognition technology. First, the encoder was trained through a public dataset to obtain a broad text feature space. Subsequently, the encoder was fine-tuned using the electronic medical record dataset, and the entity type label was characterized by a prototype network. Finally, the fingerprint feature of “entity type + entity set” was obtained by extracting the electronic medical record feature. The experimental results show that the method has excellent performance on the I2B2 dataset, surpassing other models and effectively improving the privacy protection ability of electronic medical record dataset.

Key words: data security, electronic medical records, comparative learning, named entity recognition, few-shot learning

CLC Number:

TP309

WANG Yaxin, ZHANG Jian. Fingerprint Feature Extraction of Electronic Medical Records Based on Few-Shot Named Entity Recognition Technology[J]. Netinfo Security, 2024, 24(10): 1537-1543.

Figures/Tables 8

References 23

[1]	OH J, KIM S, HO N, et al. Understanding Cross-Domain Few-Shot Learning Based on Domain Similarity and Few-Shot Difficulty[J]. Advances in Neural Information Processing Systems, 2022, 35: 2622-2636.
[2]	ZHENG Hao, WANG Runqi, LIU Jianzhuang, et al. Cross-Level Distillation and Feature Denoising for Cross-Domain Few-Shot Classification[EB/OL]. (2023-11-04)[2024-05-30]. https://arxiv.org/abs/2311.02392v1.
[3]	JANG H, LEE H, SHIN J. Unsupervised Meta-Learning via Few-Shot Pseudo-Supervised Contrastive Learning[EB/OL]. (2023-03-02)[2024-05-30]. https://arxiv.org/abs/2303.00996v1.
[4]	MIKOLOV T, CHEN Kai, CORRADO G, et al. Efficient Estimation of Word Representations in Vector Space[EB/OL]. (2013-09-07)[2024-05-30]. https://arxiv.org/abs/1301.3781v3.
[5]	ZHANG Yuntao, GONG Ling, WANG Yongcheng. An Improved TF-IDF Approach for Text Classification[J]. Journal of Zhejiang University Science, 2005, 6(1): 49-55.
[6]	MIKOLOV T, SUTSKEVER I, CHEN Kai, et al. Distributed Representations of Words and Phrases and Their Compositionality[EB/OL]. (2013-10-16)[2024-05-30]. https://arxiv.org/abs/1310.4546v1.
[7]	COLLOBERT R, WESTON J, BOTTOU L, et al. Natural Language Processing (Almost) from Scratch[J]. Journal of Machine Learning Research, 2011, 12: 2493-2537.
[8]	LAMPLE G, BALLESTEROS M, SUBRAMANIAN S, et al. Neural Architectures for Named Entity Recognition[EB/OL]. (2016-04-07)[2024-05-30]. https://arxiv.org/abs/1603.01360v3.
[9]	CHIU J P C, NICHOLS E. Named Entity Recognition with Bidirectional LSTM-CNNS[J]. Transactions of the Association for Computational Linguistics, 2016, 4: 357-370.
[10]	HUANG Zhiheng, XU Wei, YU Kai. Bidirectional LSTM-CRF Models for Sequence Tagging[EB/OL]. (2015-08-09)[2024-05-30]. https://arxiv.org/abs/1508.01991v1.
[11]	CUI Leyang, WU Yu, LIU Jian, et al. Template-Based Named Entity Recognition Using BART[EB/OL]. (2021-06-03)[2024-05-30]. https://arxiv.org/abs/2106.01760v1.
[12]	DAS S S S, KATIYAR A, PASSONNEAU R J, et al. CONTaiNER: Few-Shot Named Entity Recognition via Contrastive Learning[EB/OL]. (2021-09-15)[2024-05-30]. https://arxiv.org/abs/2109.07589v2.
[13]	LI Yongqi, YU Yu, QIAN Tieyun. Type-Aware Decomposed Framework for Few-Shot Named Entity Recognition[EB/OL]. (2023-02-13)[2024-05-30]. https://arxiv.org/abs/2302.06397v2.
[14]	SNELL J, SWERSKY K, ZEMEL R S. Prototypical Networks for Few-Shot Learning[EB/OL]. (2017-03-15)[2024-05-30]. https://arxiv.org/abs/1703.05175.
[15]	GUTMANN M, HYVÄRINEN A. Noise-Contrastive Estimation: A New Estimation Principle for Unnormalized Statistical Models[J]. Journal of Machine Learning Research, 2010, 9: 297-304.
[16]	VAN D O A, LI Yazhe, VINYALS O, et al. Representation Learning with Contrastive Predictive Coding[EB/OL]. (2018-07-10)[2024-05-30]. https://arxiv.org/abs/1807.03748v2.
[17]	SRINIVASA R S, CHO J, YANG Chouchang, et al. CWCL: Cross-Modal Transfer with Continuously Weighted Contrastive Loss[EB/OL]. (2023-09-26)[2024-05-30]. https://api.semanticscholar.org/CorpusID:262826006.
[18]	WEISCHEDEL R, PALMER M, MARCUS M, et al. OntoNotes Release 5.0[EB/OL]. (2013-10-16)[2024-05-30]. https://doi.org/10.35111/xmhb-2b84.
[19]	STUBBS A, UZUNER Ö. Annotating Longitudinal Clinical Narratives for De-Identification: The 2014 I2b2/UTHealth Corpus[J]. Journal of Biomedical Informatics, 2015, 58: S20-S29.
[20]	TÄNZER M, RUDER S, REI M. Memorisation Versus Generalisation in Pre-Trained Language Models[EB/OL]. (2021-04-16)[2024-05-30]. https://arxiv.org/abs/2105.00828v2.
[21]	YANG Yi, KATIYAR A. Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning[EB/OL]. (2020-10-06)[2024-05-30]. https://arxiv.org/abs/2010.02405v1.
[22]	MA Tingting, JIANG Huiqiang, WU Qianhui, et al. Decomposed Meta-Learning for Few-Shot Named Entity Recognition[EB/OL]. (2022-04-12)[2024-05-30]. https://arxiv.org/abs/2204.05751v2.
[23]	MA Jie, BALLESTEROS M, DOSS S, et al. Label Semantics for Few Shot Named Entity Recognition[EB/OL]. (2022-04-16)[2024-05-30]. https://arxiv.org/abs/2203.08985v1.

数据集	领域	类	语句	实体
OntoNotes	通用	18	76700	104200
I2B2	医学	23	140800	29200

对比学习损失函数	模式	1-shot		5-shot
对比学习损失函数	模式	召回率	F1分数	召回率	F1分数
NCE损失函数	IO	12.03 %	5.45 %	12.51 %	6.89 %
	BIO	14.22 %	6.12 %	18.86 %	9.37 %
	BIOES	13.35 %	6.31 %	13.68 %	7.37 %
InfoNCE 损失函数	IO	18.90 %	8.69 %	28.17 %	15.38 %
	BIO	23.60 %	10.65 %	34.93 %	17.14 %
	BIOES	23.74 %	11.38 %	31.20 %	16.82 %
CWCL 损失函数	IO	41.04 %	19.20 %	49.52 %	26.86 %
	BIO	43.15 %	19.84 %	51.60 %	25.38 %
	BIOES	36.72 %	17.87 %	44.17 %	23.74 %

模型	F1分数
模型	1-shot	5-shot
ProtoBERT^[20]	12.27 %	17.03 %
NNshot^[21]	14.19 %	21.65 %
DecomposedMetaNER^[22]	16.53 %	19.78 %
FSLS^[23]	17.96 %	23.29 %
本文模型	19.20 %	25.38 %