Transfer learning for biomedical named entity recognition with BioBert

Anthi Symeonidou; Viachaslau Sazonau; Paul Groth

Transfer learning for biomedical named entity recognition with BioBert

Anthi Symeonidou
, Viachaslau Sazonau
, Paul Groth

Research output: Contribution to journal › Conference article › peer-review

Abstract

We apply a transfer learning approach to biomedical named entity recognition and compare it with traditional approaches (dictionary, CRF, BiLTSM). Specifically, we build models for adverse drug reaction recognition on three datasets. We tune a pre-trained transformer model, BioBERT, on these datasets and observe the absolute F1-score improvements of 6.93, 10.46 and 13.31. This shows that, with a relatively small amount of annotated data, transfer learning can help in specialized information extraction tasks.

Original language	English
Journal	CEUR Workshop Proceedings
Volume	2451
State	Published - 2019
Externally published	Yes
Event	15th International Conference on Semantic Systems, SEMPDS 2019 - Karlsruhe, Germany Duration: Sep 9 2019 → Sep 12 2019

Keywords

Adverse drug reaction
BIO tagging
BioBERT
Drug safety
Named entity recognition
Text mining
Transfer learning

Cite this

@article{025ddc8c00e843d1afa94ada3700ef67,

title = "Transfer learning for biomedical named entity recognition with BioBert",

abstract = "We apply a transfer learning approach to biomedical named entity recognition and compare it with traditional approaches (dictionary, CRF, BiLTSM). Specifically, we build models for adverse drug reaction recognition on three datasets. We tune a pre-trained transformer model, BioBERT, on these datasets and observe the absolute F1-score improvements of 6.93, 10.46 and 13.31. This shows that, with a relatively small amount of annotated data, transfer learning can help in specialized information extraction tasks.",

keywords = "Adverse drug reaction, BIO tagging, BioBERT, Drug safety, Named entity recognition, Text mining, Transfer learning",

author = "Anthi Symeonidou and Viachaslau Sazonau and Paul Groth",

note = "Publisher Copyright: Copyright {\textcopyright} 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).; 15th International Conference on Semantic Systems, SEMPDS 2019 ; Conference date: 09-09-2019 Through 12-09-2019",

year = "2019",

language = "Ingl{\'e}s",

volume = "2451",

journal = "CEUR Workshop Proceedings",

issn = "1613-0073",

}

TY - JOUR

T1 - Transfer learning for biomedical named entity recognition with BioBert

AU - Symeonidou, Anthi

AU - Sazonau, Viachaslau

AU - Groth, Paul

PY - 2019

Y1 - 2019

N2 - We apply a transfer learning approach to biomedical named entity recognition and compare it with traditional approaches (dictionary, CRF, BiLTSM). Specifically, we build models for adverse drug reaction recognition on three datasets. We tune a pre-trained transformer model, BioBERT, on these datasets and observe the absolute F1-score improvements of 6.93, 10.46 and 13.31. This shows that, with a relatively small amount of annotated data, transfer learning can help in specialized information extraction tasks.

AB - We apply a transfer learning approach to biomedical named entity recognition and compare it with traditional approaches (dictionary, CRF, BiLTSM). Specifically, we build models for adverse drug reaction recognition on three datasets. We tune a pre-trained transformer model, BioBERT, on these datasets and observe the absolute F1-score improvements of 6.93, 10.46 and 13.31. This shows that, with a relatively small amount of annotated data, transfer learning can help in specialized information extraction tasks.

KW - Adverse drug reaction

KW - BIO tagging

KW - BioBERT

KW - Drug safety

KW - Named entity recognition

KW - Text mining

KW - Transfer learning

UR - https://www.scopus.com/pages/publications/85072883722

M3 - Artículo de la conferencia

AN - SCOPUS:85072883722

SN - 1613-0073

VL - 2451

JO - CEUR Workshop Proceedings

JF - CEUR Workshop Proceedings

T2 - 15th International Conference on Semantic Systems, SEMPDS 2019

Y2 - 9 September 2019 through 12 September 2019

ER -

Transfer learning for biomedical named entity recognition with BioBert

Abstract

Keywords

Other files and links

Fingerprint

Cite this