Automating Citation Placement with Natural Language Processing and Transformers

In scientific writing, references are crucial in supporting claims, spotlighting evidence, and highlighting research gaps. However, where to add a reference and which reference to cite are subjectively chosen by the papers’ authors; thus the automation of the task is challenging and requires proper investigations. This paper focuses on the automatic placement of references, considering its diverse approaches depending on writing style and community norms, and investigates the use of transformers and Natural Language Processing heuristics to predict i) if a reference is needed in a scientific statement, and ii) where the reference should be placed within the statement. For this investigation, this paper investigates two techniques, namely Mask-filling (MF) and Named Entity Recognition (NER), and provides insights on how to solve this task.