Linking Textual Data: Tools und Workflows

Verknüpfung von Textdaten mit anderen Datenarten

Why Link Textual Data in Social Sciences?

The measure “Linking Textual Data” aims to realize the potentials of combining established, mostly structured data and textual data in social science workflows. While established types of data such as survey data or administrative data remain a crucial cornerstone of social science research, the analysis of textual data has become more common in recent years. Among other things, text analysis enables researchers to take into account a continuous, granular and potentially long-term stream of information and measurements of socially relevant issues based on, for example, newspaper articles or parliamentary speeches. The analysis of structured data thus stands to benefit from additional information provided by textual analysis, whereas textual analysis greatly gains by enriching it with information found in established types of data.

In practice, this linking of different data types remains challenging, however. Their combination relies on robust and efficient mechanisms to identify corresponding entities and concepts such as persons, organizations or locations in both structured data and textual data. Yet, common approaches to identify such entities in textual data like Entity Linking which are actively developed in disciplines like Natural Language Processing are not widely established in social science research. Potential technical barriers, limited accessibility and often unknown performance in social science research continue to pose challenges. This limits their adoption in the field and obstructs the path toward a potentially powerful addition to the toolset of social science research. In consequence, established types of data and textual data often remain unconnected.

To address these challenges, the measure “Linking Textual Data” develops tools, data and workflows to make linkage between these types of data in social science research more accessible. The created resources will facilitate innovative research by contributing to the interconnectedness of the various types of data common in KonsortSWD and beyond, thus benefiting both individual researchers and research data infrastructures by further increasing the interoperability and usefulness of utilized data.

R Packages & Tools for Entity Linking

Developed in the measure, the R packages dbpedia (https://github.com/PolMine/dbpedia) and LinkTools (https://github.com/PolMine/LinkTools) facilitate Entity Linking (via the established Entity Linking tool “DBpedia Spotlight” (https://www.dbpedia-spotlight.org/)) and Record Linkage respectively. In a nutshell, the goal of both packages is the assignment of unique identifiers to concepts of interest (e.g., persons, organizations and locations) in textual data either on the level of document metadata or within the continuous text of the documents. This approach facilitates robust linking of textual data with structured data. The packages are developed with the needs of social scientists in mind. Accordingly, the integration of these approaches into common social science workflows is an important motivation. Among others, both packages allow for the local deployment of tools, thus making the analysis of large text corpora possible. They support various input formats for textual data and take into account requirements of validation and reproducibility. With these packages, textual data can be enriched with external information to facilitate new research by making use of existing but, until now, disconnected resources.

In addition to dbpedia and LinkTools, we also develop and maintain various other R packages which are used to create, manage, disseminate and analyze large annotated textual corpora within R:

polmineR (analysis of large corpora in the CWB corpus format) (https://cran.r-project.org/package=polmineR)
RcppCWB (R Wrapper for the Corpus Workbench) (https://cran.r-project.org/package=RcppCWB)
cwbtools (installation and management of CWB corpora) (https://github.com/PolMine/cwbtools)
bignlp (linguistic annotation of large corpora) (https://github.com/PolMine/bignlp)

These packages are openly developed on GitHub. Aside from the code itself, the GitHub repositories contain further information on the installation and usage of the packages.

Textual Datasets & GermaParl Corpus

Exploring strategies to make linked textual data available to other researchers is an important motivation of “Linking Textual Data”. Aside from the general importance of sharing data to foster reproducibility and reusability, shareable linked textual data is also an important prerequisite for the development of common workflows, the documentation of tools and the overall dissemination of the discussed approaches. In this context, additional resources are developed.

GermaParl, a corpus of plenary protocols of the German Bundestag prepared within the PolMine project (https://polmine.github.io/) at the University of Duisburg-Essen, serves as an example dataset to showcase how Entity Linking can be applied to a substantively relevant and large text corpus using the dbpedia R package. To this end, the corpus has been extended in the measure, now covering all plenary protocols between 1949 and 2025, and enriched with unique identifiers for persons, organizations and locations. Moreover, GermaParl is also the foundation for workflows we develop to illustrate how linked textual data can be used in social science analysis.

We release GermaParl on Zenodo (https://doi.org/10.5281/zenodo.15495748) in two formats: A “plain” XML format and a “Corpus Workbench” format which includes unique identifiers for various entities. Currently, a third exchange format is implemented: GermaParl in the ParlaMint XML format (https://www.clarin.eu/parlamint) accommodates comprehensive linguistic annotation, including unique identifiers for persons, organizations and locations and thus acts as a model of not only how to enrich large textual corpora but also how to make them available to other researchers in an interoperable format. In addition, this new version of GermaParl will also feature Entity Linking toward the Wikidata knowledge graph, thereby broadening the scope of potentially linkable entities.

Other efforts to disseminate linked data are also explored. These include the possibilities of remote access services for remote script-based analyses as well as the provision of textual data in data formats which facilitate their graphical representation on centrally hosted platforms.

Workflows and Community Outreach

Conveying how linking textual data can be realized in practice and how linked textual data can contribute to social science research is an important goal of the measure. To this end, we created comprehensive documentation of our tools which we share as vignettes in the packages themselves. Additionally, we prepared sample workflows which we presented in various formats. Aside from presentations of data, tools and workflows as well as substantive research at conferences and workshops, we also demonstrated the potentials of GermaParl as a linked resource, both as part of the “Cookin’ with GermaParl” webinar series and in an in-person user workshop in April 2026.

Future Developments & Maintenance

Entity Linking is an active research field and as such, tools, data and workflows must be kept up to date to retain their usefulness and relevance. In addition, the reliance on the continuous availability of shared resources such as large Knowledge Graphs used in Entity Linking can pose challenges, but importantly also motivates agility. A major goal of the second funding phase of KonsortSWD – NFDI4Society is thus the maintenance of our resources and adjustments to new challenges and arising possibilities.

Summary

The analysis of both structured data like survey data and textual data can provide great insights into the social world. However, potential goes unused when we are not able to robustly link these two realms of data. The measure “Linking Textual Data” addresses this gap with tools, data and workflows aimed particularly at the needs of social science research.

Project Website

Find updates on the work of the measure at https://polmine.github.io/

Selected Tools and Data

dbpedia R Package: https://github.com/PolMine/dbpedia
LinkTools R Package: https://github.com/PolMine/LinkTools
GermaParl Corpus of Plenary Protocols of the German Bundestag: https://doi.org/10.5281/zenodo.15495748

Central Publications

Blätte, Andreas, Julia Rakers, and Christoph Leonhardt. 2022. ‘How GermaParl Evolves: Improving Data Quality by Reproducible Corpus Preparation and User Involvement’. In Proceedings of the Workshop ParlaCLARIN III within the 13th Language Resources and Evaluation Conference, eds Darja Fišer, Maria Eskevich, Jakob Lenardič, and Franciska de Jong. Marseille, France: European Language Resources Association, 7–15. https://aclanthology.org/2022.parlaclarin-1.2.
Leonhardt, Christoph, and Andreas Blätte. 2023. ‘Evaluating the Quality of the GermaParl Corpus of Plenary Protocols (v2.0.0)’. In Proceedings of the 3rd Workshop on Computational Linguistics for the Political and Social Sciences, eds Christopher Klamm, Gabriella Lapesa, Valentin Gold, Theresa Gessler, and Simone Paolo Ponzetto. Ingolstadt, Germany: Association for Computational Linguistics, 88–100. https://aclanthology.org/2023.cpss-1.9.
Leonhardt, Christoph, and Andreas Blätte. 2024. ‘The dbpedia R Package: An Integrated Workflow for Entity Linking (for ParlaMint Corpora)’. In Proceedings of the IV Workshop on Creating, Analysing, and Increasing Accessibility of Parliamentary Corpora (ParlaCLARIN) @ LREC-COLING 2024, eds Darja Fišer, Maria Eskevich, and David Bordon. Turin, Italy: ELRA and ICCL, 133–44. https://aclanthology.org/2024.parlaclarin-1.20.
Leonhardt, Christoph, and Andreas Blätte. 2026. ‘Towards ParlaMint-DE: Improving the Interoperability of the GermaParl Corpus of Plenary Protocols of the German Bundestag’. In Proceedings of the ParlaCLARIN V Workshop on Interoperability, Multilinguality, and Multimodality in Parliamentary Corpora @ LREC 2026, eds Maria Eskevich, Vincent Vandeghinste, and David Bordon. Palma de Mallorca, Spain: ELRA Language Resources Association (ELRA), 31–43.