GEval: A Modular and Extensible Evaluation Framework for Graph Embedding Techniques

Maria Angela Pellegrino, Abdulrahman Altabba, Martina Garofalo, Petar Ristoski, Michael Cochez

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


While RDF data are graph shaped by nature, most traditional Machine Learning (ML) algorithms expect data in a vector form. To transform graph elements to vectors, several graph embedding approaches have been proposed. Comparing these approaches is interesting for 1) developers of new embedding techniques to verify in which cases their proposal outperforms the state-of-art and 2) consumers of these techniques in choosing the best approach according to the task(s) the vectors will be used for. The comparison could be delayed (and made difficult) by the choice of tasks, the design of the evaluation, the selection of models, parameters, and needed datasets. We propose GEval, an evaluation framework to simplify the evaluation and the comparison of graph embedding techniques. The covered tasks range from ML tasks (Classification, Regression, Clustering), semantic tasks (entity relatedness, document similarity) to semantic analogies. However, GEval is designed to be (easily) extensible. In this article, we will describe the design and development of the proposed framework by detailing its overall structure, the already implemented tasks, and how to extend it. In conclusion, to demonstrate its operating approach, we consider the parameter tuning of the KGloVe algorithm as a use case.
Original languageEnglish
Title of host publicationThe Semantic Web
EditorsAndreas Harth, Sabrina Kirrane, Axel-Cyrille Ngonga Ngomo, Heiko Paulheim, Anisa Rula, Anna Lisa Gentile, Peter Haase, Michael Cochez
Place of PublicationCham
PublisherSpringer International Publishing
Number of pages18
ISBN (Print)978-3-030-49461-2
StatePublished - 2020
Externally publishedYes


Dive into the research topics of 'GEval: A Modular and Extensible Evaluation Framework for Graph Embedding Techniques'. Together they form a unique fingerprint.
  • DL: ICAI Discovery Lab

    van Harmelen, F. (CoPI), De Rijke, M. (CoI), Siebert, M. (CoI), Hoekstra, R. (CoPI), Tsatsaronis, G. (CoPI), Groth, P. (CoPI), Cochez, M. (CoI), Pernisch, R. (CoI), Alivanistos, D. (CoI), Mansoury, M. (CoI), van Hoof, H. (CoI), Pal, V. (CoI), Pijnenburg, T. (CoI), Mitra, P. (CoI), Bey, T. (CoI) & de Waard, A. (CoPI)


    Project: Research

Cite this