Publications

Path-based link prediction methods reconstruct missing links between two vertices of a knowledge graph. They reconstruct a missing link by finding a path through the knowledge graph connecting both vertices. The path is the reasoning of the link prediction method. However, path-based link prediction methods are vulnerable to Clever Hans biases. They learn invalid reasoning patterns if these patterns are dominant and generalize well to the training and validation set. As a result, performance drops when evaluated on the real-world distribution. Whether reasoning is valid, that is, semantically relevant to the missing link and not merely factually correct, depends on the semantic concept underlying the link, which is mostly accessible through human knowledge. The paper’s approach makes the understanding of a human oracle of valid reasoning accessible while learning to predict missing links. This paper proposes the path-based link prediction method LiEr. LiEr learns human-oracle-aligned reasoning within the knowledge graph domain from preference-based human feedback. The paper demonstrates that LiEr’s prediction capability is on par with other state-of-the-art link prediction methods, while its reasoning aligns more closely with human-specified notions of valid reasoning across various benchmark reasoning tasks. In addition, a novel benchmark knowledge graph with a Clever Hans bias is introduced to evaluate the alignment of link prediction methods with human oracle-regulated understanding of valid reasoning. The paper contributes by introducing preference-based human-in-the-loop learning to path-based link prediction in knowledge graphs. It aligns the model’s behavior with human understanding of valid reasoning.