Ranking-Guided Autoregressive Modeling for Multimodal Tabular Anomaly Detection

Tabular data on the Web is frequently heterogeneous, multimodal, and published in a decentralised way, making it prone to structural and semantic inconsistencies. As such datasets increasingly serve as sources for knowledge graph construction, ensuring structural and semantic consistency prior to integration becomes critical. One approach to dealing with these issues is anomaly detection, a technique that identifies irregular patterns in a given dataset. Although several approaches exist for anomaly detection on tables, methods that capture interactions between different modalities, rather than treating each modality separately, are rare. This paper begins to address this gap by introducing a new Multimodal Large Language Model (MLLM) architecture designed for anomaly detection on multimodal tabular datasets. By combining a dual objective that is comprised of an autoregressive and a pairwise ranking loss, the model is able to efficiently learn both patterns of textual and visual data and understand cross-modal correlations. Experimental results show a performance gain of 58.4% for our architecture on a diverse set of anomaly categories. Our code and datasets are available at: https://github.com/Antonis-Georgakopoulos/multimodal-tabular-ad.