Multimodal Foundation Model for German Property Invoice Checking

Short description of the innovation study

PropertyExpert GmbH (PX) is a service provider in the insurance sector, specialising in invoice checking for property damages. With its current computer vision and natural language processing models, PX already manages to process 30% of incoming claims automatically. Part of the model family is a small-sized language foundation model for semi-structured documents, specialised in processing German craft language and handling numbers effectively. In this innovation study, PX aims to substantially improve this foundation model by adding multimodality and combining it with Llama 3.1 to incorporate question-answering and explanation generation capabilities. This enhancement will significantly increase the automation rate in the invoice checking process by improving the detection of incorrect invoices and automatically generating explanations for negative verdicts.

Organisations involved: 

Technology expert: PropertyExpert GmbH