AuDQ: Auditable Data Quality through Agentic Rule Discovery

By Jacopo Biggiogera, Nicolas Colombo, Krati Saxena, Bhaskarjit Sarmah, Stefano Pasquali

Rating

1667
Battle Count: 50

Relevance

6/10
Highly relevant for data integrity in trading pipelines, risk management, and regulatory compliance (e.g., AML, settlement cycles). Ensures that input data for quantitative models is semantically correct and auditable, reducing operational risk.

Implementation Complexity

8/10
Requires a sophisticated agentic architecture with orchestrators, subagents, ontology generation, and a central artifact repository. Integration with LLMs and database profiling tools adds significant engineering overhead.

Reproducibility

3/5
The paper describes the architecture, prompts, and evaluation protocol in detail. It uses public datasets (Berka, rel-HM). However, no public code repository URL is provided in the text, and the specific implementation of the agentic harness is proprietary to Domyn.

Abstract

High-stakes tabular pipelines rely on scattered, static quality checks that are hard to evolve and audit against policy intent. Regulated domains particularly suffer as the LLM-era automation raises expectations for traceability and accountability. We propose a vision for an ontology-grounded, repository-native data quality layer in which rules are first-class artifacts: versioned, provenance-linked to governance texts, and re-playable through immutable execution traces. As a first step toward validating this vision, we present a concrete implementation of its agentic rule-discovery core with two rule acquisition modes (Full-Agentic and Guided) and a human-in-the-loop (HITL) review interface. We define an evaluation framework and run experiments addressing whether our fully autonomous, divide-and-conquer agentic rule discovery produces rules of the same quality and diversity as those specified by a human expert, reporting preliminary results on two real world industry datasets in the finance and retail domains.

The interactive Everscope explorer (charts, battles, favorites) loads below.