Authors: Patrick Collins, Brian Campbell, Eric Wallace, Chaitanya Srinivas, Niharika
Abstract: The rapid growth of enterprise data has increased the complexity of documenting datasets, data structures, business definitions, data pipelines, metadata, and data lineage across heterogeneous information environments. Traditional data documentation practices frequently depend on manual processes, domain experts, and static documentation methods, resulting in incomplete, inconsistent, outdated, and difficult-to-maintain data descriptions. This research proposes an Automated Enterprise Data Documentation Using Generative Artificial Intelligence framework that leverages generative AI and natural language processing to automate the creation, enrichment, and maintenance of enterprise data documentation. The proposed framework integrates data discovery, schema analysis, metadata extraction, semantic interpretation, data lineage identification, business-term generation, and natural-language documentation into a unified architecture. Generative AI models are utilized to transform technical metadata and structural information into human-readable descriptions of datasets, tables, columns, relationships, transformation processes, and business rules. The framework also incorporates validation mechanisms to improve documentation accuracy, consistency, traceability, and governance. Automated documentation can be continuously updated when data structures, pipelines, or metadata change, thereby reducing documentation maintenance effort and improving data discoverability. The proposed approach supports data engineers, data stewards, analysts, governance teams, and business users by providing accessible and context-aware descriptions of enterprise data assets. The framework aims to establish a scalable and intelligent documentation process that improves metadata management, knowledge sharing, data governance, and organizational readiness for AI-driven data management.
DOI: http://doi.org/10.5281/zenodo.23075640
