Authors: Srinivasa Chakravarthy Seethala
Abstract: This research paper investigates the role of machine learning enhanced query optimization techniques in improving the performance, scalability, and reliability of SQL and NoSQL databases used for high volume big data processing in financial and healthcare applications. Traditional rule based and cost based query optimizers often struggle to adapt to dynamic workloads, heterogeneous data distributions, and rapidly evolving access patterns that characterize modern financial transactions and healthcare data ecosystems. This study addresses the research problem of how adaptive machine learning driven optimization models can overcome these limitations by learning from historical query execution patterns, system telemetry, and workload characteristics. A mixed methodology is adopted, combining conceptual framework design, algorithmic modeling, and comparative performance analysis across representative SQL and NoSQL environments handling large scale transactional and analytical workloads. The findings demonstrate that machine learning enhanced optimizers significantly reduce query latency, improve resource utilization efficiency, and enhance workload predictability under peak data volumes compared to traditional optimization approaches. The paper highlights key innovations such as predictive cost modeling, adaptive index selection, and real time execution plan refinement driven by supervised and reinforcement learning techniques. From an academic perspective, this research contributes to the evolving discourse on intelligent data management systems by extending optimization theory into data driven adaptive architectures. From an industry standpoint, the results provide actionable insights for designing resilient, high performance data platforms capable of supporting mission critical financial and healthcare operations where accuracy, compliance, and responsiveness are paramount.
DOI: http://doi.org/10.5281/zenodo.17938651
