Authors: Dr. C K Gomathy, Vishnuvel Ragavan K E C, Ghiridharan S
Abstract: Big Data computing has become a cornerstone technology driving digital transformation across industries. This paper provides a comprehensive exploration of Big Data computing paradigms, architectural frameworks, processing technologies, and contemporary challenges. We examine the evolution from traditional data warehousing to modern cloud-native architectures, analyze key processing frameworks including Apache Spark, Hadoop, Flink, and Kafka, and discuss real-time analytics capabilities. Furthermore, this paper addresses critical challenges including data privacy, security, scalability, and regulatory compliance, while highlighting emerging trends such as AI-ML integration, federated learning, and edge computing. Our findings demonstrate that hybrid approaches combining on-premise and cloud solutions are becoming mainstream, with approximately 65% of enterprises adopting Hadoop and Spark in tandem. This research concludes by identifying future research directions necessary to address emerging complexities in distributed data systems and regulatory landscapes.
DOI: http://doi.org/10.5281/zenodo.17838338
