Authors: Rajesh Chauhan, Akshay Bhardwaj, Rohit Kumar Verma
Abstract: The spread of false information on digital platforms has surged and there is a growing demand for the adoption of accurate and deployable automated false information detection systems. But models learned in one news domain can easily suffer significant performance drop when transferred to other domains out of the scope of its training. This study compares four classical machine learning models and five deep learning architectures for within and cross domain fake-news detection. Five publicly available benchmark datasets, which include over 150,000 labelled instances, are used for experiments: LIAR, ISOT Fake News, FakeNewsNet GossipCop, WELFake, and Fake and Real News Dataset. Their models are evaluated based on classification accuracy, F1 score, cross domain performance retention, computational cost, data requirements and interpretability. The best fine-tuned RoBERTa model obtained the highest accuracy score of 97.8% on ISOT and 84.9% on the transfer task from ISOT to GossipCop, outperforming the linear SVM model by 13.7 percentage points. However, classical models are still suitable in resource-limited and interpretability sensitive scenarios, and BiLSTM with additive attention is a balanced model. The results show that model selection should not only evaluate the predictive performance of the model but also take into account the operational constraints.
DOI: http://doi.org/10.5281/zenodo.21308343
