Authors: Aryan Bhatt, Aryan Verma
Abstract: Artificial Intelligence (AI) methods, specifically Large Language Models (LLMs), are increasingly being employed by developers and students to produce source code. Though helpful, such AI-produced code is problematic in terms of plagiarism, originality, and academic honesty. Hence, differentiating between code written by humans and code generated by AI has become vital for the prevention of plagiarism. This article provides an empirical evaluation of current AI detection tools to determine how well they can detect AI-generated code in educational and coding environments. The findings indicate that most of the tools are ineffective and not generalizable enough to be useful for detecting plagiarism. In order to deal with this problem, we suggest a number of solutions, such as fine-tuning LLMs and machine learning-based classification based on static code metrics and code embeddings obtained from Abstract Syntax Trees (AST). Our top-performing model outperforms current detectors (e.g., GPTSniffer) and gets an F1 score of 82.55. In addition to that, we carry out an ablation study to study the contribution of different source code features to detection accuracy.
DOI:
