LLM-BasedUnitTestGeneration:ASurveyof Approaches,EvaluationPractices,andReportingGaps

Uncategorized

Authors: Khushal Soni

Abstract: Unit tests are cheap to run but expensive to write, and large language models (LLMs) have become a popular way to write them automatically. Papers on the topic now use different languages, benchmarks and metrics, so their headline numbers cannot be lined up against each other. This survey reviews nine LLM-based unit test generation approaches published between 2023 and 2026, together with one benchmark and three earlier surveys. It groups the approaches into five families: prompt-only generation, generate-validate-repair loops, analysis or coverage-guided generation, hybrids with search-based testing, and industrial pipelines. For each study it records the setting, the technique and the result as reported by the authors, and it separates what the authors measured from what this survey interprets. The comparison shows that coverage is the most common metric, that mutation score is rarely central, that human or developer judgement appears in only a few studies, and that closed commercial models make some results hard to repeat. A small but telling data point is that, in one study, only 24.8% of the tests written by default ChatGPT passed when executed. The paper closes with a reporting checklist and a list of research directions. It is a literature survey, and no experiments were run.

DOI: https://doi.org/10.5281/zenodo.24364824

× How can I help you?