dorsal/arxiv
View SchemaAutonomous QA Agent: A Retrieval-Augmented Framework for Reliable Selenium Script Generation
| Authors | Dudekula Kasim Vali |
|---|---|
| Categories | |
| ArXiv ID | 2601.06034vv1 |
| URL | https://arxiv.org/abs/2601.06034 |
| License | http://creativecommons.org/licenses/by/4.0/ |
Abstract
Software testing is critical in the software development lifecycle, yet translating requirements into executable test scripts remains manual and error-prone. While Large Language Models (LLMs) can generate code, they often hallucinate non-existent UI elements. We present the Autonomous QA Agent, a Retrieval-Augmented Generation (RAG) system that grounds Selenium script generation in project-specific documentation and HTML structure. By ingesting diverse formats (Markdown, PDF, HTML) into a vector database, our system retrieves relevant context before generation. Evaluation on 20 e-commerce test scenarios shows our RAG approach achieves 100% (20/20) syntax validity and 90% (18/20, 95% CI: [85%, 95%], p < 0.001) execution success, compared to 30% for standard LLM generation. While our evaluation is limited to a single domain, our method significantly reduces hallucinations by grounding generation in actual DOM structure, demonstrating RAG's potential for automated UI testing.
{
"annotation_id": "c57a950a-ff79-4421-9350-11e58377ea84",
"date_created": "2026-02-17T05:53:04.727000Z",
"date_modified": "2026-02-17T05:53:04.727000Z",
"file_hash": "707cc2795606e85a67e0d1acfa8b1592019d5bf6652942ce44aed65128ea16da",
"private": false,
"record": {
"abstract": "Software testing is critical in the software development lifecycle, yet translating requirements into executable test scripts remains manual and error-prone. While Large Language Models (LLMs) can generate code, they often hallucinate non-existent UI elements. We present the Autonomous QA Agent, a Retrieval-Augmented Generation (RAG) system that grounds Selenium script generation in project-specific documentation and HTML structure. By ingesting diverse formats (Markdown, PDF, HTML) into a vector database, our system retrieves relevant context before generation. Evaluation on 20 e-commerce test scenarios shows our RAG approach achieves 100% (20/20) syntax validity and 90% (18/20, 95% CI: [85%, 95%], p \u003c 0.001) execution success, compared to 30% for standard LLM generation. While our evaluation is limited to a single domain, our method significantly reduces hallucinations by grounding generation in actual DOM structure, demonstrating RAG\u0027s potential for automated UI testing.",
"arxiv_id": "2601.06034",
"authors": [
"Dudekula Kasim Vali"
],
"categories": [
"cs.SE",
"cs.AI"
],
"license": "http://creativecommons.org/licenses/by/4.0/",
"title": "Autonomous QA Agent: A Retrieval-Augmented Framework for Reliable Selenium Script Generation",
"url": "https://arxiv.org/abs/2601.06034",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "439a0d7a-c46f-45f0-9ccc-6a680c18183b",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}