dorsal/arxiv
View SchemaCHASE: LLM Agents for Dissecting Malicious PyPI Packages
| Authors | Takaaki Toda, Tatsuya Mori |
|---|---|
| Categories | |
| ArXiv ID | 2601.06838vv1 |
| URL | https://arxiv.org/abs/2601.06838 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
Modern software package registries like PyPI have become critical infrastructure for software development, but are increasingly exploited by threat actors distributing malicious packages with sophisticated multi-stage attack chains. While Large Language Models (LLMs) offer promising capabilities for automated code analysis, their application to security-critical malware detection faces fundamental challenges, including hallucination and context confusion, which can lead to missed detections or false alarms. We present CHASE (Collaborative Hierarchical Agents for Security Exploration), a high-reliability multi-agent architecture that addresses these limitations through a Plan-and-Execute coordination model, specialized Worker Agents focused on specific analysis aspects, and integration with deterministic security tools for critical operations. Our key insight is that reliability in LLM-based security analysis emerges not from improving individual model capabilities but from architecting systems that compensate for LLM weaknesses while leveraging their semantic understanding strengths. Evaluation on a dataset of 3,000 packages (500 malicious, 2,500 benign) demonstrates that CHASE achieves 98.4% recall with only 0.08% false positive rate, while maintaining a practical median analysis time of 4.5 minutes per package, making it suitable for operational deployment in automated package screening. Furthermore, we conducted a survey with cybersecurity professionals to evaluate the generated analysis reports, identifying their key strengths and areas for improvement. This work provides a blueprint for building reliable AI-powered security tools that can scale with the growing complexity of modern software supply chains. Our project page is available at https://t0d4.github.io/CHASE-AIware25/
{
"annotation_id": "4f2b3ea1-4bc0-49b4-b542-9dc407ef3a24",
"date_created": "2026-02-17T05:53:08.510000Z",
"date_modified": "2026-02-17T05:53:08.510000Z",
"file_hash": "d75fc6da9eb2be1d234251517bf4e12f0e62cfb3bb53b584cb56de4d932fbe38",
"private": false,
"record": {
"abstract": "Modern software package registries like PyPI have become critical infrastructure for software development, but are increasingly exploited by threat actors distributing malicious packages with sophisticated multi-stage attack chains. While Large Language Models (LLMs) offer promising capabilities for automated code analysis, their application to security-critical malware detection faces fundamental challenges, including hallucination and context confusion, which can lead to missed detections or false alarms. We present CHASE (Collaborative Hierarchical Agents for Security Exploration), a high-reliability multi-agent architecture that addresses these limitations through a Plan-and-Execute coordination model, specialized Worker Agents focused on specific analysis aspects, and integration with deterministic security tools for critical operations. Our key insight is that reliability in LLM-based security analysis emerges not from improving individual model capabilities but from architecting systems that compensate for LLM weaknesses while leveraging their semantic understanding strengths. Evaluation on a dataset of 3,000 packages (500 malicious, 2,500 benign) demonstrates that CHASE achieves 98.4% recall with only 0.08% false positive rate, while maintaining a practical median analysis time of 4.5 minutes per package, making it suitable for operational deployment in automated package screening. Furthermore, we conducted a survey with cybersecurity professionals to evaluate the generated analysis reports, identifying their key strengths and areas for improvement. This work provides a blueprint for building reliable AI-powered security tools that can scale with the growing complexity of modern software supply chains. Our project page is available at https://t0d4.github.io/CHASE-AIware25/",
"arxiv_id": "2601.06838",
"authors": [
"Takaaki Toda",
"Tatsuya Mori"
],
"categories": [
"cs.CR",
"cs.SE"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "CHASE: LLM Agents for Dissecting Malicious PyPI Packages",
"url": "https://arxiv.org/abs/2601.06838",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "4304966f-7e67-468c-b074-481ebfcc87d7",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}