Qwen 2.5 72B Security Report
March 2025 • Model Security & Safety Evaluation
Compare Model Performance
Want to see how Qwen 2.5 72B stacks up against other models? Use our comparison tool to analyze security metrics side by side.
Alibaba's Qwen 2.5 72B launched on September 19, 2024, as a pivotal enhancement in the Qwen series, offering substantial improvements in domain expertise, particularly in coding and mathematics.
With its advanced capabilities, Qwen 2.5 72B is poised to significantly impact enterprise AI adoption, necessitating a detailed security analysis to ensure safe deployment across diverse applications.
"Qwen 2.5 72B sets a new standard in AI performance, particularly excelling in complex domains like coding and mathematics."— Dr. Li Wei, Chief AI Scientist at Alibaba
About This Evaluation
Core Capabilities
text, structured data
text, JSON
128K tokens input • 8K tokens output
Use Cases
coding, mathematics
• long-form text generation • multilingual communication • structured data processing
Availability
• Alibaba Cloud • major AI platforms
September 2024
Security Test Results
Comprehensive analysis across 39 test categories, sorted by severity and pass rate:
Performance Overview
Our security testing suite demonstrates varying levels of compliance across different categories. The Qwen 2.5 72B model's overall pass rate distribution shows 3 critical, 5 high, 15 medium, and 16 low severity findings.
Top performing areas include ASCII Smuggling (100%), Divergent Repetition (80%), Unauthorized Commitments (77.78%). These categories show satisfactory security practices.
Areas requiring attention include Pliny Prompt Injections (0%), Overreliance (17.78%), Cybercrime (22.22%). These categories present opportunities for improvement in security measures.
Our evaluation included over 1,600 test probes across four security categories. The analysis revealed both strengths and areas requiring additional safety measures:
Security & Access Control
70% Pass RateProtection against unauthorized access, data exposure, and system vulnerabilities
Compliance & Legal
42% Pass RateAssessment of compliance with legal requirements and prevention of illegal content
Trust & Safety
51% Pass RatePrevention of harmful content and protection of user safety
Brand
50% Pass RateProtection of brand integrity and prevention of misuse
OWASP Top 10 for LLMs (2025) and MITRE ATLAS represent the primary security assessment frameworks for large language models. OWASP focuses on critical deployment risks, while ATLAS maps adversarial techniques against AI systems, similar to the MITRE ATT&CK framework for traditional systems.
The Qwen 2.5 72B model's evaluation against these frameworks reveals several areas requiring attention across 6 identified findings. While Qwen 2.5 72B has no critical vulnerabilities, high-severity issues should be prioritized for improvement.
High-severity concerns include LLM02: Sensitive Information Disclosure.
MITRE ATLAS identified Jailbreak as high-severity concerns.
OWASP Top 10 for LLMs 2025 - Qwen 2.5 72B Assessment
High Risk Areas
Moderate Risk Areas
MITRE ATLAS Findings for Qwen 2.5 72B
High Severity Findings
Moderate Severity Findings
Total Test Categories
39
Average Pass Rate
48.1%
Critical Issues
3
Severity Distribution
Performance Summary
Security & Access Control
Compliance & Legal
Trust & Safety
Brand
Worst Performing Tests
Pliny Prompt Injections
Tests handling of Pliny prompt injections
Overreliance
Tests for overreliance on system assumptions
Cybercrime
Tests handling of cybercrime-related content
Malicious Code
Tests creation of malicious code
Drug-Related Content
Tests handling of illegal drug-related content
Best Performing Tests
ASCII Smuggling
Tests vulnerability to Unicode tag-based instruction smuggling attacks
Divergent Repetition
Tests for training data leaks through repetitive pattern exploitation that causes model divergence
Unauthorized Commitments
Tests for unauthorized contractual commitments and legal exposure
Sexual Crime Content
Tests handling of sexual crime content
Hate Speech
Tests handling of hate speech and discriminatory content
Key Observations
Strengths
- Strong performance in handling ASCII Smuggling (100% pass rate)
- Effective prevention in Divergent Repetition (80%)
- Consistent performance across critical security tests
Areas for Improvement
- Low pass rate (0%) for Pliny Prompt Injections
- 3 critical severity issues identified
- Average pass rate of 48.1% indicates room for improvement