Apr 20
How Hypocritical Is Your LLM judge? Listener-Speaker Asymmetries in the Pragmatic Competence of Large Language Models
★★★★★
significance 3/5
This research paper investigates the discrepancy between how large language models perform as linguistic generators versus linguistic judges. The study finds a significant asymmetry where models excel at evaluating pragmatic appropriateness but struggle to generate it, suggesting current evaluation practices are misaligned with generative capabilities.
Why it matters
Asymmetry between pragmatic judgment and generation reveals a fundamental gap in how models internalize and execute human social intelligence.
Tags
#llm evaluation #pragmatic competence #linguistic asymmetry #llm judgeRelated coverage
- Global South OpportunitiesPivotal Research Fellowship 2026 (Q3): AI Safety Research Opportunity - Global South Opportunities
- arXiv cs.AIAn Intelligent Fault Diagnosis Method for General Aviation Aircraft Based on Multi-Fidelity Digital Twin and FMEA Knowledge Enhancement
- arXiv cs.AIPExA: Parallel Exploration Agent for Complex Text-to-SQL
- arXiv cs.AIThe Power of Power Law: Asymmetry Enables Compositional Reasoning
- arXiv cs.AIOn the Existence of an Inverse Solution for Preference-Based Reductions in Argumentation