Multi-Agent System for Comprehensive Soccer Understanding

Jiayuan Rao1*, Zifeng Li1*, Haoning Wu1, Ya Zhang1, Yanfeng Wang1 , Weidi Xie1
1School of Artifitial Intelligence, Shanghai Jiao Tong University, China
Technical Report
Paper Code 🤗 Dataset arXiv
Teaser

Overview. (a) A user example of our multi-agent system, SoccerAgent, on the proposed diverse and challenging SoccerBench; (b) An example of the reasoning chain and workflow of SoccerAgent.

Abstract

Recent advancements in AI-driven soccer understanding have demonstrated rapid progress, yet existing research predominantly focuses on isolated or narrow tasks. To bridge this gap, we propose a comprehensive framework for holistic soccer understanding. Specifically, we make the following contributions in this paper: (i) we construct SoccerWiki, the first large-scale multimodal soccer knowledge base, integrating rich domain knowledge about players, teams, referees, and venues to enable knowledge-driven reasoning; (ii) we present SoccerBench, the largest and most comprehensive soccer-specific benchmark, featuring around 10K standardized multimodal~(text, image, video) multi-choice QA pairs across 13 distinct understanding tasks, curated through automated pipelines and manual verification; (iii) we introduce SoccerAgent, a novel multi-agent system that decomposes complex soccer questions via collaborative reasoning, leveraging domain expertise from SoccerWiki and achieving robust performance; (iv) extensive evaluations and ablations that benchmark state-of-the-art MLLMs on SoccerBench, highlighting the superiority of our proposed agentic system.

Benchmark

Benchmark

Proposed Benchmark Overview. SoccerBench QA Generation Pipeline. We construct multi-choice QA samples based on SoccerWiki and other existing datasets Some representative examples for each task are presented for reference.

Benchmark Generation

Data Statistics of SoccerBench. For each, we present its name, QA type, source materials, and curation strategies. Here, SN and SR-1988 represent the SoccerNet and Soccer-Replay-1988, respectively, while LLM denotes DeepSeek-v3.

Method

Architecture

SoccerAgent Architecture Overview. We design a multi-agent system to decompose and solve the given multi-modal soccer-related questions step by step with a distributed toolbox.

Results

Results1

Quantitative Comparisons on SoccerBench. Here, * indicates that we use a Commercial API (GPT-4o) as a tool in the recommend tool chain to solve the corresponding task..

Results2

Ablations on SoccerAgent. Here, gray background indicates the default configuration of SoccerAgent, while TD and EX denote Task descriptions and Execution Examples, respectively.

Qualitative Results

Results3

Qualitative Results. Here, we demonstrate several representative examples showing the entire process of tool planning and tool execution of different soccer understanding tasks.

BibTeX


        @article{rao2025soccceragent,
          title={Multi-Agent System for Comprehensive Soccer Understanding},
          author={Rao, Jiayuan and Li, Zifeng and Wu, Haoning and Zhang, Ya and Wang, Yanfeng and Xie, Weidi},
          journal={arXiv preprint arXiv:2505.03735},
          year={2025}
        }