AIMS: An Agentic AI Framework for Sim-to-Real Multi-Modal ISAC

Abstract

Multi-modal integrated sensing and communication (ISAC) enables environmental perception and reliable connectivity for intelligent wireless networks. Data-driven multi-modal ISAC models depend heavily on annotated real-world data to learn relationships across sensing and wireless observations, thereby constraining scalable deployment. Although synthetic data generation reduces the burden, adapting existing simulation pipelines to a target deployment requires consistent scene, sensing, wireless, and learning configurations, while mismatches among these coupled components impair sim-to-real transferability. To address the challenge, we propose an agentic artificial intelligence (AI) framework for sim-to-real multi-modal ISAC, named AIMS. Given a natural-language deployment request specifying the target task, deployment conditions, and real-data budget, AIMS derives a deployment-specific sim-to-real configuration and coordinates its execution to produce a deployment-specific task model. A two-agent architecture coordinates scene construction with task learning. A scene construction agent generates geographically grounded, synchronized sensing and wireless records from shared physical states, while a scene understanding agent configures task-relevant modalities and mixture-of-experts (MoE) learning for zero-shot inference or few-shot adaptation. Structured domain knowledge guides dependency-aware planning, while validation evidence supports feedback-driven revision of affected decisions. Experiments on the real-world DeepSense 6G dataset demonstrate improved vehicle detection and beam prediction over the considered simulation and fusion baselines. A separate orchestration benchmark evaluates task interpretation, dependency reasoning, and feedback-driven replanning across diverse deployment requests, showing improved plan correctness with structured domain knowledge and validation feedback.

The authors' own abstract, copied from arxiv.org.

How to cite it

Built from the details listed on this page. Check them against the paper before you submit.

Bian, Y., Zhang, K., Guo, W., Wang, Z., Song, S., Zhang, J., & Letaief, K. B. (2026). AIMS: An Agentic AI Framework for Sim-to-Real Multi-Modal ISAC. arXiv. https://arxiv.org/abs/2609.39964

Bian, Yijie, et al. "AIMS: An Agentic AI Framework for Sim-to-Real Multi-Modal ISAC." arXiv, 2026, https://arxiv.org/abs/2609.39964.

@article{bian2026aims,
  title = {AIMS: An Agentic AI Framework for Sim-to-Real Multi-Modal ISAC},
  author = {Yijie Bian and Kai Zhang and Wei Guo and Zixin Wang and Shenghui Song and Jun Zhang and Khaled B. Letaief},
  journal = {arXiv},
  year = {2026},
  url = {https://arxiv.org/abs/2609.39964},
}

What it is

Proposes AIMS, an agentic AI framework that turns a plain-language deployment request into a sim-to-real configuration for multi-modal integrated sensing and communication (ISAC) in wireless networks. A scene construction agent and a scene understanding agent coordinate sensing, wireless and learning components, and experiments on the real-world DeepSense 6G dataset show improved vehicle detection and beam prediction over simulation and fusion baselines.

Topics