SkySim: A ROS2-based Simulation Environment for Natural Language Control of Drone Swarms using Large Language Models

Aditya Shibu; Marah Saleh; Mohamed Al-Musleh; Nidhal Abdulaziz

doi:10.13140/RG.2.2.34344.84484

SkySim: A ROS2-based Simulation Environment for Natural Language Control of Drone Swarms using Large Language Models

Aditya Shibu
, Marah Saleh
, Mohamed Al-Musleh
, Nidhal Abdulaziz

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Unmanned Aerial Vehicle (UAV) swarms offer versatile applications in logistics, agriculture, and surveillance, yet controlling them requires expert knowledge for safety and feasibility. Traditional static methods limit adaptability, while Large Language Models (LLMs) enable natural language control but generate unsafe trajectories due to lacking physical grounding. This paper introduces SkySim, a ROS2-based simulation framework in Gazebo that decouples LLM high-level planning from low-level safety enforcement. Using Gemini 3.5 Pro, SkySim translates user commands (e.g., "Form a circle") into spatial waypoints, informed by real-time drone states. An Artificial Potential Field (APF) safety filter applies minimal adjustments for collision avoidance, kinematic limits, and geo-fencing, ensuring feasible execution at 20 Hz. Experiments with swarms of 3, 10, and 30 Crazyflie drones validate spatial reasoning accuracy (100% across tested geometric primitives), real-time collision prevention, and scalability. SkySim empowers non-experts to iteratively refine behaviors, bridging AI cognition with robotic safety for dynamic environments. Future work targets hardware integration.

Original language	English
Title of host publication	Drones and Unmanned Systems
Subtitle of host publication	Proceedings of the 2nd International Conference on Drones and Unmanned Systems (DAUS' 2026)
Editors	Sergey Y. Yurish
Publisher	International Frequency Sensor Association (IFSA)
Pages	150-156
Number of pages	7
ISBN (Print)	9788409820313
DOIs	https://doi.org/10.13140/RG.2.2.34344.84484
Publication status	Published - 27 Feb 2026
Event	2nd International Conference on Drones and Unmanned Systems 2026 - Salzburg, Austria Duration: 25 Feb 2026 → 27 Feb 2026 https://daus-conference.com/

Conference

Conference	2nd International Conference on Drones and Unmanned Systems 2026
Abbreviated title	DAUS' 2026
Country/Territory	Austria
City	Salzburg
Period	25/02/26 → 27/02/26
Internet address	https://daus-conference.com/

Keywords

cs.RO
cs.HC
Micro and Mini drone control
Large Language Models (LLMs)
UAV Swarms
Natural Language Control
ROS2
Gazebo Simulations

Access to Document

10.13140/RG.2.2.34344.84484

Cite this

Shibu, A., Saleh, M., Al-Musleh, M., & Abdulaziz, N. (2026). SkySim: A ROS2-based Simulation Environment for Natural Language Control of Drone Swarms using Large Language Models. In S. Y. Yurish (Ed.), Drones and Unmanned Systems: Proceedings of the 2nd International Conference on Drones and Unmanned Systems (DAUS' 2026) (pp. 150-156). International Frequency Sensor Association (IFSA). https://doi.org/10.13140/RG.2.2.34344.84484

Shibu, Aditya ; Saleh, Marah ; Al-Musleh, Mohamed et al. / SkySim: A ROS2-based Simulation Environment for Natural Language Control of Drone Swarms using Large Language Models. Drones and Unmanned Systems: Proceedings of the 2nd International Conference on Drones and Unmanned Systems (DAUS' 2026). editor / Sergey Y. Yurish. International Frequency Sensor Association (IFSA), 2026. pp. 150-156

@inproceedings{78af91e00a2c413f80d8f4f0b98fce1c,

title = "SkySim: A ROS2-based Simulation Environment for Natural Language Control of Drone Swarms using Large Language Models",

abstract = "Unmanned Aerial Vehicle (UAV) swarms offer versatile applications in logistics, agriculture, and surveillance, yet controlling them requires expert knowledge for safety and feasibility. Traditional static methods limit adaptability, while Large Language Models (LLMs) enable natural language control but generate unsafe trajectories due to lacking physical grounding. This paper introduces SkySim, a ROS2-based simulation framework in Gazebo that decouples LLM high-level planning from low-level safety enforcement. Using Gemini 3.5 Pro, SkySim translates user commands (e.g., {"}Form a circle{"}) into spatial waypoints, informed by real-time drone states. An Artificial Potential Field (APF) safety filter applies minimal adjustments for collision avoidance, kinematic limits, and geo-fencing, ensuring feasible execution at 20 Hz. Experiments with swarms of 3, 10, and 30 Crazyflie drones validate spatial reasoning accuracy (100\% across tested geometric primitives), real-time collision prevention, and scalability. SkySim empowers non-experts to iteratively refine behaviors, bridging AI cognition with robotic safety for dynamic environments. Future work targets hardware integration.",

keywords = "cs.RO, cs.HC, Micro and Mini drone control, Large Language Models (LLMs), UAV Swarms, Natural Language Control, ROS2, Gazebo Simulations",

author = "Aditya Shibu and Marah Saleh and Mohamed Al-Musleh and Nidhal Abdulaziz",

year = "2026",

month = feb,

day = "27",

doi = "10.13140/RG.2.2.34344.84484",

language = "English",

isbn = "9788409820313",

pages = "150--156",

editor = "Yurish, \{Sergey Y.\}",

booktitle = "Drones and Unmanned Systems",

publisher = "International Frequency Sensor Association (IFSA)",

address = "Spain",

note = "2nd International Conference on Drones and Unmanned Systems 2026, DAUS' 2026 ; Conference date: 25-02-2026 Through 27-02-2026",

url = "https://daus-conference.com/",

}

Shibu, A, Saleh, M, Al-Musleh, M & Abdulaziz, N 2026, SkySim: A ROS2-based Simulation Environment for Natural Language Control of Drone Swarms using Large Language Models. in SY Yurish (ed.), Drones and Unmanned Systems: Proceedings of the 2nd International Conference on Drones and Unmanned Systems (DAUS' 2026). International Frequency Sensor Association (IFSA), pp. 150-156, 2nd International Conference on Drones and Unmanned Systems 2026, Salzburg, Austria, 25/02/26. https://doi.org/10.13140/RG.2.2.34344.84484

SkySim: A ROS2-based Simulation Environment for Natural Language Control of Drone Swarms using Large Language Models. / Shibu, Aditya; Saleh, Marah; Al-Musleh, Mohamed et al.
Drones and Unmanned Systems: Proceedings of the 2nd International Conference on Drones and Unmanned Systems (DAUS' 2026). ed. / Sergey Y. Yurish. International Frequency Sensor Association (IFSA), 2026. p. 150-156.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - SkySim: A ROS2-based Simulation Environment for Natural Language Control of Drone Swarms using Large Language Models

AU - Shibu, Aditya

AU - Saleh, Marah

AU - Al-Musleh, Mohamed

AU - Abdulaziz, Nidhal

PY - 2026/2/27

Y1 - 2026/2/27

N2 - Unmanned Aerial Vehicle (UAV) swarms offer versatile applications in logistics, agriculture, and surveillance, yet controlling them requires expert knowledge for safety and feasibility. Traditional static methods limit adaptability, while Large Language Models (LLMs) enable natural language control but generate unsafe trajectories due to lacking physical grounding. This paper introduces SkySim, a ROS2-based simulation framework in Gazebo that decouples LLM high-level planning from low-level safety enforcement. Using Gemini 3.5 Pro, SkySim translates user commands (e.g., "Form a circle") into spatial waypoints, informed by real-time drone states. An Artificial Potential Field (APF) safety filter applies minimal adjustments for collision avoidance, kinematic limits, and geo-fencing, ensuring feasible execution at 20 Hz. Experiments with swarms of 3, 10, and 30 Crazyflie drones validate spatial reasoning accuracy (100% across tested geometric primitives), real-time collision prevention, and scalability. SkySim empowers non-experts to iteratively refine behaviors, bridging AI cognition with robotic safety for dynamic environments. Future work targets hardware integration.

AB - Unmanned Aerial Vehicle (UAV) swarms offer versatile applications in logistics, agriculture, and surveillance, yet controlling them requires expert knowledge for safety and feasibility. Traditional static methods limit adaptability, while Large Language Models (LLMs) enable natural language control but generate unsafe trajectories due to lacking physical grounding. This paper introduces SkySim, a ROS2-based simulation framework in Gazebo that decouples LLM high-level planning from low-level safety enforcement. Using Gemini 3.5 Pro, SkySim translates user commands (e.g., "Form a circle") into spatial waypoints, informed by real-time drone states. An Artificial Potential Field (APF) safety filter applies minimal adjustments for collision avoidance, kinematic limits, and geo-fencing, ensuring feasible execution at 20 Hz. Experiments with swarms of 3, 10, and 30 Crazyflie drones validate spatial reasoning accuracy (100% across tested geometric primitives), real-time collision prevention, and scalability. SkySim empowers non-experts to iteratively refine behaviors, bridging AI cognition with robotic safety for dynamic environments. Future work targets hardware integration.

KW - cs.RO

KW - cs.HC

KW - Micro and Mini drone control

KW - Large Language Models (LLMs)

KW - UAV Swarms

KW - Natural Language Control

KW - ROS2

KW - Gazebo Simulations

U2 - 10.13140/RG.2.2.34344.84484

DO - 10.13140/RG.2.2.34344.84484

M3 - Conference contribution

SN - 9788409820313

SP - 150

EP - 156

BT - Drones and Unmanned Systems

A2 - Yurish, Sergey Y.

PB - International Frequency Sensor Association (IFSA)

T2 - 2nd International Conference on Drones and Unmanned Systems 2026

Y2 - 25 February 2026 through 27 February 2026

ER -

Shibu A, Saleh M, Al-Musleh M , Abdulaziz N. SkySim: A ROS2-based Simulation Environment for Natural Language Control of Drone Swarms using Large Language Models. In Yurish SY, editor, Drones and Unmanned Systems: Proceedings of the 2nd International Conference on Drones and Unmanned Systems (DAUS' 2026). International Frequency Sensor Association (IFSA). 2026. p. 150-156 doi: 10.13140/RG.2.2.34344.84484

SkySim: A ROS2-based Simulation Environment for Natural Language Control of Drone Swarms using Large Language Models

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this